Practical Data Science with Python Training

Chapter 1 - Python for Data Science

• Python Data Science-Centric Libraries
• SciPy
• NumPy
• pandas
• Scikit-learn
• Matplotlib
• Seaborn
• Python Dev Tools and REPLs
• IPython
• Jupyter Notebooks
• Anaconda
• Summary

Chapter 2 - Defining Data Science

• What is Data Science?
• Data Science, Machine Learning, AI?
• The Data-Related Roles
• The Data Science Ecosystem
• Tools of the Trade
• Who is a Data Scientist?
• Data Scientists at Work
• Examples of Data Science Projects
• An Example of a Data Product
• Applied Data Science at Google
• Data Science Gotchas
• Summary

Chapter 3 - Data Processing Phases

• Typical Data Processing Pipeline
• Data Discovery Phase
• Data Harvesting Phase
• Data Priming Phase
• Exploratory Data Analysis
• Model Planning Phase
• Model Building Phase
• Communicating the Results
• Production Roll-out
• Data Logistics and Data Governance
• Data Processing Workflow Engines
• Apache Airflow
• Data Lineage and Provenance
• Apache NiFi
• Summary

Chapter 4 - Practical Introduction to NumPy

• NumPy
• The First Take on NumPy Arrays
• The ndarray Data Structure
• Getting Help
• Understanding Axes
• Indexing Elements in a NumPy Array
• Understanding Types
• Re-Shaping
• Commonly Used Array Metrics
• Commonly Used Aggregate Functions
• Sorting Arrays
• Vectorization
• Vectorization Visually
• Broadcasting
• Broadcasting Visually
• Filtering
• Array Arithmetic Operations
• Reductions: Finding the Sum of Elements by Axis
• Array Slicing
• 2-D Array Slicing
• Slicing and Stepping Through
• The Linear Algebra Functions
• Summary

Chapter 5 - Practical Introduction to pandas

• What is pandas?
• The Series Object
• Accessing Values and Indexes in Series
• Setting Up Your Own Index
• Using the Series Index as a Lookup Key
• Can I Pack a Python Dictionary into a Series?
• The DataFrame Object
• The DataFrame's Value Proposition
• Creating a pandas DataFrame
• Getting DataFrame Metrics
• Accessing DataFrame Columns
• Accessing DataFrame Rows
• Accessing DataFrame Cells
• Using iloc
• Using loc
• Examples of Using loc
• DataFrames are Mutable via Object Reference!
• The Axes
• Deleting Rows and Columns
• Adding a New Column to a DataFrame
• Appending / Concatenating DataFrame and Series Objects
• Example of Appending / Concatenating DataFrames
• Re-indexing Series and DataFrames
• Getting Descriptive Statistics of DataFrame Columns
• Navigating Rows and Columns For Data Reduction
• Getting Descriptive Statistics of DataFrames
• Applying a Function
• Sorting DataFrames
• Reading From CSV Files
• Writing to the System Clipboard
• Writing to a CSV File
• Fine-Tuning the Column Data Types
• Changing the Type of a Column
• What May Go Wrong with Type Conversion
• Summary

Chapter 6 - Repairing and Normalizing Data

• Repairing and Normalizing Data
• Dealing with the Missing Data
• Sample Data Set
• Getting Info on Null Data
• Dropping a Column
• Interpolating Missing Data in pandas
• Replacing the Missing Values with the Mean Value
• Scaling (Normalizing) the Data
• Data Preprocessing with scikit-learn
• Scaling with the scale() Function
• The MinMaxScaler Object
• Summary

Chapter 7 - Data Visualization in Python

• Data Visualization
• Data Visualization in Python
• Matplotlib
• Getting Started with matplotlib
• The matplotlib.pyplot.plot() Function
• The matplotlib.pyplot.bar() Function
• The matplotlib.pyplot.pie () Function
• The matplotlib.pyplot.subplot() Function
• A Subplot Example
• Figures
• Saving Figures to a File
• Seaborn
• Getting Started with seaborn
• Histograms and KDE
• Plotting Bivariate Distributions
• Scatter plots in seaborn
• Pair plots in seaborn
• Heatmaps
• A Seaborn Scatterplot with Varying Point Sizes and Hues
• ggplot
• Summary

Chapter 8 - Descriptive Statistics Computing in Python

• Descriptive Statistics
• Non-uniformity of a Probability Distribution
• Using NumPy for Calculating Descriptive Statistics Measures
• Finding Min and Max in NumPy
• Using pandas for Calculating Descriptive Statistics Measures
• Correlation
• Regression and Correlation
• Covariance
• Getting Pairwise Correlation and Covariance Measures
• Finding Min and Max in pandas DataFrame
• Summary

Chapter 9 - Data Science and ML Terminology and Concepts

• In-Class Discussion
• Types of Machine Learning
• Terminology: Features and Observations
• Representing Observations
• Terminology: Labels
• Terminology: Continuous and Categorical Features
• Continuous Features
• Categorical Features
• Features Types Visually
• Common Distance Metrics
• The Euclidean Distance
• What is a Model
• Data Scientist's Typical Activities
• The Typical Machine Learning Process
• Supervised vs Unsupervised Machine Learning
• Supervised Machine Learning Algorithms
• Unsupervised Machine Learning Algorithms
• Which Algorithm to Choose?
• The scikit-learn Package
• scikit-learn Estimators, Models, and Predictors
• Model Evaluation
• The Error Rate
• Confusion Matrix
• The Binary Classification Confusion Matrix
• Multi-class Classification Confusion Matrix Example
• Feature Engineering
• Scaling of the Features
• Feature Blending (Creating Synthetic Features)
• The 'One-Hot' Encoding Scheme
• Example of 'One-Hot' Encoding Scheme
• Bias-Variance (Underfitting vs Overfitting) Trade-off
• The Modeling Error Factors
• One Way to Visualize Bias and Variance
• Underfitting vs Overfitting Visualization
• Balancing Off the Bias-Variance Ratio
• Regularization in scikit-learn
• Regularization, Take Two
• Dimensionality Reduction
• The Advantages of Dimensionality Reduction
• A Better Algorithm or More Data?
• The LIBSVM format
• Data Splitting into Training and Test Datasets
• ML Model Tuning Visually
• Data Splitting in scikit-learn
• Cross-Validation Technique
• Summary

Chapter 10 - Classification Algorithms (Supervised Machine Learning)

• Classification (Supervised ML) Use Cases
• Classifying with k-Nearest Neighbors
• k-Nearest Neighbors Algorithm Visually
• Decision Trees
• Decision Tree Terminology
• Decision Tree Classification in the Context of Information Theory
• Using Decision Trees
• Properties of the Decision Tree Algorithm
• The Simplified Decision Tree Algorithm
• Random Forest
• Properties of the Random Forest Algorithm
• Support Vector Machines (SVMs)
• SVM Classification Visually
• Properties of SVMs
• Dealing with Non-Linear Class Boundaries
• Logistic Regression (Logit)
• The Sigmoid Function
• Logistic Regression Classification Example
• Logistic Regression's Problem Domain
• Naive Bayes Classifier (SL)
• Naive Bayesian Probabilistic Model in a Nutshell
• Bayes Formula
• Document Classification with Naive Bayes
• Summary

Chapter 11 - Linear Regression

• Regression Analysis
• Regression vs Correlation
• Regression vs Classification
• Simple Linear Regression Model
• Linear Regression Illustration
• Least-Squares Method (LSM)
• Gradient Descent Optimization
• Multiple Regression Analysis
• Evaluating Regression Model Accuracy
• The R2 Model Score
• The MSE and RMSD Model Scores
• Gradient Boosting
• XGBoost
• Summary

Chapter 12 - Unsupervised Machine Learning Algorithms

• PCA
• PCA and Data Variance
• PCA Properties
• Importance of Feature Scaling Visually
• Unsupervised Learning Type: Clustering
• Clustering vs Classification
• Clustering Examples
• k-means Clustering
• k-means Clustering in a Nutshell
• k-means Characteristics
• Global vs Local Minimum Explained
• Summary

Chapter 13 - (Optional) Quick Introduction to Python for Data Engineers

• What is Python?
• Additional Documentation
• Which version of Python am I running?
• Python Dev Tools and REPLs
• IPython
• Jupyter
• Jupyter Operation Modes
• Jupyter Common Commands
• Anaconda
• Python Variables and Basic Syntax
• Variable Scopes
• PEP8
• The Python Programs
• Getting Help
• Variable Types
• Assigning Multiple Values to Multiple Variables
• Null (None)
• Strings
• Finding Index of a Substring
• String Splitting
• Triple-Delimited String Literals
• Raw String Literals
• String Formatting and Interpolation
• Boolean
• Boolean Operators
• Numbers
• Looking Up the Runtime Type of a Variable
• Divisions
• Assignment-with-Operation
• Comments:
• Relational Operators
• The if-elif-else Triad
• An if-elif-else Example
• Conditional Expressions (a.k.a. Ternary Operator)
• The While-Break-Continue Triad
• The for Loop
• try-except-finally
• Lists
• Main List Methods
• Dictionaries
• Working with Dictionaries
• Sets
• Common Set Operations
• Set Operations Examples
• Finding Unique Elements in a List
• Enumerate
• Tuples
• Unpacking Tuples
• Functions
• Dealing with Arbitrary Number of Parameters
• Keyword Function Parameters
• The range Object
• Random Numbers
• Python Modules
• Importing Modules
• Installing Modules
• Listing Methods in a Module
• Creating Your Own Modules
• Creating a Runnable Application
• List Comprehension
• Zipping Lists
• Working with Files
• Reading and Writing Files
• Reading Command-Line Parameters
• Accessing Environment Variables
• What is Functional Programming (FP)?
• Terminology: Higher-Order Functions
• Lambda Functions in Python
• Example: Lambdas in the Sorted Function
• Other Examples of Using Lambdas
• Regular Expressions
• Using Regular Expressions Examples
• Python Data Science-Centric Libraries
• Summary