Vettri's Notes

Machine Learning Algorithms

1 · Foundations

What machine learning is and why it works — the types of learning, and the Python toolkit behind them.

2 · Data Preprocessing

Missing values, categorical encoding, feature scaling, outliers, and the train-test split.

3 · Linear Regression

Least-squares by hand and in scikit-learn, the matrix formulation, and what the model assumes.

4 · Optimisation

Loss functions, gradient descent, the learning rate, and how many rows each update sees.

Scoring a fitted regression — the three error averages that live in the units of the target, the two ratio-based scores that do not, and the residual plots that catch a good score sitting on top of a bad fit.

6 · Validation Strategy

Evaluating model generalisation — why single train-test splits can be a lottery, the mechanics of K-Fold and Stratified K-Fold cross-validation, and the strict protocols required to prevent data leakage.

7 · Logistic Regression

Transitioning to classification — why linear regression fails on discrete targets, the mathematical necessity of the sigmoid curve, and the mechanics of binary and multiclass decision boundaries.

8 · Classification Metrics

Measuring classifier performance — decomposing predictions with the 2x2 confusion matrix, navigating the accuracy paradox on imbalanced data, and tuning thresholds with precision, recall, F-scores, and ROC-AUC.

9 · Generalization

Ensuring model robustness — balancing the bias-variance tradeoff, controlling model complexity using L1, L2, and Elastic Net penalties, and understanding how the curse of dimensionality impacts model fit.