Overfitting, Underfitting, and Regularization
Memorizing noise versus learning generalizable structure, the bias-variance tradeoff, and L1, L2, and Elastic Net regularization.
Regularization Strength and Solvers
Tuning hyperparameter lambda and C via grid search, warm-start path algorithms, and understanding scikit-learn solver-penalty compatibility.
Curse of Dimensionality
The empty space phenomenon, distance concentration, and overcoming high dimensionality via feature selection and PCA projection.