29 docs tagged with "scikit-learn"

One decision left — how many rows the model looks at before each step. "Stochastic is faster" turns out to need a unit attached, because batch beat single-row descent on passes, on seconds and on final cost, single-row descent could not converge at all at a constant step size, and yet mini-batch 32 reached a loose target after reading 4% of one pass where batch read 50 full passes and still missed.

Understanding the 2x2 confusion matrix, extracting TP, TN, FP, FN, and diagnosing the dangerous Accuracy Paradox on imbalanced datasets.

Comparing Leave-One-Out (LOO-CV), Leave-P-Out (LPO-CV), and K-Fold Cross-Validation, their statistical mechanics, and computational tradeoffs.

The empty space phenomenon, distance concentration, and overcoming high dimensionality via feature selection and PCA projection.

Preventing data leakage with scikit-learn Pipelines, unbiased hyperparameter tuning with Nested Cross-Validation, and specialized splitting strategies.

Turning categories into numbers without inventing an order — how much label encoding really costs, why it depends on the model, what cardinality does to memory, and the leak hiding inside target encoding.

Why MAE, MSE and RMSE can report three different numbers for the same predictions, why RMSE is never below MAE, what their ratio tells you about outliers, and why scikit-learn hands back a negative mean squared error.

Synthesizing precision and recall via the harmonic mean, custom weighting with F-beta, and evaluating imbalanced cohorts with Balanced Accuracy and MCC.

Which models need scaling and which are provably indifferent, why one outlier ruins MinMaxScaler, what log transforms actually fix, and how large the scaling leak really is.

Guidelines for choosing K, resolving class imbalance with Stratified K-Fold, and analyzing the complete cross-validation pipeline.

The loss function is how you tell a model what "good" means, and changing it changes the answer. Three losses gave three different slopes on identical data; absolute error ignored an outlier at 60, 200 and 10000 alike; counting mistakes turned out to be unminimisable, with 1998 of 2000 gradients exactly zero; and squared error on a sigmoid was 11014x weaker than cross-entropy exactly where the model was most wrong.

Why a value is missing decides what you may do about it — MCAR, MAR and MNAR, what mean imputation costs a distribution, when a smarter imputer pays, and the fit/transform contract that keeps it honest.

Extending binary metrics to multiclass problems via Macro, Micro, and Weighted averaging, plus an interactive master metric selection guide.

Extending binary logistic regression to multiclass problems via One-vs-Rest and Multinomial Softmax, decision boundaries, and odds-ratio interpretation.

More than one feature — the normal equations solved by hand, why a coefficient changes when a correlated feature joins it, and the measurement showing Adjusted R² fails to penalise 100 columns of pure noise while cross-validation catches them cleanly.

What the IQR rule really flags, why outliers hide each other, the extreme point no single column can see, which treatment actually works, and the features a model cannot invent for itself.

Memorizing noise versus learning generalizable structure, the bias-variance tradeoff, and L1, L2, and Elastic Net regularization.

Two complementary metrics exposing what accuracy hides—false alarms versus missed detections, and mastering the threshold tradeoff.

Complete Python code examples using scikit-learn for all ML algorithms and techniques

Settles why R² can exceed 1 or go negative, why SST = SSR + SSE only holds for a least-squares fit, why adding a junk feature never lowers R² but does lower Adjusted R², and why scikit-learn's MAPE returns 0.0296 rather than 2.96%.

Tuning hyperparameter lambda and C via grid search, warm-start path algorithms, and understanding scikit-learn solver-penalty compatibility.

What residuals reveal about model fit, interpreting diagnostic plots, and deciding which evaluation metric to report under problem constraints.

Visualizing classifier discrimination across all cutoffs, interpreting the Area Under the Curve, and optimizing decision thresholds via Youden's J.

Fitting a line to one feature — the two coefficients derived by hand and checked against scikit-learn, the five error measures computed from the same residuals, and why a genuinely useful feature can look exactly like noise.

Deriving the logistic model from linear regression, mathematical foundations of Binary Cross-Entropy, and optimization via Gradient Descent.

The five Python libraries that do the work, how they stack, scikit-learn's fit/predict/transform contract, and the seven-step pipeline every project follows.

Why a held-out set is mandatory, how much the score depends purely on random_state, and the four situations where a random split reports a number that is simply false.

The train-test split lottery, understanding why multiple splits estimate generalization reliably, and the sealed test set protocol.

Binary classification fundamentals, why ordinary least squares linear regression fails on probabilities, and the mathematical necessity of the sigmoid curve.