Applications and Major Techniques

Topic — Across all of artificial intelligence, machine learning performs about nine core techniques. Every industry application is either one of these nine or a combination of them. The essential skill of an ML engineer is translating a business question into its corresponding technique by inspecting the shape of the required output.

---

1. The Nine Techniques at a Glance

The fastest way to identify the correct technique is to examine what shape the answer is:

#	Technique	Core Question	Output Shape	Canonical Scikit-Learn Class	Example Application
1	Regression	How much? How many?	Continuous number ($\mathbb{R}$)	LinearRegression, Ridge	Predicting house prices, demand forecasting
2	Classification	Which category?	Discrete class label	LogisticRegression, RandomForest	Churn prediction, spam detection, medical triage
3	Clustering	What natural groups exist?	Cluster ID per row	KMeans, DBSCAN	Customer segmentation, document grouping
4	Association	What co-occurs?	"If A, then B" rules	apriori (mlxtend)	Market basket analysis (frequently bought together)
5	Anomaly Detection	What doesn't belong?	Binary inlier/outlier flag	IsolationForest, OneClassSVM	Credit-card fraud, equipment sensor failure
6	Sequence Mining	What comes next?	Next item in ordered chain	MarkovModel, LSTM, RNN	Clickstream tracking, predictive search queries
7	Dimensionality Reduction	Can I compress these columns?	Lower-dimensional projection	PCA, TSNE, UMAP	Visualizing 100 features in 2D, noise reduction
8	Recommendation	What else would this user like?	Ranked list per user	Collaborative Filtering, SVD	Netflix movie recommendations, Spotify playlists
9	Generative AI	Can we generate new assets?	Novel text, audio, image	Transformers, Diffusion	Large Language Models (LLMs), image synthesis

tip

The Two-Question Decision Shortcut

1. Do you have historical answers (labels)?
   • Yes: You are in Supervised Learning (Regression, Classification, or Sequence Mining).
   • No: You are in Unsupervised Learning (Clustering, Association, Anomaly Detection, or PCA).
2. If labeled, is the answer a continuous number or a discrete name?
   • Continuous number: $\implies$ Regression (where numeric proximity matters).
   • Discrete name / category: $\implies$ Classification (where labels are distinct categories).

---

2. Interactive Problem-to-Technique Decision Wizard

Step through the diagnostic tree below to identify the exact machine learning technique, evaluation metric, and Scikit-Learn model for your task:

Problem-to-Technique Decision Wizard

Select your problem constraints to determine the matching machine learning technique:

1. Do you have historical target labels?

2. Shape of the Target Variable:

Identified Technique

Regression / Estimation

Supervised Learning

Canonical Scikit-Learn Models:
LinearRegression, Ridge, GradientBoostingRegressor

Primary Evaluation Metrics:
RMSE, MAE, R-squared

Key Engineering Insight: A predicted price of $520,000 for a $500,000 house is 96% accurate; continuous errors are smooth.

---

3. One Dataset, Five Techniques

A frequent beginner misunderstanding is believing that datasets belong to a specific technique. In reality, the business question, not the data, picks the technique.

Consider a single retail customer table:

• tenure_months: How many months they have been subscribed.
• monthly_spend: Average dollar expenditure per month.
• support_tickets: Number of help-desk calls logged.
• churned: Ground-truth historical label ($1 = \text{cancelled}, 0 = \text{retained}$).

One Dataset, Five Paradigms Sandbox

Select a machine learning paradigm to see how the exact same customer table is processed:

How much will this customer spend each month?

Target Column: monthly_spend (Continuous Numeric Y in R)

from sklearn.linear_model import LinearRegression
reg = LinearRegression().fit(df[["tenure_months"]], df["monthly_spend"])
print(f"R2 score: {reg.score(X, y):.3f}")  # R2 = 0.651

Fitted Line: spend = 2.45 * tenure + 22.10. Predicts $56.40 spend for 14-month tenure.

Mathematical Connection: Notice how the two extreme spenders ($95 and $140) pull the OLS regression line upward, reducing R2 to 0.651.

---

4. Key Distinctions & Boundary Traps

1. Classification vs. Regression

• The Trap: "Our loan default model outputs 0.83, so it's regression."
• The Principle: In loan prediction, 0.83 is the model's posterior confidence probability $P(\text{Default} \mid X)$. The underlying ground truth is discrete ($\{0, 1\}$).
• The Litmus Test: Does numeric proximity matter? If predicting $52$ for $50$ is partly right, it is Regression. If predicting Class B for Class A is completely wrong, it is Classification.

2. Association vs. Recommendation

• Association: Operates on items (Market Basket Analysis: "If a basket contains diapers, it contains beer with 60% confidence"). The shopper's identity is completely ignored.
• Recommendation: Operates on people (Collaborative Filtering: "Because user Alex watched Interstellar and has similar watch histories to user Jordan, recommend Inception").

3. Anomaly Detection vs. Classification

• When to use Classification: When you possess balanced, confirmed labels for both normal and fraudulent events ($>1\%$ frequency).
• When to use Anomaly Detection: When fraudulent events are unlabelled, extremely rare ($<0.1\%$), or when future attacks look nothing like past attacks.

---

5. Implementation Lab

tip

Lab Exercise: One Dataset, Five Techniques

An interactive, fully functional Google Colab / Jupyter Notebook is available to execute Linear Regression, Logistic Regression, KMeans Clustering, Isolation Forest Anomaly Detection, and PCA on the exact same customer dataset live in your browser.

How to run the lab:

1. Click the "Open In Colab" badge above to launch the interactive notebook.
2. Run cells sequentially to benchmark all 5 Scikit-Learn models on identical input features.

Key Experiments to Run:

• Experiment 1 (Outlier Impact on Regression): Inspect the OLS regression fit line with the 2 anomalous spenders ($R^2 = 0.651$), drop them, and watch $R^2$ jump!
• Experiment 2 (Unsupervised Churn Discovery): Examine KMeans cluster averages to see why Cluster 0 organically isolates the churned customers without ever receiving the churn label.
• Experiment 3 (PCA Variance Preservation): Verify that 2 principal components capture $97.9\%$ of total variance from the original 3-dimensional space.

---

6. Common Mistakes

1. Confusing Output Format with Task Type

• The Misconception: "A probability output like 0.85 means the algorithm is performing regression."
• Wrong Thinking: Judging a model by its mathematical floating-point format.
• The Right Principle: Check the ground-truth target. If the target in your training table is a discrete class (defaulted vs paid), the task is Classification.

2. Labeling Datasets Instead of Questions

• The Misconception: "This customer dataset is a classification dataset."
• Wrong Thinking: Believing a dataset is locked to a single technique.
• The Right Principle: Datasets are neutral containers. You can run Regression on monthly_spend, Classification on churned, and Clustering on unlabeled demographic features. The business question picks the technique.

3. Assuming Shuffling Never Matters

• The Misconception: "Shuffling dataset rows has no impact on model training."
• Wrong Thinking: Treating all data as independent and identically distributed (i.i.d.).
• The Right Principle: In Sequence Mining and time-series forecasting, temporal order carries the informational signal. Shuffling rows completely destroys the transition dynamics.

4. Association vs. Personalized Recommendations

• The Misconception: "Market basket analysis gives personalized product suggestions to users."
• Wrong Thinking: Confusing co-occurrence with individual taste profiling.
• The Right Principle: Association rules are global to the checkout basket ("If coffee, then sugar"). True recommendation systems require user interaction history to personalize suggestions.

---

7. Summary

Supervised Paradigms

• Regression: Continuous numeric targets ($\mathbb{R}$). Proximity matters ($R^2$, RMSE).
• Classification: Discrete categorical targets. Exact class matching (ROC-AUC, F1).
• Sequence Mining: Next item in temporal chain. Shuffling breaks signal.

Unsupervised Paradigms

• Clustering: Groups similar unlabeled rows (KMeans, Silhouette).
• Association: Co-occurring item rules in baskets (Support, Lift).
• Anomaly Detection: Flags rare outliers ($<0.1\%$, IsolationForest).
• Dimensionality Reduction: Projects wide feature spaces into 2D/3D (PCA).

info

Key Takeaways

• Output Shape Dictates Technique: Always identify the technique by what the answer looks like (continuous number, discrete label, cluster ID, or ranked list).
• Data Does Not Pick the Technique: A single customer dataset can simultaneously support regression, classification, clustering, anomaly detection, and PCA.
• The 0.83 Probability Trap: Output probabilities express confidence in classification; they do not turn the problem into regression.

Next Section: Types of Learning - supervised, unsupervised, and semi-supervised learning paradigms.

---

8. Active Recall

Test your understanding by answering first, then clicking to reveal the underlying principles.

Interactive Checkpoint Quiz

Interactive Checkpoint: The Loan Default Probability Trap

A bank builds an ML model that outputs 0.83 for a mortgage applicant. A junior developer claims this is a Regression task because 0.83 is a continuous number. Is this claim correct?

Review Flashcards

1. [THEORY] How do you distinguish between Association Rules and Recommendation Systems?

• Association: Learns global co-occurrence rules between items ("If coffee in basket, then sugar with 70% confidence"). The specific individual shopper is ignored.
• Recommendation Systems: Builds personalized ranked lists tailored to people based on their unique individual history and taste similarity to other users.

2. [ANALYZE] Why did KMeans clustering identify the churned customers without ever seeing the churned labels?

• Natural Behavioral Separation: The input features (tenure_months and support_tickets) had natural mathematical boundaries.
• The churned customers had short tenure ($<6$ months) and high support calls ($>4$ tickets).
• Because their feature vectors formed an isolated cluster in coordinate space, KMeans grouped them together purely from geometric proximity without needing a label!

3. [THEORY] Which machine learning technique becomes impossible if you shuffle the order of your dataset rows? Why?

• Sequence Mining: (and time-series forecasting).
• Why: In sequence mining, the temporal order of events carries the primary informational signal (e.g. click $A \to$ click $B \to$ purchase). Shuffling rows breaks the transition probabilities, making sequential modeling impossible.

4. [PRACTICE] In our customer lab, linear regression scored $R^2 = 0.651$ while Isolation Forest flagged exactly two rows. Explain the connection.

• One technique's noise is another technique's answer: The linear regression model was dragged off its fit line by the two extreme spenders ($6$mo/$95 and $36$mo/$140), depressing $R^2$.
• The exact same extreme spenders were identified by Isolation Forest as statistical outliers. Removing them would restore the linear regression $R^2$ to $>0.90$.

5. [THEORY] What does PCA mean when it reports that PC1 captures 75.0% and PC2 captures 22.9% of variance? What was sacrificed?

• Preserved Variance: The first two principal components together retain $75.0\% + 22.9\% = \mathbf{97.9\%}$ of the original 3-dimensional data variance.
• Sacrificed: We discarded $2.1\%$ of information (noise/minor variation) and sacrificed direct human interpretability, as PC1 and PC2 are abstract linear combinations of the original column names.