Part B: Long Analytical & Numerical Problems
Course - Machine Learning Algorithms (21CSC555J)
Source Document - University Question Bank (Part B)
Mark Weightage - 16 Marks per Question (or 4m / 8m / 10m / 12m modular sub-parts)
Structure - Aligned 1-to-1 withPart_B_Questions.pdfacross Sets 1, 2, 3, and 4
Set 1 (Questions 1 to 5)
Question 1 (16 Marks)
Question Statement
- Explain in detail about different types of learning methods in machine learning with example (16 Marks)
(OR)
a) Derive Logistic-Regression classification model from Linear Regression (8 Marks)
b) Write short notes on: a) Sigmoidal transfer function b) ODD c) Loss function (8 Marks)
Solution 1: Learning Methods in Machine Learning (Main Option)
- Supervised Learning:
- Principle: Learns mapping function from labelled pairs minimizing empirical risk:
- Sub-types: Regression (continuous target , e.g., house prices) and Classification (discrete target , e.g., fraud detection).
- Unsupervised Learning:
- Principle: Discovers latent clusters, manifolds, or probability density from unlabelled inputs .
- Sub-types: Clustering (K-Means, DBSCAN), Dimensionality Reduction (PCA, t-SNE), and Density Estimation (GMMs).
- Semi-Supervised Learning:
- Principle: Exploits a small labelled set () alongside massive unlabelled data () under the cluster and manifold smoothness assumptions (e.g., medical MRI segmentation).
- Reinforcement Learning (RL):
- Principle: Formulated as a Markov Decision Process . An agent takes actions under policy to maximize expected discounted cumulative reward:
Solution 1 (OR): Logistic Regression Derivation and Core Primitives
a) Derivation from Linear Regression
- Linear Incompatibility: Standard linear regression violates probability bounds .
- Odds Ratio: Let . The odds of occurrence is:
- Logit Link Function: Taking the natural logarithm maps odds to the full real line :
- Sigmoid Inversion:
b) Short Notes
- Sigmoidal Transfer Function : S-shaped curve mapping with midpoint and derivative .
- Odds: Multiplicative probability ratio (). A unit increase in feature multiplies odds by .
- Binary Cross-Entropy Loss: Strictly convex cost function avoiding local minima:
Question 2 (16 Marks)
Question Statement
- (i) Difference between classification and regression models? (4 marks)
(ii) Explain the techniques available for handling the class imbalance? (12 marks)
(OR) - Explain the concept of confusion matrix and compute the below measures with definition? (5 marks)
| Positive (0) | Negative (1) | |
|---|---|---|
| Positive (0) | 205 | 53 |
| Negative (1) | 42 | 195 |
- a) Accuracy b) Recall c) Precision d) F1 score e) False Positive Rate
Solution 2: Classification vs. Regression & Class Imbalance (Main Option)
(i) Classification vs. Regression
| Dimension | Classification | Regression |
|---|---|---|
| Target Variable () | Discrete categories ( or ) | Continuous values () |
| Loss Function | Cross-Entropy, Hinge Loss | Mean Squared Error (MSE), Mean Absolute Error (MAE) |
| Evaluation Metrics | Accuracy, Precision, Recall, , ROC-AUC | , RMSE, MAE, MAPE |
(ii) Techniques for Handling Class Imbalance
- Data-Level Resampling:
- SMOTE (Synthetic Minority Over-sampling): Synthesizes minority instances along feature segments:
- NearMiss Undersampling: Selects majority instances closest to minority boundaries.
- Algorithm-Level Adjustments:
- Cost-Sensitive Loss: Scales loss inversely proportional to class frequencies: .
- Focal Loss: Modulates cross-entropy by to suppress easy negative instances.
- Threshold Moving: Lowering classification threshold (e.g., from to ) to optimize minority recall.
Solution 2 (OR): Binary Confusion Matrix Computation ()
Given Matrix (Annotated with Totals):
| Predicted Positive (0) | Predicted Negative (1) | Row Total (Actual) | |
|---|---|---|---|
| Actual Positive (0) | () | () | |
| Actual Negative (1) | () | () | |
| Column Total (Predicted) | Total |
Metric Summary Table:
| Metric | Governing Formula | Exact Substitution | Decimal Value | Percentage |
|---|---|---|---|---|
| Accuracy | ||||
| Recall (Sensitivity) | ||||
| Precision (PPV) | ||||
| -Score | ||||
| False Positive Rate (FPR) |
Question 3 (16 Marks)
Question Statement
- (i) Write in detail about decision tree splitting criteria (8 marks): a) Gini index b) Information gain
(ii) Explain the hyper parameters of Decision Tree? (8 marks)
(OR)
i) Explain the concept of decision tree classifier and solve the given dataset using decision tree classifier using Information gain (10 marks)
| Specialized skill | Academic | Salary Expectation | Communication Skill | Job Status |
|---|---|---|---|---|
| Yes | Poor | Low | Average | Not Selected |
| Yes | Poor | High | Good | Selected |
| No | High | High | Average | Selected |
| No | Average | Medium | Poor | Not Selected |
| Yes | High | Medium | Good | Selected |
| No | Average | Low | Average | Not Selected |
- (ii) Explain the concept of post pruning and pre-pruning with example? (6 marks)
Solution 3: Splitting Criteria & Hyperparameters (Main Option)
(i) Splitting Criteria: Gini Index vs. Information Gain
- Gini Impurity (CART): Measures probability of misclassifying a randomly chosen element: Range: for binary splits. Minimizes computation by avoiding logarithms.
- Information Gain (ID3 / C4.5): Measures reduction in Shannon Entropy:
(ii) Decision Tree Hyperparameters
max_depth: Limits maximum vertical depth to stop learning sample-specific noise.min_samples_split: Minimum samples required to attempt a split at an internal node.min_samples_leaf: Minimum samples mandated in every terminal leaf.max_features: Number of candidate features evaluated at each split ().
Solution 3 (OR): Decision Tree on Job Status Dataset ()
1. Base Dataset Entropy ()
Total : .
2. Attribute Information Gain Summary:
| Candidate Attribute | Value Subsets | Sub-Entropy () | Remainder Entropy | Information Gain | Decision |
|---|---|---|---|---|---|
Academic | High () Average () Poor () | Selected as Root (Tie Winner) | |||
Salary Expectation | High () Low () Medium () | Alternate Root | |||
Communication Skill | Good () Poor () Average () | Rejected | |||
Specialized skill | Yes () No () | Rejected |
3. Subtree Splits under Academic
Academic = High(Rows 3, 5): Pure Leaf: SelectedAcademic = Average(Rows 4, 6): Pure Leaf: Not SelectedAcademic = Poor(Rows 1, 2): Split onSalary Expectation:High(Row 2): SelectedLow(Row 1): Not Selected
(ii) Pre-Pruning vs. Post-Pruning
- Pre-Pruning: Early stopping based on bounds (
max_depth,min_samples_split). Fast, but risks underfitting. - Post-Pruning: Grows the full tree, then collapses subtrees bottom-up using cost-complexity penalty .
Question 4 (16 Marks)
Question Statement
- Explain in detail about bagging ensemble techniques classification? (10 marks)
(ii) Define stacking and Voting ensemble model? (6 marks)
(OR) - Describe the random forest model with example and justify the hyper parameter importance? (16 marks)
Solution 4: Bagging, Stacking & Voting (Main Option)
- Bagging (Bootstrap Aggregation):
- Resampling: Draws independent samples of size with replacement. Each tree sees unique rows, leaving as Out-of-Bag (OOB) validation samples.
- Aggregation: Consolidated via plurality vote: .
- Voting Classifiers:
- Hard Voting: Plurality vote among base model predictions ().
- Soft Voting: Averages predicted probabilities: .
- Stacking (Stacked Generalization):
- Heterogeneous base estimators (SVM, Random Forest, XGBoost) generate cross-validated predictions.
- A Meta-Learner (Logistic Regression / ElasticNet) trains on these predictions to assign optimal regional weights.
Solution 4 (OR): Random Forest Algorithm & Hyperparameter Tuning
- Algorithmic Framework: Combines Bootstrap Resampling with Random Feature Subspacing ().
- Ensemble Variance Reduction Formula: Random feature subspacing drives pairwise tree correlation , reducing overall ensemble variance without inflating bias.
- Hyperparameters:
n_estimators: More trees monotonically reduce variance without overfitting.max_features: Smaller decorrelates trees (standard ).max_depth&min_samples_split: Regularizes individual tree complexity.
Question 5 (16 Marks)
Question Statement
- a) Discuss the distance measures used in machine learning model.
b) Write the k-nearest neighbor algorithm with suitable example
(OR) - a) Describe the issues of supervised learning algorithm.
b) Write short notes on: i) Multinomial Naïve Bayes ii) Gaussian Naïve Baye
Solution 5: Distance Metrics and KNN (Main Option)
a) Distance Measures in Machine Learning
- Euclidean (): (straight-line distance, sensitive to scaling).
- Manhattan (): (grid-based, outlier-robust).
- Minkowski (): .
- Cosine Distance: (angular orientation, invariant to text length).
b) K-Nearest Neighbors (KNN) Algorithm
- Store training instances .
- For query , compute distance to all training points.
- Identify the nearest instances and assign majority vote:
- Example: Classifying a customer (Income ₹50,000, Credit ). If nearest neighbors are , the loan is Approved.
Solution 5 (OR): Supervised Issues & Naïve Bayes Variants
a) Core Issues in Supervised Learning
- Bias-Variance Dilemma: Underfitting vs. Overfitting trade-off.
- Curse of Dimensionality: Exponential volume growth makes points equidistant in high dimensions.
- Class Imbalance: Rare target events get overwhelmed by majority classes.
b) Naïve Bayes Variants
- Multinomial Naïve Bayes: Discrete token count frequencies evaluated using Laplace smoothing:
- Gaussian Naïve Bayes: Real-valued continuous features modeled via normal distribution:
Set 2 (Questions 2 to 5)
Question 2 (16 Marks)
Question Statement
- Differentiate supervised and unsupervised machine learning model. (4 marks)
c) Difference between classification and regression models? (4 marks)
d) Write the short notes and mathematical form of ODD and Logit function? (8 marks)
(OR)
(ii) Discuss briefly the logistic regression in the machine learning model with a suitable example. (12 marks)
(iii) define train, test and validation dataset (4 marks)
Solution 2: Paradigms, Task Types & Logit Function (Main Option)
Supervised vs. Unsupervised Learning (4 Marks)
| Feature | Supervised Learning | Unsupervised Learning |
|---|---|---|
| Data | Labelled pairs | Unlabelled feature vectors |
| Goal | Prediction / Extrapolation | Structure / Cluster Discovery |
| Examples | SVM, Random Forest, Logistic Regression | K-Means, PCA, DBSCAN |
Classification vs. Regression (4 Marks)
- Classification: Discrete category target (e.g., Spam vs. Ham).
- Regression: Continuous scalar target (e.g., Stock price).
Odds and Logit Function (8 Marks)
- Odds Ratio: .
- Logit Function: .
Solution 2 (OR): Logistic Regression & Data Partitioning
(ii) Logistic Regression with Example (12 Marks)
Outputs calibrated posterior probabilities optimized via Binary Cross-Entropy loss. For an applicant with credit profile , if , the loan is approved.
(iii) Train, Validation, and Test Dataset Definitions (4 Marks)
Question 3 (16 Marks)
Question Statement
- Explain the concept of confusion matrix and compute the below measures for all the target class labels with definition? (16 marks)
| Predicted \ Actual | Apple | Orange | Mango |
|---|---|---|---|
| Apple | 7 | 8 | 9 |
| Orange | 1 | 2 | 3 |
| Mango | 3 | 2 | 1 |
- a) Accuracy b) Recall c) Precision d) F1 score e) False Positive Rate
(OR)
- What is the impact of data class Imbalance on ML model. Describe the techniques used to rectify it with an suitable example.
- a) Write the decision tree algorithm.
b) Describe briefly the entropy method of determining the feature importance in machine learning model.
Solution 3: Multi-Class Confusion Matrix (Main Option)
Total observations . Overall Accuracy .
One-vs-Rest Decomposition for Each Fruit:
| Class | TP | FP (Row Sum - TP) | FN (Col Sum - TP) | TN () | Recall () | Precision () | -Score | FPR () |
|---|---|---|---|---|---|---|---|---|
| Apple | 7 | |||||||
| Orange | 2 | |||||||
| Mango | 1 |
Solution 3 (OR): Class Imbalance & Decision Tree Algorithm
- Class Imbalance Impact & Remediation: Accuracy paradox; solved via SMOTE oversampling, NearMiss undersampling, and Cost-Sensitive loss reweighting.
- Decision Tree Algorithm (ID3 / CART):
- Evaluate all features; compute Information Gain / Gini reduction.
- Split node on attribute maximizing gain; recurse on subsets until pure or stopping criteria met.
- Entropy Method for Feature Importance: Features that yield the largest cumulative reductions in entropy across all tree splits hold the highest feature importance.
Question 3 (Continued / Alternate) (16 Marks)
Question Statement
- (i) Construct Decision tree for the given dataset using Information gain and Entropy. (12 marks)
| Day | Past Trend | Open Interest | Trading Volume | Return |
|---|---|---|---|---|
| 1 | Positive | Low | High | Up |
| 2 | Negative | High | Low | Down |
| 3 | Positive | Low | High | Up |
| 4 | Positive | High | High | Up |
| 5 | Negative | Low | High | Down |
| 6 | Positive | Low | Low | Down |
| 7 | Negative | High | High | Down |
| 8 | Negative | Low | High | Down |
| 9 | Positive | Low | Low | Down |
| 10 | Positive | High | High | Up |
- (ii) How decision tree handles the overfitting? (4 marks)
Solution: Decision Tree on Stock Return Dataset
1. Base Dataset Entropy ()
: .
2. Root Selection via Information Gain
- (Highest Root)
3. Tree Architecture & Decision Rules
Past Trend = Negative(Days 2, 5, 7, 8): Pure Leaf: DownPast Trend = Positive(Days 1, 3, 4, 6, 9, 10): Split onTrading Volume:High(Days 1, 3, 4, 10): Pure Leaf: UpLow(Days 6, 9): Pure Leaf: Down- (Complete separation)
Question 4 (16 Marks)
Question Statement
- Describe the random forest model with example and justify the hyper parameter importance? (16 marks)
OR - How feature importance playing major role in predicting the target variable? (6 marks)
Explain the detailed concept and features of a) gradient boosting and b) Adaboosting? (10 marks)
Solution 4: Boosting Paradigms & Feature Importance
- Feature Importance Role (6 Marks):
- Enables model interpretability and regulatory compliance (e.g., credit approval).
- Identifies uninformative features for dimensionality reduction.
- Calculated via Mean Decrease in Impurity (MDI) or Permutation Importance.
- AdaBoost vs. Gradient Boosting (10 Marks):
- AdaBoost: Reweights misclassified samples iteratively: .
- Gradient Boosting: Fits base regression trees directly to the pseudo-residuals (negative loss gradients):
Question 5 (16 Marks)
Question Statement
. (i) Explain the working principle of K nearest neighbour classifier? (10 marks)
(ii) what are the different distance measures are used in the KNN and how it impacts the neighbours on KNN model? (6 marks)
(OR)
5. Explain Naïve bayes classifier in detail and solve the following dataset with the bayes model?
Solution 5: Naïve Bayes Classifier with Worked Example
- Formulation: Applies Bayes' Theorem with class-conditional feature independence:
- Worked Spam Example:
- Given message .
- Priors: .
- Likelihoods: ; .
Set 3 (Questions 5 to 5)
Question 7 (16 Marks)
Question Statement
- Explain the concept of confusion matrix and compute the below measures with definition? (16 marks)
| Positive (0) | Negative (1) | |
|---|---|---|
| Positive (0) | 85 | 48 |
| Negative (1) | 47 | 75 |
- c) Accuracy d) Recall c) Precision d) F1 score e) False Positive Rate
Solution 7: Confusion Matrix B ()
Matrix Element Identifications:
Metric Summary Table:
| Metric | Governing Formula | Exact Substitution | Decimal Value | Percentage |
|---|---|---|---|---|
| Accuracy | ||||
| Recall (Sensitivity) | ||||
| Precision (PPV) | ||||
| -Score | ||||
| False Positive Rate (FPR) |
Set 4 (Questions 9 to 5)
Question 9 & 1 (16 Marks)
Question Statement
- Explain in detail about logistic regression classifier, advantages and applications with suitable example? (16 marks)
(OR) - Write the short notes and mathematical form of ODD and Logit function? (4 marks)
(ii) Explain the non-linear machine learning models in detail? (8 marks)
(iii) define train, test and validation dataset (4 marks)
Solution: Non-Linear Machine Learning Models (8 Marks)
- Kernel Support Vector Machines (Cover's Theorem): Non-linear mapping via Mercer kernels () establishes linear separability in high dimensions.
- Decision Trees & Ensembles: Recursively partitions the feature space into axis-aligned rectangular regions, producing non-linear piecewise constant boundaries.
- Neural Networks (Universal Approximation Theorem): Combining linear weights with non-linear activations (ReLU, Sigmoid) allows multi-layer perceptrons to approximate any continuous function.
Question 10 & 2 (16 Marks)
Question Statement
- (i) Difference between classification and regression models? (4 marks)
(ii) Explain the techniques available for handling the class imbalance? (12 marks)
(OR)
. 2. Explain the concept of confusion matrix and compute the below measures with definition? (5 marks)
| Positive (0) | Negative (1) | |
|---|---|---|
| Positive (0) | 65 | 43 |
| Negative (1) | 32 | 55 |
- e) Accuracy f) Recall c) Precision d) F1 score e) False Positive Rate
Solution: Confusion Matrix C ()
Matrix Element Identifications:
Metric Summary Table:
| Metric | Governing Formula | Exact Substitution | Decimal Value | Percentage |
|---|---|---|---|---|
| Accuracy | ||||
| Recall (Sensitivity) | ||||
| Precision (PPV) | ||||
| -Score | ||||
| False Positive Rate (FPR) |
Examiner's Note
This page mirrors all Part B sets from Part_B_Questions.pdf. All confusion matrix tables feature annotated row and column totals matching the PDF screenshots, and all decision tree problems include step-by-step Information Gain tables and Mermaid diagrams.