Part A: Short Answer Questions
Course - Machine Learning Algorithms (21CSC555J)
Source Document - University Question Bank (Part A)
Mark Weightage - 2 Marks per Question
Structure - Aligned 1-to-1 withPart_A_Questions.pdfacross Sets 1, 2, 3, and 4
Set 1 (Questions 1 to 10)
Question 1 (2 Marks)
Question Statement
Describe the independent and dependent variable with example? (2 Marks)
- Independent Variable (): The input, predictor, or controlled feature manipulated or observed to evaluate its effect on the system. It is assumed to be exogenous.
- Dependent Variable (): The outcome, response, or target variable whose variation is modeled and predicted as a function of :
- Concrete Example: In housing price prediction, the living area in square feet () is the independent variable, while the final selling price in dollars () is the dependent variable.
Question 2 (2 Marks)
Question Statement
Define the logistic regression with mathematical notation? (2 Marks)
- Definition: Logistic Regression is a supervised classification model that estimates the posterior probability of a binary categorical outcome via the Sigmoid function.
- Mathematical Formulation: where is the weight vector, is the bias, and is the log-odds (logit).
- Decision Rule: if ; otherwise .
Question 3 (2 Marks)
Question Statement
Define Variance and Bias? (2 Marks)
For an estimator predicting true target across training sets :
- Bias (Underfitting Error): The systematic deviation of the model's expected prediction from the ground truth: High bias stems from overly simplistic assumptions, causing the model to miss underlying trends.
- Variance (Overfitting Error): The variability of the model's predictions across different random training splits: High variance stems from excessive model complexity, causing the model to memorize localized sample noise.
Question 4 (2 Marks)
Question Statement
Explain the ROC curve with neat diagram? (2 Marks)
- Definition: The Receiver Operating Characteristic (ROC) curve evaluates classifier discrimination by plotting the True Positive Rate (TPR) against the False Positive Rate (FPR) across all decision thresholds .
- Coordinate Axes:
- Y-Axis:
- X-Axis:
- Area Under the Curve (AUC): An indicates perfect separability; represents random guessing.
Question 5 (2 Marks)
Question Statement
How to find the Optimised threshold for output probability? (2 Marks)
The default threshold is often suboptimal under class imbalance or asymmetric misclassification costs. Optimal thresholds are located via:
- Youden's Statistic (ROC-Based): Maximizes the vertical distance from the random chance line:
- Precision-Recall Optimization: Maximizes the harmonic mean: .
- Cost-Utility Minimization: Minimizes total monetary penalty:
Question 6 (2 Marks)
Question Statement
Define Gini Index with mathematical notation? (2 Marks)
- Definition: The Gini Impurity (Index) measures the probability that a randomly chosen element from a dataset would be incorrectly labelled if labelled randomly according to the class distribution.
- Mathematical Formula: where is the number of classes and is the relative proportion of class in set .
- Bounds: For binary classification, . A pure node has ; a balanced node has maximum impurity .
Question 7 (2 Marks)
Question Statement
Signify the importance of non-linear machine learning models used in supervised classification. (2 Marks)
- Overcoming Linear Inseparability: Real-world datasets (e.g., XOR patterns, concentric circles, complex fraud topologies) cannot be separated by an affine hyperplane (). Linear models suffer from severe high bias (underfitting) on such tasks.
- Automated Feature Interactions: Non-linear models (Decision Trees, Kernel SVMs, Neural Networks, Ensembles) learn curved, multi-modal boundaries without requiring manual polynomial feature engineering.
Question 8 (2 Marks)
Question Statement
Differentiate bagging and boosting technique in machine learning model. (2 Marks)
| Feature | Bagging (Bootstrap Aggregating) | Boosting |
|---|---|---|
| Model Building | Base learners train in parallel and independently. | Base learners train sequentially and adaptively. |
| Primary Goal | Variance Reduction (stabilizes complex trees). | Bias Reduction (converts weak learners to strong). |
| Sample Weights | Uniform random bootstrap sampling (). | Adaptive: misclassified points receive higher weights. |
| Canonical Example | Random Forest, Bagged CART | AdaBoost, Gradient Boosting, XGBoost |
Question 9 (2 Marks)
Question Statement
Write the assumptions of Naïve Bayes classifier? (2 Marks)
- Class-Conditional Feature Independence (Core Assumption): Features are assumed mutually independent given the class label :
- Equal Feature Weighting: All predictors contribute equally and independently to the final posterior decision in log-odds space.
Question 10 (2 Marks)
Question Statement
List the distance measures used in machine learning model. (2 Marks)
For vectors :
- Euclidean Distance ( Norm): Straight-line geometric distance:
- Manhattan Distance ( Norm): City-block / taxicab distance (robust to outliers):
- Minkowski Distance ( Norm): Generalization parameter :
- Cosine Distance: Measures angular orientation (invariant to document length):
Set 2 (Questions 1 to 10)
Question 1 (2 Marks)
Question Statement
Write the short notes on classification with example? (2 Marks)
- Definition: Classification is a supervised learning task where an algorithm learns a mapping function from continuous or discrete feature vectors to discrete categorical target labels .
- Categories:
- Binary: Two mutually exclusive classes (e.g., Default vs. Non-Default).
- Multi-Class: More than two classes (e.g., Digit recognition ).
- Concrete Example: Email spam filtering, where an incoming email vector is classified into discrete categories: or .
Question 2 (2 Marks)
Question Statement
What cost function is used for evaluating the loss of the logistic regression model? (2 Marks)
- Cost Function: The Binary Cross-Entropy Loss (Log Loss), derived via Maximum Likelihood Estimation (MLE): where and .
- Why Not MSE? Combining Mean Squared Error with the non-linear sigmoid generates a non-convex surface with local minima. Cross-Entropy guarantees a strictly convex surface with a unique global minimum.
Question 3 (2 Marks)
Question Statement
What is Overfitting and Underfitting? (2 Marks)
- Underfitting (High Bias, Low Variance):
- Symptom: High training error and high test error ().
- Cause: Model is too simple to capture the underlying pattern (e.g., fitting a straight line to quadratic data).
- Overfitting (Low Bias, High Variance):
- Symptom: Near-zero training error but high test error ().
- Cause: Model is overly complex and memorizes localized noise and outliers instead of generalizable trends.
Question 4 (2 Marks)
Question Statement
State the Kappa Statistics to measure the model performance. (2 Marks)
- Definition: Cohen's Kappa () measures classification agreement between predictions and ground truth while explicitly adjusting for agreement occurring purely by chance:
- Parameters:
- : Observed agreement proportion ().
- : Expected chance agreement calculated from marginal totals.
- Interpretation: (perfect agreement), (pure random chance). It is far more reliable than raw accuracy on imbalanced datasets.
Question 5 (2 Marks)
Question Statement
Define Up Sampling and Down Sampling? (2 Marks)
Data-level resampling techniques used to resolve class imbalance:
- Up-Sampling (Over-sampling): Increases the minority class count to match the majority class. Modern approaches use SMOTE (Synthetic Minority Over-sampling Technique) to interpolate synthetic feature vectors between -nearest minority neighbors.
- Down-Sampling (Under-sampling): Decreases majority class instances to match minority frequency by randomly removing or strategically filtering majority samples (e.g., NearMiss, Tomek Links).
Question 6 (2 Marks)
Question Statement
Define KNN model with example? (2 Marks)
- Definition: K-Nearest Neighbors (KNN) is an instance-based, non-parametric lazy learner. It retains all training instances in memory and classifies a query point by taking the majority vote among its closest neighbors:
- Concrete Example: Classifying a loan applicant by Income (₹50,000) and Credit Score (). If the closest historical applicants are , the model predicts Approve.
Question 7 (2 Marks)
Question Statement
How decision tree handles the overfitting? (2 Marks)
Decision trees control overfitting through two primary mechanisms:
- Pre-Pruning (Early Stopping): Stops tree expansion during construction if stopping constraints are reached:
- Setting
max_depth,min_samples_split,min_samples_leaf, ormax_leaf_nodes.
- Setting
- Post-Pruning (Cost-Complexity Pruning): Grows the tree to maximum depth, then collapses non-critical subtrees bottom-up using a complexity penalty: where is classification error and is leaf count.
Question 8 (2 Marks)
Question Statement
Signify the advantages of random forest technique. (2 Marks)
- Variance Reduction Without Overfitting: Adding more trees monotonically stabilizes variance without increasing test error.
- Tree Decorrelation: Random feature subspacing () prevents dominant features from dictating every tree split, driving pairwise correlation .
- Built-in Out-of-Bag (OOB) Validation: Provides an unbiased error estimate during training without requiring a separate holdout set.
- Native Feature Importance: Computes feature contributions automatically via Mean Decrease in Impurity (MDI).
Question 9 (2 Marks)
Question Statement
Define Bayes Theorem (2 Marks)
- Definition: Bayes' Theorem calculates the posterior probability of a hypothesis given observed evidence by inverting likelihood probabilities.
- Mathematical Formula:
- Component Terms:
- : Posterior Probability (updated belief after observing evidence).
- : Likelihood of evidence given hypothesis is true.
- : Prior Probability (initial baseline belief).
- : Marginal Evidence (normalization factor: ).
Question 10 (2 Marks)
Question Statement
What are the various steps involved in building a machine learning model? (2 Marks)
- Problem Definition: Specify the business target, problem type (classification/regression), and evaluation metric.
- Data Collection & Cleaning: Ingest raw data; resolve missing values, corrupt records, and outliers.
- Exploratory Data Analysis (EDA): Assess feature distributions, skewness, correlations, and class balance.
- Feature Engineering & Preprocessing: Encode categorical variables, scale numerical features, split into train/val/test sets.
- Model Training: Fit candidate baseline and ensemble algorithms.
- Hyperparameter Tuning: Optimize model parameters via stratified cross-validation.
- Evaluation & Deployment: Audit performance on held-out test data; deploy via REST API with performance drift monitoring.
Set 3 (Questions 11 to 20)
Question 11 (2 Marks)
Question Statement
Write the short notes on classification with example? (2 Marks)
- Definition: Classification is a supervised learning task where an algorithm learns a mapping function from continuous or discrete feature vectors to discrete categorical target labels .
- Categories: Binary (e.g., Pass/Fail) and Multi-Class (e.g., Disease categories).
- Example: Predicting whether a transaction is Legitimate () or Fraudulent ().
Question 12 (2 Marks)
Question Statement
Define the logistic regression with mathematical notation? (2 Marks)
- Definition: A linear model for binary classification that outputs probabilities using the logistic sigmoid function:
- Logit Form: .
Question 13 (2 Marks)
Question Statement
What is Overfitting and Underfitting? (2 Marks)
- Underfitting: High bias, low capacity; model fails to capture training patterns ().
- Overfitting: High variance, excessive capacity; model fits training noise and generalizes poorly ().
Question 14 (2 Marks)
Question Statement
Explain the ROC curve with neat diagram? (2 Marks)
- Axes: True Positive Rate () on Y-axis vs. False Positive Rate () on X-axis across all thresholds .
- AUC: Measures ranking quality ( is ideal; is random chance).
Question 15 (2 Marks)
Question Statement
Define Up Sampling and Down Sampling? (2 Marks)
- Up-Sampling: Increasing minority class representation by duplicating or synthetically interpolating new instances (e.g., SMOTE).
- Down-Sampling: Decreasing majority class representation by randomly or strategically removing instances (e.g., NearMiss).
Question 16 (2 Marks)
Question Statement
How decision tree handles the overfitting? (2 Marks)
- Pre-Pruning: Early stopping based on
max_depth,min_samples_split, ormin_impurity_decrease. - Post-Pruning: Growing the full tree and pruning subtrees using Cost-Complexity penalty .
Question 17 (2 Marks)
Question Statement
What is ensemble learning and write the types? (2 Marks)
- Definition: Combining multiple diverse base models to improve predictive performance and robustness.
- Types: Bagging (Bootstrap Aggregation), Boosting (Adaptive Sequential), Stacking (Meta-Learner), and Voting (Hard/Soft).
Question 18 (2 Marks)
Question Statement
Signify the advantages of random forest technique. (2 Marks)
- Variance Reduction: Aggregating decorrelated trees eliminates variance without increasing bias.
- Feature Subspacing: Choosing random subsets of features () at each split prevents dominant features from dictating all trees.
- Built-in OOB Error: Unbiased internal validation from the unselected bootstrap instances.
Question 19 (2 Marks)
Question Statement
Define KNN model with example? (2 Marks)
- Definition: An instance-based non-parametric classifier that assigns a query point the majority label among its nearest neighbors:
- Example: In customer segmentation, classifying a new shopper by Euclidean proximity to the closest historical profiles.
Question 20 (2 Marks)
Question Statement
What are the types of naïve bayes classifier? (2 Marks)
- Gaussian Naïve Bayes: Continuous real-valued features modeled with normal distribution .
- Multinomial Naïve Bayes: Discrete integer count/frequency features (standard for Bag-of-Words text).
- Bernoulli Naïve Bayes: Binary boolean features () indicating word presence or absence.
Set 4 (Questions 1 to 10)
Question 1 (2 Marks)
Question Statement
Difference between Supervised and Unsupervised Learning with examples? (2 Marks)
| Feature | Supervised Learning | Unsupervised Learning |
|---|---|---|
| Data | Labelled pairs | Unlabelled feature vectors |
| Goal | Predict target for unseen inputs | Discover natural clusters or latent representations |
| Examples | Linear Regression, Logistic Regression, SVM | K-Means, PCA, DBSCAN, Autoencoders |
Question 2 (2 Marks)
Question Statement
What cost function is used for evaluating the loss of the logistic regression model? (2 Marks)
- Function: Binary Cross-Entropy (Log Loss):
- Property: Strictly convex with respect to , avoiding the local minima of Mean Squared Error.
Question 3 (2 Marks)
Question Statement
Define Variance and Bias? (2 Marks)
- Bias: Error from erroneous assumptions; difference between expected model prediction and ground truth ().
- Variance: Error from sensitivity to training data fluctuations; spread of predictions around the mean ().
Question 4 (2 Marks)
Question Statement
Define Recall and Precision (2 Marks)
- Precision: Proportion of positive alerts that are true:
- Recall: Proportion of actual positives successfully detected:
Question 5 (2 Marks)
Question Statement
What is Data imbalance and how it affects the model performance? (2 Marks)
- Definition: Severe disproportion between classes (e.g., negative, positive).
- Impact: Generates the Accuracy Paradox (predicting the majority class yields deceptively high accuracy while missing all critical minority events) and shifts decision boundaries away from the minority class.
Question 6 (2 Marks)
Question Statement
Define Gini Index with mathematical notation? (2 Marks)
- Formula:
- Meaning: Measures node impurity. for a pure node; for an equally balanced two-class split.
Question 7 (2 Marks)
Question Statement
Difference between the adaboost and gradient boosting (2 Marks)
| Dimension | AdaBoost | Gradient Boosting |
|---|---|---|
| Adaptive Mechanism | Updates sample weights based on classification error. | Fits base trees directly to pseudo-residuals (loss gradients). |
| Loss Support | Restricted primarily to Exponential Loss. | Optimizes arbitrary differentiable losses (MSE, Log-Loss, Huber). |
| Base Learner | Decision stumps (depth 1). | Shallow regression trees (depth 3 to 6). |
Question 8 (2 Marks)
Question Statement
Define Random forest with its hyper parameters? (2 Marks)
- Definition: An ensemble of decision trees trained on bootstrap samples with random feature subspacing at each node split.
- Key Hyperparameters:
n_estimators: Number of trees in the forest.max_features: Number of randomly sampled candidate features at each split ().max_depth: Maximum allowed tree depth to constrain tree complexity.min_samples_split: Minimum samples required to split an internal node.
Question 9 (2 Marks)
Question Statement
Define Bayes Theorem (2 Marks)
- Formula:
- Terms: is the posterior, is the likelihood, is the prior, and is the marginal evidence.
Question 10 (2 Marks)
Question Statement
What are the Distance measures are used in the KNN model? (2 Marks)
- Euclidean Distance (): (straight line).
- Manhattan Distance (): (grid based).
- Minkowski Distance (): .
- Cosine Distance: (angular orientation).
Examiner's Note
This page mirrors the exact 4 sets from Part_A_Questions.pdf. Structuring every 2-mark question with bold definitions, explicit equations, and clear examples guarantees maximum rubric hit rates during university evaluations.