CT1 Sample Questions — Batch 4: Theoretical Concepts

This document provides rigorous, textbook-style solutions for the theoretical questions (Q7, Q8, and Q9) from the IFS UNIT - 1 & 2 sample questions sheet. All concepts, definitions, and properties are explained in a rigorous academic tone, using LaTeX for all mathematical formulations and structural diagrams.

---

Question 7: Sampling Methodologies and Frameworks

Sampling is the process of selecting a subset of observations (a sample) from a larger population to infer characteristics about the population. Mathematically, let $\Omega$ represent the population space, and $S \subset \Omega$ represent the sample space of size $n$, drawn from a population of size $N$.

---

Part A: Probability Sampling Methods

In probability sampling, every element in the population has a known, non-zero probability of selection. This is the foundation of inferential statistics.

1. Simple Random Sampling (SRS)

Definition: Every possible subset of size $n$ has an equal probability of being selected.

• Simple Random Sampling with Replacement (SRSWR): Each selected unit is returned to the population before the next draw. The probability of selecting any unit on any draw is constant at $1/N$.
• Simple Random Sampling without Replacement (SRSWOR): Selected units are not returned. The probability of selecting unit $i$ on draw $j$ is conditional, but the overall marginal selection probability for any unit is $n/N$.
• Mathematical Form: The number of possible samples in SRSWOR is $\binom{N}{n}$.
• Data Science Example: Drawing a random sample of $10,000$ user transaction records from a database of $1,000,000$ rows using a pseudo-random number generator (e.g., df.sample(n=10000, replace=False)).

2. Stratified Random Sampling

Definition: The heterogeneous population is divided into mutually exclusive and collectively exhaustive subgroups called strata based on an auxiliary variable. An SRS is then drawn independently from each stratum.

• Mathematical Form: If stratum $h$ has population size $N_h$, we select sample size $n_h$. Under proportional allocation: $n_h = n \left(\frac{N_h}{N}\right)$
• Benefits: Minimizes intra-stratum variance and maximizes inter-stratum variance, leading to a smaller standard error than SRS for the same sample size.
• Data Science Example: Evaluating customer satisfaction in an e-commerce platform. The population is stratified by tier: Free, Bronze, Silver, Gold. A proportional SRS is drawn from each tier to prevent under-representing high-value, smaller-sized segments.

3. Systematic Sampling

Definition: Selecting every $k$-th element from an ordered list of the population, starting from a randomly selected element between $1$ and $k$.

• Mathematical Form: Let $k = \lfloor N/n \rfloor$ represent the sampling interval. Select a random starting integer $r \in [1, k]$. The sample units are: $S = \{r, r+k, r+2k, \dots, r+(n-1)k\}$
• Risk: If the population exhibits a periodic pattern (seasonality/cyclicality) that matches a multiple of $k$ (a phenomenon known as periodicity), the sample will be highly biased.
• Data Science Example: Quality control on an automated factory conveyor belt. An engineer inspects every $100$-th motherboard coming off the line.

4. Cluster Sampling

Definition: The population is divided into heterogeneous subgroups called clusters, which ideally mirror the population's diversity. A random sample of clusters is selected, and all elements within the chosen clusters are surveyed (one-stage cluster sampling) or randomly sampled (two-stage cluster sampling).

• Mathematical Comparison:
  • Stratified: Sample some elements from all strata (homogeneous groups).
  • Cluster: Sample all elements from some clusters (heterogeneous groups).
• Data Science Example: Conducting regional A/B testing across brick-and-mortar stores. Instead of sampling individuals nationwide, 10 physical retail stores (clusters) are randomly selected, and all shoppers in those stores are monitored.

---

Part B: Non-Probability Sampling Methods (Brief Overview)

Non-probability sampling relies on non-random criteria, meaning elements have unknown or zero probabilities of selection. These methods are prone to selection bias.

Method	Definition	Data Science Example
Convenience	Selecting easily accessible units.	Scraping reviews from only the first page of a website.
Quota	Selecting units until a specific target threshold is met.	Recruiting 50 Android and 50 iOS users from a local café.
Judgmental	Selecting units based on researcher expertise.	Hand-picking "typical" representative outlier cases for debugging.
Snowball	Existing subjects recruit future subjects.	Finding niche beta testers for a specialized developer API.

---

Question 8: Theoretical Frameworks of Probability

Probability is the mathematical quantification of uncertainty. Mathematically, it is defined over a probability space $(\Omega, \mathcal{F}, P)$, where $\Omega$ is the sample space, $\mathcal{F}$ is the $\sigma$-algebra of events, and $P$ is the probability measure.

---

1. Classical (A Priori / Laplace) Approach

Definition: If a random experiment has $N$ mutually exclusive, collectively exhaustive, and equally likely outcomes, and $M$ of these outcomes are favorable to event $A$: $P(A) = \frac{M}{N} = \frac{\text{Number of favorable outcomes}}{\text{Total number of possible outcomes}}$

• Assumption: Outcoms must be symmetric and equally likely (symmetric prior).
• Limitation: Cannot be applied to infinite sample spaces or asymmetric systems (e.g., loaded dice).
• Example: The probability of drawing an Ace of Spades from a standard shuffled deck of 52 cards is $P(\text{Ace of Spades}) = 1/52$.

2. Empirical (Frequentist / Relative Frequency) Approach

Definition: The probability of an event is the limiting value of its relative frequency of occurrence in a sequence of $n$ independent, identical trials as $n$ approaches infinity. $P(A) = \lim_{n \to \infty} \frac{f(A)}{n}$ where $f(A)$ is the frequency of event $A$ occurring in $n$ trials.

• Law of Large Numbers (LLN): As $n$ grows, the sample proportion $\frac{f(A)}{n}$ converges almost surely to the true underlying population probability.
• Limitation: Requires repeatable, identical trials. Cannot determine the probability of unique, non-repeatable events (e.g., "The probability that a specific company goes bankrupt in 2027").
• Example: Flipping a coin $10,000$ times and finding it lands on heads $5,002$ times, giving an empirical probability of $P(\text{Heads}) \approx 0.5002$.

3. Subjective Approach

Definition: Probability represents an individual’s personal degree of belief or confidence that a specific outcome will occur, based on available evidence, experience, and intuition.

• Methodology: Often calibrated using betting odds or decision-theory utility functions. Coherence is maintained via Bayesian updating as new data arrives.
• Limitation: Lacks objective, verifiable scientific basis; two rational agents can assign different subjective probabilities to the same event.
• Example: An AI researcher estimating that there is a $65\%$ probability that Artificial General Intelligence (AGI) will be achieved by 2030.

4. Axiomatic (Kolmogorov) Approach

Definition: Modern probability theory is defined purely axiomatically. A probability measure $P$ is a real-valued function defined on $\mathcal{F}$ that satisfies three fundamental axioms:

1. Axiom of Non-Negativity: For any event $A \in \mathcal{F}$, the probability is non-negative: $P(A) \ge 0$
2. Axiom of Normalization: The probability of the entire sample space $\Omega$ is 1: $P(\Omega) = 1$
3. Axiom of Countable Additivity: For any countable sequence of pairwise disjoint (mutually exclusive) events $A_1, A_2, \dots$: $P\left(\bigcup_{i=1}^\infty A_i\right) = \sum_{i=1}^\infty P(A_i)$

---

Question 9: Statistical Metrics of Central Tendency, Dispersion, and Distribution

A comprehensive statistical characterization of a univariate dataset requires analyzing its location (central tendency), scale (dispersion), and shape (distribution).

---

Part A: Measures of Central Tendency (Location)

These measures find the "center" or typical value of a distribution.

       Skewed and Symmetric Distributions Comparison
       
     Symmetric (Normal)            Right-Skewed
        |      |      |               |
        |  M = Md = Mo                |  Mo  Md  M
        |      |      |               |   |   |  |
   -----+------*------+-----     -----+---*---*--*-----+-----

1. Arithmetic Mean ($\mu$ or $\bar{x}$)

• Mathematical Form: $\bar{x} = \frac{1}{n} \sum_{i=1}^n x_i$
• Property: Highly sensitive to outliers; represents the "center of gravity" of the physical distribution.
• Best Use Case: Symmetric distributions with no extreme values (e.g., sensor calibration temperatures).

2. Median ($M_d$)

• Mathematical Form: Let the ordered observations of a dataset of size $n$ be $x_{(1)}, x_{(2)}, \dots, x_{(n)}$ in ascending order.
  • If $n$ is odd: The median is the single middle value located at the $\frac{n+1}{2}$-th position: $M_d = x_{\left(\frac{n+1}{2}\right)}$
  • If $n$ is even: The median is the arithmetic mean of the two middle values located at the $\frac{n}{2}$-th and $\left(\frac{n}{2}+1\right)$-th positions: $M_d = \frac{x_{\left(\frac{n}{2}\right)} + x_{\left(\frac{n}{2}+1\right)}}{2}$
• Property: Robust to extreme outliers; minimizes the sum of absolute deviations: $\operatorname{argmin}_a \sum_{i=1}^n |x_i - a| = M_d$
• Best Use Case: Highly skewed interval/ratio scale data (e.g., household incomes, software bug-fix times).

3. Mode ($M_o$)

• Definition: The most frequently occurring value in the dataset.
• Property: The only measure of central tendency applicable to nominal (categorical) data.
• Best Use Case: Qualitative classes (e.g., most popular operating system used by developers).

---

Part B: Measures of Dispersion (Scale)

These measures quantify the spread, variability, or uncertainty in the data.

1. Range ($R$)

• Mathematical Form: $R = X_{\text{max}} - X_{\text{min}}$
• Property: Simple to calculate but extremely sensitive to outliers; ignores the distribution of data between the boundaries.

2. Quartile Deviation (QD)

• Mathematical Form: $QD = \frac{Q_3 - Q_1}{2}$
• Property: Measures the spread of the middle $50\%$ of the data; highly robust to extreme outliers.

3. Variance ($\sigma^2, s^2$) and Standard Deviation ($\sigma, s$)

• Mathematical Form:
  • Population Variance: $\sigma^2 = \frac{1}{N}\sum (x_i - \mu)^2$
  • Sample Variance: $s^2 = \frac{1}{n-1}\sum (x_i - \bar{x})^2$ (where the $n-1$ denominator is Bessel's Correction to ensure an unbiased estimator).
• Property: Standard deviation is in the same physical units as the underlying data. Highly sensitive to outliers due to the squaring of deviations.

4. Coefficient of Variation (CV)

• Mathematical Form: $CV = \left( \frac{\sigma}{\mu} \right) \times 100\%$
• Property: A dimensionless measure of relative dispersion, enabling direct comparison of variability between datasets with different scales or units (e.g., comparing the variability of animal weights in grams vs. human weights in kilograms).

---

Part C: Measures of Distribution Shape (Moments)

Moments describe the geometric properties of a distribution. Let $\mu_k$ represent the $k$-th central moment: $\mu_k = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^k$

1. Skewness (Asymmetry)

Skewness measures the degree of asymmetry of a distribution around its mean.

• Karl Pearson's Coefficient of Skewness: $S_{k,P} = \frac{\text{Mean} - \text{Mode}}{\text{SD}} \approx \frac{3(\text{Mean} - \text{Median})}{\text{SD}}$
• Moment-based Skewness ($\gamma_1$ or $\beta_1$): $\gamma_1 = \frac{\mu_3}{\mu_2^{1.5}}$
• Interpretation:
  • $\gamma_1 > 0$: Positive (Right) Skew. Right tail is longer; $\text{Mode} < \text{Median} < \text{Mean}$.
  • $\gamma_1 < 0$: Negative (Left) Skew. Left tail is longer; $\text{Mean} < \text{Median} < \text{Mode}$.
  • $\gamma_1 = 0$: Perfect symmetry (e.g., Normal Distribution).

2. Kurtosis (Peakedness & Tail-Weight)

Kurtosis measures the "tailedness" of the distribution, reflecting the presence of extreme outliers.

• Moment-based Kurtosis ($\beta_2$): $\beta_2 = \frac{\mu_4}{\mu_2^2}$
• Excess Kurtosis ($\gamma_2$): $\gamma_2 = \beta_2 - 3$
• Classification:
  • Leptokurtic ($\gamma_2 > 0$): Heavy-tailed, high density of extreme outliers (e.g., Student's t-distribution, financial asset returns).
  • Mesokurtic ($\gamma_2 = 0$): Standard tails, matching the normal distribution.
  • Platykurtic ($\gamma_2 < 0$): Thin-tailed, low density of outliers, flatter peak (e.g., uniform distribution, Bernoulli trials).

Exam tip: Memorize the sign rules for skewness and excess kurtosis. These are extremely common 2-mark questions (e.g., "Define Leptokurtic and state its relation to $\beta_2$"). Always write: "A leptokurtic distribution has $\beta_2 > 3$ or excess kurtosis $\gamma_2 > 0$."