93 Machine Learning & DS interview questions with answers

All Machine Learning & DS interview questions

• What is Mixture of Experts (MoE)? How does it differ from dense models?

• What is Flash Attention? Why does it speed up training/inference without changing results?

• What is RoPE (Rotary Position Embedding)? Why is it preferred over learned positional embeddings?

• Scaling laws and emergent abilities: why do larger LLMs unlock new capabilities?

• What is the bias-variance tradeoff?

  A model's expected test error decomposes into three parts: Bias², Variance, and irreducible error (noise σ² that cannot be removed): E[(y − f̂(x))²] = Bias(f̂(x))² + Var(f̂(x)) + σ² - Bias — error from a model too simple to capture the true relationship. High bias…

• What is overfitting and how do you prevent it?

  Overfitting is when a model memorizes noise and quirks of the training set instead of the general pattern — scoring very high on training data but poorly on new data. Telltale sign: a large gap between training error and validation/test error. (The opposite, underfitting, is…

• What is a confusion matrix? Distinguish precision, recall, and F1-score.

  For binary classification, a confusion matrix is a 2×2 table matching true vs predicted labels, with 4 cells: TP (correct positive), TN (correct negative), FP (false positive — predicted positive but actually negative), FN (false negative — a missed positive). - Precision = TP /…

• What are the ROC curve and AUC?

  A probabilistic classifier needs a threshold to turn probabilities into 0/1 labels. The ROC curve plots the relationship between two quantities as the threshold sweeps from 0 to 1: - Y-axis: TPR = Recall = TP / (TP + FN). - X-axis: FPR = FP…

• What is cross-validation? Compare k-Fold, Leave-One-Out, and Hold-Out.

  Cross-validation estimates generalization by splitting data and training/testing on different portions, giving a less luck-dependent estimate than a single split. - Hold-out — one split into train/test (e.g. 80/20). Fast, but the estimate has high variance: results depend heavily on where the split lands. -…

• What is regularization? Distinguish L1 (Lasso), L2 (Ridge), and Elastic Net.

• Distinguish feature engineering from feature selection.

  Feature engineering is creating and transforming features so the model learns more easily: encoding categoricals (one-hot, target encoding), scaling numerics, building interaction or polynomial features, extracting parts from datetimes, log-transforming skewed distributions, or crafting domain-knowledge features. It is often the highest-impact step for quality in…

• What is the curse of dimensionality and how do you address it?

• How do you handle an imbalanced dataset? How does SMOTE work?

  Imbalanced data (e.g. 99% negative, 1% positive as in fraud detection) makes accuracy misleading — a model that always predicts the majority still hits 99%. Address both the metric and the data: - Pick suitable metrics — precision, recall, F1, PR-AUC instead of accuracy. -…

• What are the roles of train/validation/test sets, and what is data leakage?

  Split data into three parts, each with a distinct role: - Train — used to fit the model parameters. - Validation — used to select models and tune hyperparameters. Because you make decisions from it, the validation score is already "contaminated" with information. - Test…

• Gradient descent: distinguish batch, stochastic (SGD), and mini-batch.

  Gradient descent updates parameters in the direction opposite the loss gradient. The three variants differ in how much data each update uses: - Batch GD — computes the gradient over the entire dataset before each single update. Accurate direction, smooth convergence, but each step is…

• What is the precision-recall tradeoff and threshold tuning?

  A classifier usually outputs a probability, then compares it to a threshold to assign a positive/negative label. The default 0.5 threshold is rarely optimal. Tradeoff: - Lower the threshold → label positive more readily → recall rises (catch more positives) but precision falls (more false…

• Regression metrics: MSE, RMSE, MAE, and R²?

  Four common metrics for evaluating regression models (with y the true value and ŷ the prediction): - MSE (Mean Squared Error) = mean of (y − ŷ)². Penalizes large errors heavily (squared), sensitive to outliers. Its units are the target squared, so it is hard…

• How does linear regression work, and what assumptions does it rely on?

  Linear regression models the relationship between a continuous target y and features X as a linear function: y = β₀ + β₁x₁ + … + βₚxₚ + ε. The coefficients β are usually estimated by ordinary least squares (OLS) — minimizing the sum of squared…

• Why is logistic regression a classification algorithm despite the word "regression" in its name?

  Because it estimates the probability that a sample belongs to the positive class, then compares it to a threshold to produce a label. Internally it still computes a linear combination z = β₀ + βᵀx like linear regression, but passes it through the sigmoid/logistic function…

• Distinguish bagging from boosting; how do random forests and bootstrapping relate?

• How do AdaBoost, gradient boosting, and XGBoost differ?

• How does K-Means work, and how do you choose the optimal number of clusters k?

  K-Means partitions data into k clusters by iterating two steps: 1. Assign: each point to its nearest centroid by Euclidean distance. 2. Update: each new centroid = the mean of the points in its cluster. Repeat until the assignment stabilizes. The objective is to minimize…

• What does PCA do, and why does it maximize variance?

• How does K-Nearest Neighbors work, and why is KNN called a "lazy" algorithm?

  KNN predicts by finding the k nearest training points to the query sample (by a distance metric, usually Euclidean), then taking a majority vote (classification) or an average (regression) over them. It is a "lazy" learner because it builds no model at training time —…

• How do SVMs work, and what is the kernel trick?

• How does a decision tree choose splits (entropy/gini), and why is pruning needed?

  A decision tree splits the data recursively on features. At each node it picks the split that makes the child branches as label-pure as possible, measured by an impurity metric: - Gini impurity: 1 − Σ pᵢ² — the probability of misclassifying if you labeled…

• What is the difference between supervised, unsupervised, and reinforcement learning?

  - Supervised: learns from labeled data (each input paired with the correct output). The goal is to learn a mapping to predict labels for new data. It covers classification (discrete labels) and regression (continuous values). Examples: linear/logistic regression, SVM, decision trees. - Unsupervised: the data…

• How does Naive Bayes work, and what is its "naive" independence assumption?

  Naive Bayes is a probabilistic classifier based on Bayes' theorem: it picks the class y with the highest posterior probability, P(y x) ∝ P(y) · Π P(xⱼ y). The "naive" assumption is conditional independence: the features are independent of each other given the class. This…

• In ensemble learning, how do voting and stacking differ?

  Both combine multiple base models to predict better than any single one, but they differ in how they aggregate: - Voting: merges predictions with a fixed, non-learned rule. - Hard voting: take the label by majority vote. - Soft voting: average the predicted probabilities and…

• Compare K-Means, DBSCAN, and hierarchical clustering.

  - K-Means: partitions data into a preset k clusters around centroids; fast and scalable. But it assumes spherical, similarly sized clusters, is sensitive to outliers, and requires choosing k upfront. - DBSCAN (density-based): groups high-density points (at least minPts within radius ε) and treats sparse-region…

• What is the Central Limit Theorem and why does it matter?

  The CLT states that for a large enough sample size n, the distribution of the sample mean (x̄) is approximately normal, regardless of the shape of the underlying population — provided observations are independent, identically distributed, and the population has finite variance. - The mean…

• What is a p-value? Distinguish Type I and Type II errors.

  A p-value is the probability of observing data at least as extreme as what you got, assuming the null hypothesis (H0) is true. A small p-value means the data is unlikely under H0 → evidence against H0. Compare it to the significance level α: if…

• Does correlation imply causation? Distinguish covariance and correlation.

  Correlation measures the strength and direction of a linear relationship between two variables. Correlation does not imply causation: a link can arise from a confounder, reverse causation, or coincidence. Only a controlled / randomized experiment can establish causation. Covariance vs correlation coefficient: - Covariance captures…

• How does hypothesis testing work (H0/H1, significance level α)?

  Hypothesis testing is a procedure that uses sample data to decide between two competing claims: 1. State the hypotheses. H0 (null) describes the default / no-effect state; H1 (alternative) is what you want to demonstrate (one- or two-sided). 2. Choose a significance level α (commonly…

• What is the normal distribution, and what is a z-score for?

  The normal (Gaussian) distribution is a symmetric bell-shaped distribution fully described by its mean μ and standard deviation σ, written N(μ, σ²). The 68–95–99.7 (empirical) rule: - ~68% of data falls within μ ± 1σ - ~95% within μ ± 2σ - ~99.7% within μ…

• What is a confidence interval, and how do you interpret it correctly?

  A confidence interval (CI) is a range estimate for a population parameter computed from sample data. For a mean: x̄ ± z · (σ/√n) (use t with the sample standard deviation s when σ is unknown). z is the critical value for the confidence level…

• What are Bayes' theorem and conditional probability?

  Conditional probability P(AB) is the probability of A given that B occurred: P(AB) = P(A∩B) / P(B). Bayes' theorem lets you "flip" the condition, updating beliefs after seeing evidence: P(AB) = P(BA) · P(A) / P(B) - P(A): the prior (belief before). - P(BA): the…

• What sampling methods exist, and what is sampling bias?

  Probability sampling methods (each unit has a known chance of selection): - Simple random — every individual has an equal chance. - Stratified — split the population into subgroups (strata), then sample within each; ensures subgroups are represented. - Cluster — split into clusters, randomly…

• Descriptive statistics: distinguish mean/median/mode and variance/std.

  Measures of central tendency: - Mean (average) — sum divided by count; sensitive to outliers. - Median — the middle value when sorted; robust to outliers and skew. - Mode — the most frequent value; works for categorical data too. For skewed data (like income),…

• What are skewness and kurtosis?

  Both describe the shape of a distribution beyond its mean and variance. Skewness measures asymmetry: - Right / positive skew — a long right tail, mean median (e.g. income, response times). - Left / negative skew — a long left tail, mean < median. -…

• A/B testing: how do you design the experiment and read the results correctly?

• What are vanishing and exploding gradients, why do they happen, and how do you fix them?

• Compare SGD + momentum, Adagrad, RMSProp, and Adam; when do you use each?

  All are gradient-descent variants that differ in how they use gradient history to set the step. - SGD + momentum: accumulates a decayed moving average of gradients (a "velocity") → smooths updates, speeds up along consistent directions, dampens oscillation. Uses a single global learning rate…

• What is batch normalization, how does it work, and what benefits does it bring?

• How do dropout and early stopping combat overfitting?

  Both are regularization techniques — reducing overfitting without extra data. - Dropout: during training, each forward pass randomly drops (zeroes out) each neuron with probability p. The network cannot rely on a few co-adapted units → this effectively trains an ensemble of subnetworks that share…

• Explain convolution, kernel, stride, padding, and pooling in a CNN.

  A CNN processes grid-like data (images) via local connectivity and weight sharing. - Kernel / filter: a small weight matrix slid over the input; at each position it computes a dot product → detecting a local pattern (edge, texture). The weights are learned. - Convolution:…

• How do RNN, LSTM, and GRU differ, and what problem does LSTM solve?

• How does a GAN work, and what are the roles of the generator and discriminator?

• How do an autoencoder and a variational autoencoder (VAE) differ?

• ReLU, sigmoid, tanh, and softmax: characteristics and when to use each?

  An activation function injects non-linearity into the network. Without it, stacked linear layers collapse into a single linear map. - Sigmoid 1/(1+e^(-x)): output in (0, 1). Use it for a binary probability at the output. In hidden layers it saturates easily: when the input is…

• How does backpropagation work?

  Backpropagation is the algorithm that computes the gradient of the loss with respect to every weight efficiently, so gradient descent can update them. Two passes: 1. Forward pass: feed the input through the network, computing each layer's output and the final loss. 2. Backward pass:…

• Why does weight initialization matter, and how does Xavier/Glorot differ from He initialization?

• How do cross-entropy and MSE differ, and when do you use each?

  The loss function defines what "wrong" means and produces the gradients that drive learning. - MSE (mean squared error) mean((y − ŷ)²): for regression (continuous targets). Penalizes large errors quadratically; sensitive to outliers. Pairs with a linear output. - Cross-entropy (log loss): for classification. Measures…

• Distinguish epoch, batch, and iteration; what are learning rate and scheduling?

  - Epoch: one full pass over the entire training set. - Batch (mini-batch): the group of samples processed before one weight update; batchsize sets its size. - Iteration: one weight update = one processed batch. iterations per epoch = ceil(N / batchsize). Example: 10,000 samples,…

• What is a Markov Decision Process (MDP), and what does the Bellman equation state?

• Distinguish Q-Learning from SARSA (off-policy vs on-policy).

• Compare value-based and policy-based methods in RL.

• What is the exploration vs exploitation trade-off, and how does ε-greedy work?

  In RL, the agent must balance: - Exploitation — pick the best known action to maximize immediate reward. - Exploration — try uncertain actions to gather information that may reveal a better choice later. Pure exploitation → getting stuck in a local optimum (never trying…

• Explain the core RL concepts: reward, policy, value function, discount factor γ.

  - Reward r — a scalar signal the environment returns each step, measuring the immediate goodness of an action. The agent aims to maximize the cumulative reward (return), not any single-step reward. - Policy π — the agent strategy, mapping states → actions. It can…

• Distinguish model-free from model-based RL.

• What is a Deep Q-Network (DQN)? The core idea and its stabilizing tricks.

• What is ARIMA, and why does stationarity matter in time series?

• Distinguish collaborative filtering from content-based filtering in recommender systems.

  Content-based filtering — recommends based on item features and the user own preference profile. If you like sci-fi movies, it suggests other sci-fi movies. - Pros: needs no data from other users; handles new items (as long as they have features); easy to explain. -…

• What strategies exist for handling missing data?

  First understand the missingness mechanism: MCAR (missing completely at random), MAR (depends on other observed variables), MNAR (depends on the missing value itself) — this determines whether a strategy introduces bias. Main strategies: - Deletion — drop rows (listwise) or columns. Simple but loses data;…

• Compare one-hot, label, and target encoding for categorical variables.

  Many models accept only numeric input, so categorical variables need encoding. - One-hot encoding — each value becomes its own binary column. It imposes no fake ordering, fitting nominal variables and linear models. Downside: a dimensionality blow-up at high cardinality (many distinct values). - Label…

• Feature scaling: distinguish normalization from standardization, and when is it needed?

  Scaling brings numeric features onto a comparable range so no feature dominates just because its units are large. - Normalization (Min-Max) — squeeze values into a fixed range, usually [0, 1], using x′ = (x − min) / (max − min). It preserves the distribution…

• Distinguish parametric from non-parametric models.

  The difference is whether the model assumes a fixed functional form up front (with a finite number of parameters that does not grow with the data) or not. Parametric — assume a functional form and learn a fixed number of parameters. Examples: linear/logistic regression, Naive…

• From a learning curve, how do you tell whether a model has high bias or high variance, and what do you do in each case?

  A learning curve plots training error and validation error against the number of training samples. The shape of the two curves tells you where the problem is. High bias (underfitting): both curves plateau at a high error and sit close together. Adding data barely helps…

• Why should you build a simple baseline before reaching for a complex model? What goes into a baseline?

  A baseline is the reference point that tells you whether a complex model actually adds value. Without one, an AUC of 0.85 means nothing. Baselines worth running first: - Constant / majority: always predict the most frequent class (classification) or the mean (regression). In scikit-learn:…

• Which models require feature scaling and which do not? Why?

  Scaling matters for any model where feature magnitude feeds directly into the objective function or into a distance. Needs scaling: - Distance-based models: KNN, K-Means, SVM with an RBF kernel. An income feature in millions will dominate age in a Euclidean distance. - Gradient-descent models:…

• A fraud-detection task has 0.5% positive samples. Why does 99.5% accuracy say nothing, and which metrics should you report?

  Because a model that predicts everything as negative already reaches 99.5% accuracy while catching zero fraudulent transactions. On imbalanced data, accuracy is dominated by the majority class and does not reflect performance on the class you care about. Metrics to use instead: - Recall (sensitivity):…

• When should you use PR-AUC instead of ROC-AUC? Why can ROC-AUC look good on highly imbalanced data?

  The difference is in the denominator of the x-axis. - ROC plots TPR against FPR, where FPR = FP / (FP + TN). When the negative class is huge, TN is enormous, so FPR barely moves even as FP grows by thousands. The ROC curve…

• What forms of data leakage are common in practice, and how do you prevent them?

  Leakage is information that would not exist at real prediction time slipping into training. The classic symptom: unusually high offline scores that collapse in production. Common forms: 1. Preprocessing before splitting. Calling scaler.fit() or imputing on the full dataset and only then doing traintestsplit →…

• A classifier outputs probabilities. Why is 0.5 not the right default threshold, and how do you choose one?

  0.5 is just the convention baked into predict(), not the result of any optimization. The right threshold depends on the cost of the two error types and the class ratio — neither of which the model knows. How to choose it properly: 1. Build a…

• For imbalanced data, how do you choose between oversampling, undersampling and class weights? When is no intervention needed?

  When to do nothing: imbalance is not a defect by itself. If you use an appropriate metric (PR-AUC, recall at a fixed precision) and choose your own threshold, a probabilistic model trained on the raw distribution is often enough. In many cases lowering the threshold…

• Why can you not use random k-fold on time-series data? How do you validate it correctly?

  Because random k-fold shuffles away the time order: the model trains on June data and then predicts March. That is temporal leakage, and the cross-validation score will be far better than what the model achieves when actually forecasting an unseen future. The right approach —…

• A model performs well on the test set but degrades after a few months in production. What are the likely causes and how do you investigate?

  Three families of causes, distinguished by what actually changed: 1. Data drift (covariate shift) — the input distribution P(X) changed while P(yX) held. For example the customer base expanded to a new region, or an app redesign left a feature permanently empty. 2. Concept drift…

• How should outliers be handled? Should they always be removed?

  No. The first question is whether the outlier is a data error or a genuine rare event. - age = 200, price = -1: entry or unit errors → fix or drop. - A 5-billion-VND transaction in a fraud dataset: that is exactly what you…

• How do you explain a model to non-technical stakeholders? What is wrong with a random forest's default feature importance?

• How do you encode a categorical variable with tens of thousands of levels (product code, district, user id)? How does target encoding leak and how do you fix it?

• When is manual feature engineering worth the effort, and when should you let the model learn representations?

• A colleague reruns your training code and gets different results. What sources of non-reproducibility should you check, and how do you fix them?

• What does it take to put a trained model into production (serving, monitoring, retraining)?

• When you receive a new dataset, what steps does your EDA (exploratory data analysis) follow, and why must the data be cleaned before it goes into a model?

  EDA is reading and understanding the data before doing anything with it: what it looks like, what is wrong with it, which variables relate to the target. The goal is to know what you are holding, not yet to build a model. The usual order:…

• Which chart types do you use in data analysis, and what do you base the choice of chart on?

  Pick the chart by the question you want to answer, not by taste: comparing groups, trend over time, distribution, relationship between two variables, or share of a total. Question Chart Example --------- Which group is bigger? Bar (vertical; horizontal when labels are long) Revenue by…

• How does OCR (optical character recognition) work? What steps make up an OCR pipeline?

  Modern OCR does two main jobs: text detection finds the regions that contain text, then text recognition reads the character sequence in each region. Around them sit image preprocessing and text post-processing. 1. Preprocessing: grayscale, denoise, binarise (Otsu), deskew, and fix perspective when a document…

• In object detection, what is IoU, and how does Non-Maximum Suppression (NMS) use IoU to remove duplicate bounding boxes? Implement NMS from scratch.

  IoU (Intersection over Union) measures how much two boxes overlap: the intersection area divided by the union area, from 0 (no contact) to 1 (identical). NMS uses IoU to remove duplicates: a detector usually fires many boxes around one object, and NMS keeps the highest-scoring…

• Why does the gradient of cross-entropy loss combined with softmax, taken with respect to the logits, simplify to p − y? Prove it.

  With logits z, probabilities p = softmax(z) and one-hot label y, the derivative of cross-entropy with respect to the logits is ∂L/∂zi = pi − yi: prediction minus label. It is that clean because the derivative of log cancels exactly the pk produced by the…

• What is data augmentation? What are common augmentation techniques for image data, and when should you use them?

  Data augmentation creates extra variants of training images with transformations that do not change the label: a horizontally flipped cat is still a cat. The model sees more forms of the same object, so it memorises fewer incidental details of the training set, i.e. less…

• How do AI, Machine Learning and Deep Learning differ, and how are they related?

  The three concepts nest like circles: AI ⊃ Machine Learning ⊃ Deep Learning. AI is the goal (machines doing tasks that need human intelligence), ML is one way to reach it (learning patterns from data instead of hand-writing rules), and DL is a branch of…

• Derive the loss function (log loss) and gradient of Logistic Regression from maximum likelihood.

  Logistic Regression assumes the label y ∈ {0, 1} follows a Bernoulli distribution with probability p = σ(wᵀx + b). The negative log-likelihood of that distribution is exactly log loss (binary cross-entropy), and its gradient simplifies to Σ (pi − yi) xi. Step 1, the…

• When do you reduce dimensionality with PCA, t-SNE or UMAP, and how do you choose how many dimensions to keep?

  Choose by purpose: PCA for real dimensionality reduction before a model (linear, fast, applies to new data), t-SNE almost only for plotting data in 2D/3D so humans can see clusters, and UMAP in between: non-linear like t-SNE but faster and usable as a preprocessing step…

• How does the EM (Expectation-Maximization) algorithm work? How do you apply EM to fit a Gaussian Mixture Model?