41 Cross-validation and Data Leakage Interview Questions

All 41 questions

How would you structure your validation splits to prevent data leakage or look ahead bias?
Asked in 5 of 320 interviews · last asked Sep 11 · Scenario · Senior
Show answer
I would split data to match how the model will be used. For time-based predictions, I would train on earlier data and validate on later data. For predictions on new users or devices, I would keep each group in one split. I would fit preprocessing only on training data and keep a final test set untouched during tuning.
- Use rolling time splits to test performance across several future periods.
- Leave a time gap when training labels overlap the validation period.
- Use only features available at prediction time, not information added later.
- Fit imputation, scaling, and feature selection separately within each training fold.
What the interviewer checks: The interviewer checks whether your validation reflects real use and keeps future or held-out information out of training.
Other ways it was asked
- “How would you structure your validation splits to prevent data leakage or look-ahead bias?”
- “How would you structure validation splits to prevent data leakage or look-ahead bias?”
What are the differences between training, validation, and test data?
Asked in 4 of 320 interviews · last asked Sep 15 · Concept · Junior
Show answer
Training data is used to learn the model’s parameters. Validation data is used to tune settings and choose between models. Test data is kept separate until those choices are final, then used to estimate performance on unseen data.
- Training changes learned parameters; validation guides choices such as hyperparameters and when to stop training.
- Using test results to tune the model makes the final performance estimate less trustworthy.
- Fit preprocessing steps on training data only, then apply them to validation and test data.
- Avoid duplicate records across splits; separate users and preserve time order when the task requires it.
What the interviewer checks: The interviewer checks whether you understand each split’s role and how to avoid data leakage.
Other ways it was asked
- “What is the difference between training data and test data?”
- “What is the difference between training data and testing data?”
- “What is the difference between test data and validation data?”
What validation techniques can you use to evaluate a model, and how does cross-validation work?
Asked in 3 of 320 interviews · last asked Aug 21 · Concept · Junior
Other ways it was asked
- “Which validation techniques can you use to evaluate a machine-learning model?”
- “What coding techniques or other methods would you use to validate a machine-learning model?”
- “Can you describe what cross-validation is?”
What is data leakage, and how would you identify and prevent it in a modeling workflow?
Asked in 3 of 320 interviews · last asked Oct 18 · Scenario · Mid-level
Other ways it was asked
- “How do you identify data leakage in a proposed modeling workflow?”
- “What is data leakage in machine learning? Give a real-world example and explain how you would prevent it.”
- “How would you prevent data leakage when building a machine learning model?”
In scikit-learn, how do you correctly run cross-validation when preprocessing steps such as scaling and one-hot encoding are involved, to avoid data leakage?
Asked in 2 of 320 interviews · last asked Jun 3 · Tooling · Mid-level
Other ways it was asked
- “In scikit-learn, how do you avoid data leakage when doing preprocessing and cross-validation, especially with scaling and imputation?”
How do you validate transaction cost and slippage assumptions in a rates backtest against realized fills?
Asked once · last asked Oct 10 · Scenario · Senior
Other ways it was asked
- “How do you model transaction costs and slippage in a rates backtest, such as for U.S. Treasury futures or swaps, and validate those assumptions against realized fills?”
What data would you use to validate a campus placement prediction model?
Asked once · last asked Mar 27 · Scenario · Mid-level
How do you design validation when there's a risk of leakage, such as with time-series data, multiple rows per user, or repeated interactions in a recommender system?
Asked once · last asked May 3 · Scenario · Senior
In scikit-learn, how do you prevent target leakage when doing time-series cross-validation with feature scaling and encoding, and what APIs do you use to enforce that?
Asked once · last asked May 3 · Tooling · Senior
How would you structure a scikit-learn pipeline that includes lag or rolling features from a timestamped series without leaking future information, given that a pipeline doesn't natively know about time order?
Asked once · last asked May 3 · Scenario · Senior
In a custom transformer, how would you handle the boundary between training and validation so the first validation rows can use the last k training observations for lag or rolling features without refitting on validation targets?
Asked once · last asked May 3 · Scenario · Senior
How do you prevent data leakage when doing target encoding for a high-cardinality categorical feature, especially under time-based cross-validation?
Asked once · last asked May 3 · Scenario · Senior
When splitting a dataset into groups, should the groups have similar data distributions or equal numbers of samples?
Asked once · last asked May 7 · Concept · Junior
When you use GridSearchCV with a pipeline and a ColumnTransformer, where should fitting happen, and what data should never be used to learn preprocessing parameters?
Asked once · last asked Jun 3 · Tooling · Senior
How do you detect and prevent data leakage in a time-to-event prediction problem, especially with feature generation and train-validation splitting?
Asked once · last asked Jun 15 · Scenario · Senior
How would you implement a scikit-learn workflow, including input data, preprocessing steps, and validation of the results?
Asked once · last asked Aug 12 · Behavioral · Mid-level
Other ways it was asked
- “Describe a scikit-learn workflow you implemented, including its input data, preprocessing steps, and validation results.”
How would you prevent data leakage or frame dependence from making a confidence interval for a model's precision improvement too optimistic?
Asked once · last asked Aug 12 · Scenario · Senior
How would you prevent label leakage and spurious shortcuts, for example artifact cases always having a specific operator ID?
Asked once · last asked Aug 17 · Scenario · Senior
How do you detect and prevent label leakage when features are time-based and target labels are derived from future events?
Asked once · last asked Aug 19 · Scenario · Senior
In time-series forecasting, what is the primary concern when using future information in training?
Asked once · last asked Sep 3 · Concept · Mid-level
How do you choose a train, validation, and test split for a computer vision dataset?
Asked once · last asked Sep 29 · Scenario · Mid-level
How should you cross-validate a time-series model, and what are the leakage traps?
Asked once · last asked Oct 9 · Concept · Senior
How do you prevent lookahead and survivorship bias in rates backtests, especially in roll logic and historical instrument availability?
Asked once · last asked Oct 10 · Scenario · Senior
Suppose you have 50 candidate features and tune hyperparameters over 2020 to 2024. How do you structure the research process to minimize data snooping and quantify selection bias?
Asked once · last asked Oct 10 · Scenario · Senior
How do you detect and correct lookahead bias in a fundamental or macro signal when input series are revised, released with lags, and have point-in-time vintages as well as final values?
Asked once · last asked Oct 10 · Scenario · Senior
When building a multi-factor model for rates or FX signals, how do you test whether apparent alpha is actually unintended exposure to latent factors such as level, slope, curvature, or risk-on/risk-off, especially across regime shifts?
Asked once · last asked Oct 10 · Scenario · Senior
How do you tell whether an accuracy value comes from training data or test data?
Asked once · last asked Oct 12 · Concept · Junior
What are the most common backtest-to-production mismatches, and how would you test for them before launch?
Asked once · last asked Oct 15 · Scenario · Senior
Other ways it was asked
- “What are the most common backtest-to-production mismatches you've seen, and how do you test for them before launch?”
How do you validate that your curve construction (OIS discounting, swap par curve, interpolation and extrapolation) doesn't introduce spurious signal or P&L, especially when instruments drop in and out?
Asked once · last asked Oct 15 · Scenario · Senior
How would you statistically test whether convexity and basis adjustments are overfit to a particular market regime, including the design of out-of-sample validation?
Asked once · last asked Oct 15 · Concept · Senior
Other ways it was asked
- “How do you statistically test that your convexity/basis adjustments aren't overfit to a specific regime? And what out-of-sample design do you use?”
With 50 candidate signals and walk-forward validation, how would you select a final model and report results while controlling selection bias, considering methods such as White’s Reality Check, deflated Sharpe, or nested cross-validation?
Asked once · last asked Oct 15 · Scenario · Senior
Other ways it was asked
- “You have 50 candidate signals and run walk-forward validation, how do you choose the final model while controlling for selection bias? For example, White's Reality Check, deflated Sharpe, or nested CV. What do you report?”
In your backtests, how do you detect and prevent subtle lookahead from trading at the close using signals computed with same-day close data, especially across assets with different market hours?
Asked once · last asked Oct 15 · Scenario · Senior
How do you model transaction costs and market impact so they're not implicitly using future liquidity, for example volume/ADV, and how do you stress them to avoid optimistic backtests?
Asked once · last asked Oct 15 · Scenario · Senior
How do you quantify bias from stale or revised macroeconomic data, such as CPI or payrolls, when building predictors, including implementing real-time data vintages and release-time alignment?
Asked once · last asked Oct 15 · Scenario · Senior
Other ways it was asked
- “How do you quantify bias from stale or revised macro data like CPI or payrolls when building predictors? Specifically, how do you implement real-time vintages and release-time alignment?”
How would you structure a walk-forward in Python so feature engineering and standardization, for example z-score scaling, are fit only on the training window and then applied to the test window without leakage?
Asked once · last asked Oct 15 · Write code · Senior
Imagine you're reviewing a teammate's memo claiming strong walk-forward results. What specific assumptions, leakage checks, and limitations would you require them to document before you sign off?
Asked once · last asked Oct 15 · Scenario · Senior
You receive a strategy write-up with unclear claims. What outline would you require so the next revision is unambiguous and reproducible end to end?
Asked once · last asked Oct 15 · Scenario · Senior
How would you detect and mitigate label leakage in a supervised learning pipeline, especially when features are aggregated over time such as rolling windows?
Asked once · last asked Oct 23 · Scenario · Senior
How does cross-validation help estimate model performance, and why is it important?
Asked once · last asked Aug 21 · Concept · Junior
What is data leakage?
Asked once · last asked Aug 22 · Concept · Junior
How would you handle optimizing a model purely for historical accuracy?
Asked once · last asked Aug 24 · Scenario · Mid-level

How this list is made

Every question here was asked by a real interviewer in a real job interview that a candidate ran MeetAssist in. After the call, the interviewer's questions are pulled from the transcript. Practice sessions, tests, exams and anything that was not a job interview are dropped.

Questions are rewritten so they make sense on their own: names of people, employers and clients are removed, and speech-to-text mistakes are fixed. Small talk, logistics and questions about one candidate's own résumé are left out. Rephrasings of the same question are merged, and the count is the number of different interviews it came up in.

The interviews come from candidates who use MeetAssist, so they lean towards software, data and AI roles: the shares describe these interviews, not every Machine Learning interview. Answers to the most asked questions are written and checked by an editor; a "common mistake" is shown only when reviews of real answers in these interviews show it, with the count.

320 interviews, 221 candidates, Nov 2025 to Oct 2026. Updated every month with new interviews.

All 41 questions

How would you structure your validation splits to prevent data leakage or look ahead bias?

What are the differences between training, validation, and test data?

What validation techniques can you use to evaluate a model, and how does cross-validation work?

What is data leakage, and how would you identify and prevent it in a modeling workflow?

In scikit-learn, how do you correctly run cross-validation when preprocessing steps such as scaling and one-hot encoding are involved, to avoid data leakage?

How do you validate transaction cost and slippage assumptions in a rates backtest against realized fills?

What data would you use to validate a campus placement prediction model?

How do you design validation when there's a risk of leakage, such as with time-series data, multiple rows per user, or repeated interactions in a recommender system?

In scikit-learn, how do you prevent target leakage when doing time-series cross-validation with feature scaling and encoding, and what APIs do you use to enforce that?

How would you structure a scikit-learn pipeline that includes lag or rolling features from a timestamped series without leaking future information, given that a pipeline doesn't natively know about time order?

In a custom transformer, how would you handle the boundary between training and validation so the first validation rows can use the last k training observations for lag or rolling features without refitting on validation targets?

How do you prevent data leakage when doing target encoding for a high-cardinality categorical feature, especially under time-based cross-validation?

When splitting a dataset into groups, should the groups have similar data distributions or equal numbers of samples?

When you use GridSearchCV with a pipeline and a ColumnTransformer, where should fitting happen, and what data should never be used to learn preprocessing parameters?

How do you detect and prevent data leakage in a time-to-event prediction problem, especially with feature generation and train-validation splitting?

How would you implement a scikit-learn workflow, including input data, preprocessing steps, and validation of the results?

How would you prevent data leakage or frame dependence from making a confidence interval for a model's precision improvement too optimistic?

How would you prevent label leakage and spurious shortcuts, for example artifact cases always having a specific operator ID?

How do you detect and prevent label leakage when features are time-based and target labels are derived from future events?

In time-series forecasting, what is the primary concern when using future information in training?

How do you choose a train, validation, and test split for a computer vision dataset?

How should you cross-validate a time-series model, and what are the leakage traps?

How do you prevent lookahead and survivorship bias in rates backtests, especially in roll logic and historical instrument availability?

Suppose you have 50 candidate features and tune hyperparameters over 2020 to 2024. How do you structure the research process to minimize data snooping and quantify selection bias?

How do you detect and correct lookahead bias in a fundamental or macro signal when input series are revised, released with lags, and have point-in-time vintages as well as final values?

When building a multi-factor model for rates or FX signals, how do you test whether apparent alpha is actually unintended exposure to latent factors such as level, slope, curvature, or risk-on/risk-off, especially across regime shifts?

How do you tell whether an accuracy value comes from training data or test data?

What are the most common backtest-to-production mismatches, and how would you test for them before launch?

How do you validate that your curve construction (OIS discounting, swap par curve, interpolation and extrapolation) doesn't introduce spurious signal or P&L, especially when instruments drop in and out?

How would you statistically test whether convexity and basis adjustments are overfit to a particular market regime, including the design of out-of-sample validation?

With 50 candidate signals and walk-forward validation, how would you select a final model and report results while controlling selection bias, considering methods such as White’s Reality Check, deflated Sharpe, or nested cross-validation?

In your backtests, how do you detect and prevent subtle lookahead from trading at the close using signals computed with same-day close data, especially across assets with different market hours?

How do you model transaction costs and market impact so they're not implicitly using future liquidity, for example volume/ADV, and how do you stress them to avoid optimistic backtests?

How do you quantify bias from stale or revised macroeconomic data, such as CPI or payrolls, when building predictors, including implementing real-time data vintages and release-time alignment?

How would you structure a walk-forward in Python so feature engineering and standardization, for example z-score scaling, are fit only on the training window and then applied to the test window without leakage?

Imagine you're reviewing a teammate's memo claiming strong walk-forward results. What specific assumptions, leakage checks, and limitations would you require them to document before you sign off?

You receive a strategy write-up with unclear claims. What outline would you require so the next revision is unambiguous and reproducible end to end?

How would you detect and mitigate label leakage in a supervised learning pipeline, especially when features are aggregated over time such as rolling windows?

How does cross-validation help estimate model performance, and why is it important?

What is data leakage?

How would you handle optimizing a model purely for historical accuracy?

More Machine Learning interview questions

How this list is made

Walk into your next interview ready.