79 MLOps and Model Deployment Interview Questions

All 79 questions

How should a deployed model adapt or gracefully degrade when its input data or user behavior changes?
Asked in 5 of 320 interviews · last asked Sep 14 · Scenario · Senior
Show answer
Monitor input data, user behavior, and model performance after deployment. If changes hurt quality, retrain on recent, representative data and test before a gradual rollout. While quality is uncertain, use safe rules, ask for human review, or stop making risky predictions. Keep rollback available if a new model performs worse.
- A change in input data does not always mean prediction quality has dropped.
- Track real outcomes when available; model confidence alone may miss failures.
- Validate updates before release, then monitor a small rollout before expanding.
- Define fallback triggers and safe behavior before problems happen.
What the interviewer checks: The interviewer is checking whether you can detect change, update safely, and protect users when predictions become unreliable.
Other ways it was asked
- “If data patterns or human behavior shift after deployment, how would you adapt the system or make it degrade gracefully?”
- “If underlying data patterns or human behaviors shift post deployment, how will your system adapt or gracefully degrade?”
- “How should a deployed predictive system adapt or gracefully degrade when data patterns or human behaviors shift?”
How do you mathematically handle live data that's highly sparse, imbalanced, or unrepresentative of the environment your model encounters post launch?
Asked in 3 of 320 interviews · last asked Sep 11 · Scenario · Senior
Other ways it was asked
- “How would you mathematically handle live data that is sparse, balanced, or unrepresentative of the environment a model encounters after launch?”
- “What methods would you use to handle live data that is sparse, imbalanced, or unrepresentative of a model's post-launch environment?”
How would you distinguish data drift, concept drift, and temporary data issues when production performance drops?
Asked in 3 of 320 interviews · last asked Aug 19 · Scenario · Senior
Other ways it was asked
- “In a production ML system, how would you detect and handle data drift versus concept drift, and what metrics or signals would you monitor for each?”
- “Suppose your ML model's accuracy drops from 91% to 78% in production. How would you determine whether this is data drift, concept drift, or a temporary data issue?”
- “In production, how would you detect and respond to data drift versus concept drift for a deployed classifier?”
How would you deploy a machine learning model for business intelligence or analytics engineering, and how would it be used?
Asked in 2 of 320 interviews · last asked Aug 31 · Behavioral · Senior
Other ways it was asked
- “Describe a machine learning model you personally deployed to production within your area of specialization.”
- “Tell me about a machine learning model you deployed in your area of specialization and how it was used.”
How would you investigate a model that performs well during development but poorly in production?
Asked in 2 of 320 interviews · last asked Sep 23 · Scenario · Mid-level
Other ways it was asked
- “How would you debug a model that performs well in training but poorly in production?”
- “How do you handle a model that performs well during development but poorly during production?”
- “Your ML model has 95% accuracy in testing but only 75% in production. What would you do?”
What causes data drift?
Asked in 2 of 320 interviews · last asked Sep 7 · Concept · Mid-level
Other ways it was asked
- “What is data drift?”
How would you train a machine learning model using Xcode or Python scripts, and integrate it with Core ML?
Asked once · last asked Feb 27 · Tooling · Mid-level
How would you communicate with a business unit to gather the information needed and assess whether a model is working effectively?
Asked once · last asked Mar 12 · Scenario · Mid-level
How would you identify and address failure modes in a machine-learning project?
Asked once · last asked Apr 15 · Scenario · Senior
Besides accuracy, what signs could indicate that a machine-learning model is underperforming or something is going wrong?
Asked once · last asked Apr 15 · Scenario · Mid-level
What's your typical approach to packaging and serving a model in batch versus real time, and how do you ensure training-serving feature parity?
Asked once · last asked May 3 · Design · Senior
Once a model is in production, what monitoring do you set up for data drift, performance, calibration, and latency, and what thresholds or triggers cause you to retrain or roll back?
Asked once · last asked May 3 · Scenario · Senior
How would you monitor a model when ground-truth labels arrive with a long delay or are partially missing, including your choice of proxies and evaluation setup?
Asked once · last asked May 3 · Scenario · Senior
Other ways it was asked
- “How do you handle monitoring when ground-truth labels arrive with a long delay or are partially missing? What proxies or evaluation setup do you use?”
What are the stages of an end-to-end machine learning model lifecycle, from development through implementation?
Asked once · last asked May 13 · Concept · Mid-level
In an end-to-end pipeline, how would you design training and serving feature engineering so that exactly the same transformations run in production and prevent train-serving skew?
Asked once · last asked May 24 · Design · Senior
How would you implement model monitoring to detect data drift and performance degradation in production, and what metrics would you alert on?
Asked once · last asked May 24 · Scenario · Senior
How would you roll out a new model using an A/B test, and what offline and online criteria would you require before fully promoting it?
Asked once · last asked May 24 · Scenario · Senior
Suppose you're building a simple cats-and-dogs classification problem and achieve 90% accuracy on your local pipeline, but accuracy drops to 65% when the model sees real production data. What might be causing this, and how would you address it?
Asked once · last asked May 30 · Scenario · Mid-level
In an end-to-end pipeline, how do you structure data preprocessing and feature generation so training and inference use identical transformations, and how do you version them?
Asked once · last asked Jun 3 · Design · Senior
How do you handle schema drift and missing or novel categories in production while keeping your feature transformations backward-compatible with past model versions?
Asked once · last asked Jun 3 · Scenario · Senior
When your recommendation model is updated, how do you detect and handle distribution shift or metric drift in production, and what triggers a rollback versus continued monitoring?
Asked once · last asked Jun 15 · Scenario · Senior
How do you ensure that your models remain robust over time in the presence of concept drift or changing data distributions?
Asked once · last asked Jun 26 · Scenario · Senior
Suppose your production model's AUC stays stable, but calibration degrades over a month. What are the most likely causes, and what remediation would you implement first?
Asked once · last asked Jun 30 · Scenario · Senior
If your model started performing poorly after six months, how would you investigate that scenario?
Asked once · last asked Jul 16 · Scenario · Senior
How would you select, train, and deploy a semi-supervised learning model for customer churn prediction, and how would you ensure reliability and monitor performance after deployment?
Asked once · last asked Jul 16 · Scenario · Senior
How would you handle computational resource limitations when carrying out a data science project?
Asked once · last asked Jul 16 · Scenario · Junior
How would you build a QA dashboard for tracking AI product quality over time?
Asked once · last asked Jul 16 · Design · Senior
How would you monitor and evaluate a machine learning model deployed on Google Cloud or in a production environment?
Asked once · last asked Jul 20 · Scenario · Senior
Can you talk about how you organize an exploratory research project in Weights & Biases so you can understand models, prompts, and results?
Asked once · last asked Jul 30 · Tooling · Mid-level
What are the main security risks in machine learning and deep learning?
Asked once · last asked Aug 11 · Concept · Mid-level
Walk me through a concrete case where model metrics improved, but you still rejected the change. What was the hidden failure mode, and how did you detect it?
Asked once · last asked Aug 11 · Behavioral · Senior
For a real-time video detector, how would you balance accuracy against latency, and what would you profile or test to catch data-integrity problems and accuracy regressions after an optimization?
Asked once · last asked Aug 12 · Scenario · Senior
How would you convert an ONNX model into a TensorRT engine, and what would you check if accuracy dropped after FP16 or INT8 optimization?
Asked once · last asked Aug 12 · Scenario · Senior
If offline metrics look incredible, but live business KPIs stall after launch, how would you diagnose the disconnect?
Asked once · last asked Aug 13 · Scenario · Senior
When leading model retraining in production, how do you choose between scheduled and drift-triggered retraining, and which signals or thresholds do you trust most?
Asked once · last asked Aug 13 · Scenario · Senior
How do you handle label delay and noisy labels when setting up drift monitoring and deciding whether to retrain a model?
Asked once · last asked Aug 13 · Scenario · Senior
Suppose offline metrics improve after retraining, but online business KPIs drop. How would you investigate whether the cause is data shift, thresholding, feedback loops, or instrumentation?
Asked once · last asked Aug 13 · Scenario · Senior
How would you set up an online drift detector for a deep model that triggers retraining while minimizing false alarms?
Asked once · last asked Aug 19 · Design · Senior
How would you choose a model to run on bare-metal GPU servers for a telecom use case?
Asked once · last asked Aug 20 · Scenario · Senior
How do you determine whether a model's predictions are practically useful for business decisions, beyond traditional evaluation metrics?
Asked once · last asked Aug 22 · Scenario · Mid-level
When offline metrics improve but user satisfaction drops after launch, how do you determine whether the cause is data drift, a rubric mismatch, or a product or UI issue, and what experiment would you run first?
Asked once · last asked Aug 24 · Scenario · Senior
Tell me about a time you deployed a model to production and communicated its performance and limitations to nontechnical stakeholders.
Asked once · last asked Aug 24 · Behavioral · Mid-level
How would you structure a pipeline for model evaluation and deployment?
Asked once · last asked Aug 26 · Design · Mid-level
What could be three possible causes if a model has high offline accuracy but fails in production?
Asked once · last asked Aug 27 · Scenario · Senior
How would you optimize a TensorFlow or PyTorch model for real-time inference using techniques such as quantization, pruning, or knowledge distillation, and measure the resulting performance improvements?
Asked once · last asked Aug 27 · Tooling · Senior
Other ways it was asked
- “Can you explain the process of optimizing a TensorFlow or PyTorch model for real-time inference? What specific techniques, like quantization, pruning, or knowledge distillation, have you used, and what performance improvements did you achieve?”
How would you choose models for each stage of a pipeline while managing cost, latency, and output reliability?
Asked once · last asked Aug 27 · Design · Senior
If detection runs in the cloud rather than on the edge device, how can that help filter false positives?
Asked once · last asked Aug 28 · Scenario · Mid-level
How do you ensure that feature importance insights align with what the business team expects or can act upon during feedback discussions?
Asked once · last asked Aug 29 · Scenario · Mid-level
How do you ensure that stakeholders understand and trust the model outputs, regardless of which model is chosen?
Asked once · last asked Aug 31 · Scenario · Senior
How would you monitor data drift and decide when to retrain a model?
Asked once · last asked Sep 10 · Scenario · Senior
How would you explain to a nontechnical executive why a dashboard displays particular metrics or why an AI system produced a specific result?
Asked once · last asked Sep 13 · Scenario · Mid-level
What does the software development life cycle look like for an AI project?
Asked once · last asked Sep 14 · Concept · Mid-level
How would you identify and evaluate opportunities to integrate AI into a product?
Asked once · last asked Sep 14 · Scenario · Senior
How do you take a new model from training to production, including quality gates, drift detection, and detection of silent degradation?
Asked once · last asked Sep 16 · Design · Senior
How do you connect a predictive model's success criteria to business outcomes such as revenue or cost savings?
Asked once · last asked Sep 22 · Case · Senior
How would you determine whether an organization is ready to use AI-based analytics?
Asked once · last asked Sep 23 · Scenario · Senior
How should teams divide responsibilities for model selection and output evaluation in an AI solution?
Asked once · last asked Sep 24 · Scenario · Senior
How would you evaluate whether an AI pipeline is improving outcomes while balancing its costs?
Asked once · last asked Sep 25 · Design · Senior
Imagine you're reviewing a technical document describing a new machine learning model. What specific elements would you scrutinize to ensure it's clear, accurate, and useful for both technical and nontechnical audiences?
Asked once · last asked Sep 26 · Scenario · Senior
What are model extraction and training-data extraction attacks?
Asked once · last asked Sep 30 · Concept · Senior
How would you debug and improve low precision and recall in a production model after training?
Asked once · last asked Oct 1 · Scenario · Senior
What key metrics or indicators would you use to determine whether an AI-driven security solution is ready for full production integration?
Asked once · last asked Oct 2 · Scenario · Senior
Other ways it was asked
- “What key metrics or indicators would you focus on to determine if the AI-driven solution is ready to be fully integrated into production?”
How do you determine whether an AI system is ready to be deployed to real users?
Asked once · last asked Oct 5 · Scenario · Senior
How would you implement model routing, and how should a classifier decide which model to use?
Asked once · last asked Oct 6 · Design · Senior
A feature is still a float, has no null values, and passes all checks, but model performance has declined. What could be causing this?
Asked once · last asked Oct 18 · Scenario · Senior
How would you turn a data insight into a recommendation for changing an app or its features and validate the recommendation's effectiveness after implementation?
Asked once · last asked Oct 19 · Behavioral · Mid-level
Other ways it was asked
- “Can you describe a specific example of an insight you discovered from your data that led to a concrete change in the app or its features? How did you validate that your recommendation was effective after implementation?”
How would you implement a machine learning model end to end, from defining the problem to selecting an appropriate model and deploying it?
Asked once · last asked Oct 19 · Behavioral · Mid-level
Other ways it was asked
- “Can you describe a project where you implemented a machine learning model end to end? What was the problem you were solving? And how did you select the appropriate model for your use case?”
What specific metrics or tools do you use to detect hallucinations and model drift?
Asked once · last asked Oct 20 · Tooling · Senior
How would you investigate a machine-learning model that predicts low pollution despite readings indicating severe pollution?
Asked once · last asked Oct 21 · Scenario · Mid-level
How would you evaluate the effectiveness of changes to an AI model before full deployment?
Asked once · last asked Oct 23 · Scenario · Senior
How would you measure the success of a pilot integrating an AI tool into a bank?
Asked once · last asked Oct 23 · Scenario · Senior
How would you assure a bank's board and operational teams of an AI tool's reliability before full-scale deployment?
Asked once · last asked Oct 23 · Scenario · Senior
After deploying an AI tool, what key performance indicators would you monitor to evaluate its long-term success?
Asked once · last asked Oct 23 · Scenario · Senior
If an AI tool has performance gaps, what continuous improvement strategies would you consider?
Asked once · last asked Oct 23 · Scenario · Senior
Why can model accuracy drop after deployment?
Asked once · last asked Sep 14 · Concept · Junior
Other ways it was asked
- “Why will the model’s accuracy drop after deployment?”
- “Would you deploy a model whose test accuracy is 98% but drops to 70% after deployment?”
- “What could cause the model accuracy to drop from 98% on the test set to 70% after deployment, and how would you improve it?”
Is it really important to include AI and ML if we’re not planning to scale much right now?
Asked once · last asked Apr 11 · Scenario · Mid-level
What is a minimum viable model in machine learning?
Asked once · last asked Aug 5 · Concept · Junior
In what ways can a cheaper AI model be better than a more expensive one?
Asked once · last asked Aug 14 · Concept · Mid-level
In machine learning deployment, what environmental differences can cause model accuracy to drop?
Asked once · last asked Sep 14 · Concept · Mid-level

How this list is made

Every question here was asked by a real interviewer in a real job interview that a candidate ran MeetAssist in. After the call, the interviewer's questions are pulled from the transcript. Practice sessions, tests, exams and anything that was not a job interview are dropped.

Questions are rewritten so they make sense on their own: names of people, employers and clients are removed, and speech-to-text mistakes are fixed. Small talk, logistics and questions about one candidate's own résumé are left out. Rephrasings of the same question are merged, and the count is the number of different interviews it came up in.

The interviews come from candidates who use MeetAssist, so they lean towards software, data and AI roles: the shares describe these interviews, not every Machine Learning interview. Answers to the most asked questions are written and checked by an editor; a "common mistake" is shown only when reviews of real answers in these interviews show it, with the count.

320 interviews, 221 candidates, Nov 2025 to Oct 2026. Updated every month with new interviews.

All 79 questions

How should a deployed model adapt or gracefully degrade when its input data or user behavior changes?

How do you mathematically handle live data that's highly sparse, imbalanced, or unrepresentative of the environment your model encounters post launch?

How would you distinguish data drift, concept drift, and temporary data issues when production performance drops?

How would you deploy a machine learning model for business intelligence or analytics engineering, and how would it be used?

How would you investigate a model that performs well during development but poorly in production?

What causes data drift?

How would you train a machine learning model using Xcode or Python scripts, and integrate it with Core ML?

How would you communicate with a business unit to gather the information needed and assess whether a model is working effectively?

How would you identify and address failure modes in a machine-learning project?

Besides accuracy, what signs could indicate that a machine-learning model is underperforming or something is going wrong?

What's your typical approach to packaging and serving a model in batch versus real time, and how do you ensure training-serving feature parity?

Once a model is in production, what monitoring do you set up for data drift, performance, calibration, and latency, and what thresholds or triggers cause you to retrain or roll back?

How would you monitor a model when ground-truth labels arrive with a long delay or are partially missing, including your choice of proxies and evaluation setup?

What are the stages of an end-to-end machine learning model lifecycle, from development through implementation?

In an end-to-end pipeline, how would you design training and serving feature engineering so that exactly the same transformations run in production and prevent train-serving skew?

How would you implement model monitoring to detect data drift and performance degradation in production, and what metrics would you alert on?

How would you roll out a new model using an A/B test, and what offline and online criteria would you require before fully promoting it?

Suppose you're building a simple cats-and-dogs classification problem and achieve 90% accuracy on your local pipeline, but accuracy drops to 65% when the model sees real production data. What might be causing this, and how would you address it?

In an end-to-end pipeline, how do you structure data preprocessing and feature generation so training and inference use identical transformations, and how do you version them?

How do you handle schema drift and missing or novel categories in production while keeping your feature transformations backward-compatible with past model versions?

When your recommendation model is updated, how do you detect and handle distribution shift or metric drift in production, and what triggers a rollback versus continued monitoring?

How do you ensure that your models remain robust over time in the presence of concept drift or changing data distributions?

Suppose your production model's AUC stays stable, but calibration degrades over a month. What are the most likely causes, and what remediation would you implement first?

If your model started performing poorly after six months, how would you investigate that scenario?

How would you select, train, and deploy a semi-supervised learning model for customer churn prediction, and how would you ensure reliability and monitor performance after deployment?

How would you handle computational resource limitations when carrying out a data science project?

How would you build a QA dashboard for tracking AI product quality over time?

How would you monitor and evaluate a machine learning model deployed on Google Cloud or in a production environment?

Can you talk about how you organize an exploratory research project in Weights & Biases so you can understand models, prompts, and results?

What are the main security risks in machine learning and deep learning?

Walk me through a concrete case where model metrics improved, but you still rejected the change. What was the hidden failure mode, and how did you detect it?

For a real-time video detector, how would you balance accuracy against latency, and what would you profile or test to catch data-integrity problems and accuracy regressions after an optimization?

How would you convert an ONNX model into a TensorRT engine, and what would you check if accuracy dropped after FP16 or INT8 optimization?

If offline metrics look incredible, but live business KPIs stall after launch, how would you diagnose the disconnect?

When leading model retraining in production, how do you choose between scheduled and drift-triggered retraining, and which signals or thresholds do you trust most?

How do you handle label delay and noisy labels when setting up drift monitoring and deciding whether to retrain a model?

Suppose offline metrics improve after retraining, but online business KPIs drop. How would you investigate whether the cause is data shift, thresholding, feedback loops, or instrumentation?

How would you set up an online drift detector for a deep model that triggers retraining while minimizing false alarms?

How would you choose a model to run on bare-metal GPU servers for a telecom use case?

How do you determine whether a model's predictions are practically useful for business decisions, beyond traditional evaluation metrics?

When offline metrics improve but user satisfaction drops after launch, how do you determine whether the cause is data drift, a rubric mismatch, or a product or UI issue, and what experiment would you run first?

Tell me about a time you deployed a model to production and communicated its performance and limitations to nontechnical stakeholders.

How would you structure a pipeline for model evaluation and deployment?

What could be three possible causes if a model has high offline accuracy but fails in production?

How would you optimize a TensorFlow or PyTorch model for real-time inference using techniques such as quantization, pruning, or knowledge distillation, and measure the resulting performance improvements?

How would you choose models for each stage of a pipeline while managing cost, latency, and output reliability?

If detection runs in the cloud rather than on the edge device, how can that help filter false positives?

How do you ensure that feature importance insights align with what the business team expects or can act upon during feedback discussions?

How do you ensure that stakeholders understand and trust the model outputs, regardless of which model is chosen?

How would you monitor data drift and decide when to retrain a model?

How would you explain to a nontechnical executive why a dashboard displays particular metrics or why an AI system produced a specific result?

What does the software development life cycle look like for an AI project?

How would you identify and evaluate opportunities to integrate AI into a product?

How do you take a new model from training to production, including quality gates, drift detection, and detection of silent degradation?

How do you connect a predictive model's success criteria to business outcomes such as revenue or cost savings?

How would you determine whether an organization is ready to use AI-based analytics?

How should teams divide responsibilities for model selection and output evaluation in an AI solution?

How would you evaluate whether an AI pipeline is improving outcomes while balancing its costs?

Imagine you're reviewing a technical document describing a new machine learning model. What specific elements would you scrutinize to ensure it's clear, accurate, and useful for both technical and nontechnical audiences?

What are model extraction and training-data extraction attacks?

How would you debug and improve low precision and recall in a production model after training?

What key metrics or indicators would you use to determine whether an AI-driven security solution is ready for full production integration?

How do you determine whether an AI system is ready to be deployed to real users?

How would you implement model routing, and how should a classifier decide which model to use?

A feature is still a float, has no null values, and passes all checks, but model performance has declined. What could be causing this?

How would you turn a data insight into a recommendation for changing an app or its features and validate the recommendation's effectiveness after implementation?

How would you implement a machine learning model end to end, from defining the problem to selecting an appropriate model and deploying it?

What specific metrics or tools do you use to detect hallucinations and model drift?

How would you investigate a machine-learning model that predicts low pollution despite readings indicating severe pollution?

How would you evaluate the effectiveness of changes to an AI model before full deployment?

How would you measure the success of a pilot integrating an AI tool into a bank?

How would you assure a bank's board and operational teams of an AI tool's reliability before full-scale deployment?

After deploying an AI tool, what key performance indicators would you monitor to evaluate its long-term success?

If an AI tool has performance gaps, what continuous improvement strategies would you consider?

Why can model accuracy drop after deployment?

Is it really important to include AI and ML if we’re not planning to scale much right now?

What is a minimum viable model in machine learning?

In what ways can a cheaper AI model be better than a more expensive one?

In machine learning deployment, what environmental differences can cause model accuracy to drop?