76 Data Pipeline Monitoring and Troubleshooting Interview Questions

All 76 questions

How would you investigate and recover missing report records after a successful pipeline run, and prevent recurrence?
Asked in 4 of 307 interviews · last asked Sep 28 · Behavioral · Senior
Show answer
I would trace missing record IDs from the source through each stage to the report. I would check filters, joins, rejected rows, and load windows. After fixing the cause, I would replay affected data without creating duplicates and reconcile report results before release. I would add completeness checks, alerts, and tests for the failure.
- Compare record IDs, counts, and totals for the same time window; successful execution does not prove completeness.
- Check incremental load markers, late arrivals, and report refreshes, not just pipeline logs.
- Backfill the affected range safely, then confirm missing records are restored and business totals match.
- Block report publication when completeness checks fail, and alert the team.
What the interviewer checks: The interviewer is checking whether you can trace data loss, recover safely, verify report accuracy, and prevent repeat failures.
Other ways it was asked
- “How would you find the root cause of a data pipeline that completed successfully but produced incomplete results, and what would you change?”
- “How would you investigate a successfully completed data pipeline that produced incomplete results, and what would you change to prevent a recurrence?”
- “What changes would you make to prevent a data pipeline from producing incomplete results again?”
How would you design pipeline monitoring and alerting, assess existing coverage, and identify gaps?
Asked in 3 of 307 interviews · last asked Sep 22 · Scenario · Senior
Show answer
I would build monitoring into each pipeline from the start, covering failures, delays, data quality, and resource use. I would set freshness and quality targets, then send actionable alerts to an owner with recovery steps. To assess coverage, I would map existing checks to pipeline stages and targets, review past incidents, and test failure cases to find gaps.
- Track job status, retries, run time, freshness, row counts, and data quality.
- Set alert thresholds from business needs; page only for urgent issues and group repeated alerts.
- Check every critical source, processing step, and output for missing monitoring or unclear ownership.
- Test alerts with safe failure simulations, including cases where jobs succeed but data is late or wrong.
What the interviewer checks: The interviewer is checking whether you can detect real data problems, limit alert noise, and systematically find monitoring gaps.
Other ways it was asked
- “How would you design observability and monitoring for a data platform, including when to implement it, how to assess current coverage, and how to identify gaps?”
- “How do you monitor data pipelines and configure alerting?”
- “How would you implement monitoring and alerting for a data engineering environment?”
If the Bronze layer completed successfully but the Gold layer has 40% fewer records than usual, how would you investigate where the problem occurred?
Asked once · last asked Oct 18 · Scenario · Senior
Other ways it was asked
- “If the Bronze layer has the expected record count but the Silver layer does not, what would you investigate?”
How would you troubleshoot and handle an error in a production Apache NiFi flow?
Asked once · last asked Feb 6 · Scenario · Mid-level
Other ways it was asked
- “How do you implement logging and error handling in Apache NiFi flows?”
How can you configure Apache NiFi to notify you when a critical processor fails?
Asked once · last asked Feb 6 · Tooling · Mid-level
How do you identify data pipeline failure modes and choose appropriate monitoring to detect them?
Asked once · last asked Mar 3 · Scenario · Senior
How do you identify the root cause of a data pipeline failure and implement a fix?
Asked once · last asked Mar 3 · Scenario · Senior
How do you make responses to recurring data pipeline failures more efficient, and when do you document runbooks or standard operating procedures?
Asked once · last asked Mar 3 · Scenario · Senior
Can you tell me about a time you had to troubleshoot a data pipeline or refresh problem in the Azure environment, and what you did?
Asked once · last asked Apr 24 · Behavioral · Mid-level
What specific observability signals would you instrument across the pipeline, such as freshness, volume, schema drift, and label distribution, and what thresholds or alerting logic would you use for each?
Asked once · last asked Apr 25 · Scenario · Senior
When defining SLAs or SLOs with stakeholders for an ETL pipeline, which metrics do you commit to—latency, freshness, completeness, and accuracy—and how do you monitor and alert on them?
Asked once · last asked May 3 · Scenario · Senior
How are you notified when a job fails or something goes wrong in the application?
Asked once · last asked Nov 10 · Tooling · Mid-level
Where do you configure email alerts in IICS?
Asked once · last asked Nov 10 · Tooling · Junior
If a workflow fails in production but the same workflow succeeds in development, how would you investigate the issue in PowerCenter?
Asked once · last asked Nov 10 · Scenario · Senior
If the repository service fails to start after Informatica PowerCenter is installed on a server, what steps would you take?
Asked once · last asked Nov 10 · Scenario · Senior
If the repository service is running but multiple users cannot connect, what might be the cause, and how would you troubleshoot it?
Asked once · last asked Nov 10 · Scenario · Senior
If an Airflow task is not triggering, how can you find the issue?
Asked once · last asked May 27 · Scenario · Mid-level
If a task fails in Airflow, how can you rerun it?
Asked once · last asked May 27 · Tooling · Mid-level
If a pipeline fails at 70% completion, how can you resume it from where it failed?
Asked once · last asked May 27 · Scenario · Mid-level
A user-activity pipeline tracks mobile app and Windows login events with attributes such as location and OS version. What could cause its reported user count to exceed counts from other tracking sources?
Asked once · last asked Jun 9 · Scenario · Senior
If an Apache Iceberg manifest file were deleted, what would happen to the data, and would you still be able to access it?
Asked once · last asked Jun 9 · Scenario · Senior
What issues in the data pipeline or model design could cause regional content to be available but not reflected in recommendations in new markets?
Asked once · last asked Jun 17 · Scenario · Mid-level
How would you diagnose and resolve a technical challenge encountered while building or maintaining a data pipeline?
Asked once · last asked Jun 24 · Scenario · Mid-level
Other ways it was asked
- “What specific technical challenge did you face while building or maintaining the pipeline, how did you diagnose it, and what did you do to solve it?”
Can you describe a concrete production issue involving data latency, a schema mismatch, or a pipeline failure?
Asked once · last asked Jun 24 · Scenario · Mid-level
Other ways it was asked
- “Can you give a concrete example of a production issue you encountered, such as data latency, a schema mismatch, or a pipeline failure?”
- “How would you detect and resolve a production data pipeline issue, such as data latency, a schema mismatch, or a pipeline failure?”
How do you create observability for data loads and pipelines in Snowflake?
Asked once · last asked Jul 16 · Tooling · Senior
Spark writes succeed, but an Iceberg metadata commit fails. What object storage issue could be causing the failure?
Asked once · last asked Jul 16 · Scenario · Senior
If a third-party API is unavailable for six hours, how would you backfill the missing data?
Asked once · last asked Jul 20 · Scenario · Senior
Other ways it was asked
- “If the Facebook API is unavailable for six hours, how would you backfill the missing data?”
How would you detect and handle errors in an affiliate commission pipeline?
Asked once · last asked Jul 27 · Scenario · Senior
When a website's page structure changes and your scraping selectors no longer match the content, what is your go-to approach to fix the issue quickly?
Asked once · last asked Jul 28 · Scenario · Mid-level
Other ways it was asked
- “When you encounter an unexpected page structure, meaning the selectors you set up no longer match the content, what's your go to approach to fix that quickly?”
How would you handle failure scenarios such as network interruptions or data corruption during ingestion and transformation, and ensure the pipeline remains resilient with data integrity maintained?
Asked once · last asked Aug 11 · Scenario · Senior
If a third-party PII-redaction dependency fails and normalized events can no longer be generated, how should the system respond and recover?
Asked once · last asked Aug 14 · Scenario · Senior
Tell me about a difficult bug or edge case in ETL refresh logic and how you isolated and fixed it.
Asked once · last asked Aug 29 · Behavioral · Mid-level
What signals or metrics would you track to confirm that an ETL refresh process remains stable over time?
Asked once · last asked Aug 29 · Scenario · Mid-level
What changes would you make after detecting drift in an ETL refresh process?
Asked once · last asked Aug 29 · Behavioral · Mid-level
Other ways it was asked
- “What changes did you make after detecting drift in an ETL refresh process?”
When a regression appears in a data pipeline, how do you investigate it?
Asked once · last asked Aug 29 · Scenario · Mid-level
Imagine you must make a decision in 30 minutes: keep the pipeline running or halt it? What minimal evidence and thresholds would you use?
Asked once · last asked Aug 31 · Scenario · Senior
When you decide to halt the pipeline, how would you communicate the impact, evidence, and next steps to engineering versus nontechnical stakeholders such as operations or leadership?
Asked once · last asked Aug 31 · Scenario · Mid-level
How do we store the error data in the database so we can track and display the import results to the user?
Asked once · last asked Sep 2 · Design · Mid-level
How would you update a CSV import's progress feedback to show users what is happening during a long-running process?
Asked once · last asked Sep 2 · Design · Mid-level
How would you investigate and recover 20,000 missing updates in a Snowflake CDC pipeline?
Asked once · last asked Sep 9 · Scenario · Senior
How would you systematically find where 1 million missing records disappeared between source and Snowflake?
Asked once · last asked Sep 9 · Scenario · Senior
How would you debug a data pipeline that completed successfully but did not update the downstream dashboard?
Asked once · last asked Sep 10 · Scenario · Senior
How do you distinguish a data error from a timing or latency issue, and what service-level or cutoff rules would you use to address it?
Asked once · last asked Sep 14 · Scenario · Senior
What common issues do you encounter in Microsoft Fabric data pipelines, and how do you address them?
Asked once · last asked Sep 16 · Scenario · Mid-level
How would you use SQL and pipeline metadata to investigate a data reconciliation difference?
Asked once · last asked Sep 20 · Scenario · Senior
Tell me about a time a critical error or security issue occurred in a data flow you managed. How did you handle it from start to finish?
Asked once · last asked Sep 20 · Behavioral · Senior
What is your action plan if a critical Informatica job fails repeatedly overnight and impacts operations?
Asked once · last asked Sep 21 · Scenario · Senior
How do you handle exceptions in an Oracle GL journal integration?
Asked once · last asked Sep 21 · Scenario · Mid-level
How would you handle an EDM-to-Fusion integration when the source system is down during the scheduled job window?
Asked once · last asked Sep 21 · Scenario · Senior
How would you recover missing records after discovering incomplete raw landing data, without creating duplicates or disrupting reports?
Asked once · last asked Sep 21 · Scenario · Senior
How would you approach supporting a complex Oracle PL/SQL or SQL batch process that you did not originally build?
Asked once · last asked Sep 22 · Scenario · Senior
When an alert fires for a data pipeline, how do you investigate the issue, which logs do you check first, and how do you fix it without reintroducing duplicates?
Asked once · last asked Sep 22 · Scenario · Senior
If a data pipeline fails during a reporting window, how would you communicate the issue and who would you involve?
Asked once · last asked Sep 22 · Scenario · Mid-level
How would you decide whether it is safe to resume updates to tables exposed to downstream consumers?
Asked once · last asked Sep 22 · Scenario · Senior
If you could improve one thing about a data pipeline to make it more robust or easier to operate, what would you change?
Asked once · last asked Sep 22 · Design · Senior
Suppose your hourly Airflow pipeline normally completes in forty minutes, but today it has been running for two hours without any code deployment. How would you troubleshoot it?
Asked once · last asked Sep 23 · Scenario · Senior
The source dataset contains today’s new records, but the Power BI report still shows yesterday’s figures. How would you investigate?
Asked once · last asked Sep 23 · Scenario · Mid-level
A data pipeline fails overnight and the support task is escalated to you in the morning. What would you do in the first hour?
Asked once · last asked Sep 23 · Scenario · Mid-level
Can you tell us about a data pipeline that went badly wrong? What went wrong, how did you detect it, and how would you monitor pipeline health?
Asked once · last asked Sep 23 · Behavioral · Senior
At which stages could errors occur as data moves from its source to a Power BI report?
Asked once · last asked Sep 28 · Concept · Mid-level
If the source owner’s extraction count is higher than the shipment count in your raw landing table, what evidence would you bring to the platform engineer and source owner, and how would you divide the investigation?
Asked once · last asked Sep 29 · Scenario · Senior
If an ADF activity succeeds and raw data is present in Snowflake but a curated table has not changed for two days, how would you distinguish a legitimate no-change period from a failed transformation or stuck incremental checkpoint?
Asked once · last asked Sep 29 · Scenario · Senior
If Power BI and the reporting table show 9,000 shipments but the raw landing data and operational manifest show 10,000 for the same cutoff, where would you investigate first, and how would you account for the missing 1,000 before release?
Asked once · last asked Sep 29 · Scenario · Senior
If a downstream team says expected data has not arrived in NetReveal but you see no obvious failure, how would you approach the situation?
Asked once · last asked Sep 29 · Scenario · Senior
What actions do you take when a replication flow fails?
Asked once · last asked Sep 30 · Scenario · Senior
If a Power BI dashboard refreshes successfully but the last three days of data are missing while other regions appear unaffected, how would you investigate and what would you communicate during the investigation?
Asked once · last asked Oct 1 · Scenario · Mid-level
If records are present in the raw layer but a code change caused an inner join to the region dimension to drop them from the curated dataset, how would you restore the missing data and prevent this from happening again?
Asked once · last asked Oct 1 · Scenario · Senior
A pipeline normally processes 1,000,000 interactions daily, but today it reports success while the Gold layer contains only 600,000 records. How would you investigate the incident and identify the root cause?
Asked once · last asked Oct 18 · Scenario · Senior
Other ways it was asked
- “A pipeline processes 1,000,000 interactions daily and reports success, but the Gold layer contains only 600,000 records. How would you investigate the incident and identify the root cause?”
What unexpected challenges might arise in a rocket landing prediction pipeline, and how would you address them?
Asked once · last asked Oct 20 · Behavioral · Mid-level
Other ways it was asked
- “What was the most surprising challenge you faced in a rocket landing prediction pipeline, and how did you address it?”
How would you isolate and fix a difficult bug in a real-time measurement pipeline?
Asked once · last asked Oct 20 · Behavioral · Mid-level
Other ways it was asked
- “What was the hardest bug you encountered in a real-time measurement pipeline, and how did you isolate and fix it?”
After rolling back a data pipeline change, how would you identify and handle any data loss or corrupted records?
Asked once · last asked Oct 23 · Scenario · Senior
In a spot check, you find a few unredacted emails only in one upstream system’s records. How do you isolate root cause and prevent recurrence across future pipelines?
Asked once · last asked Oct 23 · Scenario · Senior
How do you decide whether to configure failure alerts for each Apache NiFi flow?
Asked once · last asked Feb 6 · Tooling · Mid-level
How would you handle data pipeline failures or retries?
Asked once · last asked Jun 24 · Scenario · Mid-level
Other ways it was asked
- “How did you handle pipeline failures or retries?”
How would you implement automated email notifications in a data pipeline, step by step?
Asked once · last asked Aug 10 · Tooling · Mid-level
What are common reasons for missing fields in scraped data?
Asked once · last asked Aug 29 · Concept · Junior

How this list is made

Every question here was asked by a real interviewer in a real job interview that a candidate ran MeetAssist in. After the call, the interviewer's questions are pulled from the transcript. Practice sessions, tests, exams and anything that was not a job interview are dropped.

Questions are rewritten so they make sense on their own: names of people, employers and clients are removed, and speech-to-text mistakes are fixed. Small talk, logistics and questions about one candidate's own résumé are left out. Rephrasings of the same question are merged, and the count is the number of different interviews it came up in.

The interviews come from candidates who use MeetAssist, so they lean towards software, data and AI roles: the shares describe these interviews, not every ETL and Data Pipelines interview. Answers to the most asked questions are written and checked by an editor; a "common mistake" is shown only when reviews of real answers in these interviews show it, with the count.

307 interviews, 169 candidates, Nov 2025 to Oct 2026. Updated every month with new interviews.

All 76 questions

How would you investigate and recover missing report records after a successful pipeline run, and prevent recurrence?

How would you design pipeline monitoring and alerting, assess existing coverage, and identify gaps?

If the Bronze layer completed successfully but the Gold layer has 40% fewer records than usual, how would you investigate where the problem occurred?

How would you troubleshoot and handle an error in a production Apache NiFi flow?

How can you configure Apache NiFi to notify you when a critical processor fails?

How do you identify data pipeline failure modes and choose appropriate monitoring to detect them?

How do you identify the root cause of a data pipeline failure and implement a fix?

How do you make responses to recurring data pipeline failures more efficient, and when do you document runbooks or standard operating procedures?

Can you tell me about a time you had to troubleshoot a data pipeline or refresh problem in the Azure environment, and what you did?

What specific observability signals would you instrument across the pipeline, such as freshness, volume, schema drift, and label distribution, and what thresholds or alerting logic would you use for each?

When defining SLAs or SLOs with stakeholders for an ETL pipeline, which metrics do you commit to—latency, freshness, completeness, and accuracy—and how do you monitor and alert on them?

How are you notified when a job fails or something goes wrong in the application?

Where do you configure email alerts in IICS?

If a workflow fails in production but the same workflow succeeds in development, how would you investigate the issue in PowerCenter?

If the repository service fails to start after Informatica PowerCenter is installed on a server, what steps would you take?

If the repository service is running but multiple users cannot connect, what might be the cause, and how would you troubleshoot it?

If an Airflow task is not triggering, how can you find the issue?

If a task fails in Airflow, how can you rerun it?

If a pipeline fails at 70% completion, how can you resume it from where it failed?

A user-activity pipeline tracks mobile app and Windows login events with attributes such as location and OS version. What could cause its reported user count to exceed counts from other tracking sources?

If an Apache Iceberg manifest file were deleted, what would happen to the data, and would you still be able to access it?

What issues in the data pipeline or model design could cause regional content to be available but not reflected in recommendations in new markets?

How would you diagnose and resolve a technical challenge encountered while building or maintaining a data pipeline?

Can you describe a concrete production issue involving data latency, a schema mismatch, or a pipeline failure?

How do you create observability for data loads and pipelines in Snowflake?

Spark writes succeed, but an Iceberg metadata commit fails. What object storage issue could be causing the failure?

If a third-party API is unavailable for six hours, how would you backfill the missing data?

How would you detect and handle errors in an affiliate commission pipeline?

When a website's page structure changes and your scraping selectors no longer match the content, what is your go-to approach to fix the issue quickly?

How would you handle failure scenarios such as network interruptions or data corruption during ingestion and transformation, and ensure the pipeline remains resilient with data integrity maintained?

If a third-party PII-redaction dependency fails and normalized events can no longer be generated, how should the system respond and recover?

Tell me about a difficult bug or edge case in ETL refresh logic and how you isolated and fixed it.

What signals or metrics would you track to confirm that an ETL refresh process remains stable over time?

What changes would you make after detecting drift in an ETL refresh process?

When a regression appears in a data pipeline, how do you investigate it?

Imagine you must make a decision in 30 minutes: keep the pipeline running or halt it? What minimal evidence and thresholds would you use?

When you decide to halt the pipeline, how would you communicate the impact, evidence, and next steps to engineering versus nontechnical stakeholders such as operations or leadership?

How do we store the error data in the database so we can track and display the import results to the user?

How would you update a CSV import's progress feedback to show users what is happening during a long-running process?

How would you investigate and recover 20,000 missing updates in a Snowflake CDC pipeline?

How would you systematically find where 1 million missing records disappeared between source and Snowflake?

How would you debug a data pipeline that completed successfully but did not update the downstream dashboard?

How do you distinguish a data error from a timing or latency issue, and what service-level or cutoff rules would you use to address it?

What common issues do you encounter in Microsoft Fabric data pipelines, and how do you address them?

How would you use SQL and pipeline metadata to investigate a data reconciliation difference?

Tell me about a time a critical error or security issue occurred in a data flow you managed. How did you handle it from start to finish?

What is your action plan if a critical Informatica job fails repeatedly overnight and impacts operations?

How do you handle exceptions in an Oracle GL journal integration?

How would you handle an EDM-to-Fusion integration when the source system is down during the scheduled job window?

How would you recover missing records after discovering incomplete raw landing data, without creating duplicates or disrupting reports?

How would you approach supporting a complex Oracle PL/SQL or SQL batch process that you did not originally build?

When an alert fires for a data pipeline, how do you investigate the issue, which logs do you check first, and how do you fix it without reintroducing duplicates?

If a data pipeline fails during a reporting window, how would you communicate the issue and who would you involve?

How would you decide whether it is safe to resume updates to tables exposed to downstream consumers?

If you could improve one thing about a data pipeline to make it more robust or easier to operate, what would you change?

Suppose your hourly Airflow pipeline normally completes in forty minutes, but today it has been running for two hours without any code deployment. How would you troubleshoot it?

The source dataset contains today’s new records, but the Power BI report still shows yesterday’s figures. How would you investigate?

A data pipeline fails overnight and the support task is escalated to you in the morning. What would you do in the first hour?

Can you tell us about a data pipeline that went badly wrong? What went wrong, how did you detect it, and how would you monitor pipeline health?

At which stages could errors occur as data moves from its source to a Power BI report?

If the source owner’s extraction count is higher than the shipment count in your raw landing table, what evidence would you bring to the platform engineer and source owner, and how would you divide the investigation?

If an ADF activity succeeds and raw data is present in Snowflake but a curated table has not changed for two days, how would you distinguish a legitimate no-change period from a failed transformation or stuck incremental checkpoint?

If Power BI and the reporting table show 9,000 shipments but the raw landing data and operational manifest show 10,000 for the same cutoff, where would you investigate first, and how would you account for the missing 1,000 before release?

If a downstream team says expected data has not arrived in NetReveal but you see no obvious failure, how would you approach the situation?

What actions do you take when a replication flow fails?

If a Power BI dashboard refreshes successfully but the last three days of data are missing while other regions appear unaffected, how would you investigate and what would you communicate during the investigation?

If records are present in the raw layer but a code change caused an inner join to the region dimension to drop them from the curated dataset, how would you restore the missing data and prevent this from happening again?

A pipeline normally processes 1,000,000 interactions daily, but today it reports success while the Gold layer contains only 600,000 records. How would you investigate the incident and identify the root cause?

What unexpected challenges might arise in a rocket landing prediction pipeline, and how would you address them?

How would you isolate and fix a difficult bug in a real-time measurement pipeline?

After rolling back a data pipeline change, how would you identify and handle any data loss or corrupted records?

In a spot check, you find a few unredacted emails only in one upstream system’s records. How do you isolate root cause and prevent recurrence across future pipelines?

How do you decide whether to configure failure alerts for each Apache NiFi flow?

How would you handle data pipeline failures or retries?

How would you implement automated email notifications in a data pipeline, step by step?

What are common reasons for missing fields in scraped data?

More ETL and Data Pipelines interview questions

How this list is made

Walk into your next interview ready.