What should you check before an ETL interview?
Can you explain how data moves from source to target?
Ingestion and transformation appeared in 125 of 307 interviews. Start here: can you describe how you extract, map, clean, and load data from an API, file, or database? Explain the path of the data, not just the tools you use.
- Can you explain ETL versus ELT and when you would choose each?
- Can you compare batch and stream processing?
- Can you describe medallion layers and how you would model data for a warehouse?
Can you prove the data is complete and correct?
The most asked question was how to validate data integrity and prove no data was lost during a migration. It appeared in 7 of 307 interviews.
Check whether you can explain how you would compare source and target data, find duplicates, and validate the results. Can you describe what happens when the source schema changes? Which records would you set aside for review?
For migration questions, go beyond loading the data. Can you explain how you would keep systems in sync, switch to the target system, and roll back if needed?
Can you handle changes and recover missing records?
Check whether you can explain how a pipeline picks up changed data. Can you describe watermarks, upserts, late updates, and history tracking? How would you make a retry safe without processing the same data twice?
A real interview question asked about missing report records after a successful pipeline run. Can you walk through how you would investigate the missing results, recover the records, and prevent the problem from returning? Include the checks and alerts you would use.
Can you explain how you test and run pipelines?
Can you describe how you schedule tasks, manage their dependencies, and coordinate workers? Explain how you would test changes before deployment.
For dbt, check whether you can describe models, how their results are stored, and their dependencies. Which dbt tests would you use to validate models and source data? Can you also explain project setup, version control, and deployment?
Can you write transformations and find slow steps?
Code-writing tasks appeared in 10 of 307 interviews. Check these tasks from the sample:
- Can you use PySpark to read JSON from S3, flatten it, and write it to a table?
- Can you remove records with missing values, calculate grouped averages, and write to Delta Lake with schema enforcement and partitioning?
For a slow pipeline, can you explain how you would locate the bottleneck? Describe how you would check uneven data distribution, shuffles, partitioning, and file formats.
Who gets asked ETL and Data Pipelines
The 20 most asked ETL and Data Pipelines questions
Counted by interview, not by mention: a question asked twice in one conversation counts once.
How would you validate data integrity and prove no data was lost during a migration?
Asked in 7 of 307 interviews · last asked Sep 18 · Scenario · Mid-level
Show answer
I would compare the source and target at the same snapshot or cutoff time. I would reconcile row counts and every record key, then compare values using the expected transformation rules. I would also check duplicates, nulls, and broken relationships. Before sign-off, I would confirm all changes through the cutoff were applied and save the reconciliation results.
- Matching counts alone cannot prove no data was lost; compare keys and values too.
- Check required fields, unique keys, and relationships against agreed rules.
- Track rejected rows and failed batches; resolve every unexplained difference.
- Capture ongoing source changes and reconcile through a clear final cutoff.
What the interviewer checks: The interviewer checks whether you can show complete, correct migration results with repeatable checks and saved evidence.
Other ways it was asked
- “How would you monitor data quality during a migration?”
- “How do you validate data quality during a data migration?”
- “How do you ensure data quality during a data migration?”
What is the difference between ETL and ELT, and when would you use each in data pipeline development?
Asked in 6 of 307 interviews · last asked Oct 19 · Concept · Mid-level
Show answer
ETL means extract, transform, load: data is changed before it reaches the target system. ELT means extract, load, transform: data is loaded first and changed inside the target system. Use ETL when data must be cleaned or masked before loading, or the target has limited processing power. Use ELT when the target can handle transformations efficiently.
- ETL uses a separate processing step before loading; ELT uses the target system for transformations.
- ETL can keep sensitive raw data out of the target.
- ELT can retain raw data for later changes, but needs access controls and enough computing power.
What the interviewer checks: The interviewer is checking whether you understand the processing order and can choose based on privacy and system capabilities.
Other ways it was asked
- “What is the difference between ETL and ELT?”
- “What are ETL and ELT, and how do they differ?”
How do you develop and maintain data pipelines, including ETL processes?
Asked in 5 of 307 interviews · last asked Oct 22 · Scenario · Mid-level
Show answer
I define the source data, target tables, and freshness needs, then build extraction, transformation, and loading steps. I use SQL or Python for processing and a scheduler such as Airflow to manage runs. I add data checks, safe retries, and alerts, then maintain the pipeline through monitoring, tests, version control, and updates when source data changes.
- Check required fields, duplicates, row counts, and business rules.
- Make reruns safe so retries do not duplicate or corrupt data.
- Track failures, data freshness, and runtime; alert the team when limits are missed.
- Test source schema changes and keep a way to reload missing or incorrect data.
What the interviewer checks: The interviewer checks whether you can build the full pipeline and keep its data correct and reliable over time.
Other ways it was asked
- “How do you work on data pipelines?”
- “How do you process data in an ETL pipeline, from extraction through transformation and loading?”
- “How do you build an ETL pipeline?”
Which dbt tests can you use to validate models and source data?
Asked in 5 of 307 interviews · last asked Sep 22 · Concept · Mid-level
Show answer
I use dbt’s built-in data tests: not_null, unique, accepted_values, and relationships. I configure these in YAML to check model or source columns. For business rules, I write custom reusable tests or singular SQL tests that return rows that break the rule.
- Use not_null and unique together for primary keys; unique alone does not reject nulls.
- accepted_values checks allowed values; relationships checks that non-null values exist in a referenced column.
- Data tests pass when no failing rows are returned, unless failure thresholds are configured.
- Source freshness checks run separately with dbt source freshness; they check whether source data is recent enough.
What the interviewer checks: The interviewer checks whether you can choose built-in tests and write custom checks for data quality.
Other ways it was asked
- “What data quality tests are available in dbt?”
- “How do you write data-quality tests in dbt?”
- “What data quality tests can you implement in dbt?”
What is the difference between batch processing and stream processing?
Asked in 4 of 307 interviews · last asked Oct 19 · Concept · Mid-level
Show answer
Batch processing handles a finite set of data in one run. Stream processing handles incoming data continuously, usually with lower delay. I use batch for scheduled reports and large historical loads, and streaming for alerts or dashboards that need fresh data quickly.
- Batch is often simpler to operate when results do not need to be immediate.
- Streaming must account for late events, duplicate records, and failures.
- Spark uses spark.read for batch data and spark.readStream for streaming data; Structured Streaming usually uses micro-batches.
What the interviewer checks: The interviewer checks whether you can choose an approach based on data arrival, acceptable delay, and operational complexity.
Other ways it was asked
- “What are stream processing and batch processing, and when would you use each?”
- “What is the difference between batch processing and streaming?”
- “What is the difference between batch and stream processing, and how do you read data with Spark?”
How would you implement a Slowly Changing Dimension Type 2 in an ETL process?
Asked in 4 of 307 interviews · last asked Oct 7 · Scenario · Mid-level
Show answer
I would use SCD Type 2 to keep a new dimension row for each tracked change, preserving history. I would match incoming records to current rows by business key and compare tracked fields. For a change, I would close the old row and insert a new row with a new surrogate key, validity dates, and a current flag.
- Insert new business keys as current rows; leave unchanged records alone.
- Set the old row’s end time to the new row’s start time; treat end times as exclusive.
- Expire and insert rows in one transaction, and make retries safe.
- Link facts to the dimension version valid at the fact’s event time.
What the interviewer checks: The interviewer checks whether you can preserve history and link facts to the correct dimension version.
Other ways it was asked
- “How do you implement Slowly Changing Dimension Type 2 (SCD Type 2) in a data warehouse?”
- “What are slowly changing dimensions (SCDs) in data warehousing?”
- “What is a slowly changing dimension (SCD) in data warehousing?”
How do you ensure data quality and reconciliation when integrating multiple sources into a reporting pipeline?
Asked in 4 of 307 interviews · last asked Oct 23 · Scenario · Mid-level
Show answer
I define shared data rules, standardize source formats, and validate data at ingestion, after transformation, and before reporting. I check required fields, duplicates, valid values, and links between records. I reconcile row counts and business totals against each source, allowing for expected transformation changes. I quarantine bad records and block publication when critical checks fail.
- Agree on shared keys, field meanings, and validation rules with data owners.
- Compare counts and totals for the same time window, accounting for filters, duplicates, and late arrivals.
- Keep source identifiers and audit logs so differences can be traced.
- Automate checks and alerts; assign owners to investigate failures.
What the interviewer checks: The interviewer is checking whether you can detect bad data, explain reconciliation differences, and prevent unreliable reports.
Other ways it was asked
- “How do you ensure the quality of data and the pipeline?”
- “How do you ensure data quality throughout an ETL pipeline?”
- “How do you ensure data quality and reconciliation across a data pipeline? Give an example of how you have done this.”
How would you investigate and recover missing report records after a successful pipeline run, and prevent recurrence?
Asked in 4 of 307 interviews · last asked Sep 28 · Behavioral · Senior
Show answer
I would trace missing record IDs from the source through each stage to the report. I would check filters, joins, rejected rows, and load windows. After fixing the cause, I would replay affected data without creating duplicates and reconcile report results before release. I would add completeness checks, alerts, and tests for the failure.
- Compare record IDs, counts, and totals for the same time window; successful execution does not prove completeness.
- Check incremental load markers, late arrivals, and report refreshes, not just pipeline logs.
- Backfill the affected range safely, then confirm missing records are restored and business totals match.
- Block report publication when completeness checks fail, and alert the team.
What the interviewer checks: The interviewer is checking whether you can trace data loss, recover safely, verify report accuracy, and prevent repeat failures.
Other ways it was asked
- “How would you find the root cause of a data pipeline that completed successfully but produced incomplete results, and what would you change?”
- “How would you investigate a successfully completed data pipeline that produced incomplete results, and what would you change to prevent a recurrence?”
- “What changes would you make to prevent a data pipeline from producing incomplete results again?”
What is the medallion architecture for data processing?
Asked in 3 of 307 interviews · last asked Oct 24 · Concept · Mid-level
Show answer
Medallion architecture organizes data into three layers: bronze, silver, and gold. Bronze keeps raw data from source systems. Silver cleans, checks, and combines that data. Gold holds business-ready data, often summarized for reports and analysis.
- Data quality and structure improve as data moves through the layers.
- Keeping raw data lets teams reprocess it when rules change or errors occur.
- Layers are logical stages; they do not require separate storage systems.
What the interviewer checks: The interviewer checks whether you know each layer’s role and how the layers turn raw data into trusted business data.
Other ways it was asked
- “What is medallion architecture, and how does it work?”
- “What is a medallion architecture, and how do its layers organize data?”
What is ETL?
Asked in 3 of 307 interviews · last asked Aug 9 · Concept · Junior
Show answer
ETL stands for Extract, Transform, Load. It takes data from source systems, cleans or reshapes it, and loads it into a target such as a data warehouse. It supports data ingestion by bringing source data into a consistent form for reporting and analysis.
- Extract reads data from sources such as databases, files, or APIs.
- Transform cleans, validates, and reshapes the data.
- Load writes the transformed data into the target system.
What the interviewer checks: The interviewer is checking whether you understand the three steps and how they move data from sources to a usable target.
Other ways it was asked
- “What is ETL, and how does it support data ingestion?”
What is data engineering?
Asked in 3 of 307 interviews · last asked Sep 14 · Concept · Junior
Show answer
Data engineering is building and running systems that collect, store, and prepare data for use. Data engineers create pipelines that move data from source systems, clean it, and make it ready for reports, analysis, or machine learning. They keep data reliable, secure, and available when needed.
- Build pipelines to collect, transform, and deliver data.
- Manage data storage, data quality, and access.
- Monitor pipelines and fix failures so data stays available.
What the interviewer checks: The interviewer is checking whether you understand how data engineering makes data usable and reliable for others.
Other ways it was asked
- “What is data science engineering?”
What is the difference between COPY INTO and Snowpipe in Snowflake?
Asked in 3 of 307 interviews · last asked Sep 20 · Concept · Mid-level
Show answer
COPY INTO loads staged files in batches using a virtual warehouse that you manage. Snowpipe loads new staged files continuously using Snowflake-managed compute, usually triggered by file notifications. Use COPY INTO for scheduled bulk loads and Snowpipe when fresh data should arrive with less delay.
- COPY INTO runs when you execute it or schedule it.
- Snowpipe uses a pipe object containing a COPY INTO statement.
- COPY INTO uses warehouse credits; Snowpipe has separate serverless ingestion charges.
- File-based Snowpipe is not instant row streaming; Snowpipe Streaming is a separate ingestion option.
What the interviewer checks: The interviewer checks whether you can choose a loading method based on timing, compute management, and cost.
Other ways it was asked
- “What's the difference between Snowpipe and the standard COPY INTO batch load?”
- “What is the difference between Snowpipe and COPY INTO in Snowflake?”
What is dbt, and why would you use it?
Asked in 3 of 307 interviews · last asked Sep 20 · Concept · Junior
Show answer
dbt is a tool for transforming data already loaded into a data warehouse or other supported data platform. You define models, usually in SQL, and dbt runs them in dependency order. I would use it to keep transformation logic in version control, reuse code, test data, and generate documentation.
- dbt handles the transformation step of ELT; separate tools usually extract and load the data.
- Models define transformations that dbt builds as views, tables, or other supported forms.
- Model references let dbt track dependencies and choose the correct run order.
- Tests check data rules, such as unique keys, required values, and valid relationships.
What the interviewer checks: The interviewer wants to know whether you understand dbt’s role and how it makes data transformations easier to maintain.
Other ways it was asked
- “What is dbt?”
- “Can you explain what dbt is?”
What is an idempotent data pipeline, and how does idempotency help?
Asked in 3 of 307 interviews · last asked Oct 18 · Concept · Mid-level
Show answer
An idempotent data pipeline leaves the output data in the same state when it processes the same input again. It does not create duplicates or apply the same change twice. This makes retries after failures and reprocessing safer.
- Use stable record keys to identify data already processed.
- Upsert by key or replace the affected partition instead of blindly appending.
- Repeated updates must not add the same amount to a total twice.
CodeInput: order_id=42, amount=100. First run: insert the order using order_id as a unique key. Retry: update that order to amount=100, rather than insert another row. Result: one order row with amount=100 after either run.What the interviewer checks: The interviewer checks whether you can make retries safe without duplicating data or changing results.
Other ways it was asked
- “What does idempotency mean in a data integration pipeline?”
- “What does it mean for a data pipeline to be idempotent? Give a concrete example.”
How would you design pipeline monitoring and alerting, assess existing coverage, and identify gaps?
Asked in 3 of 307 interviews · last asked Sep 22 · Scenario · Senior
Show answer
I would build monitoring into each pipeline from the start, covering failures, delays, data quality, and resource use. I would set freshness and quality targets, then send actionable alerts to an owner with recovery steps. To assess coverage, I would map existing checks to pipeline stages and targets, review past incidents, and test failure cases to find gaps.
- Track job status, retries, run time, freshness, row counts, and data quality.
- Set alert thresholds from business needs; page only for urgent issues and group repeated alerts.
- Check every critical source, processing step, and output for missing monitoring or unclear ownership.
- Test alerts with safe failure simulations, including cases where jobs succeed but data is late or wrong.
What the interviewer checks: The interviewer is checking whether you can detect real data problems, limit alert noise, and systematically find monitoring gaps.
Other ways it was asked
- “How would you design observability and monitoring for a data platform, including when to implement it, how to assess current coverage, and how to identify gaps?”
- “How do you monitor data pipelines and configure alerting?”
- “How would you implement monitoring and alerting for a data engineering environment?”
Tell me about a time you designed an end-to-end ETL pipeline, from extraction through data delivery, integrating multiple data sources while ensuring data quality.
Asked in 2 of 307 interviews · last asked Sep 20 · Behavioral · Senior
Show answer
I designed a daily sales pipeline combining database orders, API payments, and CSV refunds. I stored incremental extracts in raw storage, then cleaned, deduplicated, and joined them in a warehouse. I checked required fields, unique keys, and source totals before publishing tables for dashboards. An orchestrator managed dependencies, safe retries, and alerts; failed checks blocked publication.
- Explain how each source was extracted and how changes were tracked.
- Describe how you matched records across sources and handled missing or duplicate data.
- Show how quality checks controlled publication to downstream users.
- Explain how reruns avoided duplicates and how failures were detected and recovered.
What the interviewer checks: The interviewer is checking whether you owned the full pipeline and can explain its design, reliability, and data quality.
Other ways it was asked
- “Can you give a detailed end-to-end example of a data pipeline you built, including how you ingested and transformed data, delivered it to downstream users, and ensured data quality?”
How would you design a data ingest and transformation pipeline for a retail company using Azure Data Factory and Azure Databricks to process data from multiple sources and load the results into a Delta Lake, maximizing performance and handling failures effectively?
Asked in 2 of 307 interviews · last asked Aug 11 · Design · Senior
Show answer
I would use Azure Data Factory to ingest incremental data from each source into raw storage on ADLS. Databricks would clean, join, and aggregate it into Delta tables with bronze, silver, and gold layers. I would tune ingestion parallelism, Spark joins, and file sizes for speed. Retries, monitoring, and safe reruns would handle failures.
- Track CDC positions or watermarks per source; advance them only after successful processing.
- Deduplicate records before Delta MERGE and use stable keys so batch reruns do not create duplicates.
- Use autoscaling job compute, avoid small files, and partition large tables only where queries benefit.
- Set bounded retries, quarantine bad records, and alert on failures; use managed identities and Key Vault.
What the interviewer checks: The interviewer checks whether you can connect ingestion, transformation, performance, cost, security, and reliable recovery in one design.
Other ways it was asked
- “You're tasked with designing an ETL pipeline using Azure Databricks to process incremental data loads from Azure Data Lake. The data contains multiple large Parquet files, and you need to perform transformations such as filtering, aggregations, and data enrichment before writing the output back to Azure SQL database. How would you ensure efficient data processing, optimize costs, and handle potential failures in the pipeline?”
- “How would you structure an Azure Data Factory pipeline to optimize performance and ensure fault tolerance when ingesting large volumes of data from multiple sources, transforming it, and loading it into Azure SQL Database?”
- “How would you design an Azure Data Factory pipeline for data ingestion from on-premise SQL Server, processing in Azure Databricks, and loading into Azure Synapse Analytics to ensure security, scalability, and performance?”
What is a staging area in Snowflake?
Asked in 2 of 307 interviews · last asked Sep 18 · Concept · Junior
Show answer
A staging area, or stage, in Snowflake is a location for data files before loading them into tables. An internal stage stores files in Snowflake-managed storage; an external stage points to supported cloud storage. COPY INTO loads data from staged files into a table.
- Internal stages can be user stages, table stages, or named stages.
- External stages point to Amazon S3, Google Cloud Storage, or Azure storage.
- Stages hold files, not database rows, and can also receive files exported from tables.
What the interviewer checks: The interviewer checks whether you understand stages, their types, and their role in loading data.
Other ways it was asked
- “What is staging in Snowflake?”
In Power Query, what's the practical difference between merge queries and append queries, and when would you use each?
Asked in 2 of 307 interviews · last asked Sep 6 · Tooling · Mid-level
Show answer
Merge joins tables using matching key columns, usually to add related columns. Append stacks rows from tables into one table. Use merge to add customer details to orders, and append to combine monthly sales tables with the same structure.
- Merge join types control which matching and unmatched rows are kept.
- Append matches columns by name, not position; missing columns get null values.
- Expanding a merge can increase row counts when a key has multiple matches.
- Append keeps duplicate rows; it does not remove them automatically.
What the interviewer checks: The interviewer checks whether you can choose between joining related data and stacking similar records.
Other ways it was asked
- “What is the difference between Append and Merge in Power Query?”
If business data is maintained manually in Excel or CSV files on SharePoint, how would you build a reliable reporting pipeline that handles changes to file structure or values?
Asked in 2 of 307 interviews · last asked Sep 29 · Scenario · Mid-level
Show answer
I would automatically copy SharePoint files into versioned storage, then validate them before loading reporting tables. I would agree on column names, data types, and value rules with the business owners. Invalid files would be held back and trigger an alert, keeping the last valid report available. Approved changes would update the pipeline and its tests.
- Keep original files and source versions so reports can be traced and rebuilt.
- Check required columns, types, duplicate keys, missing values, and allowed values.
- Use stable business keys to apply corrections without creating duplicates when files are processed again.
- Reconcile row counts and business totals; show the last successful refresh so stale reports are visible.
What the interviewer checks: The interviewer is checking whether you can turn manually maintained files into controlled, traceable reporting despite unexpected changes.
Other ways it was asked
- “Some operational data arrives in Excel or CSV files maintained by business users, perhaps through SharePoint. How would you bring it into a production reporting pipeline and make sure people can trust it?”
- “Tell me about a time you worked with data maintained in multiple CSV files on SharePoint because it wasn’t captured in a core system. What problem did it cause, what did you change, and how did you make the resulting reporting trustworthy?”
- “If business data is maintained manually in Excel or CSV files on SharePoint, how would you bring it into a reliable reporting pipeline? Tell me about your experience with this kind of data and what you did when the file structure or values changed.”
ETL basics and data pipeline architecture interview questions
Start with questions about ETL versus ELT, batch versus streaming, warehouse modeling, medallion layers, and end-to-end pipeline design. 132 questions.
What are the key challenges you typically encounter when building data integration pipelines?
Asked once · last asked May 19 · Scenario · Senior
Other ways it was asked
- “What other challenges can arise when building data integration pipelines?”
Would you pass each event through the normalizer individually or process events in batches?
Asked once · last asked Jul 20 · Design · Mid-level
How would you design the tools and data pipeline for an AI scorecard agent?
Asked once · last asked Jul 25 · Design · Senior
How would you design an ETL process to move retail data from source to target?
Asked once · last asked Sep 16 · Scenario · Mid-level
What is your approach to providing an end-to-end data warehouse solution for a client?
Asked once · last asked Sep 19 · Design · Mid-level
Other ways it was asked
- “What would be your approach to implementing a data warehouse from scratch and delivering an end-to-end solution?”
How would you create a healthcare data warehouse?
Asked once · last asked Sep 19 · Design · Mid-level
Other ways it was asked
- “How would you create a healthcare data warehouse for a client?”
- “How do you create a data warehouse?”
Where should business logic be applied in a file-ingestion pipeline that loads data into multiple database tables?
Asked once · last asked Oct 7 · Design · Senior
How would you design an end-to-end Apache NiFi flow to process a 4-GB file?
Asked once · last asked Feb 6 · Tooling · Mid-level
How would you integrate Power BI with an ERP system, designing the connection and data flows end to end and evaluating the outcome?
Asked once · last asked Feb 10 · Behavioral · Senior
Other ways it was asked
- “Tell me about a project where you integrated Power BI with an ERP system. How did you design the connection and data flows, and what was the outcome?”
What source types and data models, such as fact, dimension, and event tables, are commonly used in data pipelines?
Asked once · last asked Mar 3 · Concept · Mid-level
How do you build resiliency into data solutions, and what techniques do you use?
Asked once · last asked Mar 3 · Scenario · Senior
Can you explain the data analytics process, including how data is collected and analyzed?
Asked once · last asked Mar 28 · Concept · Junior
Data ingestion and transformation interview questions
These questions cover extracting, mapping, cleaning, and loading data from APIs, files, databases, enterprise systems, and documents. 194 questions.
What does POE mean in satellite orbit data processing, and how is it used?
Asked once · last asked Sep 22 · Concept · Mid-level
Other ways it was asked
- “What does POE mean in satellite orbit data processing?”
- “What role does POE play in satellite orbit data processing?”
How would you map platform-specific identifiers to a normalized campaign ID?
Asked once · last asked Jul 20 · Design · Mid-level
Other ways it was asked
- “How would you map platform-specific identifiers, such as Facebook campaign IDs or Google objective IDs, to a normalized campaign ID?”
Why might Amazon S3 be used as an intermediary between AWS Lambda and Snowflake, and how would you integrate it?
Asked once · last asked Sep 20 · Design · Mid-level
Other ways it was asked
- “Why might you use Amazon S3 as an intermediary between AWS Lambda and Snowflake?”
- “Why might you use Amazon S3 as an intermediary instead of loading data directly from AWS Lambda into Snowflake?”
- “How would you set up data ingestion between AWS Lambda, Amazon S3, and Snowflake?”
When writing data to Iceberg with Spark, what is the difference between overwrite and dynamic partition overwrite?
Asked once · last asked Aug 6 · Concept · Mid-level
Other ways it was asked
- “When writing to an Iceberg table with Spark using dynamic partition overwrite, what happens if the input data contains multiple partitions?”
What novel approaches would you consider when building a data enrichment platform?
Asked once · last asked Aug 6 · Behavioral · Senior
Other ways it was asked
- “What novel approaches did you use to build a data enrichment platform?”
Write a PySpark script to read CSV from ADLS, remove records with missing values in a specified column, group by another column to calculate an average, and write the results to a Delta Lake table in ADLS with schema enforcement and partitioning.
Asked once · last asked Aug 11 · Write code · Mid-level
Other ways it was asked
- “Can you write a PySpark script that reads a CSV file from Azure Data Lake Storage, filters out records where a specific column has no values, groups the data by a given column to calculate the average of another column, and writes the transformed data back to a Delta Lake table in ADLS with schema enforcement and partitioning?”
- “Can you write a PySpark script that reads a CSV file from Azure Data Lake Storage, filters out empty values, groups the data by a column to calculate an average, and writes the results to Delta Lake with schema enforcement and partitioning?”
How would you approach ingesting and processing a batch of JSON files from Azure Data Lake in Databricks, including Delta Lake, partitioning, and performance optimization?
Asked once · last asked Aug 22 · Design · Senior
Other ways it was asked
- “How would you use Databricks to ingest and process a batch of JSON files stored in an Azure Data Lake, ensuring data quality and proper partitioning for efficient querying?”
In what scenarios would you use dataflows in Power BI?
Asked once · last asked Aug 28 · Tooling · Mid-level
What is the difference between a replication flow and a transformation flow in SAP Datasphere, and how would you decide which to use?
Asked once · last asked Aug 28 · Concept · Mid-level
Other ways it was asked
- “What is the difference between a replication flow and a transformation flow in SAP Datasphere, and which one would you use for this scenario?”
- “How would you use a replication flow or transformation flow to filter data in SAP Datasphere?”
How would you connect an external website to a chatbot and provide the website's data to it?
Asked once · last asked Sep 30 · Design · Mid-level
Other ways it was asked
- “How would you integrate an external company website with a chatbot so the bot can use the website's data?”
What is the standard CDS view used to extract ACDOCA into SAP Datasphere?
Asked once · last asked Sep 30 · Concept · Mid-level
Other ways it was asked
- “How do you manage and extract data from the standard CDS view for ACDOCA into SAP Datasphere?”
If your organization uses ServiceNow HRSD and an external payroll system, how would you integrate the two to ensure seamless data flow for employee payroll information?
Asked once · last asked Oct 22 · Scenario · Senior
Other ways it was asked
- “How would you integrate ServiceNow HRSD with an external payroll system while addressing security, data consistency, and error handling?”
dbt interview questions
For dbt, the questions cover models, materializations, dependencies, tests, project setup, version control, deployment, and optimization. 42 questions.
How would you safely deploy a change to a dbt model that feeds several business reports, and what would you do if the change created duplicate records after release?
Asked in 2 of 307 interviews · last asked Sep 22 · Scenario · Senior
Other ways it was asked
- “How would you deploy a dbt model change safely when the model feeds several downstream reports?”
- “A dbt model feeds several important reports. How would you release a change safely, and what would you do if the change introduced duplicate records after deployment?”
- “When deploying a dbt model change that feeds several downstream reports, what would you test before release, and how would you recover if it produced incorrect results?”
How do incremental dbt models differ from full refreshes, and when would you use each?
Asked in 2 of 307 interviews · last asked Sep 20 · Concept · Mid-level
Other ways it was asked
- “When would you use an incremental model in dbt?”
- “Can you give an example of when an incremental model in dbt is appropriate?”
- “What is the difference between an incremental dbt model and a full-refresh model?”
What is the ref function used for in dbt?
Asked in 2 of 307 interviews · last asked Sep 20 · Concept · Mid-level
Other ways it was asked
- “What is the difference between source and ref in dbt?”
How would you create a custom SQL test in dbt?
Asked once · last asked Sep 20 · Tooling · Mid-level
How would you work with end users to design data models within a medallion architecture using dbt?
Asked once · last asked Jul 16 · Scenario · Senior
What are the best practices for setting up a dbt project for a team that is new to dbt, and how would you go about establishing the project from scratch?
Asked once · last asked Jul 16 · Tooling · Mid-level
How can we configure dbt to run only the sales or marketing models and their dependencies instead of all models?
Asked once · last asked Jul 16 · Tooling · Mid-level
How would you use dbt to implement silver-to-gold transformations in a Databricks-orchestrated data platform?
Asked once · last asked Jul 28 · Scenario · Senior
How do you design dbt gold-layer models, including dimensions, facts, and dataset selection?
Asked once · last asked Jul 28 · Scenario · Senior
What is the difference between Snowflake and dbt?
Asked once · last asked Aug 31 · Concept · Junior
What does it tell you about your testing strategy if a dbt model passes all existing tests but downstream users report incorrect business numbers?
Asked once · last asked Sep 9 · Scenario · Senior
What is materialization in dbt?
Asked once · last asked Sep 14 · Concept · Mid-level
Data pipeline orchestration and deployment interview questions
Explore scheduling, task dependencies, worker coordination, development tests, CI/CD, security, and documentation with Airflow and other pipeline tools. 90 questions.
How would you transfer orchestration experience from Azure Data Factory to Airflow, identify learning gaps, and become productive safely?
Asked in 2 of 307 interviews · last asked Sep 22 · Scenario · Mid-level
Other ways it was asked
- “Which data orchestration skills transfer from Azure Data Factory to Airflow and Spark, where would you expect learning gaps, and how would you become productive without creating delivery risk?”
How do you securely manage credentials when connecting data pipelines to Salesforce and databases?
Asked once · last asked Sep 10 · Scenario · Mid-level
Other ways it was asked
- “How would you implement secure credential management for data pipelines connecting to Salesforce and databases?”
What authorization mechanisms are commonly used to secure data pipelines?
Asked once · last asked Jan 29 · Concept · Mid-level
Using Azure DevOps or GitHub, how would you set up a CI/CD pipeline for database scripts and ETL workflows, from source control through deployment to development and production?
Asked once · last asked Feb 24 · Tooling · Senior
How do you choose unit tests and scenarios to cover when developing a data solution?
Asked once · last asked Mar 3 · Scenario · Mid-level
How would you handle data from different sources or regions, and how would you orchestrate that processing?
Asked once · last asked Mar 3 · Scenario · Senior
How would you design encryption for a data pipeline, including client-side or server-side encryption and key management?
Asked once · last asked Mar 3 · Design · Senior
How would you deploy and operate data pipelines beyond the notebook stage, and what infrastructure would you use?
Asked once · last asked Mar 7 · Design · Mid-level
After the code is merged, how would you automate delivering it to the Airflow cluster or other hosts where the application will run?
Asked once · last asked Mar 20 · Scenario · Mid-level
How do you write an Apache Airflow DAG?
Asked once · last asked Apr 24 · Tooling · Mid-level
How do you structure your Python exceptions or error taxonomy so the orchestrator, for example Airflow, can distinguish retryable from non-retryable failures and alert appropriately?
Asked once · last asked Apr 25 · Tooling · Senior
How would you trigger Spark on Kubernetes to process PostgreSQL staging tables populated by Flink without relying on scheduled orchestration?
Asked once · last asked May 18 · Design · Senior
Incremental loading, CDC, and streaming interview questions
Questions on changing data cover change capture, watermarks, upserts, history tracking, late updates, idempotency, exactly-once processing, and safe retries. 95 questions.
How would you design incremental loading to handle late-arriving updates across large time windows?
Asked in 2 of 307 interviews · last asked Jul 20 · Design · Senior
Other ways it was asked
- “How would you change a data pipeline to handle revisions across large time windows?”
- “How would you redesign the data flow to handle revisions across large time windows?”
- “Suppose you’re designing an end-to-end pipeline from the source through cloud storage, Spark, a database, and a dashboard, and the source sends late-arriving updates. How would you design incremental logic that remains correct throughout data loading and the pipeline?”
How would you safely recover a partially loaded API batch and prove no records are missing or duplicated?
Asked in 2 of 307 interviews · last asked Sep 29 · Scenario · Senior
Other ways it was asked
- “Suppose the API fails after five pages have been retrieved. How would you recover the batch without losing records or creating duplicates?”
- “If a new engineer has to support an incremental ADF-to-Snowflake pipeline while you are away, what would you document so they can detect a partial batch, rerun it safely, and prove the recovered data is complete?”
- “If a nightly load fails on 8% of an API batch after some rows have landed in raw and some may have reached the curated table, what recovery sequence would you follow from containment to checkpoint commit, and how would you prove the rerun did not skip or double-count records?”
A source record is corrected today, but it belongs to last month. The dashboard still shows the old value after its scheduled refresh. How would you investigate and fix it?
Asked in 2 of 307 interviews · last asked Sep 23 · Scenario · Senior
Other ways it was asked
- “A source record from last week is corrected today, but the daily Power BI dashboard still shows the old value. How would you investigate and fix it?”
How would you ensure reliable data synchronization between MongoDB and OpenSearch?
Asked once · last asked Sep 7 · Design · Senior
Other ways it was asked
- “How would you minimize synchronization delays between MongoDB and OpenSearch?”
- “How would you design reliable synchronization between MongoDB, OpenSearch, and APIs?”
If shipment records are being added or updated while your pipeline is moving through the API pages, how would you reduce the risk of missing records or creating duplicates?
Asked once · last asked Sep 22 · Scenario · Senior
Other ways it was asked
- “If shipment records are added or updated while a pipeline is paging through an API, how would you reduce the risk of missing records or creating duplicates?”
- “If shipment records are being added or updated while your pipeline is moving through the API pages, how would you reduce the risk of missing or duplicate records?”
How would you ingest changes from an on-premises SQL Server database, and why choose direct reads or transaction-log CDC?
Asked once · last asked Sep 16 · Scenario · Senior
Other ways it was asked
- “How would you design ingestion from an on-premises SQL Server database, choosing between direct reads and transaction-log CDC?”
What is the difference between an upsert and CDC?
Asked once · last asked Mar 20 · Concept · Mid-level
What SQL would you use to apply CDC events from a source such as Kafka to a target table?
Asked once · last asked Mar 20 · Write code · Senior
When applying CDC changes with a MERGE or upsert, how would you identify which target rows to update?
Asked once · last asked Mar 20 · Concept · Mid-level
What is an incremental load, and how can you implement it for a dataset?
Asked once · last asked Apr 14 · Concept · Mid-level
What is Change Data Feed (CDF) in Databricks, and when would you use it?
Asked once · last asked Apr 17 · Concept · Mid-level
How can you use Change Data Feed (CDF) to process data incrementally in a data architecture?
Asked once · last asked Apr 17 · Scenario · Senior
ETL data quality and validation interview questions
Data quality questions cover validation, schema changes, duplicates, comparing results across systems, and setting aside records that fail checks. 96 questions.
How would you handle missing fields in data scraped from a website?
Asked in 2 of 307 interviews · last asked Aug 29 · Scenario · Mid-level
Other ways it was asked
- “How do you handle missing or inconsistently formatted fields in scraped data?”
When importing database rows into an application, how would you generate a unique key and validate missing values?
Asked once · last asked Aug 22 · Scenario · Mid-level
Other ways it was asked
- “When importing database rows into an application, how would you generate a unique key and assign values for missing fields?”
How would you address data integration and data quality risks in a financial dashboard?
Asked once · last asked Feb 18 · Scenario · Senior
How do you design data-quality or integrity tests for data pipelines?
Asked once · last asked Mar 3 · Scenario · Mid-level
Other ways it was asked
- “How do you distinguish functional tests from data-quality tests in a data pipeline?”
How would you validate that all rows from a source have arrived in a data warehouse?
Asked once · last asked Mar 3 · Scenario · Mid-level
How should a data pipeline handle a schema mismatch: continue processing with the existing schema, or stop and report an error?
Asked once · last asked Apr 14 · Scenario · Mid-level
When extracting and transforming data for AI model training, what are your key considerations for data quality, consistency, and security?
Asked once · last asked Apr 17 · Scenario · Mid-level
When you validate incoming data in Python, what checks do you treat as hard failures versus quarantined bad records, and how do you route or report them?
Asked once · last asked Apr 25 · Scenario · Senior
In Python, how would you implement a reusable data quality framework with rules, severities, and metrics that works across multiple pipelines while still supporting dataset-specific checks?
Asked once · last asked Apr 25 · Design · Senior
How would you validate an ETL pipeline end to end without relying on developers?
Asked once · last asked May 3 · Scenario · Mid-level
In production, how do you handle schema evolution, such as added or renamed fields and type changes, across ingestion, transformation, and downstream consumers without breaking the pipeline?
Asked once · last asked May 3 · Scenario · Senior
If parent and child records can arrive independently, how would you build a database framework or report to identify child records that have no corresponding parent?
Asked once · last asked May 19 · Scenario · Mid-level
Data pipeline performance optimization interview questions
Performance questions cover finding bottlenecks, handling large datasets, distributed processing, skew, shuffles, partitioning, file formats, and storage trade-offs. 54 questions.
How do you work with a large dataset, and why would you choose that approach?
Asked in 2 of 307 interviews · last asked Aug 9 · Scenario · Mid-level
Other ways it was asked
- “How would you work with a large dataset?”
What is query folding, and when is it useful?
Asked in 2 of 307 interviews · last asked Sep 15 · Concept · Mid-level
Other ways it was asked
- “What is query folding?”
What tools and strategies would you use to develop or optimize ETL pipelines to improve data processing and analytics efficiency?
Asked in 2 of 307 interviews · last asked Feb 27 · Behavioral · Senior
Other ways it was asked
- “Describe a time when you developed or optimized ETL pipelines to improve data processing and analytics efficiency. What specific tools and strategies did you use?”
- “Can you describe a specific project where you optimized ETL performance? What steps did you take to ensure the processes met the application's performance requirements?”
What do you mean by data skew?
Asked in 2 of 307 interviews · last asked Sep 11 · Concept · Mid-level
Other ways it was asked
- “Can you give an example of data skew?”
- “What is data skewness?”
How would you use Python to process terabytes of data reliably?
Asked once · last asked Aug 12 · Scenario · Senior
Other ways it was asked
- “How would you optimize Python code that processes terabytes of data?”
Can Pandas query distributed data, and what tools or approaches would you use instead?
Asked once · last asked Sep 9 · Concept · Mid-level
What performance considerations would you account for when applying CDC changes?
Asked once · last asked Mar 20 · Scenario · Senior
What problems would you anticipate when processing a 4- to 5-GB nested JSON file?
Asked once · last asked Apr 17 · Scenario · Mid-level
When you run ETL on Spark or a cloud warehouse, what steps do you take to diagnose and fix performance regressions involving skew, shuffles, partitioning, or file sizes?
Asked once · last asked May 3 · Scenario · Senior
How would you process domain tables containing roughly 1.3–1.5 billion records when the reference tables are much smaller?
Asked once · last asked May 18 · Design · Senior
How would you handle unpredictable incoming data volumes, such as 10,000 transactions one hour, a million the next, and none the following hour?
Asked once · last asked May 18 · Design · Senior
What steps would you take if a mapping or workflow that should finish in 30 minutes suddenly took two hours?
Asked once · last asked Nov 10 · Scenario · Senior
Data pipeline monitoring and troubleshooting interview questions
Work through questions about monitoring, alerts, production incidents, missing or incorrect results, finding causes, recovery, and written recovery steps. 76 questions.
If the Bronze layer completed successfully but the Gold layer has 40% fewer records than usual, how would you investigate where the problem occurred?
Asked once · last asked Oct 18 · Scenario · Senior
Other ways it was asked
- “If the Bronze layer has the expected record count but the Silver layer does not, what would you investigate?”
How would you troubleshoot and handle an error in a production Apache NiFi flow?
Asked once · last asked Feb 6 · Scenario · Mid-level
Other ways it was asked
- “How do you implement logging and error handling in Apache NiFi flows?”
How can you configure Apache NiFi to notify you when a critical processor fails?
Asked once · last asked Feb 6 · Tooling · Mid-level
How do you identify data pipeline failure modes and choose appropriate monitoring to detect them?
Asked once · last asked Mar 3 · Scenario · Senior
How do you identify the root cause of a data pipeline failure and implement a fix?
Asked once · last asked Mar 3 · Scenario · Senior
How do you make responses to recurring data pipeline failures more efficient, and when do you document runbooks or standard operating procedures?
Asked once · last asked Mar 3 · Scenario · Senior
Can you tell me about a time you had to troubleshoot a data pipeline or refresh problem in the Azure environment, and what you did?
Asked once · last asked Apr 24 · Behavioral · Mid-level
What specific observability signals would you instrument across the pipeline, such as freshness, volume, schema drift, and label distribution, and what thresholds or alerting logic would you use for each?
Asked once · last asked Apr 25 · Scenario · Senior
When defining SLAs or SLOs with stakeholders for an ETL pipeline, which metrics do you commit to—latency, freshness, completeness, and accuracy—and how do you monitor and alert on them?
Asked once · last asked May 3 · Scenario · Senior
How are you notified when a job fails or something goes wrong in the application?
Asked once · last asked Nov 10 · Tooling · Mid-level
Where do you configure email alerts in IICS?
Asked once · last asked Nov 10 · Tooling · Junior
If a workflow fails in production but the same workflow succeeds in development, how would you investigate the issue in PowerCenter?
Asked once · last asked Nov 10 · Scenario · Senior
All 76 Data pipeline monitoring and troubleshooting questions
Data migration interview questions
Migration questions cover planning, tool choice, source-to-target mapping, integrity checks, keeping systems in sync, switching systems, rollback, and platform upgrades. 51 questions.
What factors can motivate moving Apache NiFi from on-premises infrastructure to the cloud, and what challenges might arise?
Asked once · last asked Feb 6 · Scenario · Mid-level
What would you want to understand before migrating R pipelines to dbt or PostgreSQL?
Asked once · last asked Sep 9 · Scenario · Mid-level
Other ways it was asked
- “What would you assess, and what issues would you look for, when migrating R pipelines to dbt or PostgreSQL?”
- “How would you approach migrating an R data pipeline to PostgreSQL?”
How do you decide whether to transform data formats when migrating data to Amazon S3?
Asked once · last asked Mar 3 · Scenario · Mid-level
When migrating data, how do you decide whether to ingest the full dataset or simplify it beforehand?
Asked once · last asked Mar 6 · Scenario · Mid-level
How would you check database tables for missing data and compare migrated data with the source file?
Asked once · last asked Mar 6 · Scenario · Mid-level
When migrating from Postgres to MySQL, how would you propagate CDC changes to keep the target system in sync?
Asked once · last asked Mar 20 · Design · Senior
How can you validate data integrity during a cloud migration using checksums or comparisons of the actual data?
Asked once · last asked Apr 14 · Concept · Mid-level
How would you migrate data from Google Cloud Storage to Amazon S3?
Asked once · last asked May 21 · Scenario · Senior
What discovery questions would you ask a customer modernizing an on-premises Hadoop platform with an 18-month Cloudera deadline and no data lineage?
Asked once · last asked Jul 16 · Scenario · Senior
What is the difference between data migration and data integration?
Asked once · last asked Jul 16 · Concept · Mid-level
How would you migrate a large dataset to AWS?
Asked once · last asked Jul 16 · Scenario · Senior
Other ways it was asked
- “Assume you have to migrate 10 TB of data from Oracle to AWS. How would you do it?”
How would you optimize the migration of a large dataset using PySpark?
Asked once · last asked Jul 16 · Scenario · Senior
Related interview questions
- Data Engineer interview questions
- AI Engineer interview questions
- Software Engineer interview questions
- Data Analyst interview questions
- Data Scientist interview questions
- SQL interview questions
- MongoDB and NoSQL interview questions
- Data Modeling interview questions
- Spark, PySpark and Databricks interview questions
- Snowflake interview questions
- Azure Data Factory interview questions
- Power BI interview questions
- Excel interview questions
How this list is made
Every question here was asked by a real interviewer in a real job interview that a candidate ran MeetAssist in. After the call, the interviewer's questions are pulled from the transcript. Practice sessions, tests, exams and anything that was not a job interview are dropped.
Questions are rewritten so they make sense on their own: names of people, employers and clients are removed, and speech-to-text mistakes are fixed. Small talk, logistics and questions about one candidate's own résumé are left out. Rephrasings of the same question are merged, and the count is the number of different interviews it came up in.
The interviews come from candidates who use MeetAssist, so they lean towards software, data and AI roles: the shares describe these interviews, not every ETL and Data Pipelines interview. Answers to the most asked questions are written and checked by an editor; a "common mistake" is shown only when reviews of real answers in these interviews show it, with the count.
307 interviews, 169 candidates, Nov 2025 to Oct 2026. Updated every month with new interviews.
Or add the Chrome extension