Data Engineering interview question · Most asked
Which data quality checks would you put on a critical pipeline?
Summary
Schema checks, uniqueness and not-null on the grain key, not-null and range checks on key measures, accepted values, referential integrity, freshness and volume against history, and reconciliation of totals with an independent source. Critical failures block publication; others warn the owner.
Longer Answer
Tools: dbt tests, Great Expectations, Soda, or data observability platforms. Start with a small set that would have caught past incidents and add checks after each new incident to keep noise low.
Key Points
- Grain uniqueness and key not-nulls.
- Freshness and volume.
- Ranges, accepted values, relationships.
- Reconcile with an independent source.
Deep dive: ETL and Data Pipelines: Data Quality Checks and Data Contracts
Explanation
What interviewers look for
A prioritised, actionable set with severities and owners.
More most-asked Data Engineering questions
- Batch or streaming: how do you decide?
- What is the difference between ETL and ELT, and when would you choose each?
- What is change data capture (CDC), and why is log-based CDC preferred over polling?
- What does it mean for a data pipeline to be idempotent, and how do you achieve it?
- How do you implement incremental extraction, and what can go wrong?
- How do you deduplicate events, and what does 'exactly-once' really mean?
- What is a star schema, and why is it used in data warehouses?
- What is the grain of a table, and what is join fan-out?