Apache Druid for data engineering interviews

Why interviewers ask about Druid

Apache Druid is the database that powers the real-time analytics tier at Netflix, Airbnb, Confluent, and a long tail of ad-tech companies. When a data engineering interviewer brings it up, they are rarely checking whether you have memorized the docs. They want to know whether you can reason about a system that sits between OLAP and streaming, that ingests from Kafka one second and serves a dashboard the next.

A typical scenario is this. Your platform team asks you to design a system where click-stream events land in a topic and a product manager refreshes a dashboard every 30 seconds expecting sub-second latency on billions of rows. You can stitch this together in Spark plus Postgres and watch it fall over by Friday, or you can pick a purpose-built engine and explain why. Druid is the canonical answer to this question, which is exactly why it shows up on DE interview loops at companies running event-driven products.

The narrower reason to study Druid is that the patterns it uses — segments, deep storage, tiered services — also show up in Pinot, BigQuery's storage layer, and the column-store half of any modern lakehouse. Learning Druid pays compounding interest across other interviews too.

Architecture in one diagram

Druid is intentionally multi-tier. Each process type does one job, and most production deployments scale them independently. This is the same design philosophy you see in Snowflake and BigQuery, just exposed as separate JVMs you operate yourself.

Two external dependencies make the whole thing work. A metadata store (Postgres or MySQL) holds the catalog of segments, datasources, and tasks — think of it as the source of truth that the Coordinator reads from. And a deep storage layer (S3, GCS, or HDFS) holds the canonical copies of every segment, so a historical node failing simply means the Coordinator reassigns its segments to a peer.

How segments work

A segment is the atomic storage unit in Druid. Every segment covers a contiguous time range — for example one hour or one day of a single datasource — and is stored as a self-contained file containing column data, dictionaries, and bitmap indexes.

datasource=events
segment id=events_2026-05-25T00:00:00Z_2026-05-25T01:00:00Z_v1_0
size on disk=180 MB
rows=42,000,000
columns=user_id, event_type, country, revenue

Inside a segment, data is stored column by column, not row by row. String columns are dictionary-encoded — every distinct value gets an integer id, and a bitmap index per value lets Druid answer WHERE country = 'US' by AND-ing two bitmaps instead of scanning a column. Numeric columns are stored as compressed arrays. The metric columns (the ones you defined as sum, count, min, etc.) are pre-aggregated at ingest time if you opt in to rollup, which is what makes the difference between a dashboard that loads in 200ms and one that times out.

Segments are immutable. You never UPDATE a row in Druid. When data needs to change, you re-ingest the affected time window and Druid swaps the new segments in atomically. This sounds limiting until you realise it is the reason every query is cache-friendly and every node is trivially replaceable.

Ingestion patterns

Druid supports three ingestion modes, and interviewers will expect you to know which one to reach for.

Real-time ingestion uses the Kafka or Kinesis indexing service. A MiddleManager spawns one task per partition, consumes events, builds an in-memory segment, and periodically flushes it to disk and then to deep storage. End-to-end latency from a Kafka write to a queryable row is typically 5-30 seconds. The classic gotcha here is that your task count must equal or exceed your topic partition count, otherwise you silently lag.

Batch ingestion reads Parquet, CSV, ORC, or JSON files (or rows from a JDBC source) and emits segments in one shot. This is the right tool for backfills and for daily loads where freshness in minutes is good enough.

Compaction tasks run on a schedule and merge many small segments into fewer larger ones. Without compaction, a busy real-time pipeline produces thousands of 50MB segments per day and your queries slow to a crawl from per-segment overhead.

The shape of an ingestion spec is worth memorizing because interviewers love to ask "what does it look like":

{
  "type": "kafka",
  "spec": {
    "dataSchema": {
      "dataSource": "events",
      "timestampSpec": { "column": "event_time", "format": "iso" },
      "dimensionsSpec": { "dimensions": ["user_id", "event_type", "country"] },
      "metricsSpec": [
        { "type": "count", "name": "events" },
        { "type": "doubleSum", "fieldName": "amount", "name": "revenue" }
      ],
      "granularitySpec": {
        "segmentGranularity": "HOUR",
        "queryGranularity": "MINUTE",
        "rollup": true
      }
    },
    "ioConfig": {
      "topic": "events",
      "consumerProperties": { "bootstrap.servers": "kafka:9092" },
      "taskCount": 8,
      "replicas": 2
    }
  }
}

The three knobs you will get asked about are segmentGranularity (how big each segment is on disk), queryGranularity (the resolution of the timestamp after rollup), and rollup itself. If queryGranularity is MINUTE, every event in the same minute with the same dimension values collapses into one row at ingest. On real click-stream data this gives 10-50x compression for free.

Querying with Druid SQL

Since version 0.10, Druid speaks SQL through a parser that translates queries into its native JSON request format. For interviews you should know both, but in production code SQL is almost always the right choice.

SELECT
  event_type,
  country,
  SUM(revenue)  AS revenue,
  SUM(events)   AS events
FROM events
WHERE __time >= TIMESTAMP '2026-05-01'
  AND __time <  TIMESTAMP '2026-05-25'
  AND country IN ('US', 'CA', 'GB')
GROUP BY 1, 2
ORDER BY revenue DESC
LIMIT 100;

Two things are non-obvious here. First, the __time column is mandatory — every Druid datasource has it, and every query should filter on it. A query without a time filter scans every segment and is the single biggest cause of "why is Druid slow" tickets. Second, Druid's query planner picks between four engine paths under the hood — Timeseries, TopN, GroupBy, Scan — and the path you get depends on the shape of your SQL. A query that groups by one dimension and asks for the top 100 by a metric will use the TopN engine, which is 5-10x faster than the general GroupBy. Adding an unnecessary ORDER BY on a second column can quietly downgrade you.

Druid vs ClickHouse

This is the comparison interviewers ask about most often, because both are open-source columnar analytics engines and they overlap heavily in marketing material. The honest answer is that they are optimised for different shapes of workload.

A useful heuristic: if the workload is time-bounded slice-and-dice on event streams with sub-second SLO, Druid wins. If the workload is arbitrary analytical SQL over a denormalized warehouse with joins, ClickHouse wins. Teams that pick wrong tend to either fight Druid's join limitations forever or fight ClickHouse's ingest lag forever.

Common pitfalls

The first pitfall is forgetting the time filter. Druid is built around the assumption that every query touches a bounded time range, and the segment pruner uses that range to skip 99% of disk. A query like SELECT COUNT(*) FROM events GROUP BY country with no WHERE __time clause will scan every segment ever ingested, which on a year of click-stream data is hours of work for a question you could answer in seconds.

The second pitfall is misconfigured rollup. Rollup is on by default and it pre-aggregates rows at ingest by the dimension set you declare. If you accidentally include a high-cardinality field like session_id or request_id in dimensions, every row becomes unique and rollup compression collapses to 1x. The fix is to drop those columns from the dimension list or store them in a separate, non-rolled-up datasource.

A third pitfall hits during scaling: too many small segments. Real-time ingest cuts a fresh segment every time it flushes, so a busy datasource accumulates thousands of 20-50MB files per day. Each one costs per-segment query overhead and memory on the broker. The fix is to schedule automatic compaction on each datasource targeting 500MB-1GB segments, and to monitor the average segment size in the web console.

The fourth pitfall is assuming Druid behaves like a transactional database. There is no row-level update, no UPSERT, and no foreign keys. If the source data corrects yesterday's records, you re-ingest yesterday's hour. Teams that do not internalize this build elaborate workarounds and slowly stop trusting their dashboards.

The fifth, more subtle pitfall is the broker as bottleneck. Brokers merge results from every historical that holds a relevant segment, and on a fan-out of hundreds of segments the merge step becomes CPU-bound on a single broker process. The fix is horizontal — add more brokers behind the router — but the symptom is easy to misdiagnose as a historicals problem.

Related reading

• ClickHouse MergeTree for data engineering interviews
• ClickHouse as a data warehouse
• Snowflake vs BigQuery for data engineers
• Kafka for data engineering interviews
• Apache Flink for data engineering interviews

If you want to drill questions like this on a daily cadence — segments, ingest specs, query plans, real interview write-ups — NAILDD is launching with 500+ data engineering problems built around exactly these patterns.

FAQ

Is Druid still actively developed in 2026?

Yes. Apache Druid ships roughly quarterly releases. The active commercial sponsor is Imply, and Pinterest, Netflix, and Confluent maintain meaningful contributions. The release notes are a good health barometer.

When should I prefer Pinot over Druid?

Apache Pinot, originally from LinkedIn, occupies almost the same niche. The honest summary is that Pinot has slightly better upsert support and a slightly cleaner deployment story for Kubernetes, while Druid has a more mature SQL layer and a larger ecosystem of connectors. For a greenfield interview answer, either is defensible — what matters is that you can justify the choice based on the workload.

How big a Druid cluster do I need to handle 100k events per second?

A rough sizing rule is one MiddleManager task per Kafka partition and one historical CPU core per 10-20 segments served concurrently. For 100k events/sec on a topic with 32 partitions, plan for at least 32 MiddleManager tasks, 6-10 historical nodes with 32GB RAM each, and 2-3 brokers. Real numbers depend on row width and rollup ratio — measure on a one-week pilot before committing to capacity.

Can Druid replace my data warehouse?

Almost never. Druid is excellent at slicing event tables by time and dimensions, but it cannot do the multi-table joins, semi-structured analytics, and ad-hoc SQL that a warehouse like Snowflake or BigQuery handles natively. The right pattern is Druid for sub-second dashboards on event streams, warehouse for everything else, and they share underlying data via the same lake.

What does a Druid interview round usually look like?

Two flavours. The first is a design question: "build me a real-time analytics platform for X" — here the interviewer wants to see you pick Druid for the right reasons and reason about ingestion, segment sizing, and broker fan-out. The second is a debugging question: "this query is slow, what would you check" — and the expected answer walks through time filter, segment count, rollup configuration, and the query path. Both reward depth over breadth.

How does Druid handle late-arriving events?

At ingest time, late events that fall into an already-flushed segment are routed into a new segment for the same time window — Druid does not reject them. The Coordinator eventually compacts overlapping segments into one. The catch is that until compaction runs, your historical queries on that time range fan out to more segments than necessary, so monitor late-event rate and tune compaction frequency accordingly.