Kafka Glossary
Explore 23 platform engineering definitions related to Kafka.
Kafka partition
Fundamental unit of parallelism and ordering within a topic.
Consumer lag (Kafka)
Difference between latest message and consumer committed offset.
Consumer group (Kafka)
Set of consumers cooperatively consuming a topic.
Consumer offset (Kafka)
Position consumer group has committed as processed.
Effective consumers (Kafka lag)
Actual consumers processing partitions, capped at partition count.
Consumer poll loop (Kafka)
Central control loop: fetch records, process, repeat.
Consumer rebalance (Kafka)
Redistributing partition assignments among group consumers.
Replication factor (Kafka)
Number of brokers storing copies of each partition.
Log retention (Kafka)
How long messages kept before deletion eligibility.
Producer compression (Kafka)
Reduces batch size before sending to broker.
Throughput vs bandwidth (Kafka)
Actual processed rate vs theoretical maximum capacity.
max.poll.interval.ms
Maximum time between poll() calls before consumer considered dead.
max.poll.records
Maximum records returned by single poll() call.
Kafka batch.size
Maximum size of message batch before sending.
kafka linger.ms
How long producer waits to accumulate messages into batch.
replica.fetch.max.bytes
Maximum bytes follower fetches from leader per request.
message.max.bytes
Maximum size of single message batch broker accepts.
Kafka rebalance protocol (Eager vs Cooperative)
Eager stops all consumers; Cooperative only moves needed partitions.
CooperativeStickyAssignor
Incremental cooperative rebalancing with minimal partition movement.
Kafka partition skew
Uneven message distribution across partitions.
Kafka time lag
Elapsed time between message production and consumption.
Kafka claim check pattern
Stores large payloads externally, puts reference in Kafka.
Kafka RecordBatch
Wire format container grouping records for transmission.