Kafka Topic Explained: Definition, Examples, and Key Concepts

Streaming data is everywhere — from payment orders to IoT sensor pings — and at the heart of most event-driven systems sits a simple but powerful abstraction: the Kafka topic. That single named log is the reason companies like Uber, Netflix, and LinkedIn can process over a million messages per second without missing a beat.

Messages per second (typical): 1 million+ ·
Data durability: Persistent, configurable retention ·
Original developer: LinkedIn (2011) ·
Default partition count: 1 per topic

Four facts about Kafka topics, one pattern: the topic is a logical container, but all the real work – ordering, scaling, fault tolerance – happens at the partition level.

The table below maps the key attributes that define how a Kafka topic behaves in production.

The implication: a Kafka topic gives you a durable, replayable event log, but the exact ordering story is more limited than a naive reading might suggest.

What is a topic in Kafka?

How are Kafka topics different from queues?

• A traditional queue (e.g., RabbitMQ, SQS) removes a message after it is consumed; a Kafka topic retains it. Retention is configurable, defaulting to 7 days according to Cloudera’s Kafka Guide.
• Multiple consumers can read the same event independently — Kafka does not delete after consumption. This publish‑subscribe model is a core differentiator described in the Apache Kafka Documentation.
• Ordering in a queue is global (first in, first out); in Kafka ordering is guaranteed per partition, not across the whole topic, as noted by Red Hat’s documentation.

What makes a topic name unique?

• Topic names are globally unique within a Kafka cluster. Convention often reflects the data: orders, page_visits, payment-events, per IBM Cloud’s Event Streams guide.
• A topic name can include a dot or underscore, but naming consistency matters for monitoring and access control — best practices are discussed in community guides.

A topic is similar to a folder in a filesystem, and the events are the files in that folder.

— Apache Kafka Documentation

The trade-off: a topic gives you a named, reusable event stream, but only if you respect that order lives inside partitions, not the topic itself.

What is the purpose of a Kafka topic?

Why use topics instead of direct messaging?

Direct messaging (point‑to‑point) couples producers and consumers tightly. A topic decouples them — a producer writes once, any number of consumers can read later. This is the fundamental architectural choice behind Apache Kafka, as explained in the Apache Kafka Documentation.

How do topics enable decoupling of producers and consumers?

• Producers write to a topic without knowing who (or how many) will read.
• Consumers can start reading from any offset — replaying past data is built in.
• Topics act as a buffer: if a consumer goes down, the topic still holds the data until the consumer comes back, per Cloudera’s Kafka Guide.

The pattern: decoupling via topics is the reason Kafka can serve both real‑time dashboards and nightly batch jobs from the same event stream without any duplication.

What is a Kafka topic example?

Example: order events topic

An e‑commerce platform might define a orders topic. Every time a customer places an order, the frontend publishes an event containing the order ID, items, and timestamp. The topic stores these events sequentially. Downstream services — inventory, billing, shipping — each subscribe independently. This pattern is described in IBM Cloud’s Event Streams documentation.

Example: sensor readings topic

In IoT, a sensor-readings topic ingests temperature, pressure, and humidity readings from thousands of devices. Each reading is a small JSON blob. Because the topic is partitioned (e.g., by device ID), reads and writes scale linearly with the number of brokers. Reference: Conduktor’s glossary on Kafka topics.

What is a Kafka topic partition?

How do partitions enable parallelism?

• A topic is split into partitions. Each partition lives on one broker as the leader, with replicas on other brokers, per Red Hat’s documentation.
• Multiple consumers in the same consumer group can each take one partition — parallelism scales with partition count, as noted by Conduktor.
• The Apache Kafka Documentation confirms that partitions are the unit of parallelism and ordering.

What is the relationship between partitions and offsets?

• Records in a partition are assigned sequential offsets — a unique ID per record within that partition.
• Consumers track their offset to know which records have been read. This allows replay from any point in time, provided data is still retained.
• Offsets are stored in a special __consumer_offsets topic.

A topic is split into one or more partitions, and partitions act as shards of the topic.

— Red Hat Streams for Apache Kafka

The catch: the more partitions a topic has, the more parallel consumers can process — but too many partitions can increase broker overhead and failover time.

What is a Kafka broker?

How do brokers store topic partitions?

• A broker is a server that hosts a subset of partitions for each topic. Cloudera’s Kafka Guide calls brokers “the storage layer.”
• Each partition has a leader broker that handles all reads and writes; follower brokers replicate the data for fault tolerance, per Yandex Cloud’s broker documentation.
• Brokers communicate with each other to reassign leadership during failures — a process also described by IBM Cloud.

What is the role of a broker in topic replication?

• The replication factor determines how many copies of each partition exist. A factor of 3 means three brokers each hold a copy.
• If the leader broker fails, a follower becomes the new leader automatically. This is covered in Google Cloud’s Managed Kafka documentation.

The trade-off: higher replication improves durability but costs more storage and network bandwidth. The default factor is 1; production clusters typically run 3.

Topics can be thought of as logical units, while partitions are the physical units.

— Conduktor Glossary — Kafka Topics Explained

For developers building event‑driven systems, the implication is clear: design topics with a mindset that partitions are the real workhorses. Naming topics thoughtfully and choosing a partition count aligned with expected throughput and consumer parallelism will save significant rework later.

Related reading: Conduktor Glossary — Kafka Topics Explained · Apache Kafka Documentation

Frequently asked questions