- The charging hardware publishes events to
EV_charging_topic. - The station status system streams metrics to
station_metrics_topic.
Key Features of Kafka Topics
| Feature | Description | Benefit |
|---|---|---|
| Message Categorization | Logical grouping of related messages | Keeps streams organized by data type |
| Immutable Log | Append-only, time-ordered storage | Guarantees ordered, auditable event replay |
| Multi-Consumer Access | Independent consumer groups with individual offsets | Enables parallel processing without conflicts |
| Decoupled Communication | Producers and consumers operate independently | Simplifies scaling and fault isolation |
| Replication | Partitions duplicated across multiple brokers | Provides high availability and durability |
1. Message Categorization
Each topic acts as a container for messages of a similar kind—whether they’re logs, IoT data, or transaction records. You can create dozens or even hundreds of topics to keep your data streams organized.Managing tens of thousands of topics can increase metadata overhead on the brokers. Monitor performance and adjust
num.network.threads or num.io.threads as needed.2. Immutable Log
Internally, a topic is stored as an append-only log. Once a message is written, it cannot be altered—only new messages can be appended. This ensures consumers always read events in the order they were produced.Use retention settings (
retention.ms, retention.bytes) to control how long messages remain available in the log. Align retention with your consumers’ processing speed to prevent data loss.3. Multi-Consumer Access
Kafka tracks each consumer group’s offset—the position in the log it has read. Multiple groups can independently consume the same topic, enabling use cases such as real-time analytics, monitoring dashboards, and batch processing on the same data.4. Decoupled Communication
Producers push messages to a topic without knowing who will consume them. Consumers pull messages at their own pace. This decoupling allows producers and consumers to scale, update, or fail independently without impacting each other.5. Replication
To ensure fault tolerance, Kafka replicates each partition of a topic across multiple brokers. If one broker fails, another replica can take over with no data loss. We’ll cover replication in more detail in a future lesson.