Tag: kafka
Browse 19 practical software engineering resources tagged with "kafka". Discover code recipes, design patterns, documentation templates, and in-depth guides to help you build, deploy, and maintain production-ready solutions involving kafka. Each resource is written for engineers who ship real systems, with copy-paste examples and practical trade-offs.
Apache Kafka for Event Streaming
Apache Kafka is a distributed event streaming platform used for building real-time data pipelines and event-driven applications. It excels at high throughput, fault tolerance, and decoupling producers from consumers.
The recipes below cover producers, consumers, topics, partitions, consumer groups, schemas, exactly-once semantics, and operational concerns. Each guide helps you build streaming systems that are observable and resilient.
Every resource includes clear explanations, copy-paste code, and practical warnings. Use them to make informed decisions, avoid production pitfalls, and speed up your delivery. If you are just getting started, read the beginner-friendly articles first; if you are experienced, jump straight to the advanced patterns and architecture guides. New resources are added regularly, so bookmark this page and check back for the latest patterns.
Dead Letter Queues
Handle failed messages gracefully with dead letter queues, retry policies, and poison pill...
Event-Driven Microservices
Design event-driven microservices with message brokers, event sourcing, CQRS, and eventual...
Event Streaming with Apache Kafka and Node.js
Build growth-ready event-driven systems using Apache Kafka with producers, consumers, consumer...
Kafka Consumer Groups with Python for Scalable Streaming
Create Kafka consumer groups in Python with partition assignment, offset management, commit...
Consume Kafka Topics with Spring Boot Stream Listeners
Build Kafka consumers in Spring Boot using @KafkaListener annotations, concurrent consumers, error...
Message Processing Idempotency
Make message processing idempotent so duplicate deliveries don't trigger side effects in...
Batch-to-Streaming Bridge
How to bridge batch and streaming pipelines with a data lake. Covers Lambda architecture, Kafka...
CDC Pattern: Stream Database Changes to Downstream
How to stream database changes to downstream consumers with CDC. Covers log-based CDC, Debezium,...
Schema Registry Evolution
How to manage schema versions for streaming pipelines with a schema registry. Covers Avro, backward...
Event-Carried State Transfer Pattern
Replicate state changes across services by publishing events that carry the full updated entity...
Idempotent Consumer Pattern
Process messages from a queue exactly once regardless of duplicates by using idempotent operations,...
Sequential Convoy Pattern
Preserve message ordering for related messages in a distributed system by grouping them into...
Dead Letter Queue Runbook
Runbook for handling and replaying dead letter queue messages in Kafka and RabbitMQ: DLQ setup,...
Kafka Topic Naming Convention Template
Template for standardizing Kafka topic names across teams: naming patterns, environment prefixes,...
Complete Guide to Kafka Stream Processing
Build real-time event streaming pipelines with Kafka. Covers producers, consumers, Kafka Streams,...
Real-Time Analytics: From Events to Dashboards in Seconds
A practical guide to real-time analytics: event collection, stream processing, data warehousing,...
Stream Processing: Event-Driven Data Pipelines with
A practical guide to stream processing: choosing between Kafka Streams, Flink, and Spark Streaming,...
Complete Guide to Apache Kafka in Production
Run Apache Kafka in production with confidence. Covers partitions, replication, consumer groups,...
Message Queues — RabbitMQ, Kafka, and SQS detailed analysis
A thorough guide to message queues: when to use RabbitMQ, Kafka, or SQS. Covers patterns,...