Achieving Low Latency with Kafka: Best Practices and Configuration (2025 Guide)

Introduction to Low Latency Kafka

Understanding Kafka Latency: Key Concepts and Metrics

What is Kafka Latency?

• Producer-to-broker latency: Time from message creation to successful write on the broker
• Broker-to-consumer latency: Time from message persistence to consumption

Kafka Latency vs. Throughput

Why Low Latency Matters

• Missed trades in financial applications
• Delayed alerts in IoT and monitoring systems
• Laggy user experiences in gaming or live analytics

Main Causes of Kafka Latency

Producer-side Factors

• Batching (queue.buffering.max.ms/linger.ms): Larger batches increase throughput but add wait time before sending.
• Compression: Reduces network usage but adds serialization/deserialization time.
• Acknowledgment (acks): Waiting for more replicas to acknowledge increases durability but adds delay.

Broker-side Factors

• Replication lag: Time taken for messages to be replicated to other brokers.
• Log flush interval: How often messages are flushed from memory to disk (log.flush.interval.ms).
• Segment size: Smaller segments can help flushes but may increase overhead.

Consumer-side Factors

• fetch.wait.max.ms: Maximum time the broker waits before responding to fetch requests.
• Consumer lag: Delay between message availability and consumption.

Infrastructure

• Network: Latency and bandwidth constraints between producers, brokers, and consumers.
• Disk I/O: SSDs outperform HDDs for log persistence and recovery.
• CPU: Impacts serialization, compression, and network handling.

Kafka Low Latency Configuration: Producer, Broker, Consumer

Producer Configuration for Low Latency

• queue.buffering.max.ms (or linger.ms in newer producers): Lower values reduce waiting time before sending.
• batch.size: Smaller batches decrease latency at the cost of throughput.
• acks: Set to 1 (leader only) for lower latency, but at the cost of durability.

1// Java Producer configuration for low latency
2Properties props = new Properties();
3props.put("bootstrap.servers", "kafka-broker:9092");
4props.put("acks", "1");
5props.put("batch.size", "16384"); // Tune as needed
6props.put("linger.ms", "1");
7props.put("compression.type", "none");
8props.put("max.in.flight.requests.per.connection", "1");
9KafkaProducer<String, String> producer = new KafkaProducer<>(props);
10

Broker Configuration for Low Latency

• log.flush.interval.ms: Lower to flush logs more frequently to disk for faster durability.
• log.flush.interval.messages: Control flush frequency by message count.
• replication.factor and min.insync.replicas: Lower for latency, higher for durability.
• segment.bytes and segment.ms: Smaller segments may help with quicker flushes.

1# Example Kafka broker server.properties
2log.flush.interval.ms=100
3log.flush.interval.messages=1000
4replication.factor=2
5min.insync.replicas=1
6segment.bytes=1073741824
7segment.ms=60000
8

Consumer Configuration for Low Latency

• fetch.wait.max.ms: Lower values reduce wait time for fetch.
• fetch.min.bytes: Lower values ensure quick responses.
• max.poll.records: Adjust for your consumption rate.

1// Java Consumer configuration for low latency
2Properties props = new Properties();
3props.put("bootstrap.servers", "kafka-broker:9092");
4props.put("group.id", "low-latency-consumer");
5props.put("fetch.wait.max.ms", "5");
6props.put("fetch.min.bytes", "1");
7props.put("max.poll.records", "100");
8KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);
9

Network and Operating System Tuning

• socket.blocking.max.ms: Lower to minimize network blocking time.
• OS-level tuning: Adjust TCP settings (e.g., TCP_NODELAY, buffer sizes), disable Nagle's algorithm, and ensure low-latency networking drivers are used.

1# Example OS-level TCP tuning (Linux)
2sysctl -w net.core.rmem_max=26214400
3sysctl -w net.core.wmem_max=26214400
4sysctl -w net.ipv4.tcp_no_delay=1
5

Hardware and Infrastructure Optimization for Low Latency Kafka

Hardware Choices

• SSDs vs HDDs: SSDs provide much better write/read latency for broker storage.
• Network bandwidth: Use at least 10GbE or higher for inter-broker and client communications.
• CPU cores: More cores support higher concurrency for producers, brokers, and consumers.

Cloud vs On-Premises Considerations

• Cloud providers may introduce additional network hops and virtualization overhead. For ultra-low latency, consider on-premises or dedicated cloud infrastructure.
• Use placement groups, provisioned IOPS, and dedicated instances where possible.

Benchmarking for Low Latency

1# Example openmessaging-benchmark command
2./bin/openmessaging-benchmark -t kafka -c config/kafka.yaml -w workloads/low-latency.yaml
3

Balancing Low Latency with Throughput and Durability

• Reduce overall throughput due to increased request overhead
• Lower durability if acknowledgments are not required from multiple replicas

• Use different topics/configurations for latency-sensitive and bulk workloads
• Profile typical and peak loads to find the right balance
• For financial trading, latency is often paramount, while analytics workloads may prioritize throughput

Real-World Latency Optimization Tips and Troubleshooting

• Diagnosing bottlenecks: Use end-to-end tracing to pinpoint where delays occur (producer, broker, consumer, network).
• Monitoring metrics: Integrate Prometheus and Grafana with Kafka for real-time visibility into latency, consumer lag, replication lag, and throughput.
• Common mistakes: Over-batching, excessive replication, or misconfigured network settings often increase latency.
• Quick checklist:
  • Set low linger.ms and fetch.wait.max.ms
  • Use SSD storage
  • Monitor consumer lag and broker replication
  • Benchmark regularly
  • Tune OS and network stack for low latency

Conclusion