Apache Kafka Cluster Setup for Big Data

Designed and deployed an Apache Kafka cluster to handle up to 8 billion records/day.

Tasks included:

Setting up and configuring multi-node Kafka clusters.

Performance tuning and monitoring.

Implemented log retention strategies and topic partitioning for high throughput.

Technologies:

Apache Kafka, Linux, Prometheus, Grafana