Large-Scale Data Transfer System – .NET & Kafka Microservices

As a senior back-end developer at Cankash.net, I designed and built a distributed ETL system that processes over 100 billion records daily (maximum 9 billion records per instance) from telecom network data.

Key achievements:

Developed and deployed 5+ background processing services using .NET Framework & .NET 6 and higher.

Transferred data to Apache Kafka, HPE Vertica, ClickHouse, and FTP servers.

Integrated RESTful APIs to control data access and operations.

Technologies:

.NET, Apache Kafka, HPE Vertica, ClickHouse, Docker, Prometheus, Grafana.