Distributed E-Invoice Processing Pipeline

Re-architected a multi-stage e-invoice processing workflow from GCP serverless services to a Dockerized, VPS-based system. The pipeline handles CSV ingestion and validation, UBL XML generation, external document submissions, status polling, and email notifications. I designed Redis and BullMQ workers with retries, backoff, parallel processing, and failure isolation, and added observability for queue health, worker status, API latency, and errors. I also migrated object storage from Google Cloud Storage to MinIO. The work focused on making long-running processing easier to monitor and recover, while separating API, data, and worker responsibilities. Stack: Node.js, Redis, BullMQ, Docker, PostgreSQL, MinIO, OpenTelemetry, Prometheus, Grafana, Loki, and Tempo.