Automated Weather Data ETL Pipeline

Built an automated ETL pipeline for collecting, processing, and storing weather data.

The workflow integrates multiple data engineering tools to automate data ingestion and transformation. Weather data is collected from external sources, processed through the pipeline, and stored in a database for further analysis.

The project demonstrates workflow orchestration and automated data processing using Airflow, Airbyte, NiFi, Python, and MySQL.

Technologies used:

Python, Apache Airflow, Apache NiFi, Airbyte, MySQL, ETL.

The project focuses on reducing manual data handling by creating a structured and automated data pipeline.