airflow pipeline

### Retail ETL Pipeline | Python & Apache Airflow

Developed an end-to-end ETL pipeline for a retail sales dataset using Python and Apache Airflow 3.

**Key Features:**

- Extracted order data from CSV files and product information from JSON files.

- Implemented data validation to identify duplicate orders, missing or invalid quantities, and unknown products.

- Cleaned and joined order and product data to calculate sales totals.

- Loaded validated sales records into a SQLite data warehouse using an idempotent loading process to prevent duplicate records.

- Generated a daily sales report showing order counts and revenue by date and product category.

- Implemented data quality checks and task dependencies using Airflow's TaskFlow API and DAGs.

- Configured scheduled execution, retries, and error handling for reliable workflow orchestration.

**Technologies:** Python, Apache Airflow 3, SQL, SQLite, CSV, JSON, ETL, Data Quality, Workflow Automation.

**Project Outcome:** Processed 11 source order records into 7 clean sales records, rejected 4 invalid records, and generated automated sales reporting outputs.