Junior Data Engineer | PySpark, Databricks, Delta Lake & AWS Lakehouse
I build production-grade Lakehouse pipelines that turn messy, unreliable data into clean, trustworthy datasets using PySpark, Databricks, Delta Lake, and AWS.
What I bring:
* End-to-end ETL/ELT pipelines with PySpark, Spark SQL, Delta Lake, and Unity Catalog
* Medallion Architecture (Bronze → Silver → Gold) with data-quality validation
* Dimensional data modeling using Star and Galaxy Schemas
* AWS data ingestion and storage with S3, IAM, and Glue
* Python and SQL for transformation, validation, and data processing
* Version-controlled pipelines using Git/GitHub
Flagship Project — Bayt El-Cash Lakehouse:
I independently designed and built a Medallion Lakehouse pipeline processing 35M+ transaction records across 11 years and 15 branches. Around 40% of the migrated records contained missing or inconsistent data, including balances, phone formats, and dates. I built a stateful ledger-reconstruction engine using PySpark Window Functions and deterministic business rules to recover missing values, then modeled the cleaned data into a Galaxy Schema with automated reconciliation checks.
I’m early in my freelance journey and building my track record, so I offer focused communication, competitive rates, and an honest approach to what I can deliver.
Tools: PySpark, Databricks, Delta Lake, Spark, AWS, SQL/PostgreSQL, Python, Unity Catalog, Git/GitHub