Data-Processing-and-Visualization-using-Spark-Cluster-on-Azure

This project provides an end-to-end data processing and visualization of visa numbers in Japan using Azure Cloud Spark Cluster with PySpark, Spark SQL and Plotly. The spark clusters are set up within a Docker container on Azure