In this notebook, there will be 3 main questions that will be addressed. For the 3 questions, I have utilised Google Colab as my IDE, and have done the programming on Colab. I have put in my thought ...
variables: s3PysparkScriptBucket: "TO_BE_SPECIFIED" ## S3 script bucket name in Dev env ex: "s3://lfg-151324446874-use1-gluescripts-bucket/pyspark/dev ...