Work Experience and Skill Levels

Experience

  • Added page for tracking form responses on admin dashboard UI.

    Updated profit margin for 30,000 grocery items using boto3 and DynamoDB.

    Tech Stack: AWS Amplify, Athena, React.js, JavaScript, DynamoDB, Python,

  • Managed data pipeline, data streams and updates for one human study's data.

    Automated plot generation to share weekly updates with study stakeholders.

    Optimized existing testing processes for team to reduce chances of manual error.

    Tech Stack: Python, SQL, PySpark, Pandas, Matplotlib, Jupyter notebooks, Airflow, Delta tables, Partitioning

  • Managed project that automated monthly accounting metric transformations.

    Migrated 10+ existing ETLs to an updated version of spark and EMR jobs.

    Presented to the entire Engineering team on Amazon Redshift.

    Developed ETL jobs that collect and aggregate data from the resale platform into datasets for business analytics teams, using Airflow and EMR clusters.

    Mentored 1 new hire and 2 interns and created documentation of development process to expedite onboarding.

    Tech Stack: Scala, MySQL, Spark, Airflow, Amazon Redshift, Amazon S3, AWS EMR clusters, DataBricks, Hive Metastore, Kinesis Streaming, Ruby, APIs

Data Engineering

Full Stack