Data Engineer

Accenture

1. Developed End to End Data Pipeline ingesting data (structured, and semi-structured) from S3 and transformed the data using Databricks (PySpark) Notebooks which also runs Snowflake SQLs and generate Extracts for the vendors after the transformations on S3 and mailed the extracts as an attachment using notebooks.
3. Worked on designing and Implementing solutions for migrating existing applications from Legacy platform (AbInitio), SAS (Statistical Analysis Software) scripts to Cloud (AWS) using Snowflake, AWS, Databricks python scripts(PySpark).
5. Implemented CI/CD pipelines to streamline the development lifecycle, enhacing efficiency and ensuring rapid, raliable releases.
7. Contributed to designing and implementing GDP (Generic Data Parsing) Process - Modern Data Lake Architecture (medallion architecture) in S3 data lake using databricks notebooks.
9. Worked on Replications of tables from on-premises databases to cloud databases using in-house tool.
11. Developed data pipeline using Snowflake to ingest data from S3 data lake, deployed storage Integrations External Stage & Stored Procedure in Python with Tasks (using Cron) to get data in snowflake tables and make dashboards.

NBN Sinhgad School of Engineering

Computer Science and Engineering