Data Engineer

Accenture

1. • Tools and Technologies: Azure Data Factory, Azure Data lake, Databricks, Bigquery, Python, Visual Studio
2. • Migrated database objects from Microsoft Synapse to Bigquery, including Stored Procedures and Views using Azure Data Factory.
3. • Developed Pyspark notebooks in Databricks to ingest data from various sources and created driver notebooks to fetch GBQ environment config files when running ADF pipelines.
4. • Analyzed triggers to fetch GBQ environment config and source files while running ADF pipelines, and applied transformation rules to source/target application data.
5. • Built pipelines using Azure Data Factory and created transformation logic to meet business requirements.
6. • Conducted end-to-end testing of applications to ensure functionality aligns with business requirements.

Data Engineer

Accenture

1. • Tools and Technologies: Spark , HDFS, ETL pipeline, HIVE
2. • Configured Hive database to store and manage data, optimizing
3. database performance and scalability, and ensuring data consistency
4. and accuracy.
5. • Setup a ETL pipeline using spark to extract data from multiple
6. sources and load it into hive database.
7. • Developed transformation code to manipulate the data stored in
8. the Hive database according to specific requirements, incorporating
9. data quality and validation checks to ensure the accuracy
10. and completeness of the processed data.

Hadoop Developer

Food Pine

1. • Tools and Technologies: HDFS, Hive
2. • Designed and developed data pipelines using Hadoop ecosystem
3. technologies to process and analyze large volumes of data from
4. multiple sources, including transactional systems, log files, and
5. customer feedback.
6. • Developed and optimized Hive queries to transform and aggregate
7. data, enabling faster and more efficient analysis of business
8. performance and customer behavior

Illinois institute of technology

Computer Scinece