Data Engineer

DocuSign

1. Designing and implementing data pipelines using Snowflake ensuring efficient data storage and retrieval
2. o Using dbtTM to develop and maintain data models and transformations, ensuring data consistency and quality
3. o Orchestrating data workflows and scheduling tasks using Airflow, improving data processing efficiency
4. o Developed Python scripts to access and retrieve data from various B2B data providers, including Dun & Bradstreet, through their REST APIs for Master Data Management (MDM) purposes. This involves authentication setup, endpoint requests, rate limiting, transformation, error handling, and seamless data loading into MDM systems.

Principal Data Engineer

Prescience Decision Solutions

1. o Researching, designing, and developing ETL and Data Pipeline components and visualization reports required for the analytical of the E-commerce domain
2. o Writing complex queries on PostgreSQL/ Redshift for the Data Pipelines and Reports
3. o Automated customer segmentation analysis for marketing activities using Python, pandas, and NumPy,
4. resulting in a 63% increase in customer conversion rate
5. o Developed and optimized PySpark data processing pipelines for batch processing
6. o Deploying the analytics code using AWS Services like Lambda, Glue, S3, Redshift, Cloud Watch
7. o Successfully implemented open-source software and tools for both internal use cases and client projects,
8. resulting in a 20% reduction in budget
9. o Mentoring the junior engineers and guiding the team members on the assigned tasks migrating of the
10. SAS Analytics code to Pyspark code / Spark SQL using Azure Databricks and Azure Services

Data Engineer

Tata Consultancy Services

1. o Responsible for the ETL and Data Engineering tasks on the product development
2. o Complex transformation logic using PostgreSQL and Spark SQL / Pyspark / SQL
3. o Implemented monitoring Proof of Concept to reduce manual intervention by 90%
4. o Designing Airflow DAG for ETL Workflow Orchestrations, Job Automation using shell scripts
5. o Working on Product Division of TCS Digital Software and Solution Group. Our group develops
6. various analytical products in different domains
7. o Implementation of Digital Persona for the customer using K-Mean clustering algorithm using Apache
8. Spark, Apache Airflow
9. o Data Streaming and Integration for the various B2B applications using Messaging Queue (MQ) Using
10. various adapters for Data ingestion and Data Streaming
11. o Responsible for managing the production team for the seamless data steaming between applications

Pondicherry Engineering College

Computer Engineering