Sarath B

Data Engineer

Irving, TX, USA

Followers

Following

With eight years of extensive and diverse experience as a data engineer in delivering innovative 
solutions across various domains and industries. Expertise in data modeling, designing and maintaining 
ETL pipelines using cloud services like Azure, AWS, and big data technologies like Hadoop, Kafka, Spark 
to tackle complex business challenges. Well-versed in a wide array of cloud services, including Azure 
Data Factory, Databricks, and AWS Services like Athena, Glue, Lambda, and more. Proficient in various 
SQL databases like MySQL, SQL Server and NOSQL databases such as DynamoDB, Cosmos DB, and 
Oracle DB, and utilized data warehouses like Azure Synapse, AWS Redshift, and Snowflake, and adept 
at creating impactful dashboards using visualization tools like QuickSight, Tableau, and Power BI. 
Possesses a strong automation skill set, leveraging Airflow, Bash scripting, and Cron jobs to streamline 
workflows. Experienced in Spark applications, SQL, Pyspark, Agile methodologies, and influencing and 
building strong relationships with stakeholders and architecture teams. Also well-versed in Hadoop 
ecosystem components and CI/CD pipeline tools.

Careers

Senior Data Engineer

Eli Lilly

Full time contract12/2022 -

Developed batch pipeline by ingesting data from, On-premises SQL Server
and Oracle Database into Data Lake Storage Gen2 using Azure Data Factory.
 Used transformation scripts in Data Factory by invoking Azure Databricks
notebooks to perform data processing, data cleansing, and profiling using
Spark SQL and PySpark.
 To Store and manage huge volumes of transformed and clean data, utilized
Azure Synapse analytics data warehouse.
 Developed reusable data pipelines in Azure Data Factory to extract and load
data from Synapse analytics into Snowflake.
 Performed complex SQL queries in Snowflake to extract valuable insights from
data stored and make data readily available for analysis.
 Integrated Apache Kafka with Azure Stream analytics to enable real-time
data streaming and analytics.
 Stored aggregated data sets into Azure Data Lake Storage Gen2, allowing for
near real-time insights and analysis.
 Staged the API and Kafka Data (JSON format) into Snowflake DB by
Flattening same for functional services.
 Implemented data migration of multilevel state data from SQL server to
owflake using Python and SnowSQL.
 Configured and optimized Snowpipe to efficiently handle high-volume data
streams, ensuring timely and accurate data ingestion into Snowflake data
warehouse.
 Scalable data warehouse architecture was implemented by utilizing Delta
Lake to ensure data quality.

Welcome to Outdefine

A free tokenized community dedicated to connecting global tech talent with remote job opportunities. Our platform is designed to help you connect, learn, and earn in the tech industry while providing the chance to collect DEF tokens. Join our vibrant community today and explore a world of possibilities for your tech career!

Join for free

Skills

SQLPythonGithubAzureAWS LambdaDatabricksMySQLPysparkNoSQL, HadoopDatabase Management

Experience5-8 years

Hourly rate$65/hr

Open to

remotehybridonsite