Professional Summary
Certified Databricks Data Engineer Associate with 2 years of hands-on experience in building scalable data pipelines and data warehouse solutions using Apache Spark, Python, SQL, AWS, and modern data warehousing tools. Passionate about software development and data engineering, with strong expertise in programming, data architecture, and ETL design. Proven ability to develop robust, efficient ETL processes that ensure data integrity and minimize anomalies in cyclical workflows.
Work Experience
Freelance
Data Engineer
Sept 2024 - Present
Remote
- Successfully designed and delivered the SmartStream ETL Pipeline in the Data Engineering domain using AWS.
- Optimized ETL pipelines in AWS Glue for orchestration, leveraging S3 to improve data performance and scalability.
- Transformed data using PySpark and Data Flow, applying Spark configurations for memory management.
- Enhanced big data processing efficiency by implementing bucketization and broadcast joins, achieving an approx 45% reduction in processing time while ensuring seamless data distribution and scalability.
R Systems International
Data Engineer
July 2023 - July 2024
Hybrid
- Built execution flow of raw data through Lambda and Step Functions and monitored multiple services in AWS.
- Engineered and implemented an efficient ETL pipeline in Databricks, ensuring reliable and scalable data processing.
- Developed and optimized code in Databricks Workflow to enhance performance, improving job efficiency by 40% and significantly reducing processing time from 7 hours to 1.5 hours.
- Collaborated in Dimension Modelling, conducted RCA for pipeline failures, identified root causes and implemented corrective measures, increasing pipeline accuracy by 35%, ensuring seamless and consistent data delivery.
Codencreative
Frontend Developer Intern
July 2023 - Sept 2023
Remote
- Crafted engaging and interactive UIs for diverse projects using HTML, CSS, JavaScript, Bootstrap, and React.
- Converted frontend codebases into React, enhancing reliability and scalability by 40% using multiple React hooks.
- Contributed to an e-commerce app, focusing on UI development and enhancing user experience using css.
Key Projects
F1 Racing
An F1 Racing Data Engineering Project for structuring data in optimised way made up by using ETL pipeline using Databricks and Azure.
Key Features:
Technologies:
IPL Data Analytics
The IPL Data Analysis project efficiently structured the Data of cricket stats, enabling deep analysis across regions and roles.
Key Features:
Technologies:
Achivements
Databricks
Certified Databricks Data Engineer Associate.
AWS Solution Artitect Associate
Completed and Earned Certificate from Udemy of AWS Solution Artitect Associate.
Hackathon
Hackathon - AutoISV Innovation Quest - Rank 3.
Codekaze
CodeKaze - Global Coding Event by Coding Ninjas Secured College Rank 1.
Techgig
Techgig - Open Coding Contest Secured Global Rank 317 over 50,000 + Participants.
Coding Profile
Data Structure & Problem Solving : Leetcode, Codechef, GeeksforGeeks. Solved 800+ Problems.
Technical Skills
Language
Big Data
Cloud
Tools
Education
Bachelor of Technology
Computer Science Engineering
Amritsar College of Engineering & Technology
2019 - 2023
CGPA: 7.77
Key Subjects: Data Structures & Algorithms, Web Development, Big Data, Software Engineering
Senior Secondary (XII)
Science (Mathematics)
St. Xavier Jr./Sr. School
2016 - 2018
73.2%