Hiring - Sr. Databricks Engineer (PySpark and Data Lake) Exp - Wilmington, DE location - In person interview

Hello Folks,

 

Hope you are doing great!

 

This is Himank Jani from ApTask.

 

We have urgent requirements with one of our client’s, please review the job description below and let me know if you have any relevant candidates on your bench, Kindly share.

 

Kindly share work authorization status and DL location with resume.

 

Job Title: Sr. Databricks Engineer (PySpark and Data Lake) Exp - 5 Positions

Location: Wilmington, DE – 5 Days onsite role

Long Term Project

 

Local candidate will get the first preference as candidate has to attend the in-person interview in any of JPMC office.

 

Overview

We are seeking a Data Engineer to lead the modernization of legacy ETL systems by migrating Ab Initio workflows to scalable, modular PySpark pipelines on Databricks. The role involves transforming complex data ecosystems into cloud-native architectures while ensuring data integrity, performance, and reliability.

🎯 Key Responsibilities

ETL Modernization & Development

•                      Analyze and migrate legacy ETL workflows from Ab Initio to PySpark-based pipelines

•                      Design and develop scalable data pipelines on Databricks

•                      Refactor monolithic processes into modular, reusable components

•                      Leverage existing enterprise datasets to avoid redundancy

Data Integration & Processing

•                      Build and maintain ETL/ELT pipelines integrating data from Snowflake and other sources

•                      Process and publish enriched datasets for downstream applications

•                      Support batch and near real-time data processing

Data Lineage & Optimization

•                      Create end-to-end data lineage and data flow diagrams

•                      Identify redundancies and drive process consolidation and optimization

•                      Ensure adherence to data governance and quality standards

Testing & Validation

•                      Develop unit, integration, and reconciliation frameworks

•                      Perform dual-run comparisons with legacy systems

•                      Validate outputs in UAT and pre-production environments

Deployment & Operations

•                      Support cutover and migration strategy from legacy systems

•                      Decommission legacy workflows and optimize scheduling (e.g., Control-M)

•                      Develop runbooks, monitoring, and operational documentation

Collaboration

•                      Work with data architects, analysts, and downstream application teams

•                      Coordinate user acceptance testing (UAT/FAT) and stakeholder sign-offs

 

 

 

 

 

 

Best Regards,

Himank Deepak Jani

ApTask | A global, diversity-certified workforce solutions provider.

Address: 120 Wood Ave South, Suite # 300, Iselin, NJ 08830

 

This e-mail and any attachments may be confidential, proprietary or legally privileged. Any review, use, disclosure, distribution or copying of this e-mail is prohibited except by or on behalf of the intended recipient. If you received this message in error or are not the intended recipient, please delete or destroy the e-mail message and any attachments or copies and notify the sender of the erroneous delivery by return e-mail. It shall not attach any liability on the sender or ApTask or its affiliates. Any views or opinions presented in this email are solely those of the sender and may not necessarily reflect the opinions of ApTask or its affiliates.

 

Candidate Data Collection Disclaimer:
At ApTask, we prioritize safeguarding your privacy. As part of our recruitment process, certain Personally Identifiable Information (PII) may be requested by our clients for verification and application purposes. Rest assured, we strictly adhere to confidentiality standards and comply with all relevant data protection laws. Please note that we only collect the necessary information as specified by each client and do not request sensitive details during the initial stages of recruitment.

If you have any concerns or queries about your personal information, please feel free to contact our compliance team at compliance@aptask.com.

Applicant Consent:
By submitting your application, you agree to ApTask's (www.aptask.com) Terms of Use and Privacy Policy, and provide your consent to receive SMS and voice call communications regarding employment opportunities that match your resume and qualifications. You understand that your personal information will be used solely for recruitment purposes and that you can withdraw your consent at any time by contacting us at 732-355-8000 or help@aptask.com. Message frequency may vary. Msg & data rates may apply.