Lead Data Engineer – Databricks@ Watsonville, CA

Greeting from Excel-EST!!
 
We are looking for Lead Data Engineer – Databricks @ Watsonville, CA
for the opening with one of our clients.
 
 
Job Title: Lead Data Engineer – Databricks
 
Location: Watsonville, CA / Remote
 
Type: Contract
 
Qualifications
JOB RESPONSIBILITIES:

Required
  • Minimum 7+ years of experience in design, development, and deployment of large-scale, distributed, and cloud-deployed software services.
  • Bachelor’s in Computer Science or related disciplines
  • Must have AWS Data Engineer with Databricks and Delta lake experience.
  • Must have deep understanding of Spark, AWS (specifically Redshift, S3, Glue, Athena) and should be having practical exposure on executing multiple projects in Delta lake
  • Must have been part of minimum 2 end to end big data projects and must have handled defined modules independently.
  • Expert in SQL and good with data modelling for relational, analytical and big data workloads.
  • Advanced programming skills with Python, Scala or Java.
  • Strong knowledge of data structures, algorithms, & distributed systems.
  • Strong experience and deep understanding of Spark internals.
  • Expert in Hive.
  • Hand on experience with one of the cloud technologies (AWS, Azure, GCP).
  • Hands on experience with at least one NoSQL database (HBase, Cassandra, MongoDB etc).
  • Experience in working with both batch and streaming datasets.
  • Knowledge of at least one ETL tool like Informatica, Apache NiFi, Airflow, DataStage etc.
  • Experience in working with Kafka or related messaging queue technology.
  • Hands on experience in writing shell scripts for automating processes.
  • Willingness to learn and adapt.
  • Delivery focused and willingness to work in a fast-paced work environment.
  • Takes initiative and responsibility for delivering complex software.
  • Knowledge of building REST API end points for data consumption.
  • Excellent oral and written communication is a must.
  • Well versed with Agile methodologies and experience in working with scrum teams.
Preferred
  • Master's in Computer Science or related disciplines
  • Experience building self-service tools for analytics would be plus.
  • Knowledge of ELK stack would be a plus.
  • Knowledge of implementing CI/CD on the pipelines is a plus.
  • Knowledge of Containerization (Docker/Kubernetes) will be plus.
  • Knowledge of building RESTful services would be an added advantage.
  • Experience working with one of the popular Public Cloud based platforms.

I would appreciate if you could reach out on my below mentioned contact coordinates, should the position be of interest to you.
 
Have a great day ahead!!
Regards,
 
Alam Rahman
Lead Recruiter
Excel-EST, LLC
Phone: +1-214-550-6407
17290 Preston Rd, Suite 300-F, Dallas TX, 75252
 
Disclaimer: We respect your Online Privacy. This is not an unsolicited mail. Under Bill S 1618 Title III passed by the 105th US Congress this mail cannot be considered Spam as long as we include Contact information and a method to be removed from our mailing list. If you are not interested in receiving our e-mails then please reply to AlamR@ExcelEST.com subject=Remove. Also mention all the e-mail addresses to be removed which might be diverting the e-mails to you. We are sorry for the inconvenience.
Excel-EST, LLC | www.excelest.com

Comments

Popular Posts