Big Data Platform Engineer // Remote (10+ Years Required) // Only USC, GC, H4EAD, GC EAD

Hello,

Please find the below requirement details.

Position: Big Data Platform Engineer (10+ Years exp required)

Location: 100% Remote

Contract Length: 1 year contract to hire

Visa's:  Only USC, GC, H4 EAD, GC EAD

Code Test - 1 Hour

Healthcare Experience REQUIRED!!!

SQL, python, pyspark, scala, Airflow

working with large data sets (100 gigs in size)

building and optimizing end to end pipelines

azure stack - would look at people without this if they are strong engineers

Required:

•        Strong verbal and written communication skills to effectively articulate messages  to internal and external teams.

Hands on experience with Object Oriented programming with Python

Experience with Python, Pyspark, Scala and SQL

Experience with designing, building, optimizing, troubleshooting end-to-end big data pipelines using structured (relational and files) and semi-structured data

Experience with building metadata driven data processing frameworks

Strong experience in SQL, python, Pyspark, Scala and shell scripting.

Experience working with Airflow.

Experience working with big data.

Ability to take ownership of a request from initial requirements, design, development and production deployment

 

Nice to have:

•        Azure

•        Azure EventHubs

•        Apace Kafka

•        Streaming data

•        CosmosDB/NoSQL Database

common data model

workflow - talk to business partners, gather requirements

use azure stack to complete this work

current common data model is a dimensional data warehouse

data factory and functions - not necessarily needed

azure devops - can have something similar like github

all other azure requirements needed

dimensional modeling

SQL - coding

spark - questions on optimization

pyspark - coding

python - coding

shell scripting - if on resume, need to be ready to speak to

 

Thanks & Regards, Saranya Technical  Recruiter

Phone: (302) 204-0565 Email: saranya@imcsgroup.net 9901 East Valley Ranch Parkway Suite 3020 Irving, Texas – 75063

Disclaimer

This electronic mail (including any attachments) may contain information that is privileged, confidential, and/or otherwise protected from disclosure to anyone other than its intended recipient(s). Any dissemination or use of this electronic mail or its contents (including any attachments) by persons other than the intended recipient(s) is strictly prohibited. If you have received this message in error, please notify us immediately by reply e-mail or e-mail unsubscribe@imcsgroup.net so that we may correct our internal records. Please then delete the original message (including any attachments) in its entirety. Thank you