| JOB DESCRIPTION Responsibilities: (The primary tasks, functions and deliverables of the role) · Design and build reusable components, frameworks and libraries at scale to support analytics products · Design and implement product features in collaboration with business and Technology stakeholders · Identify and solve issues concerning data management to improve data quality · Clean, prepare and optimize data for ingestion and consumption · Collaborate on the implementation of new data management projects and re-structure of the current data architecture · Implement automated workflows and routines using workflow scheduling tools · Build continuous integration, test-driven development and production deployment frameworks · Collaboratively review design, code, test plans and dataset implementation performed by other data engineers in support of maintaining data engineering standards · Analyze and profile data for designing scalable solutions · Troubleshoot data issues and perform root cause analysis to proactively resolve product and operational issues Requirements: (The abilities that the individual needs to perform this role effectively) Required Preferred Education · Bachelor's degree in IT or related field. · One of the following alternatives may be accepted: PhD or Law + 3 yrs; Masters + 4 yrs; Associates + 6 yrs; High School + 7 yrs exp · 5 years relevant work experience Skills · Strong understanding of data structures and algorithms · Strong understanding of solution and technical design · Has a strong problem solving and analytical mindset · Able to influence and communicate effectively, both verbally and written, with team members and business stakeholders · Able to quickly pick up new programming languages, technologies, and frameworks · Advanced experience building cloud scalable, real time and high-performance data lake solutions · Experience with relational SQL · Experience with scripting languages such as Shell, Python · Experience with source control tools such as GitHub and related dev processes · Experience with workflow scheduling tools · In-depth understanding of micro service architecture · Strong understanding of developing complex data solutions · Experience working on end-to-end solution design · Able to lead others in solving complex problems by taking a broad perspective to identify innovative solutions · Willing to learn new skills and technologies · Has a passion for data solutions Technology/Capabilities Service Comments AWS EMR (Pyspark,SparkSQL),S3,Athena, Hive, SQL Python Batch scripting, data manipulation, distributable packages Orchestration (DAG) Apache Airflow or equivalent Code Versioning Git Hub DevOps/CICD Jenkins Ingestion Nifi, Kafka Familiarity DWH Snowflake, SQL, Data Modelling Reporting Tableau & Cognos Familiarity Data Governance Mdcat, Collibra Familiarity Agile Atlassian JIRA Familiarity ----------------------------------------------------------------------------------------------------------------------------------
Sundeep Diwan
Senior IT Recruiter VBeyond Corporation
www.linkedin.com/in/sundeep-diwan-87a08b227/
+1-908-633-4217
sundeepl@vbeyond.com
Disclaimer:
We respect your Online Privacy. This is not an unsolicited mail. Under Bill S 1618 Title III passed by the 105th US Congress this mail cannot be considered Spam as long as we include contact information and a method to be removed from our mailing list. If you are not interested in receiving our e-mails then please reply to (sundeepl@vbeyond.com) subject=Remove. Also mention all the e-mail addresses to be removed which might be diverting the e-mails to you. We are sorry for the inconvenience | 
Comments
Post a Comment
Thanks