Hello Partner,
Hope you are doing good.
Position: Big Data Engineer
Location: Ashburn, VA
Duration: 1+ year
- Skill set (looking for candidates with Pyspark and NiFi experience)
- Solid experience in Hadoop
- Automation and Scripting experience.
Description:
Qualifications:
1. At least 3 years working experience in a Big Data Environment
2. Knowledge on design and development best practices in datawarehouse environments
3. Experience developing large scale distributed computing systems
4. Knowledge of Hadoop ecosystem and its components – HBase, Pig, Hive, Sqoop, Flume, Oozie, etc.
5. Experience of Pyspark & Spark SQL
6. Experience with integration of data from multiple data sources
7. Implement ETL process in Hadoop (Develop big data ETL jobs that ingest, integrate, and export data.) Converting Teradata SQL to PySpark SQL.
8. Experience in Presto, Kafka, Nifi.
Roles and Responsibilities:
1. Understanding business requirements & helping in assessing them with the development teams.
2. Creating high quality documentation supporting the design/coding tasks.
3. Participate in architecture / design discussions and Develop the ETL/ELT using PySpark and SparkSql
4. Conduct code and design reviews and provide review feedback.
5. Identify areas of improvement in framework and processes and strive them to make better.
Thanks & Regards,
Ram.m | IT Recruiter | Techgene Solutions, LLC.
O: 972-580-0247 Ext: 220 | E: ramu.m@techgene.com
G-Talk: ram.recruiter121@gmail.com
300 East Royal lane, suite # 109, Irving, Texas – 75039
Fax: (888)-361-1774 | www.Techgene.com
