Data Engineering Infrastructure and Application Lead(Azure Architect)

 

Data Engineering Infrastructure and Application Lead(Azure Architect)
Location-  Fremont, CA
Duration: C2c/Long Term.

Responsibilities

·         Hands on experience in HPC and Big Data (Hadoop based) platforms and in scoping latest hardware (cloud/on premise)

·         Design and implementation security systems and security technologies

·         Developing virtual private networks (VPNs), routers, firewalls, and related security and network topologies

·         Design of public key infrastructures (PKIs), including use of certification authorities (CAs)

·         Production support and maintenance of Active Directory Infrastructure.

·         Administration of DNS, DHCP

·         Iterate quickly through latest packages and R&D on latest information.

·         Work with Product Management and multiple business units for requirement gathering and iteratively refine architectural proposals

·         Support upstream and downstream use cases with integration with existing Big data ecosystem.

 

Desired Skills and Experience

·         Undergraduate/Ms degree in Computer Science or distributed systems

·         Experience with Linux based OS’s such as CentOS, Redhat, Ubuntu

·         Experience working as infrastructure lead in Azure ecosystem

·         Experience in working with Microsoft Azure services and application architecture and strong knowledge about Even Hub, ADLS, Blob Storage, Data Factory, Data Bricks, SQL WH etc

·         Must have strong experience in Big Data, ETL design and development, methodologies, tools, processes and best practices

·         5 years’ experience in Java/C++/Object Oriented systems programming and/or python, shell scripting (e.g. bash, PowerShell)

·         Experience building and deploying enterprise scale applications, familiarity with GPU computing

·         Understanding of Machine learning, simulation, and massive parallel compute architectures

·         Performance analysis and optimization, experience architecting and deploying highly reliable systems (HA, ACID, guaranteed delivery etc.)

·         Experience with scalable computer architectures (on premise and cloud (private cloud e.g. Openstack))

·         Experience with distributed computing/storage technologies like Hadoop (Map/Reduce, HDFS, HBase, in memory compute etc.), containerization such as docker

·         Understand requirements and goals for distributed computing. Prior exposure to MOAB/SLURM and Cyclecloud strong plusses. Experience with InfiniBand and clustered file systems

·         Design the appropriate computing and storage architecture to meet the customer goals

·         Hands-on engagement in the build out and delivery of the architecture

·         Experience operating in a Hybrid cloud environment and connectivity, prior exposure to Azure IaaS or equivalent required