Urgent Need || Big Data Developers || Nashville, TN

Title: Big Data Developers
Position Type: Contract
Location: Nashville, TN, United States
Description:

Title: Big Data Developers
Location: Nashville, TN
Duration: Long Term Contract

 
EAD GC, GC USC only

Must Have Strong AWS,Spark, Scala
 
Job Description: The Software Engineer, Data Platform is at the core a technologist with a broad skillset that brings enhanced value to the product development, and responsible in building key components of Change Healthcare's Intelligent Healthcare Data Platform.
Responsibilities:

• Designs, modifies, implements, tests, and provides operational support for enterprise software products and platforms, including software infrastructure and development tools for Data Platform components and modules in AWS-hosted environments 
• Engages in building solutions leveraging technologies including but not limited to Scala, Spark, Hive and AWS services such as EC2, S3, EMR, ELB, Kinesis, SNS, Redshift, Data Pipelines
• Provides consultation on complex projects and is considered to be a top-level contributor
• Relies on extensive experience and judgment to plan and accomplish goals according to timelines and budgets agreed upon with business stakeholders
• Communicates proactively and effectively with team members and other project stakeholders in a highly Agile environment

Communicates proactively and effectively with team members and other project stakeholders in a highly Agile environment
 As discussed the positions are the same tech stack with either a specialty in Big Data (Spark &Scala) or AWS.
These are the basic questions for AWS, Spark and Scala. Please reach me if you have any questions
 
AWS:
           we need to check whether he actually worked on AWS or not most of the people will have free tier AWS account and they will say that they have worked on it.So, this questions will make sure that he actually have realtime experience on AWS.
 

1. How to login into the EMR/ Ec2 instances from Commandline.

1. We can do it in 2 ways. 1 is using putty and providing the cluster ip address & private key(.ppk file).

Another way is to directly go to the commandline and run the SSH commmand and provide the .ppk file()
Eg:
 
Spark:

1. What is Lazy Evaluation ?

1. Apache Spark delays its evaluation till it is absolutely necessary. This is one of the key factors contributing to its speed. For transformations, Spark adds them to a DAG of computation and only when the driver requests some data, does this DAG actually gets executed.

1. What is RDD Lineage?

1. Spark does not support data replication in the memory and thus, if any data is lost, it is rebuild using RDD lineage. RDD lineage is a process that reconstructs lost data partitions. The best is that RDD always remembers how to build from other datasets.

1. What is Spark Driver?

1. Spark Driver is the program that runs on the master node of the machine and declares transformations and actions on data RDDs. In simple terms, a driver in Spark creates SparkContext, connected to a given Spark Master.The driver also delivers the RDD graphs to Master, where the standalone cluster manager runs.

1. What is Spark Executor?

1. When SparkContext connects to a cluster manager, it acquires an Executor on nodes in the cluster. Executors are Spark processes that run computations and store the data on the worker node. The final tasks by SparkContext are transferred to executors for their execution.

1. What are broadcast variables?

1. Broadcast variables allow the programmer to keep a read-only variable cached on each machine rather than shipping a copy of it with tasks. They can be used to give every node a copy of a large input dataset in an efficient manner. Spark also attempts to distribute broadcast variables using efficient broadcast algorithms to reduce communication cost.

1. Define RDD?

1. RDD is the acronym for Resilient Distribution Datasets – a fault-tolerant collection of operational elements that run parallel. The partitioned data in RDD is immutable and distributed. There are primarily two types of RDD:Parallelized Collections : The existing RDD's running parallel with one another.Hadoop datasets : perform function on each file record in HDFS or other storage system

 
 
         Scala:

1. What is "Type Inference" in Scala?

1. Types can be inferred by the Scala Compiler at compile-time. It is known as "Type Inference". Types means Data type or Result type. We use Types at many places in Scala programs like Variable types, Object types, Method/Function Parameter types, Method/Function return types etc.In simple words, determining the type of a variable or expression or object etc at compile-time by compiler is known as "Type Inference".
2. What is Unit in Scala?

1. What is the difference between Java's void and Scala's Unit?In Scala, Unit is used to represent "No value" or "No Useful value". Unit is a final class defined in "scala" package that is "scala.Unit".Unit is something similar to Java's void.
2. What is REPL in Scala?

1. What is the use of Scala's REPL? How to access Scala REPL from CMD Prompt?REPL stands for Read-Evaluate-Print Loop. We can pronounce it as 'ripple'. In Scala, REPL is acts as an Interpreter to execute Scala code from command prompt. That's why REPL is also known as Scala CLI(Command Line Interface) or Scala command-line shell.
2. What is the difference between Function and Method in Scala?

1. Scala supports both functions and methods. We use same syntax to define functions and methods, there is no syntax difference.However, they have one minor difference:We can define a method in a Scala class or trait.

• Method is associated with an object (An instance of a Class). We can call a method by using an instance of a Class. We cannot use a Scala Method directly without using object.

Function is not associated with a class or trait. It is defined in a Scala Package. We can access functions without using objects, like Java's Static Methods

Remarks:

--

Thanks & Regards

Shailbi Shukla

Recruitment & IT Solutions Inc.

Email To: shailbishukla@gmail.com
Fax :(703) 852 – 7939

8230 Boone Blvd, Suite 403, Vienna, VA 22182

--
You received this message because you are subscribed to the Google Groups "US-IT Openings" group.
To unsubscribe from this group and stop receiving emails from it, send an email to java-developer-deepu+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/java-developer-deepu/CADJ7EehbiHipJpqzgsVnjpO9_Cz3aSE4%2BjrMWZ%3DSY_Ab-fef-Q%40mail.gmail.com.