Giga Giglet: GCP Data Engineer :: Sunnyvale, CA (Onsite)- Looking for locals.

Role ::GCP Data Engineer

Location: Sunnyvale, CA (Onsite)- Looking for locals.

Mandatory skills

Spark

Scala

GCP

Airflow

Dag

ETL

Pyspark

Job Description :

1. Design, develop, and automate data processing workflows using Airflow, PySpark, and Dataproc on GCP.

2. Develop ETL (Extract, Transform, Load) processes that handle diverse data sources and formats.

3. Manage and provision GCP resources including Dataproc clusters, serverless batches, Vertex AI instances, GCS buckets, and custom images.

4. Provide platform and pipeline support to analytics and product teams, troubleshooting issues related to Spark, Big Query, Airflow DAGs, and serverless workflows.

5. Collaborate with data scientists and analysts to understand data needs and deliver robust solutions.

6. Provide timely and effective technical support to internal users (e.g., data analysts, data scientists) addressing their data-related queries and problems

7. Optimize and fine-tune data systems for high performance, reliability, and cost efficiency.

8. Perform root cause analysis for recurring issues and collaborate with data analysts and scientists to implement preventative measures to minimize future occurrences.

Required Skills:

• Strong programming skills in Python, SQL

• Hands-on experience with cloud platforms

• Expertise in GCP data tools: BigQuery, Dataproc, Vertex AI, Pub/Sub, Cloud Functions.

• Strong hands-on experience with Apache Airflow (incl. Astronomer), PySpark, and Python.

• Familiarity with SQL, SparkSQL, Hive, PL/SQL, and data modelling.

• Comfortable supporting distributed data systems and large-scale batch/stream data processing.

• Optimize and support Spark jobs and ETL pipelines running on Dataproc.

Thanks & Regards,

Durga Prasad

US IT Recruiter

American IT Systems