Position Description

1. Provide day-to-day support and guidance to software engineers, data analysts and data scientists on using GCP Dataproc and Dataflow services i.e. job and cluster creation, error handling, etc.. 

2. Help troubleshoot issues on GCP and Hadoop platforms including but not limited to Dataproc, Dataflow, Hive, Spark, Oozie, Kafka, NiFi, etc. 

3. Perform testing and PoCs for different Dataproc and Dataflow use cases 

4. Participate in monthly On-Call rotation

5. Secondary responsibilities include operational support for COTS software on Linux servers. Skills required for this work are basic Linux system administration

RESPONSIBILITIES

Primary Skills:

1. Python/Pyspark 
2. Dataflow, Dataproc
3. SQL
4. Shell Scripting

Additional Skills Preferred:

1. Terraform Coding Skills 

2. Apache Airflow 

3. Tekton Pipeline Experience 

4. Apache Beam Coding Skills 

5. Docker/Kubernetes experience

QUALIFICATIONS

Primary Skills:

1. Python/Pyspark 
2. Dataflow, Dataproc
3. SQL
4. Shell Scripting

Additional Skills Preferred:

1. Terraform Coding Skills 

2. Apache Airflow 

3. Tekton Pipeline Experience 

4. Apache Beam Coding Skills 

5. Docker/Kubernetes experience