Responsibilities of the Candidate:

• Data transformation and integration: Implement complex data transformations and integrate data from various sources.
• Collaboration: Work closely with data engineers, data scientists, and other stakeholders to gather requirements and deliver solutions.
• Quality assurance: Perform extensive testing and validation of data processing systems to ensure data quality and accuracy.
• Data warehousing: Work with large datasets stored in HDFS and Hive, applying concepts like Partitions and Bucketing.
• Platform expertise: Develop and deploy solutions on platforms such as Azure Databricks, which includes using PySpark notebooks.
• Data transformation and integration: Implement complex data transformations and integrate data from various sources.
• Collaboration: Work closely with data engineers, data scientists, and other stakeholders to gather requirements and deliver solutions.
• Quality assurance: Perform extensive testing and validation of data processing systems to ensure data quality and accuracy.
• Data warehousing: Work with large datasets stored in HDFS and Hive, applying concepts like Partitions and Bucketing.
• Platform expertise: Develop and deploy solutions on platforms such as Azure Databricks, which includes using PySpark notebooks.