Job Description

Role: Senior Data Engineer

Location: Remote

Duration: Fulltime/Permanent Job

Job Description

Must Have Technical/Functional Skills

Must Have Technical/Functional Skills

• Google Cloud Platform (GCP)
• PySpark
• Python
• Airflow
• Hive
• Kafka

Data Pipeline Development, Enhancement & Maintenance

• Design, build, and maintain scalable ETL/ELT pipelines.
• Automate data ingestion from multiple data sources.
• Ensure reliable, high-performance, and scalable data movement.
• Develop and optimize PySpark-based data processing frameworks.
• Build and manage Airflow workflows and orchestration pipelines.
• Work with Hive and Kafka-based batch and streaming data solutions.
• Implement data quality, monitoring, governance, and operational controls.
• Support production incidents, performance tuning, and continuous improvement initiatives

Roles & Responsibilities

• The Data Engineer will be responsible for designing, building, governing, and optimizing data platforms, pipelines, and integrations to ensure secure, reliable, and scalable data availability for analytics, reporting, AI/ML, and business operations.

Preferred Experience

• Strong Functional Knowledge in Retail (Search & Personalization) Added advantage.
• Strong experience with large-scale GCP Data Engineering solutions.
• Hands-on experience with Airflow DAG development and support
• Experience building and optimizing Spark/PySpark data pipelines.
• Exposure to Hive-based data processing and Kafka-driven ingestion frameworks.
• Experience supporting analytics, personalization, recommendation, or search platforms is highly desirable

Job Tags

Similar Jobs