Senior Consultant - Tech Consulting - National - CNS - TC - Data Engineer- Mumbai and Ahmedabad (Ahmedabad, GJ, IN, 380059) @ EY Global Services

<p>Job Title: Data Engineer<br>Level: Senior Consultant<br>Experience: 3-6 Years<br>Location: Mumbai and Ahmedabad</p> <p>Role Summary<br>The Data Engineer role focuses on the design, development, and maintenance of scalable,<br>reliable, and high-performance data platforms. The role requires strong hands-on expertise in<br>Python-based microservices, distributed data processing frameworks, and cloud-based big<br>data ecosystems, with a primary emphasis on building robust applications to process ETL<br>pipelines and production-grade data systems.</p> <p>Key Responsibilities<br>Design, develop, and maintain scalable Python-based microservices for data ingestion and<br>processing<br>Build and optimize microservices to support end-to-end ETL pipelines for large-scale batch and<br>streaming data<br>Develop and manage distributed data processing workfl ows using Apache Spark and PySpark<br>Create, schedule, and monitor workfl ows using Apache Airfl ow<br>Design and deploy data pipelines on AWS EMR or similar big data platforms using Python<br>Implement real-time data ingestion and streaming pipelines<br>Ensure data quality, reliability, performance, and scalability of data systems<br>Apply best practices for code quality, testing, CI/CD, and production monitoring<br>Troubleshoot production issues and perform root-cause analysis</p> <p>Required Skills &amp; Experience<br>6–8 years of experience in Data Engineering or backend data platform roles<br>Strong profi ciency in Python with experience building production-grade microservices<br>Hands-on expertise with Apache Spark and PySpark<br>Experience working with AWS EMR or equivalent cloud-based big data services. (Must have<br>knowledge on AWS eco system)<br>Strong experience with Apache Airfl ow for workfl ow orchestration<br>Deep understanding of ETL pipeline design and data modeling<br>Hands-on experience with Kafka for inter-domain communication and real-time or streaming<br>data pipelines<br>Strong experience with PostgreSQL or similar RDBMS<br>Experience working in Agile/Scrum environments</p> <p>Good to Have<br>Experience with Scala, Docker and Kubernetes<br>Exposure to cloud platforms such as AWS, Azure, or GCP<br>Experience with data lake architectures and schema evolution<br>Familiarity with monitoring and logging tools</p> <p><br>Desired Competencies<br>Strong ownership and accountability<br>Excellent problem-solving and analytical skills<br>Good communication and collaboration skills<br>Ability to work independently in a fast-paced environment</p>