About this role
Job Description: Title: Lead data engineer Experience - 7-10 Years
About the Role We are seeking a highly skilled and delivery-focused Lead GCP Data Engineer to support the design, development, and implementation of next-generation enterprise data and AI platforms on Google Cloud Platform (GCP). This role will work closely with Enterprise Architects, platform leaders, and cross-functional engineering teams to build scalable, reusable, and AI-ready data foundations that enable advanced analytics, intelligent automation, and enterprise AI adoption. The ideal candidate combines strong hands-on expertise in cloud-native data engineering, modern data platform development, semantic data enablement, and scalable pipeline engineering with the ability to lead engineering teams and drive high-quality delivery across multiple initiatives. This role is expected to play a critical leadership position within the engineering organization by driving implementation excellence, mentoring teams, and operationalizing modern data architecture patterns. Key Responsibilities 1. Enterprise Data Platform Engineering
• Design, develop, and optimize scalable cloud-native data platforms and pipelines on GCP. • Implement robust batch, streaming, and event-driven data processing solutions supporting enterprise analytics and AI use cases. • Collaborate with Enterprise Architects to translate target-state architecture into scalable engineering implementations. • Contribute to modernization of legacy data ecosystems into reusable, governed, and AI-ready cloud platforms. • Support implementation of scalable ingestion, transformation, serving, and orchestration frameworks. 2. Data Product Engineering
• Develop reusable and domain-oriented data products aligned with data mesh and data-as-a-product principles. • Implement scalable and modular data pipelines supporting multiple downstream consumers including analytics, AI/ML, and operational applications. • Contribute to implementation of: • Data contracts • Schema management • Metadata enrichment • Data quality frameworks • Reusable transformation patterns
• Enable discoverability, trust, and operational reliability of enterprise data assets. 3. Semantic Layer & Consumption Enablement
• Support implementation of semantic and business-consumption layers that simplify enterprise data access. • Collaborate with analytics and BI teams to enable standardized business metrics, reusable dimensions, and governed KPI definitions. • Contribute to semantic modeling and metadata integration initiatives supporting self-service analytics and AI consumption. • Assist in improving enterprise data usability, consistency, and discoverability across platforms. 4. GCP-Native Engineering & Development
• Develop and optimize solutions leveraging GCP-native services including: • BigQuery • Dataflow • Dataproc • DBT • Pub/Sub • Cloud Storage • Cloud Composer (Airflow) • Cloud SQL
• Build scalable ETL/ELT frameworks and real-time streaming pipelines. • Optimize data processing performance, reliability, scalability, and cost efficiency. • Implement CI/CD pipelines and engineering automation for data platform delivery. 5. AI/ML & GenAI Data Enablement
• Build AI-ready data pipelines and scalable feature engineering workflows supporting enterprise AI initiatives. • Support integration with: • Vertex AI • BigQuery ML • Vector databases • LangChain • Generative AI Studio
• Contribute to implementation of RAG architectures, semantic search, and AI-assisted data interaction patterns. • Partner with AI/ML teams to operationalize scalable ML and GenAI workflows. 6. Engineering Leadership & Delivery Excellence
• Lead day-to-day engineering activities across multiple data engineering workstreams. • Guide and mentor junior and mid-level data engineers on modern engineering best practices. • Ensure adherence to coding standards, architecture guidelines, and operational best practices. • Drive engineering quality through automated testing, observability, monitoring, and performance optimization. • Collaborate with architects, product owners, analysts, and client stakeholders to ensure successful delivery outcomes. 7. Governance, Reliability & Observability
• Implement data governance, lineage, monitoring, and observability frameworks. • Support enforcement of enterprise standards around security, reliability, scalability, and operational readiness. • Contribute to platform monitoring, incident management, and continuous improvement initiatives. • Ensure production readiness of pipelines and data services through robust testing and validation processes.
Technical Expertise Required Area Skills / Technologies Cloud Data Engineering GCP, BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Cloud SQL Data Transformation DBT, PySpark, SQL, ETL/ELT frameworks Streaming & Pipelines Apache Beam, real-time processing, event-driven architectures Semantic Layer & Modeling Semantic modeling concepts, Looker modeling, business metrics standardization AI/ML Enablement Vertex AI, BigQuery ML, LangChain, Vector Databases, GenAI integration Orchestration & Automation Cloud Composer (Airflow), CI/CD, Workflows Metadata & Governance Data Catalog, lineage, metadata management, observability frameworks Programming Python, SQL, PySpark
Qualifications
• Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or related field. • 7+ years of experience in data engineering and cloud-native data platform development. • Minimum 4+ years of hands-on experience delivering enterprise-scale solutions on GCP. • Strong expertise in building scalable batch and streaming data pipelines. • Experience working on modern enterprise data platforms supporting analytics, AI/ML, and GenAI use cases. • Good understanding of semantic layer concepts, reusable data models, and governed data consumption patterns. • Experience working within large-scale data modernization and cloud transformation initiatives. • Strong problem-solving, debugging, and performance optimization skills. • Proven ability to lead engineering teams and collaborate across architecture, product, and business functions. • Excellent communication and stakeholder management skills. • GCP certifications such as Professional Data Engineer preferred.
Location: DGS India - Mumbai - Thane Ashar IT Park Brand: Merkle Time Type: Full time Contract Type: Permanent