About this role
Location: CN-Shenzhen-HyQ Shift: Standard - 40 Hours (China) Scheduled Weekly Hours: 40 Worker Type: Permanent Job Summary: Build and maintain data pipelines and platform components across the full data stack. You will work within an established architecture, contributing reliable, well-tested code while growing your expertise across streaming, batch, storage, and analytics technologies. Job Duties: Responsibilities
• Data Pipelines: Develop, test, and deploy batch (Spark) and streaming (Flink/Spark Structured Streaming) jobs on OpenShift • Message & Stream: Create and manage Kafka topics, configure connectors, monitor consumer lag, troubleshoot message delivery • Lakehouse & Storage: Create and maintain Iceberg tables (schema changes, partitioning, maintenance); operate MinIO for data lake workloads • Query & Serving: Write SQL transformations (dbt), build Trino queries for federated analytics, manage materialized views on StarRocks/ClickHouse • Orchestration: Author and maintain Airflow/Dagster DAGs; troubleshoot pipeline failures and data quality issues • Observability: Build Grafana dashboards and Prometheus alerts for pipeline health, data freshness, and quality metrics • Kubernetes Operations: Write basic Helm charts, manage ConfigMaps/Secrets, troubleshoot pod failures under guidance • CI/CD: Maintain CI pipelines for data jobs, contribute to ArgoCD application definitions • Write clear documentation: runbooks, pipeline specs, onboarding guides • Participate in code reviews and agile ceremonies Required Skills & Experience
• 3 – 6 years in data engineering, software engineering, or DevOps • Working knowledge of Kubernetes: Pods, Deployments, Services, ConfigMaps, basic Helm usage • Practical Spark experience: PySpark or Spark SQL batch ETL, reading/writing to object storage and Iceberg tables • Basic Kafka experience: producing and consuming messages, understanding partitions and offsets • Understanding of data lake/lakehouse concepts: columnar formats (Parquet/ORC), open table formats (Iceberg/Delta Lake), partitioning, schema evolution • Experience with an S3-compatible object store (MinIO, AWS S3, Ceph) • Solid SQL skills; proficient in Python • Comfortable with Git, Docker, CI/CD, and Linux shell scripting Nice to Have
• Exposure to Trino or Presto for federated queries • Familiarity with a real-time OLAP engine: StarRocks, ClickHouse, or Doris • Experience with dbt for data transformation • Workflow orchestration: Airflow or Dagster • Interest in data observability, data mesh, and data quality frameworks
Company Introduction: ITD SZ
港交所科技（深圳）有限公司，是2016年12月28日于深圳市前海自贸区成立的外商独资企业。
作为港交所的技术子公司，港交所科技（深圳）有限公司主要是为集团及其附属公司提供计算机软件、计算机硬件、信息系统、云存储、云计算、物联网和计算机网络的开发、技术服务、技术咨询、技术转让；经济信息咨询、企业管理咨询、商务信息咨询、商业信息咨询、信息系统设计、集成、运行维护；数据库管理、大数据分析；以承接服务外包方式提供系统应用管理和维护、信息技术支持管理、数据处理等信息技术和业务流程外包服务。