Senior / Lead Data Engineer (Python, Kafka, Iceberg, ClickHouse) - Geylang

JOB SUMMARY:

The ideal candidate will have a strong technical background in data engineering, expertise in analytical tools, and proven leadership skills to mentor and guide a team of data engineers.

JOB DUTIES:

• Design and implement scalable batch and streaming data pipelines using tools such as Flink, Spark, Debezium, and Seatunnel (an open-source data integration tool).
• Ingest and process high-volume data from APIs, operational databases, and semi-structured formats (e.g., JSON, CSV, XML, logs) to support diverse analytical use cases.
• Build reusable transformation pipelines to consolidate cross-domain data (e.g., user behavior, transactions) into analytics-ready marts.
• Architect and optimize data storage and modeling layers using MinIO/S3, Iceberg, ClickHouse, and other OLAP or object storage platforms to improve query performance and data reliability.
• Maintain multi-layered data warehouse architecture (staging, core, mart) aligned with business needs.
• Ensure robust CI/CD, lineage, observability, and compliance through tools like OpenMetadata and DataHub.
• Mentor junior engineers, conduct rigorous code reviews, and promote engineering best practices across the data team.
• Work cross-functionally with product managers, analysts, business stakeholders to translate data needs into scalable pipelines and business insights.
• Stay current with data engineering trends and technologies, and continuously drive platform improvements.

JOB REQUIREMENTS:

• Bachelor’s degree in Computer Science, Information Systems, or equivalent qualification.
• 6+ years of experience in data engineering, including 2+ years in a technical lead or senior IC capacity.
• Proficient in Python, Java, and SQL, with strong expertise in schema design, performance tuning, and warehouse modeling.
• Hands-on experience with lakehouse architectures (e.g., Iceberg, Delta Lake), data warehouses (e.g., Hive, ClickHouse), and object storage (e.g., AWS S3, MinIO).
• Strong knowledge of orchestration tools (Airflow, DolphinScheduler), ETL/ELT design, and streaming frameworks (Kafka, Flink, Spark).
• Proven experience independently setting up and managing end-to-end data architecture in on-premise environments.
• Demonstrated success mentoring engineers in high-performance, cross-functional teams.
• Familiarity with Git-based workflows, CI/CD pipelines, and observability tools for production-grade data systems.
• Self-motivated with a strong ownership mindset, adaptability, and willingness to travel when needed.
• Fluent in English and Mandarin (written/spoken), with experience leading technical discussions in multilingual and cross-regional settings.