Data Engineer - Nicoll Curtin - Singapore
Nicoll Curtin - Singapore Singapore
Responsibilities:
- Design, develop, and maintain data pipelines and backend services for real-time decisioning, reporting, data collecting, and related functions.
- Data Requirements and Modeling
- Data Management and Transformation
- Produce high-quality, well-tested, and secure code.
- Develop and maintain software designed to improve data governance and security.
- Develop processes designed to ensure Data Security and Data Quality.
Requirements:
- BS/MS in Computer Science, Data Analytics, Data Management, Information Systems or related technical field or equivalent training & experience.
- 8+ years Incorporating data processing and workflow management tools into pipeline design.
- 8+ years’ experience in ETL/ELT, Data warehousing, and/ or Business Intelligence Development.
- 8+ years building and maintaining end-to-end data systems and supporting services in Python, Scala or similar.
- 8+ years using SQL to understand data and to investigate data issues or problems and provide solutions.
- 5+ years working with cloud data technologies.
- Experience with structured, semi-structured, and unstructured data.
- Experience utilizing a variety of data stores, including data warehouses, RDBMSes, in-memory caches, and searchable document Databases.
- Experience working with large data sets in SQL/Databricks/PySpark.
- Experience with Spark Streaming, and Delta (Live) Tables.
- Experience with storage systems such as Azure Storage/ Data Lakes.
- Experience with data modelling.
- Strong design, implementation, and testing skills.
- Experience developing for continuous integration and automated deployments.
- Experience developing on cloud platforms (preferably Azure) in a continuous delivery environment.
- Excellent communication and collaboration skills.
- Experience with microservices platforms (Kubernetes, Docker, Helm Charts ..).
- Experience with Event-Driven streaming systems (Kafka, Event Hub, Event Grid, Apache Flink).
- Knowledge and use of DBT.
- Experience with data vault is a plus.
- Experience with Microsoft Fabric.
- Experience with creating, editing and administering BI tools.
- 8+ years software development using languages like JavaScript, Java, or C#.
Oxford KnightSingapore
Salary: up to 250-275k SGD base
Summary
High-frequency prop trading firm with offices worldwide looking for a talented Data Engineer to design, implement and maintain the infrastructure which underpins the firm's data management processes...
TEKsystems (Allegis Group Singapore Pte Ltd)Singapore
Strong programming proficiency using Python
• understanding of distributed computing, especially in databases
• Hands-on in SQL with a deep understanding of query optimization
We are looking for a high-calibre data experts to be a part of a fast...
HaysSingapore
Your new company
A global technology company
Your new role
• Develop and establish an optimised offline data architecture tailored for large-scale recommendation systems.
• Create scalable, adaptable, high-performing, and reliable storage...