Senior Data Engineer

Dubai, United Arab Emiratesأمس
خبير
10+ سنوات

وصف الوظيفة

Job Title: Senior Data Engineer

Location: Dubai | UAE | Malaysia | Armenia | India

Role

Overview

Fintop Consulting has partnered with one of the pioneers in the financial industry and is looking for a hands-on Senior Data Engineer with strong expertise in Azure Databricks, PySpark, SQL, and Kafka-based streaming technologies.

The role involves building and managing scalable data pipelines, handling complex data integrations, and ensuring data quality and reliability across a high-volume financial data environment. The ideal candidate should have strong experience in trading or brokerage data, production support, performance optimization, and end-to-end data pipeline ownership.

Key Responsibilities

  • Design, develop, and maintain data ingestion and transformation pipelines using Azure Databricks, PySpark, SQL, and Delta Live Tables (DLT).
  • Build and manage Kafka-based streaming integrations using Confluent Cloud and its connectors.
  • Develop and enhance API-based ingestion frameworks and third-party data integrations.
  • Own data pipelines from source ingestion through Bronze and Silver layers, ensuring data quality, reliability, and SLA adherence.
  • Troubleshoot production issues, perform root cause analysis and implement permanent fixes.
  • Optimize Databricks workloads, Spark performance and cloud infrastructure costs.
  • Implement data governance and deployment practices using Unity Catalog, Databricks Workflows and CI/CD.
  • Collaborate with business and technical stakeholders to define data requirements and integration standards.
  • Mentor junior engineers and contribute to code reviews, technical design and engineering best practices.

Required

Skills & Experience

  • 10+ years of hands-on experience in data engineering.
  • Strong production experience with Azure Databricks, Apache Spark, PySpark, SQL and Delta Lake.
  • Practical experience with Delta Live Tables, Unity Catalog and Databricks Workflows.
  • Strong understanding of data modelling, incremental loads, CDC, partitioning and performance tuning.
  • Hands-on experience with Apache Kafka, preferably Confluent Cloud and Kafka Connect.
  • Experience building API-based data ingestion frameworks.
  • Proven experience optimizing Databricks performance and operational costs.
  • Strong understanding of trading and brokerage data, including trades, positions, balances, deals and accounts.
  • Working knowledge of CRM, client, partner, payments, reconciliation and financial data.
  • Experience supporting production data platforms and resolving critical data issues.

Good to Have:

  • Experience with MetaTrader (MT4/MT5) back-office data.
  • Exposure to trading bridges and liquidity platforms.
  • Experience with enterprise CRM systems.
  • Background in brokerage, trading, or regulated financial services.

المهارات المذكورة

تقدّم لهذه الوظيفة

تتولى FinTop Consulting معالجة الطلبات بالكامل على موقعها الخاص. لا يجمع KarieR.co بياناتك ولا يحوّل أي طلب.

قدّم الآن

Explore Similar Job Hubs