Engineer - Data Engineering & Analytics
Job description
Job Description
- Implement real-time ETL/ELT pipelines on enterprise data platforms, including Databricks, Snowflake, Microsoft Fabric, Cloudera, Informatica IDMC, or Oracle
- Implement data workflows using technologies such as Databricks Lakeflow, Fabric Data Factory, Azure Data Factory, Informatica Cloud Data Integration, Snowflake Streams & Tasks, and Apache Airflow
- Implement Change Data Capture (CDC) and streaming ingestion using one or more technologies, including Oracle GoldenGate, Apache Kafka, and Spark Structured Streaming
- Apply dimensional data modelling, including Kimball star schemas, to deliver analytics-ready data marts
- Implement data governance, security, data quality, and lineage using Databricks Unity Catalog, Microsoft Purview, Cloudera SDX, and Informatica Data Quality
- Apply DataOps practices, including Git-based version control, CI/CD for data pipelines, automated testing, and Infrastructure as Code
- Monitor, troubleshoot, and optimise production pipelines for performance and cloud cost efficiency, supporting the practice's 99.90% uptime SLA commitment
- Work directly with client stakeholders throughout the delivery lifecycle, including requirements gathering, data model validation, User Acceptance Testing (UAT), Go-Live, and post-Go-Live SLA support
Person Specification
- Possess a Bachelor's Degree in Data Science or a higher qualification, such as an MSc in Data Science, Data Engineering, or Artificial Intelligence, from a recognised university
- Have 2–3 years of professional experience in building and operating enterprise data pipelines, data warehouses, or lakehouses
- Possess hands-on experience with at least two of the following platforms: Databricks, Snowflake, Microsoft Fabric/Azure Data Services, Cloudera, Informatica (IDMC/PowerCenter), or Oracle (ADW/Exadata/ODI)
- Demonstrate strong experience with Apache Spark and distributed data processing at scale
- Possess a solid understanding of data modelling, data quality, and data governance principles
- Demonstrate strong communication skills and the ability to work directly with client stakeholders
- Professional certifications such as Databricks Certified Data Engineer (Associate/Professional), SnowPro Core or SnowPro Advanced: Data Engineer, Microsoft Certified: Fabric Data Engineer Associate (DP-700) or Fabric Analytics Engineer Associate (DP-600), Informatica IDMC, or Oracle Autonomous Database certifications will be considered an added advantage
- Experience in migrating legacy ETL platforms, including Informatica PowerCenter, SSIS, or ODI, or on-premises data warehouses to modern cloud lakehouse platforms will be considered an added advantage
- Strong SQL and Python (PySpark) skills, along with knowledge of Scala or Java, will be considered an added advantage
Skills mentioned
- AI
- Airflow
- Analytics
- Azure
- Azure Data Factory
- CI Cd
- Cloud
- Data Engineering
- Data Governance
- Data Pipelines
- Data Quality
- Data Science
- Data Warehousing
- Databricks
- Elt
- ETL
- Git
- Infrastructure As Code
- Java
- Kafka
- Lakehouse
- Microsoft Fabric
- Oracle
- Pyspark
- Python
- Requirements Gathering
- Scala
- Snowflake
- Spark
- SQL
- Ssis
- Troubleshooting
- Version Control
MillenniumIT ESP is a complete enterprise solutions provider with a proven track record of serving global customers for over 25 years. Founded in 1996, the company has established itself as a visionary in creating impactful technology solutions across multiple domains including Core Infrastructure, Cloud, Cyber Security, Enterprise Applications, Intelligent Automation and Data, Smart Buildings, and Managed Services. Headquartered in Colombo, Sri Lanka, MIT ESP has grown into a trusted partner for organizations worldwide, leveraging strategic partnerships with industry leaders like Cisco, Microsoft, and Oracle to deliver cutting-edge solutions. The company serves diverse industries including Banking and Financial Services, Government, Manufacturing and Retail, Telecommunications and Media, and Commercial sectors. With a team of 501-1,000 professionals, MillenniumIT ESP combines deep technical expertise with industry-specific knowledge to help enterprises reimagine and reinvent their digital transformation journeys. The company is committed to purpose-driven technology solutions that deliver measurable business outcomes, continuously expanding its global footprint while maintaining the agility and customer-centric approach that has defined its success for nearly three decades.
Apply for this job
Use the application link supplied with this listing to apply to Millennium IT ESP. Check the destination before entering personal information.