Senior Data Engineer: Lead Scalable Data Pipelines
Job description
MPA Recruitment seeks a Senior Data Engineer to design and develop data processing and persistence software for large-scale data systems. You will lead development in agile teams, shaping significant data components with a focus on quality and maintainability.
The role emphasizes hands-on coding in Java/Scala/Python, building on AWS/Azure/GCP and Databricks, with strong emphasis on ETL/ELT pipelines, CI/CD, and production readiness. Docker/Kubernetes experience is desirable.
Design and develop data processing software components for large-scale data systems.
Provide development leadership and own significant technical components.
Work with Architects and Lead Engineers to ensure non-functional needs are met.
Collaborate with teams to resolve challenges and maintain code quality and maintainability.
Coach and mentor team members and ensure operational readiness.
Strong software development experience in one of Java, Scala, or Python.
Experience with data-processing platforms from vendors such as AWS, Azure, GCP, Databricks.
Experience developing substantial components for large-scale data processing solutions and deploying into production.
Proficient in SQL and SQL extensions for analytical queries.
Solid understanding of ETL/ELT data processing pipelines and design patterns.
Aware of features and pitfalls of distributed data processing frameworks, data stores and formats.
Able to write quality, testable code and has experience of automated testing.
Experience with CI/CD techniques.
Skills mentioned
Apply for this job
Use the application link supplied with this listing to apply to MPA Recruitment. Check the destination before entering personal information.

