Data Engineer

Required Qualifications & Skills – Senior Data EngineerWe are seeking an experienced Senior Data Engineer to design, build, and maintain scalable data pipelines and distributed data processing solutions.

Mandatory / Primary

Skills

PythonPySparkAWSSQL

Experience

• 4 to 7 years of hands-on experience in Data Engineering and building scalable data solutions.
• Strong experience working with AWS-based data engineering services and cloud data platforms.

Technical Skills

• Strong hands-on expertise in Python and PySpark for large-scale data processing.
• Advanced proficiency in SQL, including writing complex queries, query optimization, and data analysis.
• Strong experience with AWS services relevant to data engineering and cloud-based data processing.
• Strong understanding of Apache Spark and distributed data processing systems.
• Experience designing, developing, and maintaining scalable and high-performance ETL/ELT data pipelines.
• Experience in data ingestion, transformation, integration, and processing from multiple data sources.
• Experience working with structured, semi-structured, and unstructured data such as JSON, logs, PDFs, and other file formats.
• Strong knowledge of relational and non-relational databases.
• Experience in data quality checks, validation, performance tuning, and optimization.

Analytical & Problem-Solving Skills

• Strong ability to perform deep-dive data analysis and troubleshoot complex data-related issues.
• Proven experience in building scalable data solutions to address complex business requirements.
• Strong problem-solving and analytical skills.

Soft Skills

• Strong verbal and written communication skills.
• Good organisational and stakeholder-management skills.
• Proactive self-starter with the ability to work independently and take ownership.

Technical Agility

• Ability to adapt quickly to new technologies and data platforms.
• Passion for staying current with emerging trends and best practices in Data Engineering, AWS, and Big Data technologies.