RedShift Data Engineer

Sydney, Sydney Regionlast month
8+ years

Job description

Job Overview

We are seeking an experienced Redshift Data Engineer with 8+ years of experience in data engineering and strong hands-on expertise with Amazon Redshift. The ideal candidate will be responsible for designing, developing, optimizing, and maintaining scalable data platforms and ETL/ELT pipelines on AWS.

Key Responsibilities
  • Design and develop scalable data pipelines and ETL/ELT workflows using AWS services.
  • Develop and optimize complex SQL queries, stored procedures, views, and data models in Amazon Redshift.
  • Perform Redshift performance tuning, query optimization, distribution/sort key optimization, and workload management.
  • Build and maintain data ingestion pipelines from multiple sources into Amazon Redshift.
  • Work with AWS Glue, S3, Lambda, Step Functions, and CloudWatch for data integration and orchestration.
  • Develop robust data transformation processes using Python and SQL.
  • Implement data quality, validation, reconciliation, and monitoring frameworks.
  • Design and maintain dimensional data models, including fact and dimension tables.
  • Work with large datasets and optimize data processing for performance and cost.
  • Implement incremental data loading and CDC (Change Data Capture) strategies.
  • Troubleshoot data pipeline failures, performance issues, and production incidents.
  • Collaborate with Data Architects, Business Analysts, Developers, and other stakeholders.
  • Follow best practices for data security, governance, access control, and AWS IAM.
  • Participate in code reviews, technical design discussions, and Agile delivery processes.
Mandatory Skills
  • 8+ years of experience in Data Engineering
  • Strong hands-on experience with Amazon Redshift
  • Advanced SQL skills
  • Strong Python programming/scripting experience
  • Experience developing ETL/ELT pipelines
  • Strong experience with AWS
  • Hands-on experience with Amazon S3
  • Experience with AWS Glue
  • Experience with data warehousing and dimensional data modeling
  • Strong knowledge of Redshift performance tuning
  • Experience with large-scale data processing
  • Experience with CI/CD and Git
  • Good understanding of data quality and data governance
Good to Have
  • Experience with AWS Lambda, Step Functions, EMR, Athena, and CloudWatch
  • Experience with Apache Spark / PySpark
  • Experience with Airflow
  • Knowledge of Kafka / streaming data
  • Experience with CDC tools such as AWS DMS, Debezium, or equivalent
  • Experience with Terraform / Infrastructure as Code
  • Exposure to Databricks, Snowflake, or other cloud data platforms
  • Experience working in Agile/Scrum environments
Technical SkillsCloud: AWS, S3, Glue, Lambda, Step Functions, CloudWatchDatabase: Amazon Redshift, PostgreSQLProgramming: Python, SQLETL/ELT: AWS Glue, Airflow, custom Python pipelinesBig Data: Spark, PySparkDevOps: Git, CI/CD, TerraformData Modeling: Star Schema, Snowflake Schema, Dimensional ModelingStreaming: Kafka – desirable

Skills mentioned

Apply for this job

Use the application link supplied with this listing to apply to XPT Software Australia Pty. Check the destination before entering personal information.

Apply Now

Explore Similar Job Hubs