Spark Scala consultant

Position: Spark Scala Consultant

location: Columbus, IN

Job Type: Contract

Job Description:

• Leads projects for design, development and maintenance of a data and analytics platform.

• Effectively and efficiently process, store and make data available to analysts and other consumers.
• Works with key business stakeholders, IT experts and subject-matter experts to plan, design and deliver optimal analytics and data science solutions.
• Works on one or many product teams at a time.

Key Responsibilities

• Designs and automates deployment of our distributed system for ingesting and transforming data from various types of sources (relational, event-based, unstructured).
• Designs and implements framework to continuously monitor and troubleshoot data quality and data integrity issues.
• Implements data governance processes and methods for managing metadata, access, retention to data for internal and external users.
• Designs and provide guidance on building reliable, efficient, scalable and quality data pipelines with monitoring and alert mechanisms that combine a variety of sources using ETL/ELT tools or scripting languages.
• Designs and implements physical data models to define the database structure. Optimizing database performance through efficient indexing and table relationships.
• Participates in optimizing, testing, and troubleshooting of data pipelines.
• Designs, develops and operates large scale data storage and processing solutions using different distributed and cloud based platforms for storing data (e.g. Data Lakes, Hadoop, Hbase, Cassandra, MongoDB, Accumulo, DynamoDB, others).
• Uses innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks in order to minimize manual and error-prone processes and improve productivity.
• Assists with renovating the data management infrastructure to drive automation in data integration and management.
• Ensures the timeliness and success of critical analytics initiatives by using agile development technologies such as DevOps, Scrum, Kanban
• Coaches and develops less experienced team members.

Experiences

REQUIRED

• Intermediate experience resulting in the following skills and knowledge:
• Familiarity analyzing complex business systems, industry requirements, and/or data regulations
• Background in processing and managing large data sets
• Design and development for a Big Data platform using open source and third-party tools
• SPARK, Scala/Java, Map-Reduce, Hive, Hbase, and Kafka or equivalent college coursework
• SQL query language
• Clustered compute cloud-based implementation experience
• Experience developing applications requiring large file movement for a Cloud-based environment and other data extraction tools and methods from a variety of sources
• Experience in building analytical solutions

PREFERRED

• Intermediate experience resulting in the following skills and knowledge:
• Experience with IoT technology
• Experience in Agile software development

Thanks and Regards,

Eric Lieo

Xchange Software Inc

10 Austin Avenue, Iselin, NJ - 08830.

Phone: 732.369.4195

Fax: 732.601.4641

*********@xchangesoft.net

www.xchangesoft.com