Data Engineer to platform foundation team

We are on a journey. We already have an established On-Prem Data Platform cluster and a recent first Cloud implementation of the Data Platform on AWS. We are not migrating to Cloud but instead creating a distributed Hybrid Data Platform that includes several cloud providers and On-Premise Data Centres. The technical platform used as a base for this is Cloudera.



The role

You will work within a team whose main mission is to implement and evolve the Scania Data Platform, which enables other solution-oriented Data Engineers and Analysts to focus on implementing Use Cases for the business based on the Data Platform. Our team's focus is to make sure that needed capabilities and scaffolding are available so development teams can work efficiently with the platform in a governed self-service approach.

Your Profile

You are an experienced Data Engineer who has participated in the setup of Hadoop-based data platforms and created data pipelines in the same. Dev-Ops and CI-CD are familiar to you.

You are comfortable in a rapidly changing environment and able to navigate at a high rate of change. Responsibility and focus on problem-solving come naturally to you and even though you want short lead times you understand the need for governance and knowledge sharing to keep the platform and the team sustainable over time.

Wanted Skills and Experience:


• At least 3years of experience in the areas below:
• Technical background in one or more languages such as Python, SQL, Scala
• Experience of Hadoop ecosystem: Eg. Hive, HBase, Impala, HDFS, Kafka, NiFi, Spark etc
• Working SQL knowledge and experience working with relational databases
• Experience building, optimizing, maintaining, and deploying ‘big data’ data pipelines and data sets
• Experience and practice from using agile working methods and DevOps - CI/CD. 
• BSc or MSc in Computer Science or related field (or equivalent experience)

Meritorious Skills and Experience


• Experience from working with cloud development, preferably AWS perhaps using Terraform
• Understanding of multiple architectural design patterns for large data sets, multidimensional models and data warehousing
• Knowledge of using and publishing REST APIs
• Linux Experience