Data Engineer with DevOps expertise

We now have a new need and are therefore looking for a consulting colleague with the qualifications below. The position is part of our consulting services, which means you will be employed by us and work with clients or internal projects and assignments.Job DescriptionWe are looking for a skilled Data Engineer with DevOps expertise to support the design, development, and automation of data products on our Azure-based Data Hub. This role will focus on building and maintaining infrastructure using Terraform, automating workflows with Azure DevOps, and developing data pipelines with Python and SQL. This consultant will work closely with cross-functional teams to ensure the smooth operation and scalability of our data infrastructure.Objectives:- Develop and maintain data pipelines and infrastructure in Azure.- Automate deployment processes using Azure DevOps and Terraform.- Ensure data products are scalable, secure, and optimized for performance.Tasks and Responsibilities:Pipeline Development:- Collaborate with data product designers to implement and optimize data models and pipelines using SQL and Python.- Programming and Code Quality: Work independently and alongside data product designers to write, test, and optimize code (SQL and Python) locally before committing to source control. Ensure that code meets high standards of quality and efficiency, with a proactive approach to troubleshooting and debugging.- Pipeline Implementation: Collaborate with data modeling and design-focused engineers to implement and optimize data models and pipelines on the Databricks platform, ensuring seamless integration and alignment with business needs.- Ensure data pipelines are efficient, reliable, and scalable on the Databricks platform.Infrastructure as Code (IaC):- Implement infrastructure as code using Terraform to manage and provision Azure resources.- Ensure infrastructure is consistent, reliable, and capable of supporting large-scale data operations.CI/CD Automation:- Automate and manage CI/CD pipelines through Azure DevOps to ensure smooth deployments and minimal downtime.- Monitor data workflows and troubleshoot any issues that arise in the deployment or pipeline processes.Cross-Functional Collaboration:- Work with development, operations, and data teams to ensure alignment on data infrastructure and operations.- Collaborative Development: Work closely with data product designers, modelers, and other engineers focused on specific areas such as data modeling, design, or architecture. Share ideas, best practices, and solutions to ensure alignment on data infrastructure and operations.- Contribute to best practices around cloud architecture, security, and data governance.Deliverables:- Fully functional data pipelines on the Databricks platform.- Infrastructure-as-code solutions using Terraform.- Automated CI/CD pipelines through Azure DevOps.- Documentation of the infrastructure and deployment processes for cross-team visibility and ongoing maintenance.Your Profile (Must-have requirements)- Hands-on experience with Python and SQL for building and managing data pipelines.- Proficiency in Terraform for infrastructure as code, particularly with Azure resources.- Strong knowledge of Azure DevOps for managing CI/CD pipelines and automating deployments.- Familiarity with Databricks and the Azure ecosystem.- Solid understanding of cloud-native architectures and data processing frameworks.- Experience with collaborative work in multi-disciplinary teams, particularly with data modelers and designers.- Strong problem-solving skills and attention to detail.- Excellent communication and teamwork abilities.Application deadline: 19-11-2024About UsWe currently employ experienced consultants in areas such as System Development, AI, Machine Learning, Testing, Cloud, Infra DevOps, and IT Project Management.Working at Deploja is something entirely different from working at an ordinary consulting company—and we are really proud of that. So, what makes us unique?We simply offer more than just a high salary and greater freedom; that’s just the beginning. With us, you’ll find the work environment and opportunities you deserve. Welcome to Deploja, where the grass is indeed greener!

Vi erbjuder dig följande, Valet är ditt!

Exciting assignments with our many clients. We focus on long-term projects, which gives you the opportunity to create value for the client while also developing yourself.
A secure monthly salary – together we’ll agree on a base salary depending on your assignments.
Company car – we encourage you to choose an electric car for a more sustainable future.
Occupational pension and great opportunities for salary conversion.
Vacation – Would you like more than 30 days of vacation, perhaps 60 days?
Private health insurance, accident insurance, and life insurance for increased security.
Skills development – you set your own budget and choose the training you wish to attend.
Quality of life – Are you struggling to balance everyday life and wish to work a little less while still earning as much, if not more?
We handle applications continuously, and due to the high workload, it's especially important to include a short motivation for the position, explaining how you meet the requirements.