Data Engineer

Data Engineer

Arbetsbeskrivning

We are looking for a GCP Data Engineering Specialist for the following responsibilities:
· Design, develop and operate production-grade data pipelines for data ingestion and processing, enabling downstream pipelines and analytics products, considering both technology and business requirements
· Design data compliance and access management of personally identifiable sensitive datasets in bigquery
· Work with and contribute to a DevOps setup (continuous integration, and deployment) on GCP
· Set up monitoring and alerting for data ops, quality and availability
· Work closely with Product Owner and business stakeholders to ensure business value realization as part of a cross-functional agile team with Product Owner, Data Scientists/Analysts/Stewards and Data/ML/Software/DevOps Engineers.
Key responsibilities:
Extract data from external source systems
Perform data profiling to validate data quality
ETL development using Googles Big Query and Matillion
Prepare data and access management for advanced analytics and self-service
Create ingestions patterns that allow integration with external data sources
Mapping tables for additional dimensions

Additionally:
Flexible with agile mindset, be prepared to handle ad hoc requests from stakeholders, due to some things are implemented on the fly Familiar with Agile, Scrum
Self-driven, action- and goal-oriented, good communication to technical and non-technical stakeholders
Tools and techniques:
Google Cloud Platform.
Data Engineering: developing data pipelines/ETL for data lakes, data warehouses and data marts.
Complex SQL for processing and analysis using Bigquery
Programming: Python and preferably also JavaScript. Other languages meriting.
DevOps, DataOps, CI/CD, Infrastructure as code/config
Apache Spark / Beam / Dataflow
Apache Airflow
Meriting
Azure SQL Database (SSMS) as data source
Azure portal (assigning roles and resources for Data scientists or other consumers, setting up Logic Apps, creating/managing data lakes)
Power Platform (Power BI, PowerApps, Power Automate)
What 3 things from the box above are most important?
· GCP Data Engineering/Warehousing
· SQL
· Programming

Sammanfattning

  • Arbetsplats: Kairos Blue AB Lund
  • 5 platser
  • Tills vidare
  • Heltid
  • Fast och rörlig lön
  • Publicerat: 19 januari 2022
  • Ansök senast: 1 februari 2022

Postadress

Gandalvs Gränd 6
Lund, 22474

Liknande jobb


Computer Vision and Perception Engineer

Computer Vision and Perception Engineer

26 november 2024

Simulation Engineer

19 september 2024

Data Analyst

25 november 2024