Data Engineer

What do we expect from you
Experience working with orchestration tool (ex: Airflow)
Experience working with a large data set
Experience building complex ETL pipelines
Experience working with cloud providers such as GCP, AWS or Azure
Strong programming skills in Spark with Scala and Python
Experience working with CI/CD tools like Jenkins and Git
Strong understanding of Software Engineering practices and principles
Excellent problem solving and communication skills.
Self-motivated and have a proven ability to take initiative to own the problems that come up and solve them.


What will you work on
Maintaining and improving the ingestion pipeline to reliably deliver billions of events daily in defined SLA
Provide support for all teams in building and optimizing their complex pipelines
Work closely with other teams to identify pain points and problems around the platform
Develop new tools and frameworks to improve the data platform
Facilitate company wide to be data-driven
Work in close collaboration with data scientists and data analysts to help support their work go to production
Work with vast projects such as building an ML platform and streaming use cases
Setup best practices and processes around software and data development



It would be great if you also have
Working with messaging system like Kafka
Knowledge in Kubernetes
Hands on experience with any streaming platform
Experience managing data warehouse in BigQuery or Redshift