Data Engineer  

Responsibilities:

• Design, build, and maintain high-performance and scalable data ingestion and transformation pipelines using modern technology such as serverless cloud data warehouse, CI/CD source control, and open source workflow orchestrator
• Maintain good communication with Software Engineers, Product Managers, and Business Stakeholders of varying functions to ensure the veracity and the timeliness of the data processed
• Collaborate closely with Data Analysts, Data Scientists and Machine Learning Engineers in delivering reliable data products
• Maintain high-quality technical documentation in company's internal Wiki and conduct regular knowledge sharing, hands-on training, and mentoring for the less experienced team members
• Be the VP's and Data Manager's technical counsellor/researcher in designing a future-ready data infrastructure

Qualifications:

• 3+ years of Data Engineering, or equivalent
• Fluency in Python, SQL, and Bash scripting for data ingestion and transformation pipeline
• Familiarity with various flavour of relational SQL databases such as Postgres and MySQL among others
• Experience in setting up batch workflow orchestrator using Apache Airflow, Prefect, Dagster, Azkaban, or equivalent technologies
• Experience in setting up ELT pipelines using a serverless Cloud Data Warehouse, preferably Google BigQuery or AWS Redshift
• Familiarity with Apache Spark for batch data processing
• Experience in setting up data pipeline CI/CD using tools such as Jenkins or Gitlab CI
• Familiarity with standard Software Engineering practices such as object oriented programming, modular design, logging, and alerting, among others