Senior Data Engineering
Compas.co.id
- Jakarta Selatan, IndonesiaJakarta Selatan, DKI Jakarta, Indonesia, 28, Jalan Haji Abdul Majid Raya, Cipete Utara Kel., Kebayoran BaruJakarta SelatanDKI JakartaIndonesiaIndonesia
- Rp 15.000.000,00 - Rp 20.000.000,00 / bulanIDR150000002000000015000000MONTH
- Penuh waktuFULL_TIME
Berakhir 2 years ago
2022-09-29T17:00:00+00:00Ditutup.
Deskripsi Pekerjaan
- Overseeing the collection, storage, management, quality, and protection of data.
- Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, writing SQL queries, etc.)
- Engineering costs optimization
- Lead, motivate, and manage data engineers teams.
- Effectively communicate the status, value, and importance of data collection to executive members and staff.
- Target: - Client data monitoring and notifications
- Crawler stability > 90%
- Mean time to repair crawler incident < 3 days
- Scalable proxy
Kualifikasi Minimum
- Min 4 years programming experience in python, NodeJs and PHP
- Min 2 years experience for web crawling and data cleaning
- Min 2 years experience in queuing and message broker (Kafka, RabbitMQ, Google Cloud Pub/Sub, NATS). min. 1 platform.
- Min 2 years experience with the UNIX environment.
- Min 1 year experience in GCP Product like Bigquery, Dataflow, PubSub, GCS
- Min 1 year experience in protecting web scrapers against site ban, IP leak, browser crash, CAPTCHA and proxy failure.
- Min 1+ years experience with container technology (Kubernetes, Docker)
- Bachelor's degree in information technology or related fields
Keahlian yang diperlukan
- PHP
- Node.js
- Python
Ringkasan Perkerjaan
- Tingkat Posisi
- Mid-Senior Level Manager
- Spesialisasi
- IT and Software
- Persyaratan tingkat pendidikan
- Lulus program Sarjana (S1)
- Respon rekruter ke lamaran
- Often
- Alamat Kantor
- 28, Jalan Haji Abdul Majid Raya, Cipete Utara Kel., Kebayoran Baru
Agar merasa aman saat melamar: carilah ikon verifikasi dan selalu lakukan riset terhadap Perusahaan yang Anda lamar. Hindari dan laporkan situasi dimana Perusahaan membutuhkan bayaran dalam proses rekrutmen mereka.
Tentang Compas.co.id
Compas is developed by the team that brought Telunjuk.com, a technology company based in Jakarta, Indonesia. We focus on e-commerce business intelligence tools that provide actionable insights to empower you to make strategic decisions for your greater online business.
Compas is developed because we care about your online business development that needs more improvement through data validation. You will be given the market insights from the data that has been collected in our system. We receive multiple insights which were provided by our sophisticated crawler engine, ensuring that we provide you with strong and reliable data, because data will lead your online business into high credibility. We also help you in comparing the performance of your online business and other competitors. Here we are to support you to be on top of the competition.