Senior Data Engineering
Compas.co.id
- Jakarta Selatan, IndonesiaJakarta Selatan, DKI Jakarta, Indonesia, 28, Jalan Haji Abdul Majid Raya, Cipete Utara Kel., Kebayoran BaruJakarta SelatanDKI JakartaIndonesiaIndonesia
- 15.000.000,00 IDR - 20.000.000,00 IDR / monthIDR150000002000000015000000MONTH
- Toàn thời gianFULL_TIME
Đã hết hạn 2 years ago
2022-09-29T17:00:00+00:00Công việc đã đóng.
Miêu Tả Về Công Việc
- Overseeing the collection, storage, management, quality, and protection of data.
- Gather and process raw data at scale (including writing scripts, web scraping, calling APIs, writing SQL queries, etc.)
- Engineering costs optimization
- Lead, motivate, and manage data engineers teams.
- Effectively communicate the status, value, and importance of data collection to executive members and staff.
- Target: - Client data monitoring and notifications
- Crawler stability > 90%
- Mean time to repair crawler incident < 3 days
- Scalable proxy
Trình độ tối thiểu
- Min 4 years programming experience in python, NodeJs and PHP
- Min 2 years experience for web crawling and data cleaning
- Min 2 years experience in queuing and message broker (Kafka, RabbitMQ, Google Cloud Pub/Sub, NATS). min. 1 platform.
- Min 2 years experience with the UNIX environment.
- Min 1 year experience in GCP Product like Bigquery, Dataflow, PubSub, GCS
- Min 1 year experience in protecting web scrapers against site ban, IP leak, browser crash, CAPTCHA and proxy failure.
- Min 1+ years experience with container technology (Kubernetes, Docker)
- Bachelor's degree in information technology or related fields
Kỹ năng cần thiết
- PHP
- Node.js
- Python
Tóm tắt công việc
- Trình Độ Công Việc
- Trình độ nhân viên cấp trung bình / cấp quản lý
- Ngành Nghề
- IT and Software
- Yêu cầu học vấn
- Tốt nghiệp Cử nhân
- Nhà tuyển dụng trả lời hồ sơ
- Often
- Địa chỉ văn phòng
- 28, Jalan Haji Abdul Majid Raya, Cipete Utara Kel., Kebayoran Baru
- Ngành nghề
- Information Technology / IT
- Vị trí đang tuyển
- 1 vị trí tuyển dụng mới
- Trang web
- https://compas.co.id/
Feel secure when applying: look for the verified icon and always do your research on a company. Avoid and report situations when employers require payment or work without compensation as part of their application process.
Về Compas.co.id
Compas is developed by the team that brought Telunjuk.com, a technology company based in Jakarta, Indonesia. We focus on e-commerce business intelligence tools that provide actionable insights to empower you to make strategic decisions for your greater online business.
Compas is developed because we care about your online business development that needs more improvement through data validation. You will be given the market insights from the data that has been collected in our system. We receive multiple insights which were provided by our sophisticated crawler engine, ensuring that we provide you with strong and reliable data, because data will lead your online business into high credibility. We also help you in comparing the performance of your online business and other competitors. Here we are to support you to be on top of the competition.