Job closed.
Job Description
As a Data Engineer, you will be partly a software engineer, a platform engineer, and a system engineer. You will design and grow the Shopee data platform to support a variety of big data applications using open-source technologies including Kafka, Hadoop, Presto, HBase, Spark, Hive, Druid, and our own creations. Some examples include a real-time data streaming platform, a unified query platform, a cluster management system, and a machine learning platform.
Job descriptions:
- Design, build, maintain, and scale our data infrastructure. In doing so, you will also have to monitor and debug the infrastructure that stream millions of data
- Explore and learn new technologies that can complement or improve our current stack
- Develop data ingestion and integration from data sources to data lake and/or data warehouse, based on data pipeline and framework, into query-able format
- Performance tuning and optimization of all data ingestion and data integration processes, including data platform and databases
- Work with regional team to allow centralised data warehousing
- Work with data analytics team to strive greater functionality in our data system
Minimum Qualifications
Requirements:
- Bachelor / Master / PhD in Computer Science or a related technical field
- At least 2 years working experience in software development, in at least one of these languages: Java, Scala, Go, Python, C/C++, under Linux / Unix
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores
- Uphold best practices and principle around clean code, testing, continuous integration
- Familiar with SQL and/or noSQL databases, eg MySQL, PostgreSQL, MongoDB etc
- Love to use and develop with open-source technologies
Skills below are optional but preferable:
- Experience with cloud platforms, eg Google Cloud Platform, Amazon Web Services
- Familiarity with writing ETL queries
- Strong scripting ability in Bash
- Familiarity with designing and operating of a robust distributed system.
- Experience in manipulating, processing and extracting value from large disconnected datasets
- Experience with big data tools: Hadoop, Spark, Kafka, etc
Jobs Summary
- Job Level
- Associate / Supervisor
- Job Category
- IT and Software
- Educational Requirement
- Bachelor's degree graduate
- Recruiter response to application
- Once in a while
- Office Address
- 26, Pacific Century Place SCBD Lot 10, Jl. Jendral Sudirman Kav 52-53, Jakarta 12190, Indonesia, South Jakarta, DKI Jakarta, Indonesia
About Shopee
Shopee is the leading e-commerce platform in Southeast Asia & Taiwan. Shopee promotes an inclusive and sustainable digital ecosystem by enabling businesses to digitalise and grow their online presence, helping more people access and benefit from digital services, and uplifting local communities.
Shopee offers an easy, secure, and engaging experience that is enjoyed by millions of people daily. Shopee is also a key contributor to the region’s digital economy with a firm commitment to helping homegrown brands and entrepreneurs succeed in e-commerce.
Shopee is part of Sea Limited (NYSE: SE), a leading global consumer internet company. Sea’s mission is to better the lives of consumers and small businesses with technology through its three core businesses: Shopee, Garena and SeaMoney.