Tokyo 23 wards
October 31, 2019
IT and technology
Title: Data Engineer
We are looking for a passionate data engineer to work on a scalable leading payment platform in Japan! This is a very exciting opportunity, as you will be trusted with important responsibility of building a safe and fast payment solution which serves millions of users.
More about the successful candidate:
- Work directly with Data Analysts and Platform Engineering Team to create reusable experimental and production data pipelines
- Understand, tune, and master the processing engines (like Spark, Hive, Cascading, etc) used day-to-day
- Keep the data whole, safe, and flowing with expertise on high volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc)
- Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit
- Explore as many new technology options for data processing, storage, and share them with the team
- Develop tools and contribute to open source wherever possible
- Previously building data pipelines ingesting and transforming large number of events per minute and terabytes of data per day.
- Experience of Spark and Kafka and know about Flink/Druid/Ignite/Presto/Athena and Preferably, you have worked with cloud based big data processing platform such as AWS EMR, Google Cloud DataProc.
- You are passionate about producing clean, maintainable and testable code part of real-time data pipeline.
- You understand how microservices work.
- You understand issues with ingesting data from applications in multiple data centres across geographies, on-premise and cloud and will find a way to solve them.
- Proficient in Java/Scala/Python/Spark
What's on offer:
- Competitive compensation package. 8M~12M
- Very international team culture
- Energetic environment, and flexible working hours
- Strong product ownership!
TokyoOte Center Bldg, 22F
jp +813 5219 6633