Data Engineer


Location:

Tokyo 23 wards

Date posted:

June 24, 2019

Employment type:

Permanent

Job reference:

NY33-BT

Specialisation:

IT and technology


Description

Title: Data Engineer


We are looking for a passionate data engineer to work on a scalable leading payment platform in Japan! This is a very exciting opportunity, as you will be trusted with important responsibility of building a safe and fast payment solution which serves millions of users.


More about the successful candidate:



  • Work directly with Data Analysts and Platform Engineering Team to create reusable experimental and production data pipelines

  • Understand, tune, and master the processing engines (like Spark, Hive, Cascading, etc) used day-to-day

  • Keep the data whole, safe, and flowing with expertise on high volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc)

  • Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit

  • Explore as many new technology options for data processing, storage, and share them with the team

  • Develop tools and contribute to open source wherever possible


Qualifications:



  • Previously building data pipelines ingesting and transforming large number of events per minute and terabytes of data per day.

  • Experience of Spark and Kafka and know about Flink/Druid/Ignite/Presto/Athena and Preferably, you have worked with cloud based big data processing platform such as AWS EMR, Google Cloud DataProc.

  • You are passionate about producing clean, maintainable and testable code part of real-time data pipeline.

  • You understand how microservices work.

  • You understand issues with ingesting data from applications in multiple data centres across geographies, on-premise and cloud and will find a way to solve them.

  • Proficient in Java/Scala/Python/Spark


What's on offer:



  • Competitive compensation package. 8M~12M

  • Very international team culture

  • Energetic environment, and flexible working hours

  • Strong product ownership!



Tokyo

Ote Center Bldg, 22F
1-1-3 Otemachi
Chiyoda-ku, Tokyo
100-0004
jp
+813 5219 6633
eCommerce Manager
Tokyo 23 wards
Permanent
July 12, 2019
Accountant
Osaka
Permanent
July 12, 2019
Sales Representative @ RPA vendor
Tokyo 23 wards
Permanent
July 12, 2019
Sales @ Data Analytics Vendor
Osaka
Permanent
July 12, 2019
Enterprise Sales (Machine Data, IoT, SIEM)
Tokyo 23 wards
Permanent
July 12, 2019
Accounting Manager
Osaka
Permanent
July 12, 2019
BigData Platform Sales
Tokyo 23 wards
Permanent
July 12, 2019
Finance Manager
Hyogo
Permanent
July 12, 2019
Account Executive
Osaka
Permanent
July 12, 2019
Sr. Financial Analyst
Tokyo 23 wards
Permanent
July 11, 2019
Head of Artificial Intelligence
Tokyo 23 wards
Permanent
July 11, 2019
SENIOR CLOUD ARCHITECT
Tokyo 23 wards
Temporary
July 11, 2019
Junior Controller
Tokyo 23 wards
Permanent
July 11, 2019
Finance Manager
Tokyo 23 wards
Permanent
July 9, 2019
Finance Controller
Okayama
Permanent
July 9, 2019
See more jobs >