Operations Track  

Developing and operating real-time applications with Oceanus

The Tencent Data team (data.qq.com) is responsible to build a reliable and scalable infrastructure for various products at Tencent including Tencent Games, Tencent Video and WeChat. We have to process over 1.6 trillion events per day and the peak throughput reaches 210 million per second. Such a large scale exposes great challenges in the developing and operating of real-time applications. In this talk, we will give an introduction to Oceanus, a platform built at Tencent to facilitate the development of real-time applications. We will introduce typical user cases and how applications are developed, deployed and monitored with Oceanus. We will also introduce the management of Flink jobs in Oceanus, together with some experience gained in our operating practice. Finally, we will discuss some efforts we have done to enhance the reliability and efficiency, e.g. master failover and local keyed streams.

Authors

Xiaogang Shi
Xiaogang Shi
Tencent

Xiaogang Shi

Xiaogang is a committer of the Apache Flink project and a senior software engineer at Tencent. His current work is focused on building a platform to facilitate the development of real-time applications at scale. He holds a Ph.D. in computer science from Peking University.