Technology Deep Dive Track
Upgrading Apache Flink Applications: State of the Union
Apache Flink streaming applications are typically designed to run indefinitely for long periods of time. As with all long-running services, the applications need to be maintained and upgraded, including improvements to adapt to changing business logic and bug fixes. With this in mind, being able to smoothen the process of upgrading Flink streaming applications, as well as enabling users to have enough flexibility in doing so, has always been one of the priorities of the Flink community.
This talk will provide an overview of common considerations when it comes to upgrading Flink streaming applications, and progress or discussions the community has made so far in tackling each one of them. Specifically, topics include 1) Application state compatibility and allowing schema evolution, 2) Application topology changes, 3) Swapping state backends, and 4) Pre-flight detection of incompatible upgrades.
Tzu-Li (Gordon) Tai is an Apache Flink PMC member and software engineer at Ververica. His main contributions in Apache Flink includes work on some of the most widely used Flink connectors (Apache Kafka, AWS Kinesis, Elasticsearch). Gordon was a speaker at conferences such as Flink Forward, Strata Data, as well as several Taiwan-based conferences on the Hadoop ecosystem and data engineering in general.