How Ververica Platform's and Apache Flink's Kubernetes HA Work and Where They Differ

Ververica Platform 2.8 or later

kind: Deployment  # this is Ververica Platform deployment, not the Kubernetes one
spec:
  template:
    spec:
      flinkConfiguration:
        high-availability: kubernetes
        high-availability.storageDir: /path/to/ha/store # optional, see more below
        high-availability.vvp-kubernetes.job-graph-store.enabled: true  # optional,see more below

Ververica Platform 2.7 or earlier

kind: Deployment  # this is Ververica Platform deployment, not the Kubernetes one
spec:
  template:
    spec:
      flinkConfiguration:
        high-availability: vvp-kubernetes
        high-availability.storageDir: /path/to/ha/store # optional, see more below
        high-availability.vvp-kubernetes.job-graph-store.enabled: true  # optional,see more below

Unlike Apache Flink's Kubernetes HA, this approach starts a single JobManager (JM) and manages it with a Kubernetes Jobresourcewith the following specification:

kind: Job
spec:
  completions: 1
  parallelism: 1

Whenever that JM crashes, i.e., not exited with 0, Kubernetes will launch another JM as a replacement. With the help of high availability storage, the new JM knows where to start. The high availability storage is configured automatically if Ververica Platform's Universal Blob Storage is configured.

In addition, with the configuration high-availability.kubernetes.job-graph-store.enabled: true or high-availability.vvp-kubernetes.job-graph-store.enabled:true, you can also store the generated job graph in the high availability storage. Upon JM failover,instead of re-executing your job's main() to generate the job graph, the stored job graph will be used.

Comparison

In all Kubernetes HA approaches described above, when the active JM (the leading JM in the case of Apache Flink's Kubernetes HA, the single JM in the case of Ververica Platform's original and new Kubernetes HA) is lost, the TaskManagers (TMs) who are talking to the failed JM will fail the tasks running on them. This leads to a job restart from the previous checkpoint. Flink's restart strategy controls how restarts are handled.

In terms of failover and recovery time, Apache Flink's Kubernetes HA can switch over to one of the stand-by JM quickly as it is already started. With Ververica Platform's Kubernetes HA, a new JM pod needs to be launched and started. But the execution of the job's main() can be skipped if you enable the job graph store. The actual recovery time depends on your job and your cluster environment.

Another difference is that Ververica Platform's Kubernetes HA runs a single leader election process for the entire JM process while Apache Flink's Kubernetes HA runs multiple of them, one for Dispatcher, one for ResourceManager, one for JobManager, etc. Flink 1.15 followed the same route as Ververica Platform did to implement amultiple-component leader election service. See FLINK-24038for more details.

The biggest advantage of Ververica Platform's Kubernetes HA is that, if you set RestoreStrategy=LATEST_STATE in your deployment, even if the job fails completely (e.g., exhausted the configured restart attempts) after the underlying issue is fixed, the job can still restart from the latest state (i.e., the previously completed checkpoint or savepoint).

Summary

This table summarizes the comparison between both Kubernetes HA approaches.

Related Information

• Ververica Platform's Kubernetes HA
• Apache Flink's Kubernetes HA
• FLINK-24038: Flink moves to single leader election in Flink 1.15
• Ververica Platform's LATEST_STATE RestoreStrategy
• Kubernetes: Job