Flink Forward 2025 Barcelona: The Future of AI is Real-Time

Buy Ticket

Start for Free
Request a Trial

How to Drain a Job in Ververica Platform

Question

How do I drain a Flink Job in Ververica Platform?

Answer

Note: This applies to Ververica Platform 2.1. The same can be achieved via a toggle button available in 2.2 and later.

When you stop your Flink job, before the pipeline stops, you may want it to stop processing any further events from sources, finish processing all in-flight events, and also commit them into sinks. This procedure is called draining the job and works by sending MAX_WATERMARK before stopping the pipeline. Since Flink 1.9, this can be achieved with

bin/flink stop -d ... <Job ID>

To drain Flink jobs running on Ververica Platform, set the following annotation in your Ververica Platform Deployment:

metadata:
  annotations:
    com.dataartisans.appmanager.controller.deployment.stop-with-drain: 'true'

Then, whenever your job transitions to SUSPENDED state (e.g., by clicking the button Suspend), it will be drained first before a savepoint is created.

If you remove this annotation again, Ververica Platform will revert back to the original behavior, i.e., no drain.

Related Information