4 characteristics of Timers in Apache Flink to keep in mind

January 18, 2019 | by Tzu-Li (Gordon) Tai

This blog post describes some basic concepts and considerations for the use of Timers in Apache Flink. Developers can register their own Timers with Flink’s ProcessFunction operator that gives access to some fundamental building blocks for streaming applications such as:

  • events (stream elements)

  • state (fault-tolerant, consistent, only on keyed stream)

  • timers (event time and processing time, only on keyed stream)

For more information on Apache Flink’s ProcessFunction, we suggest reading the Apache Flink 1.7 documentation here for more instructions and guidance.

What are Timers in Apache Flink?

Timers are what make Flink streaming applications reactive and adaptable to processing and event time changes. One of our earlier posts covers the alternative notions of time in Apache Flink and the differences between processing, ingestion and event time in more detail. When processing event streams with Timers, every time the processElement(...) is invoked, a Context object is passed, allowing you to access the element’s event time timestamp and a TimerService. You can then use the TimerService to register callbacks for future event-/processing-time instants. By doing so, once the particular time instant of the timer is reached, the onTimer(...) method will be called.


The onTimer(...) callback is called at different points in time depending on whether processing or event time is used to register the Timers in the first place. In particular:

  • When using processing time to register Timers in your Flink application, the onTimer(...) method is called when the clock time of the machine reaches the timestamp of the timer.

  • When using event time to register Timers in your Flink application, the onTimer(...) method is called when the operator’s watermark reaches or exceeds the timestamp of the timer.

Similar to the processElement(...) method, state access within the onTimer(...) callback is also scoped to the current key (i.e., the key for which the timer was registered for).

It is worth noting here that both the onTimer(...) and processElement(...) calls are synchronized, and thus it is safe to access state and modify it in both the onTimer(...)and processElement(...) methods.

4 characteristics of Timers to keep in mind

In this paragraph, we discuss the 4 basic characteristics of  Timers in Apache Flink that you should keep in mind before using them. These are the following:

1. Timers are registered on a KeyedStream

Since timers are registered and fired per key, a KeyedStream is a prerequisite for any kind of operation and function using Timers in Apache Flink.


2. Timers are automatically deduplicated

The TimerService automatically deduplicates Timers, always resulting in at most one timer per key and timestamp. This means that when multiple Timers are registered for the same key or timestamp, the onTimer() method will be called just once.


3. Timers are checkpointed

Timers are checkpointed by Flink, just like any other managed state. When restoring a job from a Flink checkpoint or savepoint, each registered Timer in the restored state that was supposed to be fired before restoration will be fired immediately.


4. Timers can be deleted

As of Flink 1.6, Timers can be paused and deleted. If you are using a version of Apache Flink older than Flink 1.5 you might be experiencing a bad checkpointing performance due to having many Timers that cannot be deleted or stopped.

4 characteristics of timers in Apache Flink

You can stop the processing time Timers using the following command:


long timestampOfTimerToStop = ...

ctx.timerService( ).deleteProcessingTimeTimer (timestampOfTimerToStop);

You can also stop the event time Timers by following the command:


long timestampOfTimerToStop = ...

ctx.timerService( ).deleteEventTimeTimer (timestampOfTimerToStop);


It is worth mentioning here that stopping a Timer has no effect if no such Timer with the given timestamp is registered.

All of the concepts mentioned above are covered extensively in the dA Apache Flink Standard and Advanced Trainings. If you want to get some hands-on experience working with Timers and building stateful streaming applications, we encourage you to check our training schedule for Q1 2019 below. If you have questions or comments for us, we look forward to hearing from you below.

Public Training Flink, Flink training, Ververica training, Apache Flink training

Ververica Contact

 

 

 

 

 

Topics: Apache Flink

Tzu-Li (Gordon) Tai
Article by:

Tzu-Li (Gordon) Tai

Find me on:

Related articles

Comments

Sign up for Monthly Blog Notifications

Please send me updates about products and services of Ververica via my e-mail address. Ververica will process my personal data in accordance with the Ververica Privacy Policy.

Our Latest Blogs

by Nico Kruber May 11, 2021

SQL Query Optimization with Ververica Platform 2.4

In my last blog post, Simplifying Ververica Platform SQL Analytics with UDFs, I showed how easy it is to get started with SQL analytics on Ververica Platform and leverage the power of user-defined...

Read More
by Jun Qin March 29, 2021

The Impact of Disks on RocksDB State Backend in Flink: A Case Study

As covered in a recent blog post, RocksDB is a state backend in Flink that allows a job to have state larger than the amount of available memory as the state backend can spill state to local disk....

Read More
by Konstantin Knauf March 10, 2021

Announcing Ververica Platform 2.4

Newest release adds full support for Flink SQL and Flink 1.12, and improves resource utilization via new shared session clusters.

Read More