Stream processing and Apache Flink Best Practices, Use Cases and more

Apache Flink, Apache Kafka, Kafka connector, exactly-once guarantees, exactly-once, stream processing

How Apache Flink manages Kafka consumer offsets

October 12, 2018

In this blog post, we explain how Apache Flink works with Apache Kafka to ensure that records from Kafka topics are processed with exactly-once guarantees, using a step-by-step example.

Checkpointing is Apache Flink’s internal mechanism to...

6 things to consider when defining your Apache Flink cluster size

October 05, 2018

This is the first episode of our new blog series called Flink Friday Tip. Flink Friday Tips provide some easy-to-digest best practices and advice on how to leverage the power of Apache Flink and make the best use of the different Flink features....

State TTL for Apache Flink: How to Limit the Lifetime of State

September 25, 2018

A common requirement for many stateful streaming applications is the ability to control how long application state can be accessed (e.g., due to legal regulations like GDPR) and when to discard it. This blog post introduces the state time-to-live...

Latest improvements for Apache Flink with the release of Flink 1.5.4 and 1.6.1

September 21, 2018

The Apache Flink® community just released two bugfix versions, Apache Flink 1.5.4 and Apache Flink 1.6.1 including many resolved issues and improvements.

A Practical Guide to Broadcast State in Apache Flink

August 02, 2018

Since version 1.5.0, Apache Flink^Ⓡ features a new type of state which is called Broadcast State. In this post, we explain what Broadcast State is, and show an example of how it can be applied to an application that evaluates dynamic patterns on...

Apache Flink 1.5.0 Release Announcement

May 25, 2018

This post originally appeared on the Apache Flink blog. It was reproduced here under the Apache License, Version 2.0. The Apache Flink community is thrilled to announce the 1.5.0 release. Over the past 5 months, the Flink community has been...

Continuous Queries on Dynamic Tables: Analyzing Data Streams with Streaming SQL

April 11, 2017

Xiaowei Jiang is a Senior Director in Alibaba’s Search Infrastructure division, and Shaoxuan Wang is a Senior Manager in Alibaba's Search Infrastructure division. Fabian Hueske (@fhueske) is a co-founder and software engineer at data Artisans and...

Savepoints, Part 2: Streaming Applications in the Pit Lane

November 16, 2016

By Fabian Hueske (@fhueske) and Mike Winters (@wints) Last month, we gave a high-level overview of Apache Flink® savepoints and touched on why and how you’d reprocess data in a streaming application. If you haven’t already read that post, or if...