Technology Deep Dive Track  

How to Join Two Data Streams?

 

Joins are one of the most common operations in SQL. However it is far from trivial how to express and execute them in Streaming environment with continuously running queries.During this talk we will first look into why Join operations are more difficult on infinite data streams. Next we will check couple of different approaches to tackle this problem like Time Windowed Joins or the recent addition to Flink SQL: Temporal Joins. Temporal Tables and Temporal Joins are new concepts that provide an efficient solution to a common problem of for example data enrichment. Before Flink 1.7 data enrichment in SQL was often impossible to express using Windowed Joins or very inefficient when using Regular Joins. With Temporal Joins Flink provide an interesting and ANSI SQL complaint alternative way how to join two data streams.

Authors

Piotr-Nowojski
Piotr Nowojski
Ververica

Piotr Nowojski

Piotr Nowojski is a Software Engineer at Ververica. Previously, he was a Software Engineer in Teradata working on Presto – distributed SQL query engine. Before that, he was developing streaming applications at huge scale in Gemius.