Training

This course is a hands-on introduction to Apache Flink for Java and Scala developers who want to learn to build streaming applications.

After taking this course you will have learned enough about Flink's core concepts, and the DataStream and SQL/Table APIs to be able to develop solutions for a wide variety of use cases, including data pipelines and ETL jobs, streaming analytics, and event-driven applications.

The programming exercises are oriented around common use cases and will give you a chance to see how these APIs are used to solve real problems.

Level

Beginner; no prior knowledge of Apache Flink is assumed.

• Stream Processing and Apache Flink

• Intro to the Streaming Runtime and Execution Model

• Foundations of the DataStream API

• Working with Keyed State

• Dynamic Tables and Streaming SQL

• Streaming Joins

• Event Time and Watermarks

• Process Functions, Side Outputs, and Timers

• Windows and Streaming Analytics

• Temporal Joins and Lookup Joins

• CDC and Changelog Streams

• Checkpointing and Exactly-once Guarantees

• Using Kafka with Flink

• Serialization

• Schema Evolution and State Migration

• Testing

For the hands-on exercises you will need a computer with at least 8 GB RAM (MacOS, Linux, or Windows), with these tools installed:

• Git
• Java 8 or 11 JDK (a JRE is not sufficient)
• An IDE for Java (or Scala) development

Format

This is a remote, instructor-led training, spanning four sessions, each about 2.5 hours long. Hands-on exercises will be assigned as homework to be done between classes.

This course is a hands-on introduction to key topics relating to putting Flink applications into production, including configuring and tuning your job, deployment, operations, and maintenance. The focus is on the Flink runtime, including its architecture, state backends, networking, and checkpointing, with an eye toward understanding how Flink works, and what can go wrong. You’ll learn what you can do to avoid problems, and how to diagnose and fix the issues you may experience, such as backpressure, checkpoint failures, and high latency.

The exercises provide a hands-on introduction to these same topics.

Level

Intermediate

In order to make the most of this training, you should already have a general understanding of event time and watermarking, and familiarity with the basics of the DataStream API (at the level of what is covered in our Apache Flink Developer Training). The troubleshooting examples and exercises all use the DataStream API, and are implemented in Java. They require recognizing what the provided applications are doing wrong, and adjusting the configuration and/or the code so that they perform better.

• Intro to Flink and its Runtime Architecture

• State Backends

• Metrics, Monitoring, and Alerting

• Troubleshooting Watermarks

• Troubleshooting Backpressure

• Optimizing Latency and Throughput

• Fault Tolerance, Tuning and Troubleshooting Checkpointing

• Managing Flink: the CLI, WebUI, and REST API

• Deploying Flink

• Scaling State (RocksDB, state TTL)

Exercises

• Debugging job failures

• Troubleshooting watermarks

• Tuning for latency

• Tuning for throughput

• Troubleshooting checkpoints

• Hands-on with the CLI and Web UI

For the hands-on exercises you will need a computer with at least 8 GB RAM (MacOS, Linux, or Windows), with these tools installed:

• Git
• Java 8 or 11 JDK (a JRE is not sufficient)
• An IDE for Java development
• Docker

Format

This is a remote, instructor-led training, delivered in two 3-hour sessions, with hands-on exercises assigned as homework to be done outside of class.