Training

About

This course is a hands-on introduction to Apache Flink for Java and Scala developers who want to learn to build streaming applications.

After taking this course you will have learned enough about Flink's core concepts and the DataStream and SQL/Table APIs to be able to develop solutions for a wide variety of use cases, including data pipelines and ETL jobs, streaming analytics, and event-driven applications.

The programming exercises are oriented around building blocks used in common use cases, and will give you a chance to see how these APIs are used to solve real problems.

Level

Beginner; no prior knowledge of Apache Flink is assumed.

Format

This is a remote, instructor-led training, spanning 4 sessions, each about 2 hours long. These instructor-led training sessions are not recorded, so please arrange your schedule so you can attend and participate.

Additional reading and hands-on exercises will be assigned as homework to be done between classes. This is a critical component of the course, and you should plan to spend at least a couple of hours after each session working with these self-paced materials. Give yourself extra time if you aren’t very familiar with Java (or Scala).

All of the slides will be shared with you.

Self-paced Content and Exercises

• Hands-on with Keyed State
• State Backends
• Hands-on with Timers
• Hands-on with Windows
• Side Outputs
• Hands-on with Flink SQL
• Table/DataStream Integration

Course Content

• Stream Processing and Apache Flink
• Intro to the Streaming Runtime and Execution Model
• Working with Keyed State
• Checkpointing and Exactly-once Guarantees
• Event Time and Watermarks
• Process Functions and Timers
• Dynamic Tables and Streaming SQL
• Windows and Streaming Analytics
• Streaming Joins
• Using Kafka with Flink
• Serialization
• Testing

Prerequisites

For the hands-on exercises you will need a computer with at least 8 GB RAM (MacOS, Linux, or Windows), with these tools installed:

• Git
• Java 8 or 11 JDK (a JRE is not sufficient)
• An IDE for Java (or Scala) development (we recommend IntelliJ)

About

This course is a hands-on introduction to key topics relating to putting Flink applications into production. The focus is on the Flink runtime, including its architecture, state backends, networking, and checkpointing, with an eye toward understanding how Flink works, and what can go wrong. You’ll learn what you can do to avoid problems, and how to diagnose and fix the issues you may experience, such as backpressure, checkpoint failures, and high latency.

The self-paced exercises provide a hands-on introduction to these same topics.

Level

Intermediate

You should already have a general understanding of event time and watermarking, and familiarity with the basics of the DataStream API (at the level covered in our Apache Flink Developer Training). The examples and exercises all use the DataStream API, and are implemented in Java. They require recognizing what the provided applications are doing wrong and adjusting the configuration and/or the code so that they perform better.

Format

This is a remote, instructor-led training, delivered in two 2-hour sessions. These instructor-led training sessions are not recorded, so please arrange your schedule so you can attend and participate.

Additional reading and hands-on exercises will be assigned as homework to be done between classes. This is a critical component of the course, and you should plan to spend several hours between the sessions working with these self-paced materials.

All of the slides will be shared with you.

Self-paced Content and Exercises

• Metrics for Monitoring and Troubleshooting
• Debugging Job Failures
• Troubleshooting Watermarks
• Tuning for Latency
• Tuning for Throughput
• Troubleshooting Checkpoints

Course Content

• State Backends
• Tuning Serialization
• Backpressure
• Optimizing Latency and Throughput
• Fault Tolerance, Tuning and Troubleshooting Checkpointing
• Mitigating Skew
• Scaling State with RocksDB
• Application Evolution
• The Flink Runtime

Prerequisites

For the hands-on exercises you will need a computer with at least 8 GB RAM (MacOS, Linux, or Windows), with these tools installed:

• Git
• Java 8 or 11 JDK (a JRE is not sufficient)
• An IDE for Java development (we recommend IntelliJ)