Training

This course is a hands-on introduction to Apache Flink for Java and Scala developers who want to learn to build streaming applications.

After taking this course you will have learned enough about Flink's core concepts, its DataStream API, and its distributed runtime to be able to develop solutions for a wide variety of use cases, including data pipelines and ETL jobs, streaming analytics, and event-driven applications. 

The programming exercises are oriented around common use cases, and will give you a chance to see how the pieces of the API work together to solve real problems.

• Introduction to Stream Processing and Apache Flink

• Runtime Architecture

• Foundations of the DataStream API

• Data Pipelines and Stateful Stream Processing

• Event Time and Watermarks

• Process Functions, Side Outputs, and Timers

• Windows and Streaming Analytics

• State Backends

• Fault Tolerance

• Connector Ecosystem

• Application Evolution: Rescaling, Upgrades, State Migration

• Intro to Flink SQL and the Table API

• Use Cases and Application Patterns

• Testing

No prior knowledge of Apache Flink is required.

For the hands-on exercises you will need a computer with at least 8 GB RAM (MacOS, Linux, or Windows), with these tools installed:

• Git

• Java 8 or 11 JDK (a JRE is not sufficient)

• An IDE for Java (or Scala) development

• Docker

Format

As a remote, instructor-led training, this is delivered as three sessions, each about 3 hours long, with the hands-on exercises assigned as homework to be done outside of class.

This course is a hands-on introduction to key topics relating to putting Flink applications into production, including configuring and tuning your job, deployment, operations, and maintenance. Also included is a solid introduction to the organization of the Flink runtime to help you tune and troubleshoot your applications. The exercises provide a hands-on introduction to these same topics.

The intended audience includes both developers and operations staff.

• Intro to Flink and its Runtime Architecture

• State Backends

• Metrics, Monitoring, and Alerting

• Troubleshooting Watermarks

• Troubleshooting Backpressure

• Optimizing Latency and Throughput

• Fault Tolerance, Tuning and Troubleshooting Checkpointing

• Managing Flink: the CLI, WebUI, and REST API

• Deploying Flink

• Scaling State (RocksDB, state TTL)

Exercises

• Debugging job failures

• Troubleshooting watermarks

• Tuning for latency

• Tuning for throughput

• Troubleshooting checkpoints

• Hands-on with the CLI and Web UI

In order to make the most of this training, you should already have a general understanding of event time and watermarking, and some familiarity with the DataStream API. The troubleshooting exercises require recognizing what the provided applications are doing wrong, and adjusting the configuration and/or the code so that they perform better.

For the hands-on exercises you will need a computer with at least 8 GB RAM (MacOS, Linux, or Windows), with these tools installed:

• Git

• Java 8 or 11 JDK (a JRE is not sufficient)

• An IDE for Java development

• Docker

Format

As a remote, instructor-led training, this is delivered in two 3-hour sessions, with hands-on exercises assigned as homework to be done outside of class.