Flink Forward San Francisco Preview: Real-time Anomaly Detection with Mux

March 02, 2017 | by Michael Winters

Last month, software engineer Scott Kidder (@hexdumpster) of Mux published a popular post on his company's blog titled "Discovering Anomalies in Real-Time with Apache Flink". The post covered Mux's process for adding anomaly-detection alerting to its product, starting with evaluating different streaming frameworks through to which Apache Flink® operators are used in the application.

Mux's event pipeline from their blog post
 A diagram from the Mux post
 

Lucky for all of us, the blog post was just a preview of things to come, and Scott will be presenting on the topic at the first-ever Flink Forward in San Francisco on April 11. From his talk's abstract:

Mux uses Apache Flink to identify anomalies in the distribution & playback of digital video for major video streaming websites. Scott Kidder will describe the Apache Flink deployment at Mux leveraging Docker, AWS Kinesis, Zookeeper, HDFS, and InfluxDB. Deploying a Flink application in a zero-downtime production environment can be tricky, so unit- & behavioral-testing, application packaging, upgrade, and monitoring strategies will be covered as well.


His talk is timely, both because of Flink's increasing popularity in real-time alerting & anomaly detection applications and because of his discussion of deployment-related issues--an area where much is happening in the Flink community.


San Francisco-based Mux is a Y Combinator graduate whose product provides monitoring and analytics for streaming video. The company was founded by experts in the space from the likes of Brightcove and Zencoder, and Scott himself has been working with video for more than 10 years. By the way, they're hiring.


To see Scott's talk along with 25 others, buy your ticket to Flink Forward San Francisco today. P.S. For more recommended reading, Scott also published a nice piece about using Flink's Amazon Kinesis connector and building the connector source. Check it out.

Topics: Flink Forward

Article by:

Michael Winters

Related articles

Comments

Sign up for Monthly Blog Notifications

Please send me updates about products and services of Ververica via my e-mail address. Ververica will process my personal data in accordance with the Ververica Privacy Policy.

Our Latest Blogs

Announcing Ververica Platform 2.5 for Apache Flink 1.13

New release includes full support for Apache Flink 1.13, with greatly expanded streaming SQL, new performance monitoring, and many new application management features.

Read More
by Nico Kruber May 11, 2021

SQL Query Optimization with Ververica Platform 2.4

In my last blog post, Simplifying Ververica Platform SQL Analytics with UDFs, I showed how easy it is to get started with SQL analytics on Ververica Platform and leverage the power of user-defined...

Read More
by Jun Qin March 29, 2021

The Impact of Disks on RocksDB State Backend in Flink: A Case Study

As covered in a recent blog post, RocksDB is a state backend in Flink that allows a job to have state larger than the amount of available memory as the state backend can spill state to local disk....

Read More