Use Case Track
Data lossless event time streaming processing for revenue calculation
One of the main characteristics of the good streaming pipeline is correctness for event time processing. Real challenges become when such pipeline should be resilient to different types of failures. In this talk, we describe how Criteo runs Flink on one of the biggest Yarn clusters in Europe and computes 100k messages per second to acknowledge revenue of our platform within the delay of 5 minutes. Real-time revenue monitoring system calculates data under 1% of discrepancies and minimizes business impact in case of revenue anomalies.
Staffs Software Engineer in Criteo, Currently technical lead of Invalid Traffic detection team. Worked for Grammarly in the past. Likes JVM and functional programming. Fun of improving development productivity.