Technology Deep Dive Track
High performance ML library based on Flink
We hope to take this opportunity to share some of our technical tips on building a high-performance library. We have implemented some common machine algorithms based on Flink and achieved comparable or better performance than SparkML. In the development of the algorithm library, we also accumulated intermediate function libraries and tools. Now, we are working with Flink community to contribute back our work to FlinkML to provide more efficient ML algorithms for Flink users. Flink developers can quickly develop more Flink ML algorithms based on our intermediate libraries and tools.
Hao is a software engineer on Airbnb’s Data Platform team. He has been leading the development and driving the adoption of real-time stream processing at Airbnb. Before that, he helped