Stream Processing & Apache Flink - News and Best Practices

Announcing Flink Community Packages

Written by Robert Metzger | 18 November 2019

Today, we are excited to announce the launch of Flink Community Packages — a website that showcases the work of the community to push forward the ecosystem surrounding Apache Flink®. Users can explore existing packages like connectors, libraries or other extensions, and also submit their own contributions to the ecosystem.

Why are we launching this?

Earlier this year, there was a proposal on the Flink mailing list to create a centralized place for the community to share, discuss and discover Flink-related projects. The idea quickly gained traction, since the existing solution was a combination of searching the internet for these code bits and a simple, static webpage on the Flink website that users could extend by opening a Pull Request on GitHub. Looking to improve this setup and allow for a more collaborative experience, we started working on implementing what is now Flink Community Packages.

As an open-source community initiative, the goal was to make it vendor-independent — by and for the Flink community. Due to legal constraints, we were unable to do so. Nevertheless, this initiative will operate independently, since the source code of the website is made publicly available, free for everyone to inspect, use and customize for their own needs.

How does it work?

Packages are organized in categories and have dedicated pages detailing the original website, source code location, associated license and tags, as well as any votes or comments from the community. We have pre-populated the website with some great examples of existing third-party packages for Flink, such as Flinkk8soperator, a Kubernetes operator developed by Lyft.

You can contribute a package or make use of the interactive features on the page by authenticating with your GitHub account. You are then allowed to contribute, vote and comment on any package.

Moving forward

With this initiative, we aim to make Flink Community Packages the place for community users to explore all the extensions available outside the core Flink repository and get recognition for contributions focused on expanding the ecosystem. This is, in our opinion, a better and more sustainable way to address such contributions, which often get lost in the fast pace of development of the core repository.

There are short-term plans to improve the website with new features, such as a “Supported Flink Version(s)” field in the detail pages and support for releases. We encourage you to reach out (or start a thread in the Flink mailing list) with feedback and ideas for improvements to the website.