Skip to content

Ververica Cloud, a fully-managed cloud service for stream processing!

Learn more

Updates to Ververica Platform


by

Ververica has been tirelessly working on several new products and projects in the past couple of months, including the release of Ververica Cloud, which you can read more about here.

We are always dedicated to working with the community, customers, gathering feedback, and using knowledge from our own expertise and experience to deliver products with the features that can deliver the best user experience and business value.

On top of offering new possibilities to work with Apache Flink with our new products, we have continued to bring new improvements to Ververica Platform, which has seen two new releases (2.9, 2.9.1, 2.10, 2.10.1). This post will take a look at what new features have been added to those releases.

What’s new in Ververica Platform 2.9

As you may remember about the change in our release process, Ververica Platform minor versions will correspond to either a new minor version of Flink or a new major Ververica Platform feature. Thus, Ververica Platform 2.9.0, 2.9.1 continues to support Apache Flink® 1.15 and Apache Flink® 1.14 under SLA. Per our support policy, Apache Flink® 1.13 is deprecated in this version and supported on a best-effort basis.

Running PyFlink Jobs

You can now write Flink jobs in Python or call Python UDFs and run them on Ververica Platform much easier with the release of 2.9, where we provide a Python interpreter and Pyflink in our built-in images out-of-the-box. 

Previously, in order to run Pyflink jobs on the platform, you had to build a new custom Flink image that included a Python interpreter and PyFlink, which you can read more about here.

Changes to the Community Edition

We are starting to change the terms of the Community Edition of Ververica Platform as we start to revamp the business direction of our products. Starting from this release, CPU cores are limited to 10 and the new Flink High Availability Service will only be available in the Stream Edition.

Deployment improvements

We regularly ask for user feedback and have now made it easier to change the state of your deployments in one location on the UI. Now all you have to do is navigate to the Deployments section or the Session Clusters section in order to manage the state (“start”, “suspend”, “cancel”) of all of your deployments. In addition, we have added a Restart button (on the Deployments overview page) which allows you to restart a running deployment with one click, instead of manually stopping a certain job and then starting it again.

In this release, we have also enabled automatic calculation of the number of TaskManagers based on the parallelism and number of task slots set. The default number of TaskManagers has also been set to 1 (taskmanager.numberofTasksSlots = 1).

What’s new in Ververica Platform 2.10

With the addition of support for Apache Flink 1.16 in Ververica Platform 2.10, we are sunsetting our support for Apache Flink 1.13.  Per our support policy, Ververica Platform 2.10 is supported for use with Apache Flink 1.14 and higher.

Flink SQL version upgraded to 1.16.1

Flink SQL has been upgraded to 1.16.1 on the platform. This means that the SQL editor now includes new syntax such as “CREATE TABLE AS SELECT”. You can have a look at the list of changes to the Table API and SQL in the release notes.

PyFlink Improvements

Besides SQL and Java, Python is another popular API supported by the Flink community and so we want to make it seamless to support PyFlink jobs on the Ververica Platform. 

The 2.10 release supports submitting PyFlink jobs natively and you can now manage PyFlink deployments directly in the UI in this release.

Deployment Improvements

In terms of deployment, we have also added a validation check of non-empty Service Account names in Kubernetes resources. This enforces a one-to-one mapping between [VVP Namespace + Deployment/Session Cluster Name] and the [Service Account Name] and allows administrators to safely let the users work with their own permissions. Previously, users can specify any Service Account to a Deployment/Session Cluster via Flink Pod Templates.

Moreover, we improved artifact fetching via HTTP Basic Auth when deploying in application mode, which means credentials are no longer revealed in the URL when performing basic authentication.

Integration with Azure Kubernetes Service

In this release, users are now able to use Azure Blob Storage to store information about checkpoints, savepoints, and artifacts directly in their Microsoft Azure accounts. By utilizing Azure workload identity technology, VVP users do not need any extra credentials to authenticate in Azure Blob Storage. The user just needs to properly configure the Kubernetes service account during the environment installation process.

This integration allows users to manage and deploy Apache Flink applications seamlessly on AKS while leveraging Azure services for authentication and resource access.

Kubernetes High Availability Service

In version 2.8 of the Ververica Platform, we implemented the long-awaited Flink Kubernetes HA option alongside the existing VVP Kubernetes HA option. The purpose was to allow for a smooth transition between the old and the new availability options, which both feature retaining the latest state. In the 2.10 release, Flink Kubernetes is set as the default and VVP Kubernetes is marked as deprecated. Please check Kubernetes High-Availability Service for more information.

More Info & Get Ververica Platform

In addition to building support for Apache Flink 1.16, we also resolved a number of Ververica Platform vulnerability issues, which you can read about in our release notes.

If you would like to try Ververica Platform, the getting started guide and the corresponding playground repository are the best places to start. We look forward to your feedback and suggestions. :)

Interested in trying out ververica platform Ververica Platform? Contact us for more information.

Ververica Academy

Daisy Tsang
Article by:

Daisy Tsang

Find me on:

Comments

Our Latest Blogs

Ververica Platform 2.13.0 is Released! featured image
by Ververica 16 May 2024

Ververica Platform 2.13.0 is Released!

Check out the newest features! The latest update to Ververica Platform has now been released and is generally available (GA) for all users.
Read More
Q&A with Filip Yonov: Unveiling the Future of Streaming Data Platforms featured image
by Kaye Lincoln 14 May 2024

Q&A with Filip Yonov: Unveiling the Future of Streaming Data Platforms

Q&A with Filip Yonov: Unveiling the Future of Streaming Data Platforms
Read More
Q&A with Damian Stevens, CTO & Co-Founder at Evoura featured image
by Kaye Lincoln 30 April 2024

Q&A with Damian Stevens, CTO & Co-Founder at Evoura

Our Flink Forward events are all about embracing the spirit of collaboration and knowledge sharing of Apache Flink® and everything streaming data! Flink Forward events unite the community to...
Read More