Announcing Ververica Platform 2.7 for Apache Flink® 1.15

The latest minor release includes full support for Flink 1.15, several improvements to the user experience, and a brand new look!

The recent release of Apache Flink 1.15 brings a number of exciting new changes which you can read more

about here. Today we are happy to announce the availability of Ververica Platform 2.7, an essential solution for teams who want to ease the development and operation of stream processing applications with Flink 1.15.

In addition to supporting the latest release of Flink, Ververica Platform 2.7 has been enhanced with several improvements to the user experience which we will cover in this post.

🌤️ Apache Flink 1.15, 🌥️ Apache Flink 1.12

With the addition of support for Apache Flink 1.15 in Ververica Platform 2.7, we are sunsetting our support for Apache Flink 1.12. Per our support policy, Ververica Platform 2.7 is supported for use with Apache Flink 1.13 and higher.

Connector improvements

The Kinesis connector provides access to Amazon Kinesis Data Streams. In Flink 1.15, the Kinesis connector was split into a separate source and sink because the old producer used the Kinesis Producer Library and has been deprecated. To use the Kinesis connector in Flink 1.15 means that you will need to use different connectors for the source and the sink and sink-specific config options may not work for the new producer-connector, and vice versa.

Moreover, a new Firehose connector was introduced, which is a sink that writes to Amazon Kinesis Data Firehose.

Flink SQL version upgraded to 1.15

Flink SQL has been upgraded to 1.15 on the platform. You can have a look at the list of changes to the Table API and SQL in the release notes.

Improvements to the user experience

Savepoint management

Savepoints are self-contained snapshots of the distributed state of your Apache Flink applications and Flink SQL queries. They are frequently used for a wide range of use cases and needs to be managed by the user (instead of the Flink runtime).

When adding an existing savepoint manually via the REST API (which can be useful if you have a savepoint created with Deployment 1 that you want to add to Deployment 2), flinkSavepointId and the annotation com.dataartisans.appmanager.controller.deployment.spec.version are no longer required fields.

Moreover, this can now be done manually via the web user interface of the platform!

SAML support

Security Assertion Markup Language (SAML) is an XML-based open standard for transferring identity data between an identity provider and a service provider, which creates benefits of improved user experience and increased security.

Ververica Platform now supports integration with SAML for authentication with an external identity provider. For more details, check out the documentation.

Operational improvements

REST API

On Ververica Platform, you can define default configurations that will be picked up when you create deployments. This is useful if you have configuration parameters that stay the same. For the PUT operation (to create or update deployments), there is now a new optional boolean parameter applyDefaults that toggles whether defaults will be applied to the provided payload. If you don’t specify anything, defaults will be applied.

Job failure expiration

The number of attempts a deployment makes to create a job is limited by maxJobCreationAttempts. You can now configure your accepted level of job instability with a new optional parameter jobFailureExpirationTime for maxJobCreationAttempts. This allows you to set an expiration time for the count so that any failures older than the specified time will not be counted.

Helm charts

We recommend installing Ververica Platform via Helm charts, which is a collection of files that describe a related set of Kubernetes resources. You can now configure custom pod annotations via the values.yaml file.

New look!

Last month, we introduced a new visual brand identity for Ververica and our products, which you can read more about here. The new design and color palette are now reflected on Ververica Platform as well!

More Info & Get Ververica Platform 2.7

If you'd like more detail about Apache Flink 1.15, you can see the full list of Apache Flink 1.15 enhancements on the Apache Flink blog. In addition to building support for Apache Flink 1.15, we also resolved a number of Ververica Platform issues, which you can read about in the Ververica Platform 2.7 release notes.

If you would like to try Ververica Platform 2.7, the getting started guide and the corresponding playground repository are the best places to start. We look forward to your feedback and suggestions. :)