Stream Processing & Apache Flink - News and Best Practices

Ververica Platform 2.11 is released!

Written by Ververica | 01 August 2023

🚀 We're thrilled to share the latest update packed with new features and enhancements to take your data processing to new heights!

Ververica Platform release 2.11 is now in General Availability (GA) for all customers and interested parties.

What’s New in Ververica Platform 2.11

Flink runtime: We've upgraded the underlying Flink runtime to Apache Flink® 1.17.1 to keep you right up-to-date with the best and most robust foundation for your data processing needs. Ververica Platform now supports Apache Flink® 1.17 and 1.16 under SLA. Apache Flink® 1.15 is deprecated in this version and supported on a best-effort basis.

Kubernetes Operator: We're excited to introduce the Ververica Platform Kubernetes Operator, which offers seamless Kubernetes native management of Flink deployments using kubectl and Helm. You can now use gitOps standard practices to automate your CI/CD workflow; we’ve added multi-version support to allow smooth Platform upgrades; and we’ve added Mirroring of Job / Deployment events to Kubernetes, so they can be easily sent to other monitoring platforms. We’re thrilled with the result and think you will be too! On top of the basic functionality of Flink deployment management, our operator out-of-the-box features additionally include Flink SQL support, batch processing, and seamless integration with the Web UI.

Azure Authentication: For our Azure fans, we're delighted to announce that we have improved support for OIDC login on Azure and added support for Azure Managed Identity to offer a passwordless option for authentication to Microsoft SQL databases.

Azure Graph Endpoint Integration: We have anchored our implementation on the Azure Graph endpoint, ensuring reliable and efficient extraction of group information from the Microsoft ecosystem. Leveraging Microsoft Graph API allows for a centralized point to retrieve group data, enabling administrators and developers to have a better grip on user management, permissions, and application roles within Ververica Platform.

Disaster Recovery Support: Your peace of mind is our priority! To complement our High Availability features we’ve added a configurable Disaster Recovery Checkpoint which is stored as a persistent checkpoint in the job. You can configure the frequency of saving the checkpoint, and because the job is stored in the database you can be confident of recovery even from a full Kubernetes failure.

Web UI Enhancements: We use your feedback to make continual improvements to our Platform UI so we can deliver the most user-friendly and intuitive experience. The entrypoint class and mainArgs are now shown on both the Configuration and Jobs tabs; we’ve added deploymentId to the Deployment details view; and you can now view the pod template for past jobs on the Jobs tab. And you can browse past job metrics/logs by start/end date with two clicks!

  • On the Jobs page, click Past Jobs > Metrics to see the metrics for finished jobs. Two clicks!

 

SQL Editor Enhancements: We’ve added a new panel to the SQL Editor UI so you can switch between different scripts to edit multiple scripts simultaneously.

  • The new SQL Scripts panel is a small step for the UI, and a giant step for the user!

 

 

Other enhancements

We continuously enhance features and usability to give you the best possible experience when working with Ververica Platform! In this release:

  • Spring Boot has been updated to v2.7.14, ensuring you're running on the latest technology.
  • The SQL ADD JAR statement is now fully supported, adding more flexibility to your data processing pipelines.
  • Ververica Platform now starts with a default universal blob storage configured.
  • We’ve made it easier to prune job history in the VVP UI.
  • And more … !

And as always, we have squashed bugs and removed and mitigated vulnerabilities.

We're always eager to hear your thoughts! Upgrade to Ververica Platform 2.11 now and experience cutting-edge capabilities, streamlined Deployment management, and more. We're here to empower your data processing journey!

Last but not least: We have released our new Community Edition for use on non-commercial projects. Find out more from the Ververica Platform  Downloads page.

🚀🌟 Let the excitement begin! 😄💥

As always, for full details of changes and for links to deployment images see the Ververica Platform 2.11 Release Notes.