How to customize Audit logs in the Ververica Platform

Question

Ververica platform has Audit logs enabled by default. How do I customize the audit logging behavior? For example,

• Turn on or off Audit logs
• Enable audit logging globally or for appmanager or gateway container only
• Configure the audit logs retention
• Configure the executing time of the cleanup job for the deletion of old audit logs

Answer

Since Ververica Platform 2.12 and later, audit logs have been enabled by default. Audit logs bring accountability to the Ververica platform, which provides a mechanism for capturing all user actions via the UI or the app. However, one may need to customize the audit log configurations in certain circumstances. For example, frequent updates to deployments or other resources within the platform could demand extensive usage of the underlying storage where the Ververica platform is running. This could impact the operation of the Ververica platform.

In this article, we'll be able to explore all options that are available to customize audit logs. 

Turn Audit Logs on and off globally:

Since Ververica platform 2.12, Audit logs are enabled globally by default. To disable the audit logs from the Ververica platform globally, the following snippet can be added to the Helm Values file values.yaml:

vvp:
  auditor:
    enabled: false

This will turn off the audit logs globally. By default, leaving value.auditor.enabled unspecified is equivalent to enabled: true.

Selective logging of appmanager and gateway containers:

If you have your Audit logs enabled globally, you only want to enable them for particular containers of the Ververica platform. First, you need to remove the configuration below from values.yaml.

vvp:
  auditor:
    enabled: false

If you want to enable the audit logs only for the appmanager container and not the gateway container, you must pass the following values as environment variables. This can be done by adding the following snippet can be added to values.yaml:

appmanager:
  env:
    - name: "vvp.auditor.enabled"
      value: "true"
gateway:
  env:
    - name: "vvp.auditor.enabled"
      value: "false"

You can turn selective audit logging on or off in the Ververica platform using the above snippet that you need.

Customize the Data retention period:

By default, all audit logs persist for 180 days. If you have a lot of frequent updates in your deployments, it can generate a lot of data. It can sometimes cause an undesirable increase in the underlying storage where the Ververica platform is running. Using a smaller data retention period than the default 180 days is recommended in that case. However, for compliance reasons, sometimes you must keep the data for a more extended period. You can either decide on global data retention or selective data retention.

For example, if you want to keep the Audit logs data only for one day, you can configure this by adding the following snippet to values.yaml:

Global data retention

env:
  - name: "vvp.auditor.data-retention.retention-period-in-days"
    value: "1"

Retention for specific containers

appmanager:
 env:
   - name: "vvp.auditor.data-retention.retention-period-in-days"
     value: "1"
gateway:
 env:
   - name: "vvp.auditor.data-retention.retention-period-in-days"
     value: "1"

Customize the executing time of the audit logs cleanup job:

Based on your retention period configuration, two cron jobs run daily and delete the logs older than the configured retention period. In case you have a lot of traffic during certain times of the day, you can also configure the cron job to only delete the older logs at a time when there is not too much load on the system and when there are no other automation performing updates to the various resources. You can configure the time the cron jobs run to delete the logs older than the configured retention period.

You can configure this by adding the following snippet to values.yaml:

Set global clean up jobs execution time

env:
   - name: "vvp.auditor.data-retention.cleanup-job.cron"
     value: "0 21 3 * * *"

Set cleanup jobs execution time for specific containers

appmanager:
 env:
   - name: "vvp.auditor.data-retention.cleanup-job.cron"
     value: "0 21 3 * * *"
gateway:
 env:
   - name: "vvp.auditor.data-retention.cleanup-job.cron"
     value: "0 11 3 * * *"

The value set in the above configuration is a Spring Cron Expression. For example, 0 0 3 * * * is the configuration for every day at 3 AM.

Restarting the platform to apply configurations

Once you have added the configurations in values.yaml, you need to restart the Ververica platform with the updated configuration. Run the helm upgrade command. 

helm upgrade --install vvp ververica/ververica-platform \
  --namespace vvp \
  --values <values.yaml>

Once the Ververica platform is in the running state, it will use the new configuration for Audit logs. 

Related Information

• Audit Logs in the Ververica Platform
• Persistence Database Storage for the Ververica Platform
• How to migrate from SQLite to Postgres database for metadata persistence for the Ververica Platform
• Spring Cron Expression.