How to integrate Hashicorp Vault with Ververica Platform

Question

My organization is interested to use Hashicorp Vault to manage sensitive information. How can I use it together with Ververica Platform?

Answer

The Vault Agent Injector service in Hashicorp Vault leverages the Kubernetes mutating admission webhook to inject secrets into Kuberntes pods via a sidecar container. Applications can remain Vault unaware as the secrets are stored on the file-system within their container.

Assuming we have set up Ververica Platform. It runs in the Kubernetes namespace vvp, and the Flink jobs will run in the Kubernetes namespace flink. We will follow this Hashicorp Vault tutorial but adapt it to our setup. For a detailed explanation of those Hashicorp Vault commands, please refer to their tutorial.

Install Hashicorp Vault in our Kubernetes Cluster

helm repo add hashicorp https://helm.releases.hashicorp.com 

helm install vault hashicorp/vault --set "server.dev.enabled=true,server.extraEnvironmentVars.VAULT_DEV_LISTEN_ADDRESS=0.0.0.0:8200" 

kubectl get pods

Create a Secret in Vault

Let us assume we want to create a secret containing username and password at the path: vvp/database/config. Login to the vault-0 pod:

kubectl exec -it vault-0 -- /bin/sh

and then run:

vault secrets enable -path=vvp kv-v2

vault kv put vvp/database/config username=vvpuser password=Top-Secret0

vault kv get vvp/database/config

Enable Kubernetes Authentication

Login to the vault-0 pod again if you exited already. With the following commands we create a role vvp-flink-job and bind the service account default and other ones in the namespace flink to the policy read-vvp-secret which is created by the second command:

vault auth enable kubernetes

vault write auth/kubernetes/config \
    token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
    kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
    kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt

vault policy write read-vvp-secret - <<EOF
path "vvp/data/database/config" {
    capabilities = ["read"]
}
EOF

vault write auth/kubernetes/role/vvp-flink-job \
    bound_service_account_names=default,job-*-flink-ha-*manager \
    bound_service_account_namespaces=flink \
    policies=read-vvp-secret \
    ttl=24h

Add Annotations to Flink Pods

Now we create a Deployment in Ververica Platform and add the following annotations through the YAML editor:

spec:
  template:
    spec:
      kubernetes:
        pods:
          annotations:
            vault.hashicorp.com/agent-inject: true
            vault.hashicorp.com/role: vvp-flink-job
            vault.hashicorp.com/agent-inject-secret-database-config.txt: vvp/data/database/config

When the Flink pod is started, log in to the Flink pod, you should see a file /vault/secrets/database-config.txt containing the secret we defined. To get that secret, your job simply reads it from the file.

kubectl exec -it -n flink <JM/TM_pod> -- cat /vault/secrets/database-config.txt

Related Information

• Injecting Secrets into Kubernetes Pods via Vault Helm Sidecar
• Create a Role in Vault