My organization is interested to use Hashicorp Vault to manage sensitive information. How can I use it together with Ververica Platform?
Note: This section applies to Ververica Platform 2.2 - 2.8 Enterprise Edition.
The Vault Agent Injector service in Hashicorp Vault leverages the Kubernetes mutating admission webhook to inject secrets into Kuberntes pods via a sidecar container. Applications can remain Vault unaware as the secrets are stored on the file-system within their container.
Assuming we have set up Ververica Platform. It runs in the Kubernetes namespace vvp
, and the Flink jobs will run in the Kubernetes namespace flink
. We will follow this Hashicorp Vault tutorial but adapt it to our setup. For a detailed explanation of those Hashicorp Vault commands, please refer to their tutorial.
helm repo add hashicorp https://helm.releases.hashicorp.com
helm install vault hashicorp/vault --set "server.dev.enabled=true,server.extraEnvironmentVars.VAULT_DEV_LISTEN_ADDRESS=0.0.0.0:8200"
kubectl get pods
Note: The second command above is different from the one in the original tutorial. This is due to a bug in Hashicorp Vault.
Let us assume we want to create a secret containing username and password at the path: vvp/database/config
. Login to the vault-0
pod:
kubectl exec -it vault-0 -- /bin/sh
and then run:
vault secrets enable -path=vvp kv-v2
vault kv put vvp/database/config username=vvpuser password=Top-Secret0
vault kv get vvp/database/config
Login to the vault-0
pod again if you exited already. With the following commands we create a role vvp-flink-job
and bind the service account default
and other ones in the namespace flink
to the policy read-vvp-secret
which is created by the second command:
vault auth enable kubernetes
vault write auth/kubernetes/config \
token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
vault policy write read-vvp-secret - <<EOF
path "vvp/data/database/config" {
capabilities = ["read"]
}
EOF
vault write auth/kubernetes/role/vvp-flink-job \
bound_service_account_names=default,job-*-flink-ha-*manager \
bound_service_account_namespaces=flink \
policies=read-vvp-secret \
ttl=24h
Note: bound_service_account_names
in the last command is the service account of Flink pods. If you do not use Kubernetes HA in Ververica Platform, the service account is default
. Otherwise, the service account is job-<jobid>-flink-ha-jobmanager
and job-<jobid>-flink-ha-taskmanager
. Here we cover both cases by using wildcards. See Create a Role in Vault.
Note: bound_service_account_namespaces
in the last command is the namespace where the Flink pods run. Similar to bound_service_account_names
, comma-separated multiple namespaces and wildcard can be used here if you want to cover multiple namespaces. See Create a Role in Vault.
Now we create a Deployment in Ververica Platform and add the following annotations through the YAML editor:
spec:
template:
spec:
kubernetes:
pods:
annotations:
vault.hashicorp.com/agent-inject: true
vault.hashicorp.com/role: vvp-flink-job
vault.hashicorp.com/agent-inject-secret-database-config.txt: vvp/data/database/config
When the Flink pod is started, log in to the Flink pod, you should see a file /vault/secrets/database-config.txt
containing the secret we defined. To get that secret, your job simply reads it from the file.
kubectl exec -it -n flink <JM/TM_pod> -- cat /vault/secrets/database-config.txt