Using Kubernetes Secrets and external secret managers

Passwords, API tokens, private keys, and other sensitive values do not belong in container images, ConfigMaps, or readable configuration files in Kubernetes configurations. Use a Secret in Kubernetes or retrieve the values from an external secret manager.

In this guide, you create a Kubernetes Secret without placing the values in a YAML file or your shell history. You mount the Secret as files in a Pod, use a secure reference as an environment variable when necessary, and learn how to connect an external secret manager using External Secrets Operator.

Choosing a storage method

Choose the storage method based on the content and how you manage the values:

• ConfigMap: use this only for non-sensitive configuration, such as a hostname, port number, or feature flag.
• Kubernetes Secret: use this for sensitive values that you manage within a single cluster. The values are stored in the Kubernetes API and can be made available as files or as environment variables through a reference.
• External secret manager: use this when you want to centrally manage, rotate, and audit Secrets, for example across multiple clusters. An operator can automatically synchronise the values with Kubernetes.

Never store a real password, token, or private key under data or stringData in a YAML file that you share or add to Git. A base64-encoded value is also easy to restore to its original value.

Requirements

For this guide, you need:

• A working Kubernetes cluster.
• kubectl, configured for the cluster.
• Optional: Helm 3 when installing External Secrets Operator.
• Optional: an external secret manager containing a stored Secret and a supported authentication method.

Creating a Kubernetes Secret

Step 1

Create a separate namespace for the examples in this guide:

kubectl create namespace secret-demo

Step 2

Create two temporary local files: one containing the database username and one containing the database password. Use the instructions for your client's operating system.

Linux and macOS (Bash or Zsh)

Use umask 077 to restrict permissions for new files to your own user. Then open the two files with nano:

umask 077
nano db-username
nano db-password

Enter only the corresponding value in each file. Save the changes and close nano with ctrl + x > y > enter.

Create the Secret and immediately delete the local files:

kubectl create secret generic database-credentials --namespace secret-demo --from-file=username=db-username --from-file=password=db-password
rm db-username db-password

Windows (PowerShell)

Create a unique temporary directory and grant access only to your current Windows user. PowerShell prompts for the password securely and writes both files as UTF-8 without a byte order mark (BOM):

$secretDirectory = Join-Path $env:TEMP "k8s-secret-$([guid]::NewGuid())"
New-Item -ItemType Directory -Path $secretDirectory | Out-Null
$currentUser = [System.Security.Principal.WindowsIdentity]::GetCurrent().Name
icacls $secretDirectory /inheritance:r /grant:r "$($currentUser):(OI)(CI)F" | Out-Null

$username = Read-Host 'Database username'
$password = Read-Host 'Database password' -AsSecureString
$credential = [PSCredential]::new('unused', $password)
$utf8 = [System.Text.UTF8Encoding]::new($false)
$usernameFile = Join-Path $secretDirectory 'db-username'
$passwordFile = Join-Path $secretDirectory 'db-password'
[System.IO.File]::WriteAllText($usernameFile, $username, $utf8)
[System.IO.File]::WriteAllText($passwordFile, $credential.GetNetworkCredential().Password, $utf8)
Remove-Variable username, password, credential

Create the Secret and immediately remove the temporary directory and variables:

kubectl create secret generic database-credentials --namespace secret-demo --from-file="username=$usernameFile" --from-file="password=$passwordFile"
Remove-Item -LiteralPath $secretDirectory -Recurse -Force
Remove-Variable secretDirectory, currentUser, utf8, usernameFile, passwordFile

The arguments have the following functions:

• generic: creates a Secret of type Opaque for general sensitive data.
• --namespace secret-demo: creates the Secret in the namespace secret-demo.
• --from-file=username=...: stores the contents of the specified file under the key username. The second argument does the same for password.

Step 3

Check that the Secret exists without retrieving its values:

kubectl get secret database-credentials --namespace secret-demo
kubectl describe secret database-credentials --namespace secret-demo

The output of describe shows the names and sizes of the keys, but not their contents. Do not use an output format such as -o yaml in logs or shared terminal sessions: the displayed base64 values are not encrypted.

Step 4

Open the file secret-reader.yaml with your text editor:

Linux and macOS:

nano secret-reader.yaml

Windows PowerShell:

notepad.exe secret-reader.yaml

Add the following configuration. The Secret is mounted in the container as read-only files:

apiVersion: v1
kind: Pod
metadata:
  name: secret-reader
  namespace: secret-demo
spec:
  containers:
    - name: app
      image: busybox:1.37.0
      command:
        - /bin/sh
        - -c
        - sleep 3600
      volumeMounts:
        - name: database-credentials
          mountPath: /run/secrets/database
          readOnly: true
  volumes:
    - name: database-credentials
      secret:
        secretName: database-credentials
        defaultMode: 0400

Save the file and close the text editor. Then create the Pod:

kubectl apply -f secret-reader.yaml

Each key in the Secret becomes a file under /run/secrets/database. readOnly: true prevents the container from modifying the files. defaultMode: 0400 grants read permission only to the owner of the files by default.

Step 5

Wait until the Pod is ready and then check that both files are readable. The command does not display their contents:

kubectl wait --for=condition=Ready pod/secret-reader --namespace secret-demo --timeout=60s
kubectl exec --namespace secret-demo secret-reader -- sh -c 'test -r /run/secrets/database/username && test -r /run/secrets/database/password && echo "Secret-bestanden zijn beschikbaar."'

You see the message ‘Secret-bestanden zijn beschikbaar.’ when the mount works.

Using a Secret as an environment variable

Prefer mounted files when the application supports them. An environment variable may become visible in diagnostic output, crash reports, or a process that reads the environment. In addition, an existing container does not receive a changed Secret value through an environment variable; you must restart the Pod.

If the application supports only environment variables, reference a Secret using secretKeyRef and never place the sensitive value directly under value.

Open the file secret-env-reader.yaml with your text editor:

Linux and macOS:

nano secret-env-reader.yaml

Windows PowerShell:

notepad.exe secret-env-reader.yaml

Add the following configuration:

apiVersion: v1
kind: Pod
metadata:
  name: secret-env-reader
  namespace: secret-demo
spec:
  containers:
    - name: app
      image: busybox:1.37.0
      command:
        - /bin/sh
        - -c
        - test -n "$DATABASE_PASSWORD" && echo 'Secret environment variable is available.'; sleep 3600
      env:
        - name: DATABASE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: database-credentials
              key: password

Save the file, close the text editor, and create the Pod:

kubectl apply -f secret-env-reader.yaml
kubectl logs secret-env-reader --namespace secret-demo

The Pod checks only whether the variable exists and does not write the sensitive value to the logs.

Connecting an external secret manager

Met External Secrets Operator synchronises values from HashiCorp Vault, Azure Key Vault, Google Secret Manager, and various other secret managers to Kubernetes Secrets. Connection and authentication configuration differs by provider.

Step 1

Add the official Helm repository and install External Secrets Operator:

helm repo add external-secrets https://charts.external-secrets.io
helm repo update
helm upgrade --install external-secrets external-secrets/external-secrets --namespace external-secrets --create-namespace --wait

The optional arguments have the following functions:

• --install: installs the release if it does not yet exist, or updates an existing release.
• --namespace external-secrets: uses the namespace external-secrets for the operator.
• --create-namespace: creates the namespace automatically if it does not yet exist.
• --wait: waits until the installed resources are ready or the Helm timeout expires.

Then check the operator Pods:

kubectl get pods --namespace external-secrets

Step 2

Create a SecretStore for your provider in secret-demo. Use the External Secrets Operator provider documentation. Where possible, use a workload identity, Kubernetes ServiceAccount, or another short-lived identity instead of a static management token.

After applying the provider configuration, check that the SecretStore is ready:

kubectl get secretstore --namespace secret-demo
kubectl describe secretstore <secretstore-naam> --namespace secret-demo

Replace <secretstore-naam> with the name from your SecretStore configuration. Continue only when the status shows that the connection is ready.

Step 3

Open the file database-external-secret.yaml with your text editor:

Linux and macOS:

nano database-external-secret.yaml

Windows PowerShell:

notepad.exe database-external-secret.yaml

Add the following configuration and replace the placeholders with the names from your SecretStore and external secret manager:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: database-credentials
  namespace: secret-demo
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: <secretstore-naam>
    kind: SecretStore
  target:
    name: database-credentials
    creationPolicy: Owner
  data:
    - secretKey: username
      remoteRef:
        key: <secretnaam-in-provider>
        property: username
    - secretKey: password
      remoteRef:
        key: <secretnaam-in-provider>
        property: password

The main fields have the following functions:

• refreshInterval: 1h: checks every hour whether the external value has changed.
• secretStoreRef: selects the namespaced SecretStore that the operator connects to.
• target.name: determines the name of the Kubernetes Secret created by the operator.
• remoteRef: maps a key and property from the external secret manager to a key in the Kubernetes Secret.

Save the file, close the text editor, and create the ExternalSecret:

kubectl apply -f database-external-secret.yaml

Step 4

Check the synchronisation without retrieving the values:

kubectl get externalsecret database-credentials --namespace secret-demo
kubectl get secret database-credentials --namespace secret-demo

The ExternalSecret must show the status ‘SecretSynced’ or ‘Ready’, and the Kubernetes Secret database-credentials must exist. Because the created Secret uses the same name and keys, you can mount it using the same Pod configuration from the earlier example.

After a rotation, the operator synchronises the new value according to refreshInterval. A mounted Secret volume is eventually updated, but the application must reload the file itself. An environment variable changes only after you restart the Pod.

Restricting access to Secrets

When managing Secrets, take the following measures:

• Following the Kubernetes guidelines for Secrets, grant users and ServiceAccounts only the minimum required permissions with RBAC. The list and watch permissions may also provide access to Secret contents.
• Grant access to the corresponding volume or key only to containers that need a Secret.
• Treat permissions to create Pods, Deployments, or similar workloads as sensitive permissions. A user who is allowed to create a Pod may be able to mount Secrets from the same namespace.
• Do not write Secret values to logs or display them in errors or diagnostic commands.
• Do not add files containing sensitive values, Kubernetes Secret manifests, or local .env files to Git.
• Rotate passwords, tokens, and keys regularly and immediately if you suspect that a value has been exposed.

Deleting Secrets

You can delete Secrets by deleting the namespace. Remove External Secrets Operator only when no other ExternalSecrets or SecretStores in the cluster depend on it. Delete a namespace as follows:

kubectl delete namespace secret-demo

Alternatively, view the available Secrets:

kubectl get secrets -n secret-demo

Then delete them:

kubectl delete secret <secretname>

You have securely made sensitive values available as a Kubernetes Secret and learned when an external secret manager is appropriate. Using Secret volumes, references, and minimum access permissions prevents passwords, tokens, and keys from ending up in configuration files or logs.