Store Nauthera's secrets with Sealed Secrets
Docs version v1 (main)Nauthera is pre-1.0, in a design-partner programUpdated
This guide seals every secret you hand to Nauthera with
Sealed Secrets: an OIDCClient's client
secret, the Helm chart's keys and passwords, an identity provider's client secret, an LDAP
directory's bind password and a client's private_key_jwt key. You need a Nauthera
release in the namespace nauthera with its operator running, and
kubectl and kubeseal on your machine. At the end, every one of them is in Git, encrypted to a key only the
cluster's controller holds, and a rotation is a commit. The main limit: a change to the
chart's encryption keys, database or cache passwords still needs a restart of the server.
The controller turns each SealedSecret into an ordinary Kubernetes Secret, and Nauthera
reads that Secret like any other.
| Secret | Who supplies it | Sealed here |
|---|---|---|
An OIDCClient's client secret | You, through spec.secret.secretRef, or the server with secret.generate | Yes, with secretRef |
| Key-encryption key, MFA seed key, database DSN, cache and SMTP passwords | You, through the chart's secrets.*.existingSecret | Yes |
An IdentityProvider's client secret | The upstream provider, through spec.clientSecretRef | Yes |
An LDAPDirectory's bind password | Your directory, through spec. | Yes |
A private_key_jwt client's private key | You, mounted into the client's pod | Yes |
| Signing keys, gRPC and ingress certificates | cert-manager | No |
Install the controller
Install the controller with its Helm chart. kubeseal looks for it as
sealed-secrets-controller in kube-system, so keep that name:
helm repo add sealed-secrets https://bitnami-labs.github.io/sealed-secrets
helm install sealed-secrets sealed-secrets/sealed-secrets \
--namespace kube-system \
--set-string fullnameOverride=sealed-secrets-controllerFetch the controller's public certificate once and commit it. Anyone with the certificate can seal a secret; only the controller can open one, so sealing needs no access to the cluster:
kubeseal --fetch-cert > sealed-secrets.pemBy default a SealedSecret is bound to its name and namespace: the controller refuses to
open one that was renamed or moved. Keep that default for everything below, since each
Secret is read from one namespace by one name.
Seal without writing the plaintext to disk
Every example follows the same pattern: kubectl create secret --dry-run=client renders
the Secret, and kubeseal encrypts it on the way through, so the plaintext never lands
in a file you might commit.
Use --from-literal, or --from-file on a file written with printf '%s', never echo.
echo leaves a trailing newline, and the operator refuses a client secret or bind password
that begins or ends with whitespace (SecretPadded), because no provider issues one.
The chart's secrets
The chart reads each of its secrets.* from a Secret you name in existingSecret. The
key defaults to the file name the chart mounts it as, so a Secret with these keys needs no
existingSecretKey:
| Values key | Default key in the Secret |
|---|---|
secrets. | keys-encryption-key |
secrets.mfaEncryptionKey | mfa-encryption-key |
secrets.databaseDsn | database-dsn |
secrets.databasePassword | database-password |
secrets.cachePassword | cache-password |
secrets.smtpPassword | smtp-password |
secrets.captchaSecret | captcha-secret |
secrets. | audit-checkpoint-key |
Generate the two 32-byte encryption keys and seal them into one Secret, nauthera-keys:
kubectl -n nauthera create secret generic nauthera-keys \
--from-literal=keys-encryption-key="$(openssl rand -hex 32)" \
--from-literal=mfa-encryption-key="$(openssl rand -hex 32)" \
--dry-run=client -o yaml \
| kubeseal --cert sealed-secrets.pem --format yaml > nauthera-keys.sealed.yamlThe plaintext of these two keys now exists only inside the SealedSecret. Before you
apply it, put a copy of each key in your recovery store, as
Operational Requirements
describes: losing the KEK makes every other organization's signing keys and every sealed
provider secret unreadable, and losing the MFA key invalidates every enrolled
authenticator. Read them back from the cluster once the Secret exists:
kubectl -n nauthera get secret nauthera-keys -o jsonpath='{.data.keys-encryption-key}' | base64 -dAdd an annotation to the template so the Secret outlives its SealedSecret. By default
the controller makes the SealedSecret the Secret's owner, so deleting it (a pruned file,
a deleted Argo CD Application) deletes the keys with it:
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
name: nauthera-keys
namespace: nauthera
spec:
encryptedData:
keys-encryption-key: AgB4…
mfa-encryption-key: AgCx…
template:
metadata:
name: nauthera-keys
namespace: nauthera
annotations:
sealedsecrets.bitnami.com/skip-set-owner-references: "true"Seal the database DSN, and any other password the deployment needs, the same way. Then point the chart at the Secrets:
secrets:
keysEncryptionKey:
existingSecret: nauthera-keys
mfaEncryptionKey:
existingSecret: nauthera-keys
databaseDsn:
existingSecret: nauthera-database
smtpPassword:
existingSecret: nauthera-smtpThe Secrets must exist before the server's pods start. With Argo CD, give the
SealedSecrets an earlier sync wave than the chart (argocd.);
with plain Helm, apply them first.
An organization's own SMTP relay (mailOrganizations[]., keys
username and password) is sealed the same way. The server reads those files again at
every delivery, so a resealed password takes effect without a restart.
Changing one of them
The chart's checksum covers only the Secret it creates itself, so changing a Secret you manage does not roll the pods. The encryption keys, the DSN and the passwords are read when the server starts: after the controller has updated the Secret, restart it.
kubectl -n nauthera rollout restart deploy/nauthera-nauthera-serverRotating the KEK or the MFA key is more than resealing: the old key must stay readable as
a retired key. Follow Keys & Secrets and
seal the retired list into the Secret secrets.keysRetiredKeys names.
OIDCClient client secrets
An OIDCClient that authenticates with
client_secret_basic or client_secret_post takes its secret from a Secret you hold when
spec.secret.secretRef names it. Seal that Secret into the client's namespace, and the
same Secret serves your application:
kubectl -n storefront create secret generic storefront-oidc \
--from-literal=client-secret="$(openssl rand -base64 48 | tr -d '\n')" \
--dry-run=client -o yaml \
| kubeseal --cert sealed-secrets.pem --format yaml > storefront-oidc.sealed.yamlThe secret must be at least 43 printable ASCII characters with no whitespace, drawn at
random: the server refuses a shorter one, since it stores a supplied secret as it stores
one it mints, as a SHA-256 digest. openssl rand -base64 48 gives 64 characters. Commit
the SealedSecret beside the client:
apiVersion: nauthera.io/v1alpha1
kind: OIDCClient
metadata:
name: storefront-web
namespace: storefront
spec:
clientId: storefront-web
redirectUris:
- https://shop.example.com/callback
grantTypes: [authorization_code]
scopes: [openid, profile, email]
tokenEndpointAuthMethod: client_secret_basic
secret:
secretRef:
name: storefront-oidc
key: client-secret # the defaultThe operator reads the value, sends it to the server and never writes or owns the Secret.
Until the controller has created the Secret, the client waits in Error under
SecretNotFound and nothing is sent. Once the server holds the secret, the resource is
Registered and status.secretFingerprint identifies the secret without revealing it.
secretRef cannot be combined with generate or secretName, and a public or
private_key_jwt client has no secret to supply.
Read the secret in the workload from the same Secret, and take the client ID and issuer
from your own configuration, since the operator writes no client-id or issuer key for
a supplied secret:
env:
- name: OIDC_ISSUER
value: https://auth.example.com
- name: OIDC_CLIENT_ID
value: storefront-web
- name: OIDC_CLIENT_SECRET
valueFrom:
secretKeyRef: {name: storefront-oidc, key: client-secret}Rotating it
Seal a new value over the same name and commit it. The operator sends it as soon as the controller updates the Secret, and it replaces the old secret at once: there is no window in which both work. The fingerprint changes when the server holds the new one:
kubectl -n storefront get oidcclient storefront-web \
-o jsonpath='{.status.secretFingerprint}{"\n"}'Restart the workloads that read it from the environment
(kubectl -n storefront rollout restart deploy/), or mount the Secret as a
volume and have the application read the file again when the token endpoint refuses the
secret it holds.
Or let the server mint it
With secret.generate: true instead, the server mints the secret and the operator writes
it into a Secret the resource owns. Nobody handles the value, so there is nothing to seal;
Delivering OIDC client secrets without storing them in Git
follows one end to end. Do not seal a Secret under that client's secretName: the
operator never writes over a Secret it does not own, and the client goes to Error under
SecretUndeliverable. Use secretRef for a secret you seal, and generate for one you
never see.
Identity provider client secrets
An IdentityProvider takes the client secret the
upstream provider (Google, Microsoft Entra ID, Okta) issued when you registered Nauthera
there. The resource names a Secret beside it with spec.clientSecretRef, under the key
client-secret by default. Seal that Secret
into the same namespace:
kubectl -n nauthera create secret generic google-oidc \
--from-literal=client-secret='<the secret Google issued>' \
--dry-run=client -o yaml \
| kubeseal --cert sealed-secrets.pem --format yaml > google-oidc.sealed.yamlCommit it next to the resource that reads it:
apiVersion: nauthera.io/v1alpha1
kind: IdentityProvider
metadata:
name: google
namespace: nauthera
spec:
slug: google
displayName: Google
providerIssuer: https://accounts.google.com
clientId: 1234567890-example.apps.googleusercontent.com
clientSecretRef:
name: google-oidc
key: client-secret
scopes: [openid, email, profile]The operator reads the Secret, sends the secret to the server, which stores it sealed
under the KEK, and never writes it anywhere else. Until the controller has created the
Secret, the resource waits in Error under SecretNotFound and sends nothing; it is
reconciled again the moment the Secret appears, so the order you apply the two in does not
matter.
kubectl -n nauthera get identityprovider googleRegistered in the reason column means the server holds the current secret.
status.secretFingerprint identifies which one, without revealing it.
Rotating it
Issue a new secret at the provider, seal it over the same name and commit it. The controller updates the Secret, the operator sees the change and sends the new secret, and the fingerprint changes:
kubectl -n nauthera get identityprovider google \
-o jsonpath='{.status.secretFingerprint}{"\n"}'Sign-in through the provider uses the new secret from then on, with no restart. Revoke the old secret at the provider once the fingerprint has changed.
LDAP bind passwords
An LDAPDirectory reads its service account's password
from spec., under the key password by default, and the CA bundle
from a ConfigMap or Secret. Only the password is secret:
kubectl -n nauthera create secret generic corp-ldap \
--from-literal=password='<the service account password>' \
--dry-run=client -o yaml \
| kubeseal --cert sealed-secrets.pem --format yaml > corp-ldap.sealed.yamlThe operator sends the password to the server with the spec, and again whenever the
Secret changes, so a password changed in the directory is rotated by resealing it, as for
a provider's client secret. Change it in the directory first, then commit the new
SealedSecret promptly: sign-in through the directory fails while the two disagree.
private_key_jwt keys
A client that authenticates with private_key_jwt holds a private key in its own pod and
publishes the public half at its jwksUri; Nauthera stores only the URL (see
Keys instead of secrets). Seal the private key into
the application's namespace:
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-256 -out client-key.pem
kubectl -n billing create secret generic billing-worker-client-key \
--from-file=client-key.pem --dry-run=client -o yaml \
| kubeseal --cert sealed-secrets.pem --format yaml > billing-worker-client-key.sealed.yaml
rm client-key.pemMount it into the pod as a read-only volume, not with subPath, so a resealed key reaches
the running pod. To rotate, publish the new public key next to the old one, wait a minute,
seal and roll out the new private key, then remove the old public key.
Keep the controller's key
Every SealedSecret in your repository opens only with the controller's private keys.
Lose them with the cluster and you must reseal everything from the plaintext, which for
the KEK and the MFA key means the recovery copy. Back the keys up, outside the cluster:
kubectl -n kube-system get secret \
-l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml > sealed-secrets-keys.yamlThe controller adds a new key every 30 days and keeps the old ones, so existing
SealedSecrets still open. Back up again after each renewal, and fetch the certificate
again so new secrets are sealed under the newest key. Restore the keys before the
controller starts on a rebuilt cluster, and every SealedSecret in Git opens there as it
did before.
Related
- Keys & Secrets: every key and secret Nauthera holds and how to rotate each one.
- GitOps & Argo CD: the same Secrets from a secret store, with the External Secrets operator.
- IdentityProvider and LDAPDirectory: the resources that read the Secrets.