Skip to content

The v1 docs follow nauthera-server's main branch and its 0.x releases on the way to 1.0, so fields and APIs can still change.

v1

GitOps & Argo CD

Docs version v1 (main)Nauthera is pre-1.0, in a design-partner programUpdated

Nauthera is built to be declared: the server is a Helm chart, and organizations, identity providers, clients and branding are Kubernetes resources the operator reconciles. That makes it a natural fit for GitOps, where Git is the only way changes reach the cluster and every change is a reviewed pull request. This page proposes a setup with Argo CD. The same split works with Flux; only the objects differ.

It builds on Ownership: the platform team owns the server, organizations and identity providers; application teams own their OIDCClient next to their code. Argo CD's AppProjects are where that split is enforced.

What goes where

WhatLives inSynced by
The resource definitions (deploy/crds/*-crd.yaml, or the chart's files/crds/), pinned to the chart versionPlatform repositorynauthera-crds
Secrets the chart reads (database DSN, Redis password, KEK, MFA key, SMTP password, and for appendOnlyAudit the schema owner's DSN) as ExternalSecretsPlatform repositorynauthera-secrets
The server and operator: the Helm chart and its values per environmentPlatform repositorynauthera-server
Organization resources, whose spec.namespaces decides which team configures which organizationPlatform repositorynauthera-tenants
IdentityProvider resources and their client secrets as ExternalSecretsPlatform repositorynauthera-tenants
BrokerMembership sides, each in its own organization's namespace, and the DNSEndpoints that prove emailDomainsPlatform repository, or each organization's identity teamnauthera-tenants
Admission policies limiting fields within a kind (see Ownership)Platform repositorynauthera-policies
Each application's OIDCClient and Branding, beside its DeploymentThe application's own repositoryOne Application per team and environment
Client secrets the operator mintsNowhere in GitThe operator writes them into the application's namespace

The client secrets are the point: with the operator, the only secrets in the GitOps flow are the platform's own, and those come from your secret store, never from Git.

Repository layout

platform-gitops/
├── bootstrap/
│   ├── root-staging.yaml            # app of apps, one per cluster
│   └── root-production.yaml
├── projects/
│   ├── nauthera-platform.yaml       # AppProject for everything below platform/
│   └── teams.yaml                   # AppProject per team (or an ApplicationSet of them)
├── platform/
│   └── nauthera/
│       ├── apps/                    # the Application objects the root syncs
│       │   ├── crds.yaml
│       │   ├── secrets.yaml
│       │   ├── server.yaml
│       │   ├── tenants.yaml
│       │   ├── policies.yaml
│       │   └── teams.yaml           # ApplicationSet for team Applications
│       ├── crds/
│       │   └── 0.x.y/          # deploy/crds/*-crd.yaml (or files/crds/) of this chart version
│       ├── secrets/
│       │   ├── base/                # ExternalSecrets, SecretStore reference
│       │   └── overlays/{staging,production}/
│       ├── server/
│       │   ├── values-common.yaml   # production profile settings you share
│       │   ├── values-staging.yaml
│       │   └── values-production.yaml
│       ├── tenants/
│       │   ├── base/
│       │   │   ├── organizations/   # staff.yaml, acme.yaml (cluster-scoped)
│       │   │   ├── identity-providers/
│       │   │   │   ├── entra.yaml
│       │   │   │   └── entra-secret.yaml   # ExternalSecret for the client secret
│       │   │   └── namespaces/      # team namespaces, admitted by spec.namespaces
│       │   └── overlays/{staging,production}/   # issuers differ per environment
│       └── policies/
│           └── oidcclient-organization.yaml
└── teams/
    └── teams.yaml                   # team, repository, namespace, organization
 
billing-service/                     # an application team's own repository
└── deploy/
    ├── base/
    │   ├── deployment.yaml          # reads the operator's Secret
    │   ├── oidcclient.yaml
    │   └── branding.yaml
    └── overlays/{staging,production}/   # issuer and redirect URIs per environment

One root Application per cluster keeps environments separate: staging and production have different issuers, keys and databases (Rollout), so nothing should let a staging change reach production except a pull request to the production overlay.

Sync order

Each layer needs the one before it. Order them with sync waves on the Application objects the root syncs:

WaveApplicationNeeds
-2nauthera-crdsNothing
-1nauthera-secretsThe External Secrets operator and a SecretStore
0nauthera-serverSecrets, cert-manager, PostgreSQL, Redis
1nauthera-tenants, nauthera-policiesThe CRDs, a Ready operator whose allowedClients entry grants organizations, broker_memberships and email_domains in the tenants' slugs
2The team ApplicationsTheir organization, the policy

Argo CD waits for an Application to be healthy before the next wave only if it can assess the health of Application resources, which it does not do by default since 1.8. Add the health check from the Argo CD docs to argocd-cm, or the waves only order the creation of the Applications.

The platform AppProject

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: nauthera-platform
  namespace: argocd
spec:
  description: The Nauthera server, operator, tenants and policies
  sourceRepos:
    - https://github.com/example/platform-gitops.git
    - ghcr.io/nauthera/charts                 # the OCI chart
  destinations:
    - server: https://kubernetes.default.svc
      namespace: nauthera
    - server: https://kubernetes.default.svc
      namespace: "*"                          # namespaces with team labels
  clusterResourceWhitelist:
    - { group: apiextensions.k8s.io, kind: CustomResourceDefinition }
    - { group: nauthera.io, kind: Organization }
    - { group: admissionregistration.k8s.io, kind: ValidatingAdmissionPolicy }
    - { group: admissionregistration.k8s.io, kind: ValidatingAdmissionPolicyBinding }
    - { group: rbac.authorization.k8s.io, kind: ClusterRole }
    - { group: rbac.authorization.k8s.io, kind: ClusterRoleBinding }
    - { group: "", kind: Namespace }
  roles:
    - name: platform
      groups: [alice@example.com, bob@example.com]
      policies:
        - p, proj:nauthera-platform:platform, applications, *, nauthera-platform/*, allow

Argo CD matches a role's groups against the claims that scopes in argocd-rbac-cm names, groups by default. Nauthera's tokens carry no groups claim, so when Argo CD signs in through Nauthera, list the team's email addresses and set scopes: "[email]" (Argo CD sign-in has the client and both ConfigMaps). With another provider in front of Argo CD, its group names work as usual.

Narrow destinations to the namespaces you actually use once they are known.

The server Application

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nauthera-server
  namespace: argocd
  annotations:
    argocd.argoproj.io/sync-wave: "0"
spec:
  project: nauthera-platform
  destination:
    server: https://kubernetes.default.svc
    namespace: nauthera
  sources:
    - repoURL: ghcr.io/nauthera/charts
      chart: nauthera-server
      targetRevision: 0.x.y            # the only line an upgrade changes
      helm:
        releaseName: nauthera
        valueFiles:
          - $values/platform/nauthera/server/values-common.yaml
          - $values/platform/nauthera/server/values-production.yaml
        values: |
          operator:
            enabled: true
            crds:
              install: false                # nauthera-crds owns them
          certManager:
            grpcTLS:
              enabled: true                 # the operator calls the AdminService over mTLS
              allowedClients:
                - identity: dns:nauthera-operator
                  operations: [clients, brandings, identity_providers, ldap_directories,
                               organizations, broker_memberships, email_domains]
                  organizations: [default, staff, acme]   # every tenant's slug
                  namespaces: ["*"]
          config:
            server:
              grpc:
                enabled: true
    - repoURL: https://github.com/example/platform-gitops.git
      targetRevision: main
      ref: values
  syncPolicy:
    # No automated sync in production: migrations are forward only, so an upgrade is
    # synced by a person, after a backup. Staging can use automated: { prune: true, selfHeal: true }.
    syncOptions:
      - CreateNamespace=false

The chart and both images are public, so Argo CD needs no credentials and the cluster no image pull secret. Declare the registry once as an OCI Helm repository:

apiVersion: v1
kind: Secret
metadata:
  name: nauthera-charts
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  type: helm
  name: nauthera
  url: ghcr.io/nauthera/charts
  enableOCI: "true"

Downloads lists every published version. The released chart pins both images by digest, so an upgrade is one change to targetRevision. Let Renovate or Dependabot open that pull request.

Put values-common.yaml together from the production profile (profiles/production.yaml) and your own settings. Argo CD reads value files from Git, not from the chart, so copy the profile's content in rather than referencing it, and review the profile's diff when you bump the chart.

Helm hooks under Argo CD

Argo CD renders the chart with helm template and applies the result; it does not run helm install. Two things change:

  • The migration Job still runs first. Argo CD maps the chart's pre-install and pre-upgrade hooks to PreSync, so with appendOnlyAudit the maintain Job runs before the new pods, as with Helm. It runs on every sync, not only on upgrades; maintain is idempotent, so that is safe.
  • There is no helm --timeout. A release whose migration builds an audit index on a large log is bounded by the Job's own appendOnlyAudit.maintenance.activeDeadlineSeconds alone. Raise it in the values for that release (Upgrading).

The resource definitions

The chart renders the CRDs as templates by default (operator.crds.install), and Helm keeps them on uninstall. Under Argo CD they are ordinary resources of the Application, and whether a prune or a deleted Application spares them depends on your Argo CD version and settings, not on Helm. Deleting a CRD deletes every OIDCClient with it, so don't leave that to chance: sync them from their own Application with pruning off, and turn the chart's copy off as above:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nauthera-crds
  namespace: argocd
  annotations:
    argocd.argoproj.io/sync-wave: "-2"
spec:
  project: nauthera-platform
  destination: { server: https://kubernetes.default.svc }
  source:
    repoURL: https://github.com/example/platform-gitops.git
    targetRevision: main
    path: platform/nauthera/crds/0.x.y
  syncPolicy:
    automated: { prune: false, selfHeal: true }
    syncOptions:
      - ServerSideApply=true      # large schemas exceed the client-side annotation limit

Bump the directory together with the chart's targetRevision, in the same pull request.

Secrets

The chart takes each of its secrets.*, ownerDatabaseDsn and auditCheckpointKey included, by existingSecret. Create those Secrets with the External Secrets operator from your secret store (or Sealed Secrets, or the Argo CD Vault plugin), never as plain manifests in Git:

apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: nauthera-keys
  namespace: nauthera
  annotations:
    argocd.argoproj.io/sync-wave: "-1"
spec:
  secretStoreRef: { kind: ClusterSecretStore, name: vault }
  target:
    name: nauthera-keys
    deletionPolicy: Retain        # losing the KEK makes tenant keys unreadable
  data:
    - secretKey: keys-encryption-key
      remoteRef: { key: nauthera/production, property: kek }
    - secretKey: mfa-encryption-key
      remoteRef: { key: nauthera/production, property: mfa-key }

Use the key names your chart version expects. cert-manager issues the token signing keys (certManager.signingKey) and, unless secrets.auditCheckpointKey gives one, the audit checkpoint key (certManager.auditCheckpointKey). To bring your own signing key, turn certManager.signingKey.enabled off, mount the PEM with extraVolumes and extraVolumeMounts, and set config.keys.provider: file and config.keys.signing_key_path to it.

An IdentityProvider's client secret is an ExternalSecret beside it, in the same directory: the operator sends the new secret whenever the Secret changes, so rotating it at the upstream provider and in the store is the whole procedure. Remember the operator refuses a value with a trailing newline (SecretPadded), which some stores add.

The KEK, the MFA seed key and the signing keys must also exist outside the cluster, in the recovery copy Operational Requirements describes. A secret store is that copy only if it is backed up separately from the cluster.

Tenants and identity providers

nauthera-tenants holds the Organization and IdentityProvider resources and the team namespaces, which each Organization admits in spec.namespaces. A sync must never be what removes a tenant, so protect them from pruning and from cascade deletion:

apiVersion: nauthera.io/v1alpha1
kind: Organization
metadata:
  name: acme
  annotations:
    argocd.argoproj.io/sync-options: Prune=false,Delete=false
spec:
  slug: acme
  displayName: Acme Inc.
  deletionPolicy: Retain

deletionPolicy: Retain keeps the tenant served even if the resource goes; the annotations keep Argo CD from deleting the resource in the first place, on a pruned file or a deleted Application. Do the same for IdentityProviders. Removing a tenant becomes a deliberate step: delete the resource by hand after the pull request that removed its file.

Team applications

Each team gets an AppProject that allows its own repository, its own namespace, and only the kinds it owns:

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: team-billing
  namespace: argocd
spec:
  sourceRepos: [https://github.com/example/billing-service.git]
  destinations:
    - server: https://kubernetes.default.svc
      namespace: billing
  clusterResourceWhitelist: []          # nothing cluster-scoped
  namespaceResourceWhitelist:
    - { group: apps, kind: Deployment }
    - { group: "", kind: Service }
    - { group: "", kind: ConfigMap }
    - { group: networking.k8s.io, kind: Ingress }
    - { group: nauthera.io, kind: OIDCClient }
    - { group: nauthera.io, kind: Branding }
  roles:
    - name: developers
      groups: [team-billing]
      policies:
        - p, proj:team-billing:developers, applications, *, team-billing/*, allow

IdentityProvider and Organization are not on the list, so a team's repository cannot declare them however it is written. The team's namespace configures only the organization whose spec.namespaces lists it, which the platform repository owns, and RBAC on the namespace stays as in Ownership. An ApplicationSet with a list or Git generator over teams/teams.yaml creates one AppProject and one Application per team and environment, so onboarding a team is a pull request adding one entry.

Things to know about the team side:

  • The operator's Secret is not in Git, and that is right. It carries an owner reference to the OIDCClient but not Argo CD's tracking label or annotation, so Argo CD shows it under the resource and never prunes it. Don't add a Secret with the same name to the repository; the operator does not write over a Secret it does not own (SecretUndeliverable).
  • Rotating a client secret stays a delete of that Secret. Argo CD does not recreate it; the operator does.
  • Deleting the Application deletes the OIDCClient with the rest. Its deletionPolicy decides what happens on the server, Retain by default. Leave it there for production clients.
  • Drift shows up twice: Argo CD reports a manifest changed by hand in the cluster, and the operator reports a client changed in the server behind its back (Drifted). Self-heal fixes the first; nauthera.io/restore fixes the second (OIDCClient).

Health checks for the operator's resources

Argo CD has no health check of its own for the nauthera.io kinds, so it gives an OIDCClient no health status and leaves it out of the Application's health, even when the server refused it (health code, Argo CD 3.6.0). Teach it the operator's status:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  resource.customizations.health.nauthera.io_OIDCClient: &nautheraHealth |
    hs = { status = "Progressing", message = "Waiting for the operator" }
    if obj.status == nil then return hs end
    if obj.status.observedGeneration ~= nil and obj.metadata.generation ~= nil
       and obj.status.observedGeneration < obj.metadata.generation then
      return hs
    end
    local msg = ""
    if obj.status.conditions ~= nil then
      for _, c in ipairs(obj.status.conditions) do
        if c.type == "Ready" then msg = (c.reason or "") .. ": " .. (c.message or "") end
      end
    end
    if obj.status.phase == "Ready" then
      hs.status = "Healthy"
    elseif obj.status.phase == "Error" then
      hs.status = "Degraded"
    end
    hs.message = msg
    return hs
  resource.customizations.health.nauthera.io_IdentityProvider: *nautheraHealth
  resource.customizations.health.nauthera.io_Organization: *nautheraHealth
  resource.customizations.health.nauthera.io_Branding: *nautheraHealth
  resource.customizations.health.nauthera.io_LDAPDirectory: *nautheraHealth
  resource.customizations.health.nauthera.io_BrokerMembership: *nautheraHealth

Now a Refused client turns its Application Degraded, with the server's objection as the message, and the team sees it where they deploy. Pending reasons such as Unreachable stay Progressing. Point Argo CD notifications for Degraded at the owning team's channel.

Promotion and upgrades

  • Changes to tenants and clients go through staging first: merge to the staging overlay, check the resource is Ready there, then open the production pull request. Overlays differ only in issuers, hosts and redirect URIs.
  • Upgrading Nauthera is a pull request that bumps the chart's targetRevision and the CRD directory together, first for staging. In production, take the backup, then sync by hand, inside a sync window that matches your change calendar. A deny window during rollout waves keeps platform changes out of them (Rollout).
  • Rolling back the server is not a revert. Migrations are forward only, so reverting targetRevision after a migration ran starts an older binary on a newer schema. Roll forward, or restore the backup (Upgrading). Reverting a tenant or client manifest is fine.
  • Check a sync is quiet. After the first sync, a second one with no Git change should show nothing to apply. A resource that always differs, such as a field the API server defaults, needs an ignoreDifferences entry, not self-heal fighting it.

Checklist

  • One root Application per cluster; staging and production never share an overlay.
  • CRDs in their own Application, with pruning off and server-side apply.
  • No Secret in Git; the KEK and MFA key have a recovery copy outside the cluster.
  • Organization and IdentityProvider carry Prune=false,Delete=false.
  • Team AppProjects allow OIDCClient and Branding only, in their own namespace.
  • Health checks for the six nauthera.io kinds, and for Application if you use waves.
  • Production server syncs by hand, after a backup, inside a sync window.
  • Notifications route Degraded resources to their owners.