# NautheraOperatorResourcesFailing

Resources the operator manages are in phase Error.

## Meaning

For fifteen minutes, at least one OIDCClient, Organization, IdentityProvider or Branding
has been in phase `Error`. The operator tried and the attempt cannot succeed on its own:
the server refused the spec, a Secret the resource needs is missing or malformed, its
namespace is not one the operator covers, the delivered client Secret could not be
written, or the server's copy was changed by someone else (`Drifted`). A server that
does not answer leaves resources `Pending`, not `Error`; that is
NautheraOperatorServerUnreachable.

## Impact

The application team whose resource failed: a new client is not registered, a change is
not applied, or its Secret is not delivered, so its users may not be able to sign in to
that application. Other resources are unaffected.

## Diagnosis

1. Find the resources: `kubectl get <kind> -A` and look for phase `Error`, for example
   `kubectl get oidcclients -A`.
2. `kubectl -n <ns> describe <kind> <name>`: the Ready condition's reason names the cause
   and its message the details. Each reason has its own runbook under
   [operator reasons](https://www.nauthera.io/docs/v1/runbooks/operator).
3. Count by reason: `sum by (kind, reason, cause) (increase(nauthera_operator_reconciles_total[1h]))`.

## Mitigation

Follow the runbook of the reason on the resource. Most are fixed in the manifest or a
Secret beside it (`Refused`, `SecretNotFound`, `SecretKeyMissing`, `SecretPadded`,
`NamespaceNotCovered`, `FixedAtCreation`); `Drifted` needs a decision between the
manifest and the change made on the server. The resource is reconciled again on its own
once the cause is gone.

## Escalation

Hand the resource to the team that owns its namespace when the cause is in their
manifest. Escalate to the platform team when the cause is the operator's permissions,
its `allowed_clients` grant or `operator.namespaces`.

---

[Alert runbooks](https://www.nauthera.io/docs/v1/runbooks/alerts) · Generated from [`internal/runbook/alerts.yaml`](https://github.com/nauthera/nauthera-server/blob/main/internal/runbook/alerts.yaml) and [`operator/internal/runbook/alerts.yaml`](https://github.com/nauthera/nauthera-server/blob/main/operator/internal/runbook/alerts.yaml) in nauthera-server.