# NautheraControlPlaneRefusals

The gRPC AdminService has refused calls for fifteen minutes.

## Meaning

Calls ended with `Unauthenticated` or `PermissionDenied` throughout the last fifteen
minutes. The server admits a caller by the client certificate's SAN against
`server.grpc.tls.allowed_clients` (`certManager.grpcTLS.allowedClients` in the chart),
and only for the operations, organizations and namespaces its entry grants. A caller
that keeps retrying a refused call is misconfigured, or something unexpected is calling.

## Impact

The refused caller's changes do not apply: for the operator, the affected manifests stay
`Refused` and the server keeps the last state it accepted. Users and relying parties are
not affected. Security: an unknown identity calling repeatedly is worth a look.

## Diagnosis

1. Which methods: `sum by (method, code) (rate(nauthera_grpc_handled_total\{job="<name>",
   code=~"Unauthenticated|PermissionDenied"\}[5m]))`.
2. Who and why: the audit records `admin.access.denied` carry the cause and the
   certificate's names (`peer_identities`); causes include `identity_not_allowed` (no
   SAN on the allowlist; the Common Name never matches), `operation_not_granted`,
   `organization_not_granted`, `namespace_not_granted` and `reserved_manager`. The
   `grpc_call` log lines show `peer_identity` and the error message.
3. On the operator's side: `kubectl get oidcclients -A -o wide` shows the reason and
   message on each refused resource.

## Mitigation

- A legitimate caller missing from the list, or an operation, organization or namespace
  missing from its entry: add it to `certManager.grpcTLS.allowedClients` and upgrade the
  release.
- A proxy that terminates TLS in front of `:9091`: the server sees the proxy's
  certificate; connect directly (see
  [the gRPC AdminService](https://www.nauthera.io/docs/v1/installation#the-grpc-adminservice)).
- A manifest naming an organization or namespace its caller is not granted: move it or
  widen the grant.
- An unknown identity: find and stop the caller; review who can obtain certificates from
  the control plane's CA.

## Escalation

Escalate to the security team if the refused identity is not one you run. Otherwise
handle it in working hours with whoever owns the operator's configuration.

---

[Alert runbooks](https://www.nauthera.io/docs/v1/runbooks/alerts) · Generated from [`internal/runbook/alerts.yaml`](https://github.com/nauthera/nauthera-server/blob/main/internal/runbook/alerts.yaml) and [`operator/internal/runbook/alerts.yaml`](https://github.com/nauthera/nauthera-server/blob/main/operator/internal/runbook/alerts.yaml) in nauthera-server.