# NautheraServerDown

No nauthera-server replica answers a Prometheus scrape.

## Meaning

Prometheus has had no successful scrape of the server's admin listener (`:9090/metrics`)
from any replica for five minutes. Either no replica is running, every replica is
failing, or Prometheus can no longer reach them (a NetworkPolicy, a changed Service or
ServiceMonitor, a selector that no longer matches).

## Impact

If the replicas are really gone, every relying party is affected: no user can sign in,
no token is issued or refreshed, and the admin API and dashboard are down. Tokens
already issued keep working at resource servers that validate them offline against a
cached JWKS. If only scraping broke, users are unaffected but every other alert is blind.

## Diagnosis

1. Tell an outage from a scrape problem: `kubectl -n <ns> get pods -l app.kubernetes.io/name=nauthera-server`
   and `curl -fsS https://<issuer>/.well-known/openid-configuration`.
2. Pods present but not Ready: `kubectl -n <ns> describe pod <pod>` for probe failures and
   events, and `kubectl -n <ns> logs <pod> --previous` for the last crash. A refused
   configuration is logged at startup and the process exits.
3. No pods: `kubectl -n <ns> get deploy,rs,events --sort-by=.lastTimestamp` for a failed
   rollout, quota or scheduling problem.
4. Pods Ready but no scrape: open Prometheus' Targets page for the job and check the
   ServiceMonitor's labels, the `metrics` port and any NetworkPolicy in front of `:9090`.

## Mitigation

- A bad rollout: `helm rollback <release>` or `kubectl -n <ns> rollout undo deploy/<name>`.
- A configuration refused at startup: fix the value the log names and roll out again.
- A dependency outage keeping pods unready: follow NautheraDatabaseUnavailable.
- A scrape-only problem: restore the ServiceMonitor or the network path; the server itself
  needs nothing.

## Escalation

This is a full outage of sign-in. Page the owner of the identity platform at once. If
the server crashes on a configuration it accepted before, or panics, open an issue with
the log of the crashed container and the server version (`nauthera-server version`).

---

[Alert runbooks](https://www.nauthera.io/docs/v1/runbooks/alerts) · Generated from [`internal/runbook/alerts.yaml`](https://github.com/nauthera/nauthera-server/blob/main/internal/runbook/alerts.yaml) and [`operator/internal/runbook/alerts.yaml`](https://github.com/nauthera/nauthera-server/blob/main/operator/internal/runbook/alerts.yaml) in nauthera-server.