# NautheraServerErrorBudgetBurnFast

Public requests fail with a 5xx fast enough to spend the error budget within days.

## Meaning

The share of public requests answered with a 5xx is above 14.4 times the error budget of
`availabilityObjective` (99.9% by default, a budget of 0.1%) over both the last hour and
the last five minutes, or above 6 times it over both six hours and 30 minutes. It is a
multi-window burn-rate alert: it fires on a real outage within minutes and clears soon
after the errors stop. Most 5xx come from a dependency (database, cache), the audit
pipeline with `audit.fail_closed`, or a defect in a new version.

## Impact

Users signing in, relying parties exchanging codes and refreshing tokens, and
administrators all see failures on the routes that error. At 14.4 times the budget about
1.4% of all requests fail with the default objective; the route breakdown says whether it
is everyone's sign-in or one endpoint.

## Diagnosis

1. Which routes and statuses: `sum by (route, code)
   (rate(nauthera_http_requests_total\{job="<name>",code=~"5.."\}[5m]))`. `route` is the
   API contract's operation name, such as `token` or `authorize`.
2. Check the dependency alerts: NautheraDatabaseUnavailable, NautheraCacheUnavailable,
   NautheraCacheErrors, NautheraAuditWritesFailing.
3. Read the failing requests' log lines: `kubectl -n <ns> logs deploy/<name> | jq
   'select(.msg=="http_request" and .status>=500)'`, and other lines with the same
   `request_id`.
4. A user's reference ID from an error page can be pasted into the admin dashboard's
   Troubleshooting page to see what the server recorded for that request.
5. Correlate with the last rollout: `kubectl -n <ns> rollout history deploy/<name>`.

## Mitigation

- A dependency outage: follow that alert's runbook.
- Started with a rollout: `helm rollback <release>`.
- `503` on admin changes only, with the audit pipeline degraded and `audit.fail_closed`
  on: follow NautheraAuditWritesFailing.
- One replica failing while others are fine: delete that pod and watch whether the
  replacement fails the same way.

## Escalation

Page the identity platform owner: this rate exhausts a month's budget in days. If the
5xx come from the server itself with healthy dependencies, open an issue with the routes
and status codes, a few `request_id`s and their log lines, and the server version
(`nauthera_build_info`).

---

[Alert runbooks](https://www.nauthera.io/docs/v1/runbooks/alerts) · Generated from [`internal/runbook/alerts.yaml`](https://github.com/nauthera/nauthera-server/blob/main/internal/runbook/alerts.yaml) and [`operator/internal/runbook/alerts.yaml`](https://github.com/nauthera/nauthera-server/blob/main/operator/internal/runbook/alerts.yaml) in nauthera-server.