Skip to content

The v1 docs follow nauthera-server's main branch and its 0.x releases on the way to 1.0, so fields and APIs can still change.

v1

NautheraServerErrorBudgetBurnFast

Docs version v1 (main)Nauthera is pre-1.0, in a design-partner program

Public requests fail with a 5xx fast enough to spend the error budget within days.

Meaning

The share of public requests answered with a 5xx is above 14.4 times the error budget of availabilityObjective (99.9% by default, a budget of 0.1%) over both the last hour and the last five minutes, or above 6 times it over both six hours and 30 minutes. It is a multi-window burn-rate alert: it fires on a real outage within minutes and clears soon after the errors stop. Most 5xx come from a dependency (database, cache), the audit pipeline with audit.fail_closed, or a defect in a new version.

Impact

Users signing in, relying parties exchanging codes and refreshing tokens, and administrators all see failures on the routes that error. At 14.4 times the budget about 1.4% of all requests fail with the default objective; the route breakdown says whether it is everyone's sign-in or one endpoint.

Diagnosis

  1. Which routes and statuses: sum by (route, code) (rate(nauthera_http_requests_total\{job="<name>",code=~"5.."\}[5m])). route is the API contract's operation name, such as token or authorize.
  2. Check the dependency alerts: NautheraDatabaseUnavailable, NautheraCacheUnavailable, NautheraCacheErrors, NautheraAuditWritesFailing.
  3. Read the failing requests' log lines: kubectl -n <ns> logs deploy/<name> | jq 'select(.msg=="http_request" and .status>=500)', and other lines with the same request_id.
  4. A user's reference ID from an error page can be pasted into the admin dashboard's Troubleshooting page to see what the server recorded for that request.
  5. Correlate with the last rollout: kubectl -n <ns> rollout history deploy/<name>.

Mitigation

  • A dependency outage: follow that alert's runbook.
  • Started with a rollout: helm rollback <release>.
  • 503 on admin changes only, with the audit pipeline degraded and audit.fail_closed on: follow NautheraAuditWritesFailing.
  • One replica failing while others are fine: delete that pod and watch whether the replacement fails the same way.

Escalation

Page the identity platform owner: this rate exhausts a month's budget in days. If the 5xx come from the server itself with healthy dependencies, open an issue with the routes and status codes, a few request_ids and their log lines, and the server version (nauthera_build_info).

Alert runbooks · Generated from internal/runbook/alerts.yaml and operator/internal/runbook/alerts.yaml in nauthera-server.