Skip to content

The v1 docs follow nauthera-server's main branch and its 0.x releases on the way to 1.0, so fields and APIs can still change.

v1

NautheraServerHighLatency

Docs version v1 (main)Nauthera is pre-1.0, in a design-partner program

The 99th percentile latency of the protocol endpoints is above one second.

Meaning

Over 15 minutes, the 99th percentile of nauthera_http_request_duration_seconds across the protocol endpoints (authorize, PAR, token, device authorization, introspection, revocation, UserInfo, end session, JWKS and discovery, by default) has stayed above the threshold (1s). Database pool saturation, a slow cache, CPU starvation and a queue of password verifications are the usual causes.

Impact

Relying parties wait longer for tokens and redirects, and some time out: a client with a short HTTP timeout on /token sees failed sign-ins although the server answers in the end. Users see slow redirects. Discovery and JWKS fetches by resource servers slow down too.

Diagnosis

  1. Which routes are slow: histogram_quantile(0.99, sum by (route, le) (rate(nauthera_http_request_duration_seconds_bucket\{job="<name>"\}[5m]))).
  2. Database pool waits: rate(go_sql_wait_duration_seconds_total\{job="<name>", db_name="nauthera"\}[5m]) and go_sql_in_use_connections against go_sql_max_open_connections.
  3. Cache latency: histogram_quantile(0.99, sum by (op, le) (rate(nauthera_cache_operation_seconds_bucket\{job="<name>"\}[5m]))).
  4. CPU: the chart sets no CPU limit by default; if you set one, look at throttling in your container metrics. Look at nauthera_http_in_flight_requests for a pile-up.
  5. Slow sign-ins only: password verifications queue behind auth.brute_force.max_concurrent_verifies; nauthera_auth_logins_total{result="throttled"} rises when they wait longer than verify_acquire_timeout (2s).

Mitigation

  • Pool saturation: follow NautheraDatabasePoolExhausted.
  • Slow cache: follow NautheraCacheErrors; check the cache's own CPU and memory.
  • CPU or load: add replicas (replicaCount, or autoscaling), or raise resources.requests.cpu so the pods are scheduled where CPU is free.
  • The threshold does not fit your environment: tune metrics.prometheusRule.alerts.NautheraServerHighLatency.threshold.

Escalation

Escalate to the identity platform owner when latency keeps rising or relying parties report timeouts. Involve the database or cache owner when their latency is the cause. If the latency is inside the server with healthy dependencies, open an issue with the per-route p99 graph, a few slow requests' request_ids (and trace IDs when tracing is on) and the server version.

Alert runbooks · Generated from internal/runbook/alerts.yaml and operator/internal/runbook/alerts.yaml in nauthera-server.