# Monitoring (/docs/ship/monitoring)

---
title: Monitoring
description: Two probes that mean different things, one histogram that answers three questions, and request logs that join to a trace. What ships on by default and what you have to turn on.
kind: guide
package: questpie
---

Your app is running. How do you know it is working? Three surfaces answer that,
and they do not all come from the same place.

| Surface            | Comes from                | Needs             |
| ------------------ | ------------------------- | ----------------- |
| Health probes      | `questpie` core routes    | Nothing           |
| Request logs       | `questpie` core, via Pino | Nothing           |
| Traces and metrics | `@questpie/observability` | An OTLP collector |

The first two are already on. The third is one config key.

## The two probes

Two routes come from the core module. There is nothing to enable. Both are
public, and both sit under your handler's base path. That is `/api` in every
starter.

| Route              | Point this at                   | Touches              |
| ------------------ | ------------------------------- | -------------------- |
| `/api/health/live` | `livenessProbe`                 | Nothing              |
| `/api/health`      | `readinessProbe`, load balancer | Database, KV, search |

**Liveness must not touch the database.** If it does, one database blip fails
the probe on every replica at once. The orchestrator restarts all of them. A
recoverable outage turns into a cold start under load. `/api/health/live`
answers from the process alone and returns its uptime in seconds.

**Readiness may.** A replica that cannot reach the database should leave
rotation. A failed readiness probe is exactly how that happens.

```yaml title="deployment.yaml"
livenessProbe:
  httpGet: { path: /api/health/live, port: 3000 }
  periodSeconds: 10
readinessProbe:
  httpGet: { path: /api/health, port: 3000 }
  periodSeconds: 5
  timeoutSeconds: 5
```

### What readiness checks

| Check      | What runs                           | On failure      |
| ---------- | ----------------------------------- | --------------- |
| `database` | `SELECT 1` through the app's client | 503             |
| `kv`       | A read of one reserved key          | 503             |
| `search`   | `app.search.isInitialized()`        | `degraded`, 200 |
| `storage`  | Nothing. Reports it is configured.  | cannot fail     |
| `queue`    | Nothing. Reports it is configured.  | cannot fail     |

Storage and queue are not contacted on purpose. Any real probe costs an
object-storage request or enqueues a job. That happens on every health check, at
load-balancer frequency. Check storage reachability from a scheduled job
instead. The queue line reads `configured` even when no queue adapter is set, so
do not alert on it.

```json
{
	"status": "ok",
	"timestamp": "2026-08-03T09:12:44.108Z",
	"checks": {
		"database": { "status": "ok", "latency_ms": 3 },
		"kv": { "status": "ok", "latency_ms": 1 },
		"search": { "status": "ok" },
		"storage": { "status": "ok", "detail": "configured (not probed)" },
		"queue": { "status": "ok", "detail": "configured (not probed)" }
	}
}
```

Read the body, not just the status. The worst check wins. `unhealthy` returns 503.
`degraded` and `ok` both return 200. Both probes answer with
`cache-control: no-store`.

`degraded` has exactly one cause. Search has not finished starting. That happens
in the background at boot, so a fresh process can report it for a moment. The
replica keeps serving, which is the point.

<Callout type="warn" title="Give the probe more than four seconds">
	The database and KV checks are each bounded at 2 seconds, and they run one
	after the other. If both hang the response takes four seconds. A shorter probe
	timeout fails before the check does.
</Callout>

<Callout type="info" title="The default KV adapter is in-process">
	On an app with no `kv` config the check reads an in-memory map, so it proves
	nothing about a shared cache. It becomes a real network round trip once you
	point KV at Redis.
</Callout>

## Request logs

Every request writes one structured line through Pino. The message is always
`HTTP request completed`. The record carries `requestId`, `traceId`, `method`,
`path`, `status`, `durationMs` and `slow`. The matched route pattern arrives as
`route` when routing found one.

Output is JSON on stdout. Pretty printing turns itself on when `NODE_ENV` is
`development`. The level defaults to `info`.

```ts title="src/questpie/server/questpie.config.ts"
import { runtimeConfig } from "questpie/app";

export default runtimeConfig({
	db: { url: process.env.DATABASE_URL! },
	app: { url: process.env.APP_URL! },
	logger: {
		requests: {
			slowThresholdMs: 1000,
			logSuccessfulRequests: false,
			ignorePaths: ["/api/health", "/api/health/live"],
		},
	},
});
```

`ignorePaths` matters more than it looks. A readiness probe every 5 seconds
across 10 replicas is 172,800 lines a day that tell you nothing.

<Callout type="warn" title="ignorePaths matches the full pathname">
	Including the base path. Under a handler mounted at `/api` the value is
	`/api/health`, not `/health`. A string has to match exactly. Pass a `RegExp`
	when it should not.
</Callout>

`logSuccessfulRequests` and `ignorePaths` only suppress successes. Neither one
can hide a failing request. `logger.requests: false` turns the lot off in one
word.

[Request logs](/docs/ship/monitoring/request-logs) has every option, the order
the filters run in, which level a request lands at, and the shape of the record.

## Correlation

Every response carries `x-request-id` and `x-trace-id`. The request id is read
from an inbound `x-request-id` or `x-correlation-id` when there is one. So an id
your proxy assigned survives. Otherwise it is a fresh UUID.

The trace id comes from `x-trace-id` first. Then from the trace id inside
`traceparent`. Failing both, it falls back to the request id.

Both land on the log line and on the request's span. So you can start from
either and find the other.

## Traces and metrics

Install `@questpie/observability` and give it an OTLP endpoint. Requests, CRUD
calls, database queries, transactions, jobs, KV and search all open spans. An
inbound `traceparent` is continued rather than replaced.

One metric is recorded. `http.server.request.duration` is a histogram in
seconds, sliced by `http.request.method`, `http.route` and
`http.response.status_code`. That one instrument is the whole RED triple. Rate
is its count. Errors are that count sliced by status. Duration is the histogram
itself. Parallel counters only duplicate the series.

All collections are served by one parameterized route. So `http.route` will not
tell `posts` apart from `users`.

Flush on shutdown or the last batch is lost. That batch is usually the
interesting one.

```ts
process.on("SIGTERM", async () => {
	await app.destroy();
	process.exit(0);
});
```

[Observability](/docs/infrastructure/observability) covers the adapter options,
sampling and pointing it at a backend.

## What to alert on

| Alert on                              | Read it from                    | Why                                                              |
| ------------------------------------- | ------------------------------- | ---------------------------------------------------------------- |
| Readiness failing on several replicas | The probe                       | One replica is one replica. Several at once is a shared cause.   |
| 5xx rate                              | The histogram, sliced by status | A 404 is a client asking for something absent. A 500 is you.     |
| p99 request duration                  | The same histogram              | The mean hides the requests people complain about.               |
| Failed job spans                      | Traces                          | A job fails outside any request, so nothing upstream reports it. |
| Database connection saturation        | Your database                   | Usually the first hard limit you hit.                            |

Queue depth is worth watching too, and QUESTPIE does not emit it. Read it from
your queue backend. A queue draining slowly is a different problem from a queue
that is failing. Only the second is urgent at 3am.

## Related

- [Request logs](/docs/ship/monitoring/request-logs), every option and the filter order.
- [Instrumented seams](/docs/infrastructure/observability/seams), every span and attribute.
- [Trace a slow request](/docs/guides/debug-a-slow-request), install to waterfall.
- [Scaling](/docs/ship/scaling), replicas, workers and connection limits.