Version: Latest

Security Hardening

Semantic Router sits on the request path between clients and model providers. Treat it as part of the application's trust boundary: it can inspect prompts, choose providers, mutate requests, and optionally retain routing data.

This guide highlights the controls that need an explicit production decision. It does not replace the identity, network, secret-management, and data-governance controls of the surrounding platform.

Map the trust boundaries

Review each boundary separately:

who can call inference endpoints;
which identity claims the Router trusts;
which models and tools each role may use;
where provider credentials are stored;
which requests may leave the local environment;
what prompts, responses, and route metadata are retained; and
who can change configuration or inspect stored data.

Protect the public listener

The maintained Envoy configuration removes internal control headers before a client request reaches the Router. Do the same when supplying a custom Envoy or gateway configuration. Internal examples include:

request_headers_to_remove:
  - x-vsr-looper-request
  - x-vsr-looper-secret
  - x-vsr-looper-decision
  - x-vsr-looper-iteration
  - x-authz-user-id
  - x-authz-user-groups

Do not expose Router management, metrics, ExtProc, or backing-store ports as public inference endpoints. Terminate client authentication at a trusted boundary and allow only that component to supply identity headers.

Configure authorization and rate limits

The Dashboard Security Policy page maps users or groups to Router roles and model access, and can define per-subject request and token limits. Saving a valid policy updates the canonical Router configuration and applies it to the active stack.

Use preview before saving when a policy changes several mappings. Keep the management surface authenticated and grant write permissions only to operators who are allowed to change live routing policy.

Relevant Dashboard permissions include:

Permission	Purpose	Default roles
`feedback.submit`	Submit routing feedback.	admin, write
`replay.read`	List replay records.	admin, write, read
`security.manage`	Change security policy.	admin
`logs.read`	Read bounded local-stack service logs.	admin, write

The Router management API distinguishes replay metadata from replay detail. The Dashboard service can retrieve complete records, then removes captured bodies and tool payloads for users who do not have configuration-write access. It does not receive permission to reveal stored secret values.

See the management API reference for endpoints and response contracts.

Keep credentials out of configuration

Use environment references in canonical YAML:

api_key: ${MODEL_API_KEY}

Do not commit literal API keys, passwords, authorization headers, credential query parameters, or URLs containing user information.

For vllm-sr serve --target k8s, the CLI places sensitive environment values in an immutable Secret revision scoped to the namespace and Helm release. Helm values and the Deployment reference the Secret by name; they do not contain the credential value. A failed upgrade keeps the previous workload and Secret active. Release-owned old revisions are removed only after they are no longer referenced.

Existing chart-native Secret references, such as a Dashboard JWT Secret, remain external objects and are not copied into the CLI-managed Secret. Use the same namespace and release ownership discipline for every manually managed Secret.

Secure the local stack's storage credentials

vllm-sr serve provisions Redis and Postgres for the local stack, so it also owns their credentials. Each stack generates its own on first start. No value ships in this repository, and nothing falls back to a shared default.

Where the material lives:

Artifact	Path under `<state-root>/.vllm-sr/storage-secrets/`	Mode
Credential state	`secrets[.<stack>].json`	`0600`
Postgres password	`postgres-password[.<stack>]`	`0600`
Redis config	`redis[.<stack>].conf`	`0644`

The directory itself is 0700 and owner-verified, so every file in it is unreachable by other users. The Redis config is 0644 on purpose: the Redis image drops to an unprivileged user before reading it, and the bind mount resolves inside the container without traversing the host's private parent.

The values reach their consumers without entering any shared surface. Postgres reads its password from the mounted file via POSTGRES_PASSWORD_FILE; Redis reads requirepass from its mounted config; Router receives the values as inherited environment names and the generated runtime config carries only ${VLLM_SR_STACK_POSTGRES_PASSWORD} and ${VLLM_SR_STACK_REDIS_PASSWORD}. They do not appear in a docker command line, a generated config file, a log record, or a report artifact. Dashboard is not given them.

These credentials authenticate network peers. They do not constrain a caller that can reach the container runtime directly: the Postgres image trusts local socket connections, so anyone able to docker exec bypasses the password. Keep container-runtime access restricted accordingly.

Network layering

The local stack runs on two bridge networks.

Container	`vllm-sr-network`	`vllm-sr-data-network`
Redis, Postgres, Milvus	no	yes
Router	yes	yes
Envoy, Dashboard, simulator	yes	no
Jaeger, Prometheus, Grafana	yes	no
OpenClaw workloads	yes	no

Router is the only container on both. Requests reach it over the application network; it reaches the stores over the data network. A named stack prefixes both names, so two stacks share neither. Milvus joins the data network even though it has no credentials of its own yet.

This closes east-west reachability. A container on the application network -- a sidecar, the simulator, an image chosen for an OpenClaw workload -- cannot open a connection to vllm-sr-redis:6379 or vllm-sr-postgres:5432 at all. The storage ports remain published on 127.0.0.1 only, which closes the same exposure from the host side.

It does not constrain a caller that can reach the container runtime. Such a caller can attach a container to any network, so the split is a boundary for workloads, not for the runtime socket.

A stack created before the split has its stores on the application network. The next vllm-sr serve attaches each running store to the data network and detaches it from the application network. If that detach fails, serve stops rather than continuing: a stack that reports the isolation without having it is worse than one that refuses to start.

Rotate

vllm-sr storage rotate

The command is scoped to one stack and follows VLLM_SR_STACK_NAME, like serve and stop. Rotate each stack separately; there is deliberately no cross-stack mode, because a partial failure would leave some stacks revoked and others not.

Rotation has a short degradation window. Postgres changes its role password in place, so existing connections continue but new ones fail until Router restarts. Redis is rebuilt against its named volume. Plan the rotation for a moment when a brief Router restart is acceptable.

Recover

The credential state is missing or malformed. The CLI fails closed rather than regenerating silently, because a regenerated credential would leave the CLI believing it has access it no longer has. Delete the state file and rerun vllm-sr serve. The stack is taken over in place: Postgres is re-keyed over its trusted local socket, Redis is rebuilt against the same named volume, and no data is lost.

Data from an older stack is not picked up. Storage data now lives in named volumes, and an existing container's volume is adopted by name when the stack is taken over. A container removed by an older CLI leaves its volume behind with no record of which container it belonged to, so it cannot be adopted automatically. Recover it manually:

docker system df -v --format '{{json .Volumes}}'

Look for volumes with Links: 0. Identify each candidate by its contents — a Postgres data directory contains PG_VERSION, a Redis one contains dump.rdb:

docker run --rm -v <volume>:/v:ro alpine ls /v

Then start a container against the identified volume, or copy its contents into the stack's named volume (vllm-sr-postgres-data / vllm-sr-redis-data, with the stack prefix for a named stack). The CLI does not guess which orphaned volume is yours.

Recipe activation reports that the credentials are not readable. Dashboard can start a managed store while activating a Recipe, but it runs as a separate, less trusted account -- it holds the container-runtime socket, which is exactly what these credentials are kept away from -- so it cannot read the credential state, and it will not guess which volume holds the data. Run vllm-sr serve from the account that owns the stack to provision the store, then activate the Recipe again.

An older CLI is used against a rotated stack. It will fail to authenticate. That is the intended outcome. Either upgrade the CLI, or reset the passwords by hand through the container runtime.

Review stored request data

Replay, response cache, memory, response history, service logs, and provider logs can all retain data derived from a request. Their settings are independent from the model's placement. A route to a local model can still write a prompt or response to a shared store.

For every enabled store:

identify which routes write to it;
inspect whether request or response bodies are captured;
set a retention and deletion policy;
restrict read and backup access;
use encryption and transport security appropriate to the data; and
test behavior when the store is unavailable.

Recipe Model Cards describe the checked-in replay and cache behavior for each maintained recipe. See Data and Storage for deployment guides.

Limit container-runtime access

Some Dashboard workflows can manage local containers. The CLI mounts a container-runtime socket only when it is a Unix socket with a safe owner and group mode; it rejects symlinks, world-accessible sockets, and unsafe group ownership. The Dashboard repeats the check inside the container and runs as a non-root user.

When the socket is missing or rejected, Router and Dashboard still start, but container-management features report the runtime as unavailable. Do not make a socket world-writable to bypass this protection. Use VLLM_SR_CONTAINER_SOCKET for a non-default rootless runtime socket and verify its user-namespace and supplementary-group mapping.

If the deployment does not need Dashboard-managed containers, do not mount a runtime socket.

Map the trust boundaries​

Protect the public listener​

Configure authorization and rate limits​

Keep credentials out of configuration​

Secure the local stack's storage credentials​

Network layering​

Rotate​

Recover​

Review stored request data​

Limit container-runtime access​

Production checklist​