# Production hardening
What ships locked down, what you still have to do, and the four decisions that separate a demo install from an exposed one.
Source: https://tokentriage.com/docs/operate/hardening
---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
Most of the hardening is already done when you install. The chart's defaults pass the
Kubernetes Pod Security Standards **restricted** profile with no overrides, the dashboard is
secured before it serves its first byte, and the container holds no credential. What is left is
a short list, and this page is that list.
Work through the baseline before you point production traffic at an install. The extended
items cost more and are worth their cost in some environments and not others; each says which.
**Prerequisites:** an installed release you can upgrade, and `helm template` to check what a
change renders. Verify each item against your own render rather than trusting this page.
## Baseline
### 1. Terminate TLS in front of the data plane
TokenTriage terminates no TLS on either plane. Virtual keys travel to the data plane as bearer
credentials, so exposing it without TLS in front puts those credentials on the wire in
cleartext.
Configure TLS at your ingress or load balancer. `ingress.data.tls` is passed through to the
`Ingress`'s `spec.tls` verbatim. Only once TLS genuinely fronts the data plane, set:
```yaml
config:
governance:
keys:
tls_terminated_upstream: true
```
That field is an operator **attestation**, not a probe. Nothing verifies it for you, which is
exactly why it is yours to set.
### 2. Keep the admin plane off the internet
The admin Service is `ClusterIP` and the chart makes that value a constant. It cannot be
overridden to `NodePort` or `LoadBalancer`, because a knob that silently did nothing on the
plane carrying the ledger would be worse than no knob. Reach it with a port-forward:
```console
$ kubectl port-forward -n tokentriage svc/tt-tokentriage-admin 9090:9090
```
`ingress.admin.enabled=true` does expose it, and the post-install notes fire a warning on that
toggle. If you must, put authentication and TLS in front and scope the source through the
ingress controller's annotations.
On a Compose or host deployment the admin plane binds `127.0.0.1` by default and the compose
ports publish to loopback. Widening either with `TT_ADMIN_BIND=0.0.0.0` hands `/tt/api/*`, the
ledger metadata and `/metrics` to anything that can reach the host. Do it only behind
authentication.
### 3. Mint a credential, and know that nothing exists until you do
The chart creates zero `Secret` objects and stores zero credentials. The dashboard serves but
no login succeeds until you mint one:
```console
$ kubectl exec -n tokentriage tt-tokentriage-0 -- \
tokentriage token new --role admin --state-dir /data/.tokentriage/dash
token id: 40bd4df95ae0d61b role: admin
This token is shown ONCE and cannot be recovered:
ttk_...
```
The plaintext exists only on your terminal; only its SHA-256 reaches the volume. Mint the
lowest role that does the job. `viewer` reads everything including exports, `operator` adds
config-emitting writes, and `admin` adds user and token management and key minting.
Verify the secured-by-default state before you mint anything, which is what the bundled test
does:
```console
$ helm test tt -n tokentriage
TEST SUITE: tt-tokentriage-test-connection
Phase: Succeeded
```
It asserts `/health` returns `200` **and** an unauthenticated `/tt/api/status` returns `401`.
The whole bootstrap depends on `kubectl exec` into pod zero. Clusters that forbid exec cannot
use it, and there is no workaround today. An opt-in, RBAC-scoped bootstrap Job is the designed
response and has not landed, because granting a chart the permission to write credentials is
the thing this design exists to avoid, so it will be something you turn on knowingly.
### 4. Supply credentials by reference, never by value
Provider keys arrive as a reference to a `Secret` you created; the chart mounts it with
`envFrom` and holds no value of its own. The config names the environment variable, and the
loader refuses an inline secret where a reference belongs:
```yaml
providerSecrets:
existingSecret: tt-provider-keys
```
On Compose the equivalent is an `env_file` you fill and keep out of version control. On the
Terraform module it is a secret ARN resolved on the instance. In all three the config carries a name.
### 5. Check what the defaults actually rendered
The hardening you did not configure is the hardening most likely to be wrong, so read it back:
```console
$ helm template tt deploy/helm/tokentriage -f my-values.yaml | grep -A 12 securityContext
```
The defaults, as rendered:
```yaml
# pod
securityContext:
runAsNonRoot: true
runAsUser: 65532
runAsGroup: 65532
fsGroup: 65532
fsGroupChangePolicy: OnRootMismatch
seccompProfile:
type: RuntimeDefault
# container
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: ["ALL"]
```
The pod also renders `automountServiceAccountToken: false`. The binary makes no calls to the
Kubernetes API, so no token is projected. `readOnlyRootFilesystem: true` is why the chart mounts
a writable `emptyDir` at `/tmp` and the volume at `/data`; the config mount is read-only.
You can enforce the profile cluster-side as well:
```console
$ kubectl label namespace tokentriage \
pod-security.kubernetes.io/enforce=restricted \
pod-security.kubernetes.io/enforce-version=latest
```
## Extended
### Set resource requests and limits
`resources` is empty by default, because the chart cannot size limits for your traffic. A
starting point for a single-node install is `requests: {cpu: 250m, memory: 256Mi}` and
`limits: {cpu: "1", memory: 512Mi}`. Treat that as a starting point rather than a
measurement; [sizing and capacity](/docs/operate/capacity) says what is and is not measured.
### Turn on a NetworkPolicy, if your CNI enforces one
`networkPolicy` is off by default and renders nothing. Enabled, it sets both policy types with
one of two presets:
| `networkPolicy.preset` | Egress | Ingress |
|---|---|---|
| `providers` (default) | DNS, HTTPS 443 to provider APIs, intra-namespace | data and admin ports |
| `airgap` | DNS and intra-namespace only; no 443 | same |
```console
$ helm upgrade tt deploy/helm/tokentriage -n tokentriage --reuse-values \
--set networkPolicy.enabled=true --set networkPolicy.preset=airgap
```
Two things about that object are worth knowing before you treat it as a boundary. Rendering a
NetworkPolicy declares intent; only a CNI that implements NetworkPolicy **enforces** it, and on
one that does not the object is silently inert. And the rendered rules are looser than the
preset names suggest, in ways the following section sets out.
### Harden the container on Compose
The Compose hardening overlay is a pure security layer that touches neither topology nor config,
so it composes onto any stack:
```console
$ docker compose -f docker-compose.yml -f deploy/compose/compose.prod.yaml \
-f deploy/compose/compose.hardened.yaml --profile test up \
--exit-code-from tests --abort-on-container-exit
[prod-tests] BOOT+HEALTH GATE: PASS (/health 200 after 0s)
```
It adds `read_only: true`, `cap_drop: [ALL]`, `no-new-privileges`, an explicit
`user: 65532:65532`, and a `noexec,nosuid,nodev` tmpfs at `/tmp`. That run is from this page's
verification pass; the binary tolerates a read-only root filesystem because everything it writes
is on the data volume or in `/tmp`.
### On OpenShift, null the identity fields
OpenShift's `restricted-v2` constraints assign each pod a UID and GID from the namespace's
range and **reject** a pod pinning them to fixed values outside it, so the chart's `65532`
defaults are refused. The shipped overlay nulls the three identity fields, which the chart's
helper prunes from the manifest entirely rather than emitting as `null`, while keeping
`runAsNonRoot`, the seccomp profile and every container-level field:
```console
$ helm install tt deploy/helm/tokentriage \
--namespace tokentriage --create-namespace \
-f deploy/helm/tokentriage/examples/openshift-values.yaml \
-f my-values.yaml
```
The overlay carries no `config:` block, so it is layered on top of your values rather than used
alone. It is verified at render level; the admission behaviour rests on the OpenShift platform
contract and is not exercised in this project's CI, which cannot enforce those constraints.
Validate it on a real cluster before you rely on it.
## What the presets do not deny
A hardening page that only lists what is switched on teaches the wrong lesson. Two of the
rendered rules are wider than their names imply, and both are visible in the render:
```yaml
egress:
- ports: [{ port: 53, protocol: UDP }, { port: 53, protocol: TCP }]
- to: [{ podSelector: {} }]
ingress:
- ports: [{ port: 8787, protocol: TCP }, { port: 9090, protocol: TCP }]
```
**The DNS rule has no `to:` selector.** Port 53 is open outbound to any reachable address, not
scoped to your cluster's resolver. It is the one rule in the `airgap` preset a reader could
mistake for "no egress at all". Narrow it with `networkPolicy.extraEgress` if you know where
your resolver lives.
**The ingress rule has no `from:` selector.** Both ports are admitted from any source in the
cluster. The expectation is that you front the data plane with an ingress or gateway; scope it
yourself through `extraIngress` if you need more.
Neither is a bug, and neither is a containment boundary on its own. The air-gap guarantee comes
from construction rather than from these rules. [Install air-gapped](/docs/operate/air-gap)
covers what that means.
## What this does not cover
The reasoning behind the auth model, the role matrix and how secrets are handled across
deployment shapes belongs to [security & secrets](/docs/operate/security). The guardrails tier
has its own containment story on [air-gap & known limits](/docs/guardrails/air-gap-and-limits).
Governance limits, which decide who may spend what, are a separate subject from who may reach
the admin plane, and live under [governance](/docs/build/governance).