# Autoscaling

Turn on the HorizontalPodAutoscaler, meet the gate that stops it growing an uncoordinated fleet, and know what a scale event moves.

Source: https://tokentriage.com/docs/operate/autoscaling

---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';

Let Kubernetes add replicas under load. The chart ships a `HorizontalPodAutoscaler` for the
StatefulSet, off by default, and turning it on means committing to a coordinated fleet. The
chart refuses to render an autoscaler that could grow past one replica without the shared state
underneath it.

**Prerequisites:** `metrics-server` installed in the cluster, and a Redis and Postgres you are
willing to point at. Read [multi-replica & scale-out](/docs/operate/scale-out) first if you have
not; this page is the mechanism, that one is the model.

## Meet the gate first

The topology guard reads the **ceiling** of the reachable fleet, the maximum of
`replicaCount`, `minReplicas` and `maxReplicas`, rather than the floor. An autoscaler with a floor of
one and a ceiling of six could still run six uncoordinated gateways under load, which is exactly
what the guard exists to prevent.

Miss the tier and nothing renders:

```console
$ helm template tt deploy/helm/tokentriage -f autoscaling-values.yaml
Error: values don't meet the specifications of the schema(s) in the following chart(s):
tokentriage:
- at '': 'allOf' failed
  - at '': validation failed
    - at '/experimental/multiReplica/enabled': value must be true
    - at '/externalState': validation failed
      - at '/externalState/postgres/enabled': value must be true
      - at '/externalState/redis/enabled': value must be true
```

The schema refuses first and names every missing piece at once. A template guard behind it
carries the longer explanation, and you meet that one only if schema validation is skipped —
[multi-replica & scale-out](/docs/operate/scale-out#the-single-node-default) quotes it in full.

## Enable it

The chart maps `externalState.redis` and `externalState.postgres` onto the binary's
`shared_state` config; that key is chart-managed, so set it through these values rather than inside
`config:`.

```yaml
autoscaling:
  enabled: true
  minReplicas: 2 # chart default; the fleet floor. Keep >= 2.
  maxReplicas: 6 # chart default; the HPA ceiling.
  targetCPUUtilizationPercentage: 80 # chart default; 0 omits the CPU metric.
  targetMemoryUtilizationPercentage: 0 # 0 omits the memory metric (the default).

experimental:
  multiReplica:
    enabled: true # required for any fleet > 1

externalState:
  redis:
    enabled: true
    external: 'allow' # the binary's own consent string
    addr: redis.data.svc:6379
    auth: { existingSecret: tt-redis, key: password }
  postgres:
    enabled: true
    external: 'allow'
    dsn: { existingSecret: tt-postgres, key: dsn }
```

```console
$ helm upgrade --install tt deploy/helm/tokentriage \
    --namespace tokentriage --create-namespace \
    -f autoscaling-values.yaml
```

Those five `autoscaling` values are the chart defaults verbatim. Two escape hatches exist for
setups the defaults do not reach: `autoscaling.extraMetrics` appends raw `autoscaling/v2` metric
entries, and `autoscaling.behavior` passes a `scaleUp`/`scaleDown` block through.

<Callout type="warning">
Keep `minReplicas` at two or more. With autoscaling on, the StatefulSet seeds at `minReplicas`
and the autoscaler owns the count from then on, so `replicaCount` is **not** the source of truth
in this mode. A floor of one is representable and collapses the coordinated fleet to a single node
whenever load is quiet.
</Callout>

## Confirm what rendered

An `autoscaling/v2` object targeting the StatefulSet, with the memory metric omitted because its
target is zero:

```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: tt-tokentriage
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: StatefulSet
    name: tt-tokentriage
  minReplicas: 2
  maxReplicas: 6
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 80
```

Then check it is attached and reading a metric:

```console
$ kubectl get hpa -n tokentriage
```

A `TARGETS` column showing `<unknown>` means `metrics-server` is missing or not yet reporting,
and the autoscaler will not act until it resolves.

## What a scale event moves

A limit with `store: share` enforces a global approximation by splitting across the live fleet,
and on **every** scale event that split re-derives from the new membership, so the fleet ceiling
holds at any N:

`fleet ≤ N × ceil(amount / N_live)`, over-admitting at most `N−1` from rounding.

The one transient worth knowing is the join. A newly started pod is not in the roster until its
first heartbeat, so for that window the split is computed against the smaller live N and the
fleet can briefly admit slightly more than the steady-state ceiling. It is bounded, fail-open,
and disclosed on the response header and the limits panel rather than smoothed over.

Token and request quotas converge over the larger fleet rather than splitting, so a scale event
changes their convergence lag and not their bound.

<Callout type="key">
Scaling coordinates **governance**. It never moves or replicates the decision ledger. Each new
pod gets its own volume and its own canonical ledger, and the fleet view merges them at read
time. Your storage arithmetic multiplies by the replica count, and so does your backup
procedure.
</Callout>

## Two consequences of an autoscaled fleet

**A disruption budget becomes valid.** A `PodDisruptionBudget` on a singleton is refused,
because an availability budget on one pod blocks node drains for no benefit. The guard keys on
the autoscaling **floor**, so a fleet with `minReplicas: 2` is not a singleton and
`pdb.enabled: true` is accepted even with `replicaCount` left at its default.

**An upgrade re-seeds and then re-scales.** A `helm upgrade` sets the StatefulSet back to
`minReplicas`, and the autoscaler climbs again toward current load. Treat `minReplicas` as the
value a redeploy settles to momentarily, not as a step-down of your live fleet.

<Details summary="Why no Service sets session affinity">

None of the Services set `sessionAffinity`, and that is deliberate rather than an omission. The
data plane is stateless per request, and each pod owns its own ledger, so pinning a client to a
pod would defeat load spreading without buying any correctness, because the fleet read merges
per-pod ledgers regardless of which pod served which request.

The one thing that *is* pod-affine is a routing affinity pin, which lives inside a request's
decision rather than in the load balancer.

</Details>

## What this does not cover

The coordination model, the disclosed bounds and the churn residuals belong to [multi-replica &
scale-out](/docs/operate/scale-out). Choosing a store per limit is [enforcement
classes](/docs/build/governance-enforcement-classes). Deciding whether you need more replicas at
all, and what saturates first, is [sizing and capacity](/docs/operate/capacity).

<NextSteps items={[
  {title: "Multi-replica & scale-out", href: "/docs/operate/scale-out", description: "What coordinates, what does not, and the residuals under replica churn."},
  {title: "Sizing and capacity", href: "/docs/operate/capacity", description: "Whether a second replica is the right answer, and what it costs in storage."},
  {title: "Enforcement classes", href: "/docs/build/governance-enforcement-classes", description: "Which limit types bind which store across a fleet."},
  {title: "Back up and restore", href: "/docs/operate/backup-restore", description: "One ledger per replica means one backup procedure per replica."}
]} />