Skip to main content
Sign In
Platforms

Kubernetes

Run the Rivet control plane on Kubernetes.

Requirements

  • A Kubernetes cluster with kubectl configured
  • Metrics server, required for the HPA and included by default in k3d, GKE, EKS, and AKS
  • An Ingress controller you can raise timeouts on

Steps

Download the manifests

npx giget@latest gh:rivet-dev/rivet/self-host/control-plane/kubernetes rivet-k8s
cd rivet-k8s

Configure the engine

In 02-engine-configmap.yaml, set public_url to the external URL the control plane will be reachable at.

In 11-postgres-secret.yaml, set the PostgreSQL password. To use a managed database instead, see Managed PostgreSQL below.

Create the admin token

openssl rand -hex 32

Save the value somewhere safe, then store it as a secret:

kubectl create namespace rivet-engine
kubectl -n rivet-engine create secret generic rivet-engine-secrets --from-literal=admin-token=YOUR_TOKEN_HERE

Deploy

kubectl apply -f .

kubectl -n rivet-engine wait --for=condition=ready pod -l app=nats --timeout=300s
kubectl -n rivet-engine wait --for=condition=ready pod -l app=postgres --timeout=300s
kubectl -n rivet-engine wait --for=condition=ready pod -l app=rivet-engine --timeout=300s

Raise the Ingress timeout

Long-lived WebSockets are dropped by the default 30 to 60 second idle timeout. On an NGINX Ingress:

nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"

Other controllers are covered in Ports. Skipping this step produces reconnect storms under load rather than an obvious failure at deploy time.

Verify

kubectl -n rivet-engine get pods

Every pod should be Running. Visit /ui on your public_url and log in with the admin token.

Create the runner config

A runner config declares a pool that workers connect to. Without one the control plane refuses every worker, so create it before deploying any:

curl -X PUT "<public_url>/runner-configs/default?namespace=default" \
  -H "Authorization: Bearer YOUR_TOKEN_HERE" \
  -H "Content-Type: application/json" \
  -d '{"datacenters":{"default":{"normal":{}}}}'

Use normal for long-lived workers. See Runner configs.

Managed PostgreSQL

To use Amazon RDS, Cloud SQL, or Azure Database instead of the bundled Postgres:

  1. Point postgres.url in 02-engine-configmap.yaml at your instance.
  2. Delete the bundled manifests: 10-postgres-configmap.yaml, 11-postgres-secret.yaml, 12-postgres-statefulset.yaml, 13-postgres-service.yaml.

Use the direct connection string, not a pooled one. See Storage.

Autoscaling

05-rivet-engine-hpa.yaml is a HorizontalPodAutoscaler on the engine Deployment. It scales between 2 and 10 replicas, targeting 60% average CPU and 80% average memory.

Memory is a backstop, not a scaling signal. It does not fall as quickly as CPU, so the manifest pairs it with a 15 minute scale-down stabilization window and removes at most one pod at a time. Without that, a memory spike holds the replica count up long after load has passed, and every removed replica severs the long-lived WebSockets it was serving.

kubectl -n rivet-engine get hpa rivet-engine

It needs the metrics server from the requirements above. Without it the targets read <unknown> and the HPA never scales.

Autoscaling is only safe here because this bundle runs on PostgreSQL and NATS. The file system (RocksDB) backend is single-node, so an engine using it must stay at one replica. See Storage.

Applying configuration changes

The ConfigMap is read at startup, so a change needs a restart:

kubectl apply -f 02-engine-configmap.yaml
kubectl -n rivet-engine rollout restart deployment/rivet-engine

Next steps

Edit this page