container.md Raw History Folder

Unprivileged container

contrib/container/Containerfile packages the runner for a rootless container host (Podman or Docker). Jobs run in Firecracker microVMs (vm isolation) inside the container. Nothing runs as root on the host.

What the image holds

ermite-runner, Firecracker and tini.
The guest kernel (vmlinux) and the base disk (base.ext4), from images.
Defaults: ERMITE_RUNNER_ISOLATION=vm, ERMITE_RUNNER_JAILER_MODE=off, ERMITE_RUNNER_STATE_DIR=/var/lib/ermite-runner, JSON logs.

Build

scripts/build-kernel.sh tmp/image
scripts/build-image.sh tmp/image
docker build -f contrib/container/Containerfile --build-arg VERSION=<tag> -t ermite-runner:<tag> .

What the container needs

Grant	Why
`/dev/kvm`	Firecracker
`/dev/net/tun`	One TAP device per job
`CAP_NET_ADMIN`	TAP devices and nftables rules, in the container's own network namespace only
A volume on `/var/lib/ermite-runner`	Registration, caches and job disks

No --privileged, no host network, no other capability. With rootless Podman, a CDI spec can expose the two device nodes.

The jailer is off: it needs root to chroot and switch uid. The container and the user namespace are the outer boundary instead, and each job still runs in its own microVM.

First start

With no registration in the state directory, the runner registers itself when a register token is set:

ERMITE_RUNNER_REGISTER_TOKEN_FILE: path of a file holding the token (a container secret), or
ERMITE_RUNNER_REGISTER_TOKEN.

The token is read only on that first start. Revoke it afterwards. Labels are fixed at registration: to change them, remove the runner on the server, delete runner.json from the state directory and restart with a fresh token.

Reaching the server

When the server runs on the same container network, set ERMITE_RUNNER_URL to its internal address and turn on guest forge access with a relay:

ERMITE_RUNNER_URL=http://forge:8080
ERMITE_RUNNER_INSECURE_HTTP=1
ERMITE_RUNNER_GUEST_FORGE=1
ERMITE_RUNNER_GUEST_FORGE_TARGET=forge:8080

Jobs then clone from http://forge:8080, which the runner relays to the server.

source

1# Unprivileged container
2
3`contrib/container/Containerfile` packages the runner for a rootless container host (Podman or Docker). Jobs run in Firecracker microVMs (`vm` isolation) inside the container. Nothing runs as root on the host.
4
5## What the image holds
6
7- `ermite-runner`, Firecracker and `tini`.
8- The guest kernel (`vmlinux`) and the base disk (`base.ext4`), from [images](images.md).
9- Defaults: `ERMITE_RUNNER_ISOLATION=vm`, `ERMITE_RUNNER_JAILER_MODE=off`, `ERMITE_RUNNER_STATE_DIR=/var/lib/ermite-runner`, JSON logs.
10
11## Build
12
13```sh
14scripts/build-kernel.sh tmp/image
15scripts/build-image.sh tmp/image
16docker build -f contrib/container/Containerfile --build-arg VERSION=<tag> -t ermite-runner:<tag> .
17```
18
19## What the container needs
20
21| Grant | Why |
22| --- | --- |
23| `/dev/kvm` | Firecracker |
24| `/dev/net/tun` | One TAP device per job |
25| `CAP_NET_ADMIN` | TAP devices and nftables rules, in the container's own network namespace only |
26| A volume on `/var/lib/ermite-runner` | Registration, caches and job disks |
27
28No `--privileged`, no host network, no other capability. With rootless Podman, a CDI spec can expose the two device nodes.
29
30The jailer is off: it needs root to chroot and switch uid. The container and the user namespace are the outer boundary instead, and each job still runs in its own microVM.
31
32## First start
33
34With no registration in the state directory, the runner registers itself when a register token is set:
35
36- `ERMITE_RUNNER_REGISTER_TOKEN_FILE`: path of a file holding the token (a container secret), or
37- `ERMITE_RUNNER_REGISTER_TOKEN`.
38
39The token is read only on that first start. Revoke it afterwards. Labels are fixed at registration: to change them, remove the runner on the server, delete `runner.json` from the state directory and restart with a fresh token.
40
41## Reaching the server
42
43When the server runs on the same container network, set `ERMITE_RUNNER_URL` to its internal address and turn on [guest forge access](sandboxes.md#guest-forge-access) with a relay:
44
45```sh
46ERMITE_RUNNER_URL=http://forge:8080
47ERMITE_RUNNER_INSECURE_HTTP=1
48ERMITE_RUNNER_GUEST_FORGE=1
49ERMITE_RUNNER_GUEST_FORGE_TARGET=forge:8080
50```
51
52Jobs then clone from `http://forge:8080`, which the runner relays to the server.