1# Development
2
3## Build and check
4
5```sh
6CGO_ENABLED=0 go build -trimpath -o ermite-runner .
7GOWORK=off go vet ./... && GOWORK=off go test -race ./...
8```
9
10- The binary must be static: the sandbox has no libc of ours.
11- Unit tests need nothing special. The drivers are tested with a fake Firecracker and a fake Docker engine.
12- Protobuf: `go generate ./internal/execstream`. `buf` and `protoc-gen-go` are pinned as `tool` directives in `go.mod`. CI lints the proto and fails when the committed generated code differs.
13- `ermite-runner sandbox-check [script]` runs one stub job in the selected mode. It is the smoke test of an installation.
14
15## Integration tests
16
17They are opt-in. They run the real stub executor. Output and exit codes come back, and nothing survives 10 jobs, including a killed one and a runner crash.
18
19| Variable | Enables |
20| --- | --- |
21| `ERMITE_RUNNER_IT_VM=1` | VM tests. Need `/dev/kvm`, Firecracker, the kernel and the base image |
22| `ERMITE_RUNNER_IT_CONTAINER=1` | Container tests. Need a rootful Docker daemon, root and the OCI image |
23| `ERMITE_RUNNER_IT_E2E=1` | End-to-end job tests in `internal/e2e` |
24| `ERMITE_RUNNER_IT_NET=1` | Packet-level firewall test |
25| `ERMITE_RUNNER_IT_DOCKER_API=1` | Docker Engine calls against any daemon, including rootless |
26| `ERMITE_RUNNER_IT_NODE_DIR` | Directory with `node-v24.21.0-linux-x64.tar.gz`, for offline hosts |
27| `ERMITE_RUNNER_TEST_IMAGE` | Image used by the container and end-to-end tests |
28
29Build the kernel and images first: see [images](images.md).
30
31### VM
32
33```sh
34# Unprivileged, in a user and network namespace. The jailer is off.
35FIRECRACKER=/path/to/firecracker IMAGE_DIR=$PWD/tmp/image scripts/it-vm-userns.sh
36
37# Privileged, with the jailer (the production configuration)
38sudo env GOWORK=off ERMITE_RUNNER_IT_VM=1 ERMITE_RUNNER_ISOLATION=vm \
39  ERMITE_RUNNER_IMAGE=$PWD/tmp/image ERMITE_RUNNER_STATE_DIR=/var/lib/ermite-runner-it \
40  "$(command -v go)" test -count=1 -v ./internal/sandbox/integration -run VM
41```
42
43### Container
44
45```sh
46sudo env GOWORK=off ERMITE_RUNNER_IT_CONTAINER=1 ERMITE_RUNNER_IMAGE=ermite-runner-ubuntu-26.04:latest \
47  DOCKER_HOST=unix:///var/run/docker.sock "$(command -v go)" test -count=1 -v ./internal/sandbox/integration -run Container
48```
49
50### End to end
51
52The real executor runs in the real image, started by the container driver. It also works with a rootless Docker daemon, where `mount` and `umount` are replaced by stand-ins.
53
54```sh
55IMAGE_TAG=ermite-runner-ubuntu-26.04:dev scripts/build-image.sh tmp/image
56ERMITE_RUNNER_IT_E2E=1 ERMITE_RUNNER_TEST_IMAGE=ermite-runner-ubuntu-26.04:dev GOWORK=off go test -race -count=1 ./internal/e2e
57```
58
59- `TestChainCorpus` runs jobs of the ermite-conformance corpus through a fake server and compares the result with recordings made on github.com. It reads `ERMITE_CONFORMANCE_DIR` (default: a sibling checkout of ermite-conformance) and never writes to it.
60
61## Notes
62
63- **Docker API.** Production `container` mode supports Docker Engine and sysbox. Podman's compatibility API is not a drop-in: it cannot disable DNS on the network that the runner creates.
64- **Jailer.** The jailer needs `--chroot-base-dir` to exist. The driver creates `<state dir>/jail`. It leaves `<state dir>/jail/<exec name>` and `/sys/fs/cgroup/firecracker` behind.
65- **Host firewalls.** A firewall that rejects `53` on `ertap*` or `erbr*` breaks guest DNS. The symptom is `dns silent` in `TestVMNetworkPolicy`.