5801291 docs(readme): document the unprivileged container deployment next to the root host service · ermite-runner

M README.md +17 -4


@@ -25,7 +25,16 @@ ermite-runner fetches jobs from an ermite server and runs workflows written for
25	25
26	26		Details: [docs/sandboxes.md](docs/sandboxes.md).
27	27
28		-	## Requirements
	28	+	## Deployment options
	29	+
	30	+	\| Option \| Runs as \| Isolation \| Needs \|
	31	+	\| --- \| --- \| --- \| --- \|
	32	+	\| Unprivileged container ([docs/container.md](docs/container.md)) \| An unprivileged user (rootless Podman or Docker) \| `vm`, no jailer \| `/dev/kvm`, `/dev/net/tun` and `CAP_NET_ADMIN` in the container \|
	33	+	\| Host service (below) \| root \| `vm` with the jailer, or `container` \| See the requirements below \|
	34	+
	35	+	The container image bundles the runner, Firecracker, the guest kernel and the base image. It is the simplest way to run `vm` mode without root on the host.
	36	+
	37	+	## Requirements (host service)
29	38
30	39		- A Linux host. The runner runs as root.
31	40		- `ip`, `nft` and `mkfs.ext4` (e2fsprogs 1.43 or newer).

@@ -43,7 +52,9 @@ The binary must be static:
43	52		CGO_ENABLED=0 go build -trimpath -o ermite-runner .
44	53		```
45	54
46		-	## Register and run
	55	+	## Register and run (host service)
	56	+
	57	+	In a container, the runner registers itself on its first start instead: see [docs/container.md](docs/container.md#first-start).
47	58
48	59		1. On the ermite server, create a registration token:
49	60

@@ -100,7 +111,7 @@ More on the service, logs and shutdown: [docs/service.md](docs/service.md).
100	111
101	112		\| Variable \| Default \| Meaning \|
102	113		\| --- \| --- \| --- \|
103		-	\| `ERMITE_RUNNER_URL` \| \| Base URL of the ermite server. Must be `https://` \|
	114	+	\| `ERMITE_RUNNER_URL` \| \| Base URL of the ermite server. `https://`, or `http://` with `ERMITE_RUNNER_INSECURE_HTTP=1` \|
104	115		\| `ERMITE_RUNNER_NAME` \| host name \| Runner name \|
105	116		\| `ERMITE_RUNNER_LABELS` \| \| Extra labels, comma separated \|
106	117		\| `ERMITE_RUNNER_ISOLATION` \| `auto` \| `auto`, `vm` or `container` \|

@@ -117,12 +128,14 @@ All settings: [docs/configuration.md](docs/configuration.md).
117	128		- Run only one runner per host.
118	129		- The registration token can read the code and secrets of every repository. Pass it through standard input.
119	130		- Without sysbox, `container` mode uses a privileged container, which is equivalent to host root. Set `ERMITE_RUNNER_DENY_PRIVILEGED=1` to refuse it.
120		-	- The state directory and all its parents must be owned by root and not writable by others.
	131	+	- As a host service, the state directory and all its parents must be owned by root and not writable by others.
	132	+	- In an unprivileged container, the jailer is off: the container and its user namespace confine Firecracker instead, and each job still runs in its own microVM.
121	133
122	134		## Documentation
123	135
124	136		- [Configuration](docs/configuration.md): every environment variable
125	137		- [Sandboxes](docs/sandboxes.md): `vm` and `container` modes, network policy, cleanup
	138	+	- [Unprivileged container](docs/container.md): run the runner in a rootless container
126	139		- [Service](docs/service.md): systemd unit, logs, shutdown
127	140		- [Images](docs/images.md): job image, guest kernel
128	141		- [Workflows](docs/workflows.md): job and service containers, cancellation, annotations

 Details: [docs/sandboxes.md](docs/sandboxes.md).
-## Requirements
+## Deployment options
+| Option | Runs as | Isolation | Needs |
+| --- | --- | --- | --- |
+| Unprivileged container ([docs/container.md](docs/container.md)) | An unprivileged user (rootless Podman or Docker) | `vm`, no jailer | `/dev/kvm`, `/dev/net/tun` and `CAP_NET_ADMIN` in the container |
+| Host service (below) | root | `vm` with the jailer, or `container` | See the requirements below |
+The container image bundles the runner, Firecracker, the guest kernel and the base image. It is the simplest way to run `vm` mode without root on the host.
+## Requirements (host service)
 - A Linux host. The runner runs as root.
 - `ip`, `nft` and `mkfs.ext4` (e2fsprogs 1.43 or newer).
 CGO_ENABLED=0 go build -trimpath -o ermite-runner .
 ```
-## Register and run
+## Register and run (host service)
+In a container, the runner registers itself on its first start instead: see [docs/container.md](docs/container.md#first-start).
 1. On the ermite server, create a registration token:
 | Variable | Default | Meaning |
 | --- | --- | --- |
-| `ERMITE_RUNNER_URL` | | Base URL of the ermite server. Must be `https://` |
+| `ERMITE_RUNNER_URL` | | Base URL of the ermite server. `https://`, or `http://` with `ERMITE_RUNNER_INSECURE_HTTP=1` |
 | `ERMITE_RUNNER_NAME` | host name | Runner name |
 | `ERMITE_RUNNER_LABELS` | | Extra labels, comma separated |
 | `ERMITE_RUNNER_ISOLATION` | `auto` | `auto`, `vm` or `container` |
 - Run only one runner per host.
 - The registration token can read the code and secrets of every repository. Pass it through standard input.
 - Without sysbox, `container` mode uses a privileged container, which is equivalent to host root. Set `ERMITE_RUNNER_DENY_PRIVILEGED=1` to refuse it.
-- The state directory and all its parents must be owned by root and not writable by others.
+- As a host service, the state directory and all its parents must be owned by root and not writable by others.
+- In an unprivileged container, the jailer is off: the container and its user namespace confine Firecracker instead, and each job still runs in its own microVM.
 ## Documentation
 - [Configuration](docs/configuration.md): every environment variable
 - [Sandboxes](docs/sandboxes.md): `vm` and `container` modes, network policy, cleanup
+- [Unprivileged container](docs/container.md): run the runner in a rootless container
 - [Service](docs/service.md): systemd unit, logs, shutdown
 - [Images](docs/images.md): job image, guest kernel
 - [Workflows](docs/workflows.md): job and service containers, cancellation, annotations

M docs/configuration.md +2 -2


@@ -50,7 +50,7 @@ The token has the scope `runner:register`. It can read the code and the secrets
50	50		\| `ERMITE_RUNNER_FIRECRACKER` \| from `PATH` \| Firecracker binary \|
51	51		\| `ERMITE_RUNNER_JAILER` \| from `PATH` \| Jailer binary \|
52	52		\| `ERMITE_RUNNER_KERNEL` \| `<image dir>/vmlinux` (`Image` on arm64) \| Guest kernel \|
53		-	\| `ERMITE_RUNNER_JAILER_MODE` \| `on` \| `off` only for development and tests. It removes the chroot, uid and namespace confinement of the VMM \|
	53	+	\| `ERMITE_RUNNER_JAILER_MODE` \| `on` \| `off` removes the chroot, uid and namespace confinement of the VMM. Use it only when an outer boundary replaces it, as in the [unprivileged container](container.md), or for development and tests \|
54	54		\| `ERMITE_RUNNER_LOCK_FILE` \| `/run/ermite-runner.lock` \| Single-instance lock. Change it for tests only \|
55	55
56	56		`DOCKER_HOST` is honoured in `container` mode.

@@ -79,7 +79,7 @@ The token has the scope `runner:register`. It can read the code and the secrets
79	79		\| `ERMITE_RUNNER_ACTIONS_CACHE_MAX` \| `2G` \| Size cap of the action archive cache. `0` turns it off \|
80	80		\| `ERMITE_RUNNER_ARTIFACTS` \| `1` \| `0` turns artifacts v4 off. Artifacts need the cache service \|
81	81		\| `ERMITE_RUNNER_NODE` \| `1` \| `0` turns the Node.js externals off. See [externals](externals.md) \|
82		-	\| `ERMITE_RUNNER_GUEST_FORGE` \| `0` \| `1` lets sandboxes reach the server host and port. Staging only. See [sandboxes](sandboxes.md#guest-forge-access) \|
	82	+	\| `ERMITE_RUNNER_GUEST_FORGE` \| `0` \| `1` lets sandboxes reach the server host and port. Read the security notes first. See [sandboxes](sandboxes.md#guest-forge-access) \|
83	83		\| `ERMITE_RUNNER_GUEST_FORGE_TARGET` \| \| `host:port` the runner relays guest forge traffic to. Empty: no relay, a service must listen on the gateway port \|
84	84
85	85		## Logging


@@ -50,7 +50,7 @@ The token has the scope `runner:register`. It can read the code and the secrets
50		\| `ERMITE_RUNNER_FIRECRACKER` \| from `PATH` \| Firecracker binary \|	50		\| `ERMITE_RUNNER_FIRECRACKER` \| from `PATH` \| Firecracker binary \|
51		\| `ERMITE_RUNNER_JAILER` \| from `PATH` \| Jailer binary \|	51		\| `ERMITE_RUNNER_JAILER` \| from `PATH` \| Jailer binary \|
52		\| `ERMITE_RUNNER_KERNEL` \| `<image dir>/vmlinux` (`Image` on arm64) \| Guest kernel \|	52		\| `ERMITE_RUNNER_KERNEL` \| `<image dir>/vmlinux` (`Image` on arm64) \| Guest kernel \|
53	-	\| `ERMITE_RUNNER_JAILER_MODE` \| `on` \| `off` only for development and tests. It removes the chroot, uid and namespace confinement of the VMM \|	53	+	\| `ERMITE_RUNNER_JAILER_MODE` \| `on` \| `off` removes the chroot, uid and namespace confinement of the VMM. Use it only when an outer boundary replaces it, as in the [unprivileged container](container.md), or for development and tests \|
54		\| `ERMITE_RUNNER_LOCK_FILE` \| `/run/ermite-runner.lock` \| Single-instance lock. Change it for tests only \|	54		\| `ERMITE_RUNNER_LOCK_FILE` \| `/run/ermite-runner.lock` \| Single-instance lock. Change it for tests only \|
55			55
56		`DOCKER_HOST` is honoured in `container` mode.	56		`DOCKER_HOST` is honoured in `container` mode.

@@ -79,7 +79,7 @@ The token has the scope `runner:register`. It can read the code and the secrets
79		\| `ERMITE_RUNNER_ACTIONS_CACHE_MAX` \| `2G` \| Size cap of the action archive cache. `0` turns it off \|	79		\| `ERMITE_RUNNER_ACTIONS_CACHE_MAX` \| `2G` \| Size cap of the action archive cache. `0` turns it off \|
80		\| `ERMITE_RUNNER_ARTIFACTS` \| `1` \| `0` turns artifacts v4 off. Artifacts need the cache service \|	80		\| `ERMITE_RUNNER_ARTIFACTS` \| `1` \| `0` turns artifacts v4 off. Artifacts need the cache service \|
81		\| `ERMITE_RUNNER_NODE` \| `1` \| `0` turns the Node.js externals off. See [externals](externals.md) \|	81		\| `ERMITE_RUNNER_NODE` \| `1` \| `0` turns the Node.js externals off. See [externals](externals.md) \|
82	-	\| `ERMITE_RUNNER_GUEST_FORGE` \| `0` \| `1` lets sandboxes reach the server host and port. Staging only. See [sandboxes](sandboxes.md#guest-forge-access) \|	82	+	\| `ERMITE_RUNNER_GUEST_FORGE` \| `0` \| `1` lets sandboxes reach the server host and port. Read the security notes first. See [sandboxes](sandboxes.md#guest-forge-access) \|
83		\| `ERMITE_RUNNER_GUEST_FORGE_TARGET` \| \| `host:port` the runner relays guest forge traffic to. Empty: no relay, a service must listen on the gateway port \|	83		\| `ERMITE_RUNNER_GUEST_FORGE_TARGET` \| \| `host:port` the runner relays guest forge traffic to. Empty: no relay, a service must listen on the gateway port \|
84			84
85		## Logging	85		## Logging

M docs/sandboxes.md +2 -2


@@ -7,7 +7,7 @@ Inside the sandbox, `ermite-runner init` is the entrypoint (PID 1 in a VM). `erm
7	7		## vm
8	8
9	9		- Firecracker runs the job in a microVM. The VM is destroyed when the job ends.
10		-	- Needs root, or `CAP_NET_ADMIN` and the rights the jailer requires.
	10	+	- With the jailer (default), needs root. Without it (`ERMITE_RUNNER_JAILER_MODE=off`), `CAP_NET_ADMIN`, `/dev/kvm` and `/dev/net/tun` are enough, as in the [unprivileged container](container.md).
11	11		- Disks:
12	12		- The runner executable is the read-only boot root.
13	13		- The shared base image is read-only.

@@ -88,7 +88,7 @@ Host side effects:
88	88
89	89		## Guest forge access
90	90
91		-	Off by default. Use it on staging hosts only.
	91	+	Off by default. Without a relay target, jobs reach whatever listens on that gateway port, so keep it to hosts where that is acceptable. With `ERMITE_RUNNER_GUEST_FORGE_TARGET`, jobs reach only the relayed target.
92	92
93	93		`ERMITE_RUNNER_GUEST_FORGE=1` lets sandboxes reach the server the way the runner does. It is for hosts where the server is reachable only through a local route, such as an `/etc/hosts` entry plus a reverse proxy on a local port. Jobs cannot otherwise run `git clone` against `github.server_url`.
94	94


@@ -7,7 +7,7 @@ Inside the sandbox, `ermite-runner init` is the entrypoint (PID 1 in a VM). `erm
7		## vm	7		## vm
8			8
9		- Firecracker runs the job in a microVM. The VM is destroyed when the job ends.	9		- Firecracker runs the job in a microVM. The VM is destroyed when the job ends.
10	-	- Needs root, or `CAP_NET_ADMIN` and the rights the jailer requires.	10	+	- With the jailer (default), needs root. Without it (`ERMITE_RUNNER_JAILER_MODE=off`), `CAP_NET_ADMIN`, `/dev/kvm` and `/dev/net/tun` are enough, as in the [unprivileged container](container.md).
11		- Disks:	11		- Disks:
12		- The runner executable is the read-only boot root.	12		- The runner executable is the read-only boot root.
13		- The shared base image is read-only.	13		- The shared base image is read-only.

@@ -88,7 +88,7 @@ Host side effects:
88			88
89		## Guest forge access	89		## Guest forge access
90			90
91	-	Off by default. Use it on staging hosts only.	91	+	Off by default. Without a relay target, jobs reach whatever listens on that gateway port, so keep it to hosts where that is acceptable. With `ERMITE_RUNNER_GUEST_FORGE_TARGET`, jobs reach only the relayed target.
92			92
93		`ERMITE_RUNNER_GUEST_FORGE=1` lets sandboxes reach the server the way the runner does. It is for hosts where the server is reachable only through a local route, such as an `/etc/hosts` entry plus a reverse proxy on a local port. Jobs cannot otherwise run `git clone` against `github.server_url`.	93		`ERMITE_RUNNER_GUEST_FORGE=1` lets sandboxes reach the server the way the runner does. It is for hosts where the server is reachable only through a local route, such as an `/etc/hosts` entry plus a reverse proxy on a local port. Jobs cannot otherwise run `git clone` against `github.server_url`.
94			94

M docs/service.md +2 -1


@@ -4,7 +4,8 @@
4	4
5	5		## Privileges
6	6
7		-	- The runner runs as root on the host. It needs TAP devices, nftables, loop mounts, the jailer and Docker.
	7	+	- As a host service, the runner runs as root. It needs TAP devices, nftables, loop mounts, the jailer and Docker.
	8	+	- To run without root on the host, use the [unprivileged container](container.md) instead.
8	9		- Jobs never run as root on the host. They run as user `runner` in a microVM or a container.
9	10
10	11		## Registration


@@ -4,7 +4,8 @@
4			4
5		## Privileges	5		## Privileges
6			6
7	-	- The runner runs as root on the host. It needs TAP devices, nftables, loop mounts, the jailer and Docker.	7	+	- As a host service, the runner runs as root. It needs TAP devices, nftables, loop mounts, the jailer and Docker.
			8	+	- To run without root on the host, use the [unprivileged container](container.md) instead.
8		- Jobs never run as root on the host. They run as user `runner` in a microVM or a container.	9		- Jobs never run as root on the host. They run as user `runner` in a microVM or a container.
9			10
10		## Registration	11		## Registration