service.md Raw History Folder

Service

contrib/ermite-runner.service is a systemd unit. For the commands to install it, see the README.

Privileges

The runner runs as root on the host. It needs TAP devices, nftables, loop mounts, the jailer and Docker.
Jobs never run as root on the host. They run as user runner in a microVM or a container.

Registration

Register in the same environment as the service, so register and run agree on the URL, name, labels, isolation and state directory.
systemd-run with the same EnvironmentFile does this.
register picks the isolation mode like run will. It sends the name, labels, version, isolation and architecture.

Logs

The runner writes to standard error, which journald collects.
Set ERMITE_RUNNER_LOG_FORMAT=json under journald. Every line about a job has job_id, run_id and attempt.
warn marks what is retried. error marks what is refused or leaked.
ERMITE_RUNNER_LOG_LEVEL=debug adds detail.
Tokens are never logged.

Stopping

systemctl stop sends SIGTERM. The runner:

Cancels running jobs with runner_shutdown.
Reports them to the server.
Removes their sandboxes.

Budget with the defaults:

Step	Time
Executor stops its steps and runs post-steps	30 s
Destroy the sandbox	30 s
Flush the log and report	30 s
Total	90 s

Jobs stop in parallel. The total does not grow with ERMITE_RUNNER_CAPACITY.
The unit sets TimeoutStopSec=120.
A job whose first heartbeat is pending at SIGTERM is not started. The server gives it to another runner after 30 s.
A crash or power cut leaves sandboxes behind. The next start reaps them.

Unit hardening

The unit sets only options that keep it in the host's mount namespace:

NoNewPrivileges
ProtectClock
RestrictSUIDSGID
LockPersonality
RestrictRealtime
SystemCallArchitectures=native
RestrictAddressFamilies (UNIX, INET, INET6, NETLINK, VSOCK)

Do not add:

Options that give the unit its own mount namespace: PrivateTmp, ProtectHome, ProtectSystem, ProtectKernelModules, ProtectKernelLogs, ProtectKernelTunables, ProtectControlGroups, PrivateDevices, ReadOnlyPaths, ReadWritePaths, InaccessiblePaths, TemporaryFileSystem, BindPaths. In container mode the runner loop-mounts the job disk and the Docker daemon, in the host namespace, must see it. A test fails if the unit sets one of them.
PrivateNetwork, RestrictNamespaces, CapabilityBoundingSet, SystemCallFilter: the runner creates network devices, mounts filesystems and hands capabilities to the jailer and Docker.
MemoryDenyWriteExecute: Firecracker maps guest memory writable and executable.
Delegate and UMask: not needed.

The security boundary of a job is its microVM or container, not the unit.

source

1# Service
2
3`contrib/ermite-runner.service` is a systemd unit. For the commands to install it, see the [README](../README.md#register-and-run).
4
5## Privileges
6
7- The runner runs as root on the host. It needs TAP devices, nftables, loop mounts, the jailer and Docker.
8- Jobs never run as root on the host. They run as user `runner` in a microVM or a container.
9
10## Registration
11
12- Register in the same environment as the service, so `register` and `run` agree on the URL, name, labels, isolation and state directory.
13- `systemd-run` with the same `EnvironmentFile` does this.
14- `register` picks the isolation mode like `run` will. It sends the name, labels, version, isolation and architecture.
15
16## Logs
17
18- The runner writes to standard error, which journald collects.
19- Set `ERMITE_RUNNER_LOG_FORMAT=json` under journald. Every line about a job has `job_id`, `run_id` and `attempt`.
20- `warn` marks what is retried. `error` marks what is refused or leaked.
21- `ERMITE_RUNNER_LOG_LEVEL=debug` adds detail.
22- Tokens are never logged.
23
24## Stopping
25
26`systemctl stop` sends SIGTERM. The runner:
27
281. Cancels running jobs with `runner_shutdown`.
292. Reports them to the server.
303. Removes their sandboxes.
31
32Budget with the defaults:
33
34| Step | Time |
35| --- | --- |
36| Executor stops its steps and runs post-steps | 30 s |
37| Destroy the sandbox | 30 s |
38| Flush the log and report | 30 s |
39| **Total** | **90 s** |
40
41- Jobs stop in parallel. The total does not grow with `ERMITE_RUNNER_CAPACITY`.
42- The unit sets `TimeoutStopSec=120`.
43- A job whose first heartbeat is pending at SIGTERM is not started. The server gives it to another runner after 30 s.
44- A crash or power cut leaves sandboxes behind. The next start reaps them.
45
46## Unit hardening
47
48The unit sets only options that keep it in the host's mount namespace:
49
50- `NoNewPrivileges`
51- `ProtectClock`
52- `RestrictSUIDSGID`
53- `LockPersonality`
54- `RestrictRealtime`
55- `SystemCallArchitectures=native`
56- `RestrictAddressFamilies` (UNIX, INET, INET6, NETLINK, VSOCK)
57
58Do not add:
59
60- Options that give the unit its own mount namespace: `PrivateTmp`, `ProtectHome`, `ProtectSystem`, `ProtectKernelModules`, `ProtectKernelLogs`, `ProtectKernelTunables`, `ProtectControlGroups`, `PrivateDevices`, `ReadOnlyPaths`, `ReadWritePaths`, `InaccessiblePaths`, `TemporaryFileSystem`, `BindPaths`. In `container` mode the runner loop-mounts the job disk and the Docker daemon, in the host namespace, must see it. A test fails if the unit sets one of them.
61- `PrivateNetwork`, `RestrictNamespaces`, `CapabilityBoundingSet`, `SystemCallFilter`: the runner creates network devices, mounts filesystems and hands capabilities to the jailer and Docker.
62- `MemoryDenyWriteExecute`: Firecracker maps guest memory writable and executable.
63- `Delegate` and `UMask`: not needed.
64
65The security boundary of a job is its microVM or container, not the unit.

service.md RawHistoryFolder