feat(apps): add allowlisted CDI devices for app services (#285)
* feat(apps): add allowlisted CDI devices for app services Manifests declare logical devices via devices = [...]; the installation maps each name to explicit CDI IDs plus exact app/service allowlists under [app_devices.<name>]. Resolution happens at activation time and encodes one native CDI DeviceRequest; revisions persist logical names only. Revoked grants and unsupported engines fail closed before any workload mutation. * refactor(apps): extract helpers to satisfy gocyclo budget Split device diff entry, preflight error tail, and reload policy validation out of diffService, isMappedPreflightError, and applyLoadedConfig. * fix(apps): address device review findings Probe engine CDI capability in preflight via a new ContainerRuntime.SupportsCDIDevices boundary method so unsupported engines fail before workload mutation. Preserve ErrRuntimeUnsupported through createContainer redaction and map it to the structured runtime-unsupported envelope. Stop echoing resolved CDI IDs in duplicate-grant errors and run union resolution at apply time. * fix(apps): harden device error sanitization Hoist the runtime-unsupported sentinel above the bind redaction branch so mixed bind+device creates keep the structured envelope. Sanitize the preflight engine probe cause so daemon endpoints never reach journals or API responses. * fix(apps): harden device policy, engine gate, and probe errors Address review findings on the CDI device work: - docs: move `devices` above the first nested service table so TOML assigns it to the service, not to the last bind entry. - domain: keep the ErrInvalidAppSpec chain through device policy allowlist validation, and reject CDI device names that are host paths or raw device nodes instead of valid CDI names. - docker: parse engine major/minor strictly, so prerelease suffixes such as "-rc1" fail closed instead of clearing the version gate; keep cancellation and deadlines from the /version probe while dropping the cause text that embeds the daemon endpoint. - deployment: preserve cancellation and deadlines from the capability probe and collapse every other cause to ErrRuntimeUnsupported alone. - appstate: load the legacy no-devices record from a verbatim fixture instead of writing through the current serializer.
- commit
- d69854dcb754cccf93590bb70635e02071fca86d
- author
- bnema · · 2026-09-14 16:18 UTC
- committer
- GitHub · · 2026-09-14 16:18 UTC
- parent
- 9ee3e24
- signature
- GPG Unverified
B5690EEEBB952194The key is unknown
32 files · +1990 -16
M docs/config/apps.md +3 -0
| @@ -31,6 +31,7 @@ name = "web" | |||
| 31 | 31 | image = "gordon.mydomain.com/blog:1.4.2" | |
| 32 | 32 | command = ["node", "server.js"] # optional override | |
| 33 | 33 | stop_grace = "30s" # optional, default 30s | |
| 34 | + | devices = ["transcode-gpu"] # 0..n logical names from [app_devices.<name>] | |
| 34 | 35 | ||
| 35 | 36 | [service.readiness] # optional explicit readiness | |
| 36 | 37 | type = "http" | |
| @@ -79,6 +80,7 @@ services = ["web"] | |||
| 79 | 80 | - App name: DNS label (lowercase alphanumerics and hyphens, max 63), must not contain `--`, reserved: `gordon`, `registry`, `admin`, `localhost`. Case-insensitive uniqueness. | |
| 80 | 81 | - Service name: `[a-z0-9_.-]`, max 63, unique within the app. | |
| 81 | 82 | - Bind name: `[a-z0-9_.-]`, max 63, must not contain `--`, unique within its service. | |
| 83 | + | - Device name: `[a-z0-9_.-]`, max 63, must not contain `--`, unique within its service. | |
| 82 | 84 | - Removing an app ends its incarnation: the name is freed, volumes and secrets are archived as retained under the old internal UUID, desired/active/intent state is cleared, and the next apply allocates a new UUID. A new app reusing the name never adopts the old secrets or volumes. | |
| 83 | 85 | ||
| 84 | 86 | ## Services | |
| @@ -115,6 +117,7 @@ Image registry names and digest syntax are validated during manifest apply, reso | |||
| 115 | 117 | - Manifests never carry host paths. A `[[service.bind]]` references an `[app_mounts.<name>]` policy the operator declares in `gordon.toml`; a bind whose name has no matching policy is rejected at apply time. See [Volumes](./volumes.md) and [Security Hardening](./security-hardening.md). | |
| 116 | 118 | - Named volumes are Gordon-owned app data: created, labeled, retained, backed up, and pruned by Gordon. Administrative binds are operator-owned host locations: Gordon mounts them and never creates, deletes, owns, backs up, or prunes them. | |
| 117 | 119 | - `[[service.bind]]` requires an absolute, normalized container `path` and an optional `readonly`. The policy's `read_only` and the bind's `readonly` force read-only together: either side wins and a manifest can never weaken its policy. Reserved destinations (`/`, `/proc`, `/sys`, `/dev`, `/boot` and their children) and paths colliding with a declared volume or another bind are rejected at apply time. | |
| 120 | + | - `devices` lists logical device names granted by `[app_devices.<name>]` policy the operator declares in `gordon.toml`; a device whose name has no matching policy, or whose policy does not allow the app+service pair, is rejected at apply time. Gordon resolves each name to explicit CDI IDs at activation time and encodes them as one native CDI `DeviceRequest`. Revisions persist the logical names, never the host resolution. A device add/remove shows as `service/<name>/devices` in diffs; reorder-only input is a no-op. Changing a mapping never recreates a running container: the next deploy serves the new resolution. Revoking a grant fails subsequent deploys closed while the running service is untouched. Device-bearing creates require Podman 5.4+ or Docker 28.3+ with native CDI configured; older or unrecognized engines return a structured `runtime-unsupported` error and never run without devices. Gordon installs no drivers, manages no quotas, and injects no `NVIDIA_*` environment: images carry their own runtime expectations. | |
| 118 | 121 | - A service with any bind runs as a single writer, like a volume-backed service: replacements never serve two generations at once. | |
| 119 | 122 | - A volume declared `readonly = true` is mounted read-only in the container; the service cannot modify protected data. | |
| 120 | 123 | - `[[service.database]]` declares databases explicitly (no image inference). Only PostgreSQL is supported, and each database declares its own backup `schedule` (`hourly`, `daily`, `weekly`, or `monthly`). | |
| @@ -31,6 +31,7 @@ name = "web" | |||||
| 31 | image = "gordon.mydomain.com/blog:1.4.2" | 31 | image = "gordon.mydomain.com/blog:1.4.2" | ||
| 32 | command = ["node", "server.js"] # optional override | 32 | command = ["node", "server.js"] # optional override | ||
| 33 | stop_grace = "30s" # optional, default 30s | 33 | stop_grace = "30s" # optional, default 30s | ||
| 34 | + | devices = ["transcode-gpu"] # 0..n logical names from [app_devices.<name>] | |||
| 34 | 35 | ||||
| 35 | [service.readiness] # optional explicit readiness | 36 | [service.readiness] # optional explicit readiness | ||
| 36 | type = "http" | 37 | type = "http" | ||
| @@ -79,6 +80,7 @@ services = ["web"] | |||||
| 79 | - App name: DNS label (lowercase alphanumerics and hyphens, max 63), must not contain `--`, reserved: `gordon`, `registry`, `admin`, `localhost`. Case-insensitive uniqueness. | 80 | - App name: DNS label (lowercase alphanumerics and hyphens, max 63), must not contain `--`, reserved: `gordon`, `registry`, `admin`, `localhost`. Case-insensitive uniqueness. | ||
| 80 | - Service name: `[a-z0-9_.-]`, max 63, unique within the app. | 81 | - Service name: `[a-z0-9_.-]`, max 63, unique within the app. | ||
| 81 | - Bind name: `[a-z0-9_.-]`, max 63, must not contain `--`, unique within its service. | 82 | - Bind name: `[a-z0-9_.-]`, max 63, must not contain `--`, unique within its service. | ||
| 83 | + | - Device name: `[a-z0-9_.-]`, max 63, must not contain `--`, unique within its service. | |||
| 82 | - Removing an app ends its incarnation: the name is freed, volumes and secrets are archived as retained under the old internal UUID, desired/active/intent state is cleared, and the next apply allocates a new UUID. A new app reusing the name never adopts the old secrets or volumes. | 84 | - Removing an app ends its incarnation: the name is freed, volumes and secrets are archived as retained under the old internal UUID, desired/active/intent state is cleared, and the next apply allocates a new UUID. A new app reusing the name never adopts the old secrets or volumes. | ||
| 83 | 85 | ||||
| 84 | ## Services | 86 | ## Services | ||
| @@ -115,6 +117,7 @@ Image registry names and digest syntax are validated during manifest apply, reso | |||||
| 115 | - Manifests never carry host paths. A `[[service.bind]]` references an `[app_mounts.<name>]` policy the operator declares in `gordon.toml`; a bind whose name has no matching policy is rejected at apply time. See [Volumes](./volumes.md) and [Security Hardening](./security-hardening.md). | 117 | - Manifests never carry host paths. A `[[service.bind]]` references an `[app_mounts.<name>]` policy the operator declares in `gordon.toml`; a bind whose name has no matching policy is rejected at apply time. See [Volumes](./volumes.md) and [Security Hardening](./security-hardening.md). | ||
| 116 | - Named volumes are Gordon-owned app data: created, labeled, retained, backed up, and pruned by Gordon. Administrative binds are operator-owned host locations: Gordon mounts them and never creates, deletes, owns, backs up, or prunes them. | 118 | - Named volumes are Gordon-owned app data: created, labeled, retained, backed up, and pruned by Gordon. Administrative binds are operator-owned host locations: Gordon mounts them and never creates, deletes, owns, backs up, or prunes them. | ||
| 117 | - `[[service.bind]]` requires an absolute, normalized container `path` and an optional `readonly`. The policy's `read_only` and the bind's `readonly` force read-only together: either side wins and a manifest can never weaken its policy. Reserved destinations (`/`, `/proc`, `/sys`, `/dev`, `/boot` and their children) and paths colliding with a declared volume or another bind are rejected at apply time. | 119 | - `[[service.bind]]` requires an absolute, normalized container `path` and an optional `readonly`. The policy's `read_only` and the bind's `readonly` force read-only together: either side wins and a manifest can never weaken its policy. Reserved destinations (`/`, `/proc`, `/sys`, `/dev`, `/boot` and their children) and paths colliding with a declared volume or another bind are rejected at apply time. | ||
| 120 | + | - `devices` lists logical device names granted by `[app_devices.<name>]` policy the operator declares in `gordon.toml`; a device whose name has no matching policy, or whose policy does not allow the app+service pair, is rejected at apply time. Gordon resolves each name to explicit CDI IDs at activation time and encodes them as one native CDI `DeviceRequest`. Revisions persist the logical names, never the host resolution. A device add/remove shows as `service/<name>/devices` in diffs; reorder-only input is a no-op. Changing a mapping never recreates a running container: the next deploy serves the new resolution. Revoking a grant fails subsequent deploys closed while the running service is untouched. Device-bearing creates require Podman 5.4+ or Docker 28.3+ with native CDI configured; older or unrecognized engines return a structured `runtime-unsupported` error and never run without devices. Gordon installs no drivers, manages no quotas, and injects no `NVIDIA_*` environment: images carry their own runtime expectations. | |||
| 118 | - A service with any bind runs as a single writer, like a volume-backed service: replacements never serve two generations at once. | 121 | - A service with any bind runs as a single writer, like a volume-backed service: replacements never serve two generations at once. | ||
| 119 | - A volume declared `readonly = true` is mounted read-only in the container; the service cannot modify protected data. | 122 | - A volume declared `readonly = true` is mounted read-only in the container; the service cannot modify protected data. | ||
| 120 | - `[[service.database]]` declares databases explicitly (no image inference). Only PostgreSQL is supported, and each database declares its own backup `schedule` (`hourly`, `daily`, `weekly`, or `monthly`). | 123 | - `[[service.database]]` declares databases explicitly (no image inference). Only PostgreSQL is supported, and each database declares its own backup `schedule` (`hourly`, `daily`, `weekly`, or `monthly`). | ||
M docs/config/reference.md +17 -0
| @@ -153,6 +153,20 @@ preserve = true # Keep volumes when containers are | |||
| 153 | 153 | # allowed_services = ["web"] # Required; exact, non-empty | |
| 154 | 154 | # root = "/srv/gordon" # Optional; default: parent of source | |
| 155 | 155 | ||
| 156 | + | # ============================================================================= | |
| 157 | + | # ADMINISTRATIVE APP DEVICES (CDI) | |
| 158 | + | # ============================================================================= | |
| 159 | + | # A named device grant is the only way an app manifest may request host | |
| 160 | + | # devices. Manifests declare devices = ["<name>"]; raw /dev paths are | |
| 161 | + | # always rejected, and the policy's allowlists are exact and non-empty. | |
| 162 | + | # Aggregate "=all" CDI selectors are rejected: name every device | |
| 163 | + | # explicitly. Device-bearing creates require Podman 5.4+ or Docker 28.3+ | |
| 164 | + | # with native CDI configured. | |
| 165 | + | # [app_devices.transcode-gpu] | |
| 166 | + | # cdi = ["example.com/gpu=GPU-device-uuid"] # Required; explicit, non-empty | |
| 167 | + | # allowed_apps = ["video"] # Required; exact, non-empty | |
| 168 | + | # allowed_services = ["transcoder"] # Required; exact, non-empty | |
| 169 | + | ||
| 156 | 170 | # REMOVED in v2.50: [routes], [attachments], [network_groups], | |
| 157 | 171 | # [[services]]-as-apps, [service_routes], [auto_route], [previews]. | |
| 158 | 172 | # Declare apps in standalone files (see ./apps.md). | |
| @@ -287,6 +301,9 @@ keep_last = 3 # Keep N newest tags per repository | |||
| 287 | 301 | | `app_mounts.<name>.allowed_services` | none | Required exact, non-empty service allowlist for this mount | | |
| 288 | 302 | | `app_mounts.<name>.read_only` | `false` | Force every bind resolved under this policy read-only; a manifest bind can never weaken it | | |
| 289 | 303 | | `app_mounts.<name>.root` | parent of `source` | Optional administrative boundary the resolved source must stay under | | |
| 304 | + | | `app_devices.<name>.cdi` | none | Required explicit, non-empty CDI device IDs granted under this logical name (no `=all` aggregate form) | | |
| 305 | + | | `app_devices.<name>.allowed_apps` | none | Required exact, non-empty app allowlist for this grant (no wildcard or empty-means-all form) | | |
| 306 | + | | `app_devices.<name>.allowed_services` | none | Required exact, non-empty service allowlist for this grant | | |
| 290 | 307 | | `services[].name` | none | Standalone service name used by `service:<service>:<port-name>` traffic refs | | |
| 291 | 308 | | `services[].image` | none | Container image for enabled standalone services | | |
| 292 | 309 | | `services[].enabled` | `false` | Whether Gordon creates, starts, and reconciles the service container | | |
| @@ -153,6 +153,20 @@ preserve = true # Keep volumes when containers are | |||||
| 153 | # allowed_services = ["web"] # Required; exact, non-empty | 153 | # allowed_services = ["web"] # Required; exact, non-empty | ||
| 154 | # root = "/srv/gordon" # Optional; default: parent of source | 154 | # root = "/srv/gordon" # Optional; default: parent of source | ||
| 155 | 155 | ||||
| 156 | + | # ============================================================================= | |||
| 157 | + | # ADMINISTRATIVE APP DEVICES (CDI) | |||
| 158 | + | # ============================================================================= | |||
| 159 | + | # A named device grant is the only way an app manifest may request host | |||
| 160 | + | # devices. Manifests declare devices = ["<name>"]; raw /dev paths are | |||
| 161 | + | # always rejected, and the policy's allowlists are exact and non-empty. | |||
| 162 | + | # Aggregate "=all" CDI selectors are rejected: name every device | |||
| 163 | + | # explicitly. Device-bearing creates require Podman 5.4+ or Docker 28.3+ | |||
| 164 | + | # with native CDI configured. | |||
| 165 | + | # [app_devices.transcode-gpu] | |||
| 166 | + | # cdi = ["example.com/gpu=GPU-device-uuid"] # Required; explicit, non-empty | |||
| 167 | + | # allowed_apps = ["video"] # Required; exact, non-empty | |||
| 168 | + | # allowed_services = ["transcoder"] # Required; exact, non-empty | |||
| 169 | + | ||||
| 156 | # REMOVED in v2.50: [routes], [attachments], [network_groups], | 170 | # REMOVED in v2.50: [routes], [attachments], [network_groups], | ||
| 157 | # [[services]]-as-apps, [service_routes], [auto_route], [previews]. | 171 | # [[services]]-as-apps, [service_routes], [auto_route], [previews]. | ||
| 158 | # Declare apps in standalone files (see ./apps.md). | 172 | # Declare apps in standalone files (see ./apps.md). | ||
| @@ -287,6 +301,9 @@ keep_last = 3 # Keep N newest tags per repository | |||||
| 287 | | `app_mounts.<name>.allowed_services` | none | Required exact, non-empty service allowlist for this mount | | 301 | | `app_mounts.<name>.allowed_services` | none | Required exact, non-empty service allowlist for this mount | | ||
| 288 | | `app_mounts.<name>.read_only` | `false` | Force every bind resolved under this policy read-only; a manifest bind can never weaken it | | 302 | | `app_mounts.<name>.read_only` | `false` | Force every bind resolved under this policy read-only; a manifest bind can never weaken it | | ||
| 289 | | `app_mounts.<name>.root` | parent of `source` | Optional administrative boundary the resolved source must stay under | | 303 | | `app_mounts.<name>.root` | parent of `source` | Optional administrative boundary the resolved source must stay under | | ||
| 304 | + | | `app_devices.<name>.cdi` | none | Required explicit, non-empty CDI device IDs granted under this logical name (no `=all` aggregate form) | | |||
| 305 | + | | `app_devices.<name>.allowed_apps` | none | Required exact, non-empty app allowlist for this grant (no wildcard or empty-means-all form) | | |||
| 306 | + | | `app_devices.<name>.allowed_services` | none | Required exact, non-empty service allowlist for this grant | | |||
| 290 | | `services[].name` | none | Standalone service name used by `service:<service>:<port-name>` traffic refs | | 307 | | `services[].name` | none | Standalone service name used by `service:<service>:<port-name>` traffic refs | | ||
| 291 | | `services[].image` | none | Container image for enabled standalone services | | 308 | | `services[].image` | none | Container image for enabled standalone services | | ||
| 292 | | `services[].enabled` | `false` | Whether Gordon creates, starts, and reconciles the service container | | 309 | | `services[].enabled` | `false` | Whether Gordon creates, starts, and reconciles the service container | | ||
M docs/config/security-hardening.md +20 -0
| @@ -63,6 +63,26 @@ The manifest references only the policy name: `[[service.bind]] name = "app-logs | |||
| 63 | 63 | - `gordon serve` reload republishes validated policies atomically. An edit that fails validation is rejected and the previous policies stay live. | |
| 64 | 64 | - Gordon never creates, deletes, chowns, backs up, or prunes the host path; ownership, permissions, and backup remain the operator's responsibility. | |
| 65 | 65 | ||
| 66 | + | ## Administrative app devices (CDI) | |
| 67 | + | ||
| 68 | + | App manifests cannot name host devices. A device grant exists only when the operator declares a named policy in `gordon.toml`: | |
| 69 | + | ||
| 70 | + | ```toml | |
| 71 | + | [app_devices.transcode-gpu] | |
| 72 | + | cdi = ["example.com/gpu=GPU-device-uuid"] | |
| 73 | + | allowed_apps = ["video"] # exact, non-empty | |
| 74 | + | allowed_services = ["transcoder"] # exact, non-empty | |
| 75 | + | ``` | |
| 76 | + | ||
| 77 | + | The manifest references only the logical name: `devices = ["transcode-gpu"]`. | |
| 78 | + | ||
| 79 | + | - `cdi` holds explicit CDI device IDs. Raw `/dev` paths, unqualified names, and aggregate `=all` selectors are rejected. | |
| 80 | + | - `allowed_apps` and `allowed_services` are exact, non-empty allowlists, so least privilege is enforced by construction. There is no wildcard, prefix, or empty-means-all form. | |
| 81 | + | - Gordon resolves logical names to CDI IDs at activation time and encodes them as one native CDI `DeviceRequest`. Revisions persist the logical names, never the host resolution. | |
| 82 | + | - The grant is re-resolved immediately before every preflight, container create, restart, and recovery. Revoking a grant blocks future deploys while the running service is untouched; changing a mapping never recreates a running container. | |
| 83 | + | - Device-bearing creates require Podman 5.4+ or Docker 28.3+ with native CDI configured. Older or unrecognized engines fail with a structured `runtime-unsupported` error and never run without devices. | |
| 84 | + | - Gordon installs no drivers, manages no quotas, and injects no device environment: images carry their own runtime expectations. | |
| 85 | + | ||
| 66 | 86 | ## Pass import plaintext handling | |
| 67 | 87 | ||
| 68 | 88 | With the `pass` backend, Gordon imports eligible plaintext `.env` files at startup. It removes a source file only after every entry is stored successfully. If a destination entry already exists or an import fails, Gordon fails closed and leaves the plaintext source in place for operator review. | |
| @@ -63,6 +63,26 @@ The manifest references only the policy name: `[[service.bind]] name = "app-logs | |||||
| 63 | - `gordon serve` reload republishes validated policies atomically. An edit that fails validation is rejected and the previous policies stay live. | 63 | - `gordon serve` reload republishes validated policies atomically. An edit that fails validation is rejected and the previous policies stay live. | ||
| 64 | - Gordon never creates, deletes, chowns, backs up, or prunes the host path; ownership, permissions, and backup remain the operator's responsibility. | 64 | - Gordon never creates, deletes, chowns, backs up, or prunes the host path; ownership, permissions, and backup remain the operator's responsibility. | ||
| 65 | 65 | ||||
| 66 | + | ## Administrative app devices (CDI) | |||
| 67 | + | ||||
| 68 | + | App manifests cannot name host devices. A device grant exists only when the operator declares a named policy in `gordon.toml`: | |||
| 69 | + | ||||
| 70 | + | ```toml | |||
| 71 | + | [app_devices.transcode-gpu] | |||
| 72 | + | cdi = ["example.com/gpu=GPU-device-uuid"] | |||
| 73 | + | allowed_apps = ["video"] # exact, non-empty | |||
| 74 | + | allowed_services = ["transcoder"] # exact, non-empty | |||
| 75 | + | ``` | |||
| 76 | + | ||||
| 77 | + | The manifest references only the logical name: `devices = ["transcode-gpu"]`. | |||
| 78 | + | ||||
| 79 | + | - `cdi` holds explicit CDI device IDs. Raw `/dev` paths, unqualified names, and aggregate `=all` selectors are rejected. | |||
| 80 | + | - `allowed_apps` and `allowed_services` are exact, non-empty allowlists, so least privilege is enforced by construction. There is no wildcard, prefix, or empty-means-all form. | |||
| 81 | + | - Gordon resolves logical names to CDI IDs at activation time and encodes them as one native CDI `DeviceRequest`. Revisions persist the logical names, never the host resolution. | |||
| 82 | + | - The grant is re-resolved immediately before every preflight, container create, restart, and recovery. Revoking a grant blocks future deploys while the running service is untouched; changing a mapping never recreates a running container. | |||
| 83 | + | - Device-bearing creates require Podman 5.4+ or Docker 28.3+ with native CDI configured. Older or unrecognized engines fail with a structured `runtime-unsupported` error and never run without devices. | |||
| 84 | + | - Gordon installs no drivers, manages no quotas, and injects no device environment: images carry their own runtime expectations. | |||
| 85 | + | ||||
| 66 | ## Pass import plaintext handling | 86 | ## Pass import plaintext handling | ||
| 67 | 87 | ||||
| 68 | With the `pass` backend, Gordon imports eligible plaintext `.env` files at startup. It removes a source file only after every entry is stored successfully. If a destination entry already exists or an import fails, Gordon fails closed and leaves the plaintext source in place for operator review. | 88 | With the `pass` backend, Gordon imports eligible plaintext `.env` files at startup. It removes a source file only after every entry is stored successfully. If a destination entry already exists or an import fails, Gordon fails closed and leaves the plaintext source in place for operator review. | ||
M gordon.toml.example +20 -0
| @@ -122,6 +122,26 @@ syslog_identifier = "gordon-access" | |||
| 122 | 122 | # allowed_services = ["web"] # Required; exact, non-empty | |
| 123 | 123 | # root = "/srv/gordon" # Optional; defaults to the source's parent | |
| 124 | 124 | ||
| 125 | + | # ============================================================================= | |
| 126 | + | # ADMINISTRATIVE APP DEVICES (CDI) | |
| 127 | + | # ============================================================================= | |
| 128 | + | # A named device grant is the only way an app manifest may request host | |
| 129 | + | # devices. The manifest declares devices = ["<name>"]; raw /dev paths, | |
| 130 | + | # privileged containers, and runtime-specific GPU flags are never used. | |
| 131 | + | # Gordon resolves each name to explicit CDI IDs at activation time and | |
| 132 | + | # encodes them as one native CDI DeviceRequest. allowed_apps and | |
| 133 | + | # allowed_services are exact and non-empty: only that app+service pair | |
| 134 | + | # receives the grant. Aggregate "=all" selectors are rejected: name every | |
| 135 | + | # device explicitly. Changing a mapping never recreates a running | |
| 136 | + | # container; the next deploy serves the new resolution. Revoking a grant | |
| 137 | + | # fails subsequent deploys closed while the running service is untouched. | |
| 138 | + | # Supported engines: Podman 5.4+ or Docker 28.3+ with native CDI | |
| 139 | + | # configured (CDI specs, toolkit, device permissions on the host). | |
| 140 | + | # [app_devices.transcode-gpu] | |
| 141 | + | # cdi = ["example.com/gpu=GPU-device-uuid"] # Required; explicit, non-empty | |
| 142 | + | # allowed_apps = ["video"] # Required; exact, non-empty | |
| 143 | + | # allowed_services = ["transcoder"] # Required; exact, non-empty | |
| 144 | + | ||
| 125 | 145 | [telemetry] | |
| 126 | 146 | enabled = false | |
| 127 | 147 | # endpoint = "http://localhost:4318" | |
| @@ -122,6 +122,26 @@ syslog_identifier = "gordon-access" | |||||
| 122 | # allowed_services = ["web"] # Required; exact, non-empty | 122 | # allowed_services = ["web"] # Required; exact, non-empty | ||
| 123 | # root = "/srv/gordon" # Optional; defaults to the source's parent | 123 | # root = "/srv/gordon" # Optional; defaults to the source's parent | ||
| 124 | 124 | ||||
| 125 | + | # ============================================================================= | |||
| 126 | + | # ADMINISTRATIVE APP DEVICES (CDI) | |||
| 127 | + | # ============================================================================= | |||
| 128 | + | # A named device grant is the only way an app manifest may request host | |||
| 129 | + | # devices. The manifest declares devices = ["<name>"]; raw /dev paths, | |||
| 130 | + | # privileged containers, and runtime-specific GPU flags are never used. | |||
| 131 | + | # Gordon resolves each name to explicit CDI IDs at activation time and | |||
| 132 | + | # encodes them as one native CDI DeviceRequest. allowed_apps and | |||
| 133 | + | # allowed_services are exact and non-empty: only that app+service pair | |||
| 134 | + | # receives the grant. Aggregate "=all" selectors are rejected: name every | |||
| 135 | + | # device explicitly. Changing a mapping never recreates a running | |||
| 136 | + | # container; the next deploy serves the new resolution. Revoking a grant | |||
| 137 | + | # fails subsequent deploys closed while the running service is untouched. | |||
| 138 | + | # Supported engines: Podman 5.4+ or Docker 28.3+ with native CDI | |||
| 139 | + | # configured (CDI specs, toolkit, device permissions on the host). | |||
| 140 | + | # [app_devices.transcode-gpu] | |||
| 141 | + | # cdi = ["example.com/gpu=GPU-device-uuid"] # Required; explicit, non-empty | |||
| 142 | + | # allowed_apps = ["video"] # Required; exact, non-empty | |||
| 143 | + | # allowed_services = ["transcoder"] # Required; exact, non-empty | |||
| 144 | + | ||||
| 125 | [telemetry] | 145 | [telemetry] | ||
| 126 | enabled = false | 146 | enabled = false | ||
| 127 | # endpoint = "http://localhost:4318" | 147 | # endpoint = "http://localhost:4318" | ||
M internal/adapters/in/appmanifest/parse.go +7 -2
| @@ -44,8 +44,10 @@ type rawService struct { | |||
| 44 | 44 | Secrets map[string]string `toml:"secrets"` | |
| 45 | 45 | Volumes []rawVolume `toml:"volume"` | |
| 46 | 46 | Binds []rawBind `toml:"bind"` | |
| 47 | - | Databases []rawDatabase `toml:"database"` | |
| 48 | - | Backup rawBackup `toml:"backup"` | |
| 47 | + | // Devices lists logical device names from `devices = [...]`. | |
| 48 | + | Devices []string `toml:"devices"` | |
| 49 | + | Databases []rawDatabase `toml:"database"` | |
| 50 | + | Backup rawBackup `toml:"backup"` | |
| 49 | 51 | } | |
| 50 | 52 | ||
| 51 | 53 | // rawReadiness mirrors [service.readiness]. | |
| @@ -267,6 +269,9 @@ func toDomainService(raw rawService) (domain.AppService, error) { | |||
| 267 | 269 | ReadOnly: b.ReadOnly, | |
| 268 | 270 | }) | |
| 269 | 271 | } | |
| 272 | + | // Preserve manifest order; domain equalDevices treats the slice as a | |
| 273 | + | // set, so reorder-only input is a no-op in diffs. | |
| 274 | + | svc.Devices = append([]string(nil), raw.Devices...) | |
| 270 | 275 | for _, db := range raw.Databases { | |
| 271 | 276 | svc.Databases = append(svc.Databases, domain.AppDatabase{ | |
| 272 | 277 | Name: db.Name, | |
| @@ -44,8 +44,10 @@ type rawService struct { | |||||
| 44 | Secrets map[string]string `toml:"secrets"` | 44 | Secrets map[string]string `toml:"secrets"` | ||
| 45 | Volumes []rawVolume `toml:"volume"` | 45 | Volumes []rawVolume `toml:"volume"` | ||
| 46 | Binds []rawBind `toml:"bind"` | 46 | Binds []rawBind `toml:"bind"` | ||
| 47 | - | Databases []rawDatabase `toml:"database"` | 47 | + | // Devices lists logical device names from `devices = [...]`. |
| 48 | - | Backup rawBackup `toml:"backup"` | 48 | + | Devices []string `toml:"devices"` |
| 49 | + | Databases []rawDatabase `toml:"database"` | |||
| 50 | + | Backup rawBackup `toml:"backup"` | |||
| 49 | } | 51 | } | ||
| 50 | 52 | ||||
| 51 | // rawReadiness mirrors [service.readiness]. | 53 | // rawReadiness mirrors [service.readiness]. | ||
| @@ -267,6 +269,9 @@ func toDomainService(raw rawService) (domain.AppService, error) { | |||||
| 267 | ReadOnly: b.ReadOnly, | 269 | ReadOnly: b.ReadOnly, | ||
| 268 | }) | 270 | }) | ||
| 269 | } | 271 | } | ||
| 272 | + | // Preserve manifest order; domain equalDevices treats the slice as a | |||
| 273 | + | // set, so reorder-only input is a no-op in diffs. | |||
| 274 | + | svc.Devices = append([]string(nil), raw.Devices...) | |||
| 270 | for _, db := range raw.Databases { | 275 | for _, db := range raw.Databases { | ||
| 271 | svc.Databases = append(svc.Databases, domain.AppDatabase{ | 276 | svc.Databases = append(svc.Databases, domain.AppDatabase{ | ||
| 272 | Name: db.Name, | 277 | Name: db.Name, | ||
M internal/adapters/in/appmanifest/parse_test.go +63 -0
| @@ -310,3 +310,66 @@ path = "/dev" | |||
| 310 | 310 | require.ErrorIs(t, err, domain.ErrInvalidAppSpec) | |
| 311 | 311 | assert.Contains(t, err.Error(), "sensitive container path") | |
| 312 | 312 | } | |
| 313 | + | ||
| 314 | + | func TestParse_Devices(t *testing.T) { | |
| 315 | + | doc := ` | |
| 316 | + | name = "demo" | |
| 317 | + | [[service]] | |
| 318 | + | name = "worker" | |
| 319 | + | image = "registry.example.com/demo/worker:1" | |
| 320 | + | devices = ["test_gpu"] | |
| 321 | + | ` | |
| 322 | + | spec, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |
| 323 | + | require.NoError(t, err) | |
| 324 | + | require.Len(t, spec.Services, 1) | |
| 325 | + | assert.Equal(t, []string{"test_gpu"}, spec.Services[0].Devices) | |
| 326 | + | } | |
| 327 | + | ||
| 328 | + | func TestParse_DevicesWrongType(t *testing.T) { | |
| 329 | + | doc := ` | |
| 330 | + | name = "demo" | |
| 331 | + | [[service]] | |
| 332 | + | name = "worker" | |
| 333 | + | image = "registry.example.com/demo/worker:1" | |
| 334 | + | devices = "test_gpu" | |
| 335 | + | ` | |
| 336 | + | _, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |
| 337 | + | require.Error(t, err) | |
| 338 | + | } | |
| 339 | + | ||
| 340 | + | func TestParse_DevicesMalformedRejected(t *testing.T) { | |
| 341 | + | doc := ` | |
| 342 | + | name = "demo" | |
| 343 | + | [[service]] | |
| 344 | + | name = "worker" | |
| 345 | + | image = "registry.example.com/demo/worker:1" | |
| 346 | + | devices = ["Bad Name!"] | |
| 347 | + | ` | |
| 348 | + | _, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |
| 349 | + | require.ErrorIs(t, err, domain.ErrInvalidAppSpec) | |
| 350 | + | } | |
| 351 | + | ||
| 352 | + | func TestParse_DevicesDuplicateRejected(t *testing.T) { | |
| 353 | + | doc := ` | |
| 354 | + | name = "demo" | |
| 355 | + | [[service]] | |
| 356 | + | name = "worker" | |
| 357 | + | image = "registry.example.com/demo/worker:1" | |
| 358 | + | devices = ["test_gpu", "test_gpu"] | |
| 359 | + | ` | |
| 360 | + | _, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |
| 361 | + | require.ErrorIs(t, err, domain.ErrInvalidAppSpec) | |
| 362 | + | assert.Contains(t, err.Error(), "duplicate device") | |
| 363 | + | } | |
| 364 | + | ||
| 365 | + | func TestParse_UnknownGPUFieldRejected(t *testing.T) { | |
| 366 | + | doc := ` | |
| 367 | + | name = "demo" | |
| 368 | + | [[service]] | |
| 369 | + | name = "worker" | |
| 370 | + | image = "registry.example.com/demo/worker:1" | |
| 371 | + | gpus = "all" | |
| 372 | + | ` | |
| 373 | + | _, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |
| 374 | + | require.Error(t, err) | |
| 375 | + | } | |
| @@ -310,3 +310,66 @@ path = "/dev" | |||||
| 310 | require.ErrorIs(t, err, domain.ErrInvalidAppSpec) | 310 | require.ErrorIs(t, err, domain.ErrInvalidAppSpec) | ||
| 311 | assert.Contains(t, err.Error(), "sensitive container path") | 311 | assert.Contains(t, err.Error(), "sensitive container path") | ||
| 312 | } | 312 | } | ||
| 313 | + | ||||
| 314 | + | func TestParse_Devices(t *testing.T) { | |||
| 315 | + | doc := ` | |||
| 316 | + | name = "demo" | |||
| 317 | + | [[service]] | |||
| 318 | + | name = "worker" | |||
| 319 | + | image = "registry.example.com/demo/worker:1" | |||
| 320 | + | devices = ["test_gpu"] | |||
| 321 | + | ` | |||
| 322 | + | spec, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |||
| 323 | + | require.NoError(t, err) | |||
| 324 | + | require.Len(t, spec.Services, 1) | |||
| 325 | + | assert.Equal(t, []string{"test_gpu"}, spec.Services[0].Devices) | |||
| 326 | + | } | |||
| 327 | + | ||||
| 328 | + | func TestParse_DevicesWrongType(t *testing.T) { | |||
| 329 | + | doc := ` | |||
| 330 | + | name = "demo" | |||
| 331 | + | [[service]] | |||
| 332 | + | name = "worker" | |||
| 333 | + | image = "registry.example.com/demo/worker:1" | |||
| 334 | + | devices = "test_gpu" | |||
| 335 | + | ` | |||
| 336 | + | _, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |||
| 337 | + | require.Error(t, err) | |||
| 338 | + | } | |||
| 339 | + | ||||
| 340 | + | func TestParse_DevicesMalformedRejected(t *testing.T) { | |||
| 341 | + | doc := ` | |||
| 342 | + | name = "demo" | |||
| 343 | + | [[service]] | |||
| 344 | + | name = "worker" | |||
| 345 | + | image = "registry.example.com/demo/worker:1" | |||
| 346 | + | devices = ["Bad Name!"] | |||
| 347 | + | ` | |||
| 348 | + | _, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |||
| 349 | + | require.ErrorIs(t, err, domain.ErrInvalidAppSpec) | |||
| 350 | + | } | |||
| 351 | + | ||||
| 352 | + | func TestParse_DevicesDuplicateRejected(t *testing.T) { | |||
| 353 | + | doc := ` | |||
| 354 | + | name = "demo" | |||
| 355 | + | [[service]] | |||
| 356 | + | name = "worker" | |||
| 357 | + | image = "registry.example.com/demo/worker:1" | |||
| 358 | + | devices = ["test_gpu", "test_gpu"] | |||
| 359 | + | ` | |||
| 360 | + | _, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |||
| 361 | + | require.ErrorIs(t, err, domain.ErrInvalidAppSpec) | |||
| 362 | + | assert.Contains(t, err.Error(), "duplicate device") | |||
| 363 | + | } | |||
| 364 | + | ||||
| 365 | + | func TestParse_UnknownGPUFieldRejected(t *testing.T) { | |||
| 366 | + | doc := ` | |||
| 367 | + | name = "demo" | |||
| 368 | + | [[service]] | |||
| 369 | + | name = "worker" | |||
| 370 | + | image = "registry.example.com/demo/worker:1" | |||
| 371 | + | gpus = "all" | |||
| 372 | + | ` | |||
| 373 | + | _, _, err := appmanifest.Parse([]byte(doc), "demo.toml") | |||
| 374 | + | require.Error(t, err) | |||
| 375 | + | } | |||
M internal/adapters/in/http/admin/handler_apps.go +12 -0
| @@ -519,6 +519,14 @@ func isMappedPreflightError(err error, op *domain.AppOperation) bool { | |||
| 519 | 519 | } | |
| 520 | 520 | return errors.Is(err, domain.ErrInvalidAppSpec) || | |
| 521 | 521 | errors.Is(err, domain.ErrBindPolicy) || | |
| 522 | + | errors.Is(err, domain.ErrDevicePolicy) || | |
| 523 | + | isPreflightErrorRest(err) | |
| 524 | + | } | |
| 525 | + | ||
| 526 | + | // isPreflightErrorRest covers the remaining preflight-mappable sentinels. | |
| 527 | + | // Split from isMappedPreflightError to keep its complexity within budget. | |
| 528 | + | func isPreflightErrorRest(err error) bool { | |
| 529 | + | return errors.Is(err, domain.ErrRuntimeUnsupported) || | |
| 522 | 530 | errors.Is(err, domain.ErrAppNotFound) || | |
| 523 | 531 | errors.Is(err, domain.ErrAppReservationConflict) || | |
| 524 | 532 | errors.Is(err, domain.ErrAppImageUnresolvable) || | |
| @@ -537,6 +545,10 @@ func (h *Handler) sendAppOpError(w http.ResponseWriter, err error) { | |||
| 537 | 545 | h.sendAppError(w, http.StatusBadRequest, "invalid-manifest", err.Error(), "", "") | |
| 538 | 546 | case errors.Is(err, domain.ErrBindPolicy): | |
| 539 | 547 | h.sendAppError(w, http.StatusBadRequest, "bind-policy-violation", err.Error(), "", "check the administrative mount authorization") | |
| 548 | + | case errors.Is(err, domain.ErrDevicePolicy): | |
| 549 | + | h.sendAppError(w, http.StatusBadRequest, "device-policy-violation", err.Error(), "", "check the administrative device authorization") | |
| 550 | + | case errors.Is(err, domain.ErrRuntimeUnsupported): | |
| 551 | + | h.sendAppError(w, http.StatusBadRequest, "runtime-unsupported", err.Error(), "", "use an engine with native CDI support") | |
| 540 | 552 | case errors.Is(err, domain.ErrAppReservationConflict): | |
| 541 | 553 | h.sendAppError(w, http.StatusConflict, "reservation-conflict", err.Error(), "", "") | |
| 542 | 554 | case errors.Is(err, domain.ErrAppImageUnresolvable): | |
| @@ -519,6 +519,14 @@ func isMappedPreflightError(err error, op *domain.AppOperation) bool { | |||||
| 519 | } | 519 | } | ||
| 520 | return errors.Is(err, domain.ErrInvalidAppSpec) || | 520 | return errors.Is(err, domain.ErrInvalidAppSpec) || | ||
| 521 | errors.Is(err, domain.ErrBindPolicy) || | 521 | errors.Is(err, domain.ErrBindPolicy) || | ||
| 522 | + | errors.Is(err, domain.ErrDevicePolicy) || | |||
| 523 | + | isPreflightErrorRest(err) | |||
| 524 | + | } | |||
| 525 | + | ||||
| 526 | + | // isPreflightErrorRest covers the remaining preflight-mappable sentinels. | |||
| 527 | + | // Split from isMappedPreflightError to keep its complexity within budget. | |||
| 528 | + | func isPreflightErrorRest(err error) bool { | |||
| 529 | + | return errors.Is(err, domain.ErrRuntimeUnsupported) || | |||
| 522 | errors.Is(err, domain.ErrAppNotFound) || | 530 | errors.Is(err, domain.ErrAppNotFound) || | ||
| 523 | errors.Is(err, domain.ErrAppReservationConflict) || | 531 | errors.Is(err, domain.ErrAppReservationConflict) || | ||
| 524 | errors.Is(err, domain.ErrAppImageUnresolvable) || | 532 | errors.Is(err, domain.ErrAppImageUnresolvable) || | ||
| @@ -537,6 +545,10 @@ func (h *Handler) sendAppOpError(w http.ResponseWriter, err error) { | |||||
| 537 | h.sendAppError(w, http.StatusBadRequest, "invalid-manifest", err.Error(), "", "") | 545 | h.sendAppError(w, http.StatusBadRequest, "invalid-manifest", err.Error(), "", "") | ||
| 538 | case errors.Is(err, domain.ErrBindPolicy): | 546 | case errors.Is(err, domain.ErrBindPolicy): | ||
| 539 | h.sendAppError(w, http.StatusBadRequest, "bind-policy-violation", err.Error(), "", "check the administrative mount authorization") | 547 | h.sendAppError(w, http.StatusBadRequest, "bind-policy-violation", err.Error(), "", "check the administrative mount authorization") | ||
| 548 | + | case errors.Is(err, domain.ErrDevicePolicy): | |||
| 549 | + | h.sendAppError(w, http.StatusBadRequest, "device-policy-violation", err.Error(), "", "check the administrative device authorization") | |||
| 550 | + | case errors.Is(err, domain.ErrRuntimeUnsupported): | |||
| 551 | + | h.sendAppError(w, http.StatusBadRequest, "runtime-unsupported", err.Error(), "", "use an engine with native CDI support") | |||
| 540 | case errors.Is(err, domain.ErrAppReservationConflict): | 552 | case errors.Is(err, domain.ErrAppReservationConflict): | ||
| 541 | h.sendAppError(w, http.StatusConflict, "reservation-conflict", err.Error(), "", "") | 553 | h.sendAppError(w, http.StatusConflict, "reservation-conflict", err.Error(), "", "") | ||
| 542 | case errors.Is(err, domain.ErrAppImageUnresolvable): | 554 | case errors.Is(err, domain.ErrAppImageUnresolvable): | ||
M internal/adapters/in/http/admin/handler_apps_test.go +34 -0
| @@ -610,3 +610,37 @@ func TestHandler_AppSecretsList_MissingAppIs404(t *testing.T) { | |||
| 610 | 610 | require.Equal(t, http.StatusNotFound, rec.Code) | |
| 611 | 611 | assert.Contains(t, rec.Body.String(), "app-not-found") | |
| 612 | 612 | } | |
| 613 | + | ||
| 614 | + | func TestHandler_AppApply_MapsDevicePolicyViolation(t *testing.T) { | |
| 615 | + | appSvc := inmocks.NewMockAppService(t) | |
| 616 | + | handler := appsTestHandler(t, appSvc) | |
| 617 | + | ||
| 618 | + | appSvc.EXPECT().Apply(mock.Anything, mock.Anything, mock.Anything, false).Return( | |
| 619 | + | nil, nil, | |
| 620 | + | fmt.Errorf("%w: app %q service %q device %q: refused by administrative device policy", domain.ErrDevicePolicy, "blog", "web", "test_gpu"), | |
| 621 | + | ).Once() | |
| 622 | + | ||
| 623 | + | rec := appsRequest(t, handler, http.MethodPost, "/admin/apps/apply", | |
| 624 | + | dto.AppApplyRequest{ManifestTOML: validAppManifest}, "admin:apps:write") | |
| 625 | + | assert.Equal(t, http.StatusBadRequest, rec.Code) | |
| 626 | + | var envelope dto.AppError | |
| 627 | + | require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &envelope)) | |
| 628 | + | assert.Equal(t, "device-policy-violation", envelope.Error) | |
| 629 | + | } | |
| 630 | + | ||
| 631 | + | func TestHandler_AppApply_MapsRuntimeUnsupported(t *testing.T) { | |
| 632 | + | appSvc := inmocks.NewMockAppService(t) | |
| 633 | + | handler := appsTestHandler(t, appSvc) | |
| 634 | + | ||
| 635 | + | appSvc.EXPECT().Apply(mock.Anything, mock.Anything, mock.Anything, false).Return( | |
| 636 | + | nil, nil, | |
| 637 | + | fmt.Errorf("%w: podman 4.9.5 below minimum 5.4", domain.ErrRuntimeUnsupported), | |
| 638 | + | ).Once() | |
| 639 | + | ||
| 640 | + | rec := appsRequest(t, handler, http.MethodPost, "/admin/apps/apply", | |
| 641 | + | dto.AppApplyRequest{ManifestTOML: validAppManifest}, "admin:apps:write") | |
| 642 | + | assert.Equal(t, http.StatusBadRequest, rec.Code) | |
| 643 | + | var envelope dto.AppError | |
| 644 | + | require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &envelope)) | |
| 645 | + | assert.Equal(t, "runtime-unsupported", envelope.Error) | |
| 646 | + | } | |
| @@ -610,3 +610,37 @@ func TestHandler_AppSecretsList_MissingAppIs404(t *testing.T) { | |||||
| 610 | require.Equal(t, http.StatusNotFound, rec.Code) | 610 | require.Equal(t, http.StatusNotFound, rec.Code) | ||
| 611 | assert.Contains(t, rec.Body.String(), "app-not-found") | 611 | assert.Contains(t, rec.Body.String(), "app-not-found") | ||
| 612 | } | 612 | } | ||
| 613 | + | ||||
| 614 | + | func TestHandler_AppApply_MapsDevicePolicyViolation(t *testing.T) { | |||
| 615 | + | appSvc := inmocks.NewMockAppService(t) | |||
| 616 | + | handler := appsTestHandler(t, appSvc) | |||
| 617 | + | ||||
| 618 | + | appSvc.EXPECT().Apply(mock.Anything, mock.Anything, mock.Anything, false).Return( | |||
| 619 | + | nil, nil, | |||
| 620 | + | fmt.Errorf("%w: app %q service %q device %q: refused by administrative device policy", domain.ErrDevicePolicy, "blog", "web", "test_gpu"), | |||
| 621 | + | ).Once() | |||
| 622 | + | ||||
| 623 | + | rec := appsRequest(t, handler, http.MethodPost, "/admin/apps/apply", | |||
| 624 | + | dto.AppApplyRequest{ManifestTOML: validAppManifest}, "admin:apps:write") | |||
| 625 | + | assert.Equal(t, http.StatusBadRequest, rec.Code) | |||
| 626 | + | var envelope dto.AppError | |||
| 627 | + | require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &envelope)) | |||
| 628 | + | assert.Equal(t, "device-policy-violation", envelope.Error) | |||
| 629 | + | } | |||
| 630 | + | ||||
| 631 | + | func TestHandler_AppApply_MapsRuntimeUnsupported(t *testing.T) { | |||
| 632 | + | appSvc := inmocks.NewMockAppService(t) | |||
| 633 | + | handler := appsTestHandler(t, appSvc) | |||
| 634 | + | ||||
| 635 | + | appSvc.EXPECT().Apply(mock.Anything, mock.Anything, mock.Anything, false).Return( | |||
| 636 | + | nil, nil, | |||
| 637 | + | fmt.Errorf("%w: podman 4.9.5 below minimum 5.4", domain.ErrRuntimeUnsupported), | |||
| 638 | + | ).Once() | |||
| 639 | + | ||||
| 640 | + | rec := appsRequest(t, handler, http.MethodPost, "/admin/apps/apply", | |||
| 641 | + | dto.AppApplyRequest{ManifestTOML: validAppManifest}, "admin:apps:write") | |||
| 642 | + | assert.Equal(t, http.StatusBadRequest, rec.Code) | |||
| 643 | + | var envelope dto.AppError | |||
| 644 | + | require.NoError(t, json.Unmarshal(rec.Body.Bytes(), &envelope)) | |||
| 645 | + | assert.Equal(t, "runtime-unsupported", envelope.Error) | |||
| 646 | + | } | |||
M internal/adapters/out/appstate/export_test.go +13 -0
| @@ -27,6 +27,19 @@ func CorruptCheckpointForTest(s *Store, rewrite func([]byte) []byte) error { | |||
| 27 | 27 | }) | |
| 28 | 28 | } | |
| 29 | 29 | ||
| 30 | + | // SeedDesiredRecordForTest writes an exact desired-record payload, creating | |
| 31 | + | // the app bucket when absent. Tests use it to plant records written by an | |
| 32 | + | // older Gordon, whose JSON omits fields the current schema has. | |
| 33 | + | func SeedDesiredRecordForTest(s *Store, app string, raw []byte) error { | |
| 34 | + | return s.db.Update(func(tx *bolt.Tx) error { | |
| 35 | + | bucket, err := appBucket(tx, app, true) | |
| 36 | + | if err != nil { | |
| 37 | + | return err | |
| 38 | + | } | |
| 39 | + | return bucket.Put(keyDesired, raw) | |
| 40 | + | }) | |
| 41 | + | } | |
| 42 | + | ||
| 30 | 43 | // CorruptDesiredForTest rewrites one app's desired record. Tests use it | |
| 31 | 44 | // to simulate a corrupt desired payload with other apps unaffected. | |
| 32 | 45 | func CorruptDesiredForTest(s *Store, app string, rewrite func([]byte) []byte) error { | |
| @@ -27,6 +27,19 @@ func CorruptCheckpointForTest(s *Store, rewrite func([]byte) []byte) error { | |||||
| 27 | }) | 27 | }) | ||
| 28 | } | 28 | } | ||
| 29 | 29 | ||||
| 30 | + | // SeedDesiredRecordForTest writes an exact desired-record payload, creating | |||
| 31 | + | // the app bucket when absent. Tests use it to plant records written by an | |||
| 32 | + | // older Gordon, whose JSON omits fields the current schema has. | |||
| 33 | + | func SeedDesiredRecordForTest(s *Store, app string, raw []byte) error { | |||
| 34 | + | return s.db.Update(func(tx *bolt.Tx) error { | |||
| 35 | + | bucket, err := appBucket(tx, app, true) | |||
| 36 | + | if err != nil { | |||
| 37 | + | return err | |||
| 38 | + | } | |||
| 39 | + | return bucket.Put(keyDesired, raw) | |||
| 40 | + | }) | |||
| 41 | + | } | |||
| 42 | + | ||||
| 30 | // CorruptDesiredForTest rewrites one app's desired record. Tests use it | 43 | // CorruptDesiredForTest rewrites one app's desired record. Tests use it | ||
| 31 | // to simulate a corrupt desired payload with other apps unaffected. | 44 | // to simulate a corrupt desired payload with other apps unaffected. | ||
| 32 | func CorruptDesiredForTest(s *Store, app string, rewrite func([]byte) []byte) error { | 45 | func CorruptDesiredForTest(s *Store, app string, rewrite func([]byte) []byte) error { | ||
M internal/adapters/out/appstate/store_test.go +69 -0
| @@ -521,3 +521,72 @@ func TestStore_RecoveryInhibitionsAreDurableAndGenerationScoped(t *testing.T) { | |||
| 521 | 521 | require.Error(t, reopened.SaveRecoveryInhibition(cancelled, domain.AppRecoveryInhibition{App: "blog"})) | |
| 522 | 522 | require.Error(t, reopened.ClearRecoveryInhibition(cancelled, "blog", "worker", "c-worker")) | |
| 523 | 523 | } | |
| 524 | + | ||
| 525 | + | func TestStore_DesiredDevicesRoundTrip(t *testing.T) { | |
| 526 | + | ctx := context.Background() | |
| 527 | + | store := newTestStore(t) | |
| 528 | + | ||
| 529 | + | intent := testIntent("blog", "rev-1") | |
| 530 | + | intent.Spec.Services[0].Devices = []string{"test_gpu"} | |
| 531 | + | require.NoError(t, store.StageApply(ctx, intent)) | |
| 532 | + | require.NoError(t, store.CommitApply(ctx, "blog", "apply-test-rev-1")) | |
| 533 | + | require.NoError(t, store.MaterializeApply(ctx, "blog", "apply-test-rev-1")) | |
| 534 | + | ||
| 535 | + | desired, ok, err := store.LoadDesired(ctx, "blog") | |
| 536 | + | require.NoError(t, err) | |
| 537 | + | require.True(t, ok) | |
| 538 | + | require.Len(t, desired.Spec.Services, 1) | |
| 539 | + | assert.Equal(t, []string{"test_gpu"}, desired.Spec.Services[0].Devices, "logical names persist in revisions") | |
| 540 | + | } | |
| 541 | + | ||
| 542 | + | // TestStore_DesiredWithoutDevicesLoads proves a desired record written by a | |
| 543 | + | // Gordon release that predates service devices still loads: the absent JSON | |
| 544 | + | // key decodes to a nil slice, never an error or a phantom device. | |
| 545 | + | func TestStore_DesiredWithoutDevicesLoads(t *testing.T) { | |
| 546 | + | ctx := context.Background() | |
| 547 | + | store := newTestStore(t) | |
| 548 | + | ||
| 549 | + | require.NoError(t, appstate.SeedDesiredRecordForTest(store, "blog", []byte(legacyDesiredWithoutDevices))) | |
| 550 | + | ||
| 551 | + | desired, ok, err := store.LoadDesired(ctx, "blog") | |
| 552 | + | require.NoError(t, err) | |
| 553 | + | require.True(t, ok) | |
| 554 | + | require.Len(t, desired.Spec.Services, 1) | |
| 555 | + | assert.Nil(t, desired.Spec.Services[0].Devices, "a missing devices key must not become a phantom device") | |
| 556 | + | } | |
| 557 | + | ||
| 558 | + | // legacyDesiredWithoutDevices is a verbatim state.db desired record from a | |
| 559 | + | // release predating service devices. Unlike current serializer output, its | |
| 560 | + | // service object carries no "Devices" key at all rather than a null one. | |
| 561 | + | const legacyDesiredWithoutDevices = `{ | |
| 562 | + | "revision": "rev-1", | |
| 563 | + | "app": "blog", | |
| 564 | + | "accepted_at": "2026-01-01T00:00:00Z", | |
| 565 | + | "source_sha256": "", | |
| 566 | + | "spec": { | |
| 567 | + | "Name": "blog", | |
| 568 | + | "Env": null, | |
| 569 | + | "Services": [ | |
| 570 | + | { | |
| 571 | + | "Name": "web", | |
| 572 | + | "Image": "img:1", | |
| 573 | + | "Command": null, | |
| 574 | + | "StopGrace": 10000000000, | |
| 575 | + | "Readiness": {"Type": "", "Path": "", "Contains": "", "Port": 0, "Timeout": 0}, | |
| 576 | + | "HTTP": null, | |
| 577 | + | "TCP": null, | |
| 578 | + | "UDP": null, | |
| 579 | + | "Secrets": null, | |
| 580 | + | "Volumes": null, | |
| 581 | + | "Binds": null, | |
| 582 | + | "Databases": null, | |
| 583 | + | "Backup": {"Postgres": null, "Volume": null} | |
| 584 | + | } | |
| 585 | + | ], | |
| 586 | + | "Networks": null | |
| 587 | + | }, | |
| 588 | + | "reservations": [{"proto": "http", "host": "blog.example.com", "service": "web", "app": "blog"}], | |
| 589 | + | "secrets_required": null, | |
| 590 | + | "secrets_env": {}, | |
| 591 | + | "status": "pending" | |
| 592 | + | }` | |
| @@ -521,3 +521,72 @@ func TestStore_RecoveryInhibitionsAreDurableAndGenerationScoped(t *testing.T) { | |||||
| 521 | require.Error(t, reopened.SaveRecoveryInhibition(cancelled, domain.AppRecoveryInhibition{App: "blog"})) | 521 | require.Error(t, reopened.SaveRecoveryInhibition(cancelled, domain.AppRecoveryInhibition{App: "blog"})) | ||
| 522 | require.Error(t, reopened.ClearRecoveryInhibition(cancelled, "blog", "worker", "c-worker")) | 522 | require.Error(t, reopened.ClearRecoveryInhibition(cancelled, "blog", "worker", "c-worker")) | ||
| 523 | } | 523 | } | ||
| 524 | + | ||||
| 525 | + | func TestStore_DesiredDevicesRoundTrip(t *testing.T) { | |||
| 526 | + | ctx := context.Background() | |||
| 527 | + | store := newTestStore(t) | |||
| 528 | + | ||||
| 529 | + | intent := testIntent("blog", "rev-1") | |||
| 530 | + | intent.Spec.Services[0].Devices = []string{"test_gpu"} | |||
| 531 | + | require.NoError(t, store.StageApply(ctx, intent)) | |||
| 532 | + | require.NoError(t, store.CommitApply(ctx, "blog", "apply-test-rev-1")) | |||
| 533 | + | require.NoError(t, store.MaterializeApply(ctx, "blog", "apply-test-rev-1")) | |||
| 534 | + | ||||
| 535 | + | desired, ok, err := store.LoadDesired(ctx, "blog") | |||
| 536 | + | require.NoError(t, err) | |||
| 537 | + | require.True(t, ok) | |||
| 538 | + | require.Len(t, desired.Spec.Services, 1) | |||
| 539 | + | assert.Equal(t, []string{"test_gpu"}, desired.Spec.Services[0].Devices, "logical names persist in revisions") | |||
| 540 | + | } | |||
| 541 | + | ||||
| 542 | + | // TestStore_DesiredWithoutDevicesLoads proves a desired record written by a | |||
| 543 | + | // Gordon release that predates service devices still loads: the absent JSON | |||
| 544 | + | // key decodes to a nil slice, never an error or a phantom device. | |||
| 545 | + | func TestStore_DesiredWithoutDevicesLoads(t *testing.T) { | |||
| 546 | + | ctx := context.Background() | |||
| 547 | + | store := newTestStore(t) | |||
| 548 | + | ||||
| 549 | + | require.NoError(t, appstate.SeedDesiredRecordForTest(store, "blog", []byte(legacyDesiredWithoutDevices))) | |||
| 550 | + | ||||
| 551 | + | desired, ok, err := store.LoadDesired(ctx, "blog") | |||
| 552 | + | require.NoError(t, err) | |||
| 553 | + | require.True(t, ok) | |||
| 554 | + | require.Len(t, desired.Spec.Services, 1) | |||
| 555 | + | assert.Nil(t, desired.Spec.Services[0].Devices, "a missing devices key must not become a phantom device") | |||
| 556 | + | } | |||
| 557 | + | ||||
| 558 | + | // legacyDesiredWithoutDevices is a verbatim state.db desired record from a | |||
| 559 | + | // release predating service devices. Unlike current serializer output, its | |||
| 560 | + | // service object carries no "Devices" key at all rather than a null one. | |||
| 561 | + | const legacyDesiredWithoutDevices = `{ | |||
| 562 | + | "revision": "rev-1", | |||
| 563 | + | "app": "blog", | |||
| 564 | + | "accepted_at": "2026-01-01T00:00:00Z", | |||
| 565 | + | "source_sha256": "", | |||
| 566 | + | "spec": { | |||
| 567 | + | "Name": "blog", | |||
| 568 | + | "Env": null, | |||
| 569 | + | "Services": [ | |||
| 570 | + | { | |||
| 571 | + | "Name": "web", | |||
| 572 | + | "Image": "img:1", | |||
| 573 | + | "Command": null, | |||
| 574 | + | "StopGrace": 10000000000, | |||
| 575 | + | "Readiness": {"Type": "", "Path": "", "Contains": "", "Port": 0, "Timeout": 0}, | |||
| 576 | + | "HTTP": null, | |||
| 577 | + | "TCP": null, | |||
| 578 | + | "UDP": null, | |||
| 579 | + | "Secrets": null, | |||
| 580 | + | "Volumes": null, | |||
| 581 | + | "Binds": null, | |||
| 582 | + | "Databases": null, | |||
| 583 | + | "Backup": {"Postgres": null, "Volume": null} | |||
| 584 | + | } | |||
| 585 | + | ], | |||
| 586 | + | "Networks": null | |||
| 587 | + | }, | |||
| 588 | + | "reservations": [{"proto": "http", "host": "blog.example.com", "service": "web", "app": "blog"}], | |||
| 589 | + | "secrets_required": null, | |||
| 590 | + | "secrets_env": {}, | |||
| 591 | + | "status": "pending" | |||
| 592 | + | }` | |||
M internal/adapters/out/docker/runtime.go +166 -0
| @@ -148,6 +148,15 @@ func (r *Runtime) CreateContainer(ctx context.Context, config *domain.ContainerC | |||
| 148 | 148 | if err != nil { | |
| 149 | 149 | return nil, err | |
| 150 | 150 | } | |
| 151 | + | // Device-bearing creates require a CDI-capable engine. The gate runs | |
| 152 | + | // only when devices are requested: ordinary apps never pay for it | |
| 153 | + | // and never fail it. | |
| 154 | + | if len(config.CDIDevices) > 0 { | |
| 155 | + | if err := r.requireCDISupport(ctx); err != nil { | |
| 156 | + | return nil, err | |
| 157 | + | } | |
| 158 | + | } | |
| 159 | + | deviceRequests := buildDeviceRequests(config) | |
| 151 | 160 | ||
| 152 | 161 | // Create container configuration | |
| 153 | 162 | containerConfig := &container.Config{ | |
| @@ -168,6 +177,7 @@ func (r *Runtime) CreateContainer(ctx context.Context, config *domain.ContainerC | |||
| 168 | 177 | Ulimits: []*units.Ulimit{ | |
| 169 | 178 | {Name: "nofile", Soft: 65536, Hard: 65536}, | |
| 170 | 179 | }, | |
| 180 | + | DeviceRequests: deviceRequests, | |
| 171 | 181 | } | |
| 172 | 182 | if config.PidsLimit > 0 { | |
| 173 | 183 | resources.PidsLimit = &config.PidsLimit | |
| @@ -334,6 +344,162 @@ func buildBindMounts(config *domain.ContainerConfig) ([]mount.Mount, error) { | |||
| 334 | 344 | return mounts, nil | |
| 335 | 345 | } | |
| 336 | 346 | ||
| 347 | + | // requireCDISupport fails closed when the connected engine cannot serve | |
| 348 | + | // native CDI device requests. Family and version come from the daemon's | |
| 349 | + | // own /version response; Gordon requires Podman 5.4+ or Docker 28.3+ for | |
| 350 | + | // device-bearing creates. The version is a capability gate, not hardware | |
| 351 | + | // proof: CDI specs, toolkit, and node permissions must still be correct | |
| 352 | + | // on the host. | |
| 353 | + | func (r *Runtime) requireCDISupport(ctx context.Context) error { | |
| 354 | + | version, err := r.client.ServerVersion(ctx, client.ServerVersionOptions{}) | |
| 355 | + | if err != nil { | |
| 356 | + | return engineProbeError(ctx, err) | |
| 357 | + | } | |
| 358 | + | if err := checkCDISupport(version, r.runtimeName); err != nil { | |
| 359 | + | return err | |
| 360 | + | } | |
| 361 | + | return nil | |
| 362 | + | } | |
| 363 | + | ||
| 364 | + | // engineProbeError maps a failed /version probe onto the CDI capability | |
| 365 | + | // gate. Cancellation and deadlines survive so callers can still tell an | |
| 366 | + | // aborted probe from an incapable engine; every other cause collapses to | |
| 367 | + | // ErrRuntimeUnsupported with the cause text dropped, because transport | |
| 368 | + | // errors embed the daemon endpoint and the operator's home path. | |
| 369 | + | func engineProbeError(ctx context.Context, err error) error { | |
| 370 | + | var ctxErr error | |
| 371 | + | switch { | |
| 372 | + | case errors.Is(err, context.Canceled), errors.Is(ctx.Err(), context.Canceled): | |
| 373 | + | ctxErr = context.Canceled | |
| 374 | + | case errors.Is(err, context.DeadlineExceeded), errors.Is(ctx.Err(), context.DeadlineExceeded): | |
| 375 | + | ctxErr = context.DeadlineExceeded | |
| 376 | + | } | |
| 377 | + | if ctxErr != nil { | |
| 378 | + | return fmt.Errorf("%w: cannot verify engine version: %w", domain.ErrRuntimeUnsupported, ctxErr) | |
| 379 | + | } | |
| 380 | + | return fmt.Errorf("%w: cannot verify engine version", domain.ErrRuntimeUnsupported) | |
| 381 | + | } | |
| 382 | + | ||
| 383 | + | // SupportsCDIDevices implements out.ContainerRuntime. Deployment calls it | |
| 384 | + | // in preflight for device-bearing revisions so an unsupported engine | |
| 385 | + | // fails before any workload mutation; CreateContainer keeps the same gate | |
| 386 | + | // as defense in depth. | |
| 387 | + | func (r *Runtime) SupportsCDIDevices(ctx context.Context) error { | |
| 388 | + | return r.requireCDISupport(ctx) | |
| 389 | + | } | |
| 390 | + | ||
| 391 | + | // checkCDISupport verifies one daemon version response against the CDI | |
| 392 | + | // support matrix. Family detection prefers the daemon's own components | |
| 393 | + | // (Podman Engine vs Engine) and falls back to the socket-path hint only | |
| 394 | + | // when the response names nothing recognizable. | |
| 395 | + | func checkCDISupport(version client.ServerVersionResult, socketHint string) error { | |
| 396 | + | family, daemonVersion := cdiEngineIdentity(version, socketHint) | |
| 397 | + | var minimum string | |
| 398 | + | switch family { | |
| 399 | + | case "podman": | |
| 400 | + | minimum = "5.4" | |
| 401 | + | case "docker": | |
| 402 | + | minimum = "28.3" | |
| 403 | + | default: | |
| 404 | + | return fmt.Errorf("%w: unrecognized engine %q version %q: device requests require Podman 5.4+ or Docker 28.3+ with native CDI configured", domain.ErrRuntimeUnsupported, family, daemonVersion) | |
| 405 | + | } | |
| 406 | + | if compareEngineVersion(daemonVersion, minimum) < 0 { | |
| 407 | + | return fmt.Errorf("%w: %s %s below minimum %s: device requests require Podman 5.4+ or Docker 28.3+ with native CDI configured (check engine upgrade, CDI specs, toolkit, and device permissions)", domain.ErrRuntimeUnsupported, family, daemonVersion, minimum) | |
| 408 | + | } | |
| 409 | + | return nil | |
| 410 | + | } | |
| 411 | + | ||
| 412 | + | // cdiEngineIdentity names the daemon family and its version from the | |
| 413 | + | // /version response. Podman answers with a "Podman Engine" component; | |
| 414 | + | // Docker answers with an "Engine" component and its version at top | |
| 415 | + | // level. Unknown responses keep the raw version with an "unknown" | |
| 416 | + | // family so the caller fails closed. | |
| 417 | + | func cdiEngineIdentity(version client.ServerVersionResult, socketHint string) (family, daemonVersion string) { | |
| 418 | + | for _, component := range version.Components { | |
| 419 | + | switch component.Name { | |
| 420 | + | case "Podman Engine": | |
| 421 | + | return "podman", component.Version | |
| 422 | + | case "Engine": | |
| 423 | + | return "docker", component.Version | |
| 424 | + | } | |
| 425 | + | } | |
| 426 | + | if strings.Contains(strings.ToLower(version.Platform.Name), "podman") { | |
| 427 | + | return "podman", version.Version | |
| 428 | + | } | |
| 429 | + | switch socketHint { | |
| 430 | + | case "podman", "docker": | |
| 431 | + | return socketHint, version.Version | |
| 432 | + | } | |
| 433 | + | return "unknown", version.Version | |
| 434 | + | } | |
| 435 | + | ||
| 436 | + | // compareEngineVersion compares dotted major.minor versions. A missing | |
| 437 | + | // or unparsable version compares below any minimum: the gate fails | |
| 438 | + | // closed rather than trusting an engine it cannot identify. | |
| 439 | + | func compareEngineVersion(version, minimum string) int { | |
| 440 | + | parse := func(value string) (int, int) { | |
| 441 | + | parts := strings.SplitN(value, ".", 3) | |
| 442 | + | if len(parts) < 2 { | |
| 443 | + | return -1, -1 | |
| 444 | + | } | |
| 445 | + | major, ok := parseEngineVersionComponent(parts[0]) | |
| 446 | + | if !ok { | |
| 447 | + | return -1, -1 | |
| 448 | + | } | |
| 449 | + | minor, ok := parseEngineVersionComponent(parts[1]) | |
| 450 | + | if !ok { | |
| 451 | + | return -1, -1 | |
| 452 | + | } | |
| 453 | + | return major, minor | |
| 454 | + | } | |
| 455 | + | major, minor := parse(version) | |
| 456 | + | minMajor, minMinor := parse(minimum) | |
| 457 | + | switch { | |
| 458 | + | case major != minMajor: | |
| 459 | + | return major - minMajor | |
| 460 | + | default: | |
| 461 | + | return minor - minMinor | |
| 462 | + | } | |
| 463 | + | } | |
| 464 | + | ||
| 465 | + | // parseEngineVersionComponent parses one numeric major/minor component. | |
| 466 | + | // Anything but digits is refused, so prerelease suffixes such as "-rc1" | |
| 467 | + | // and malformed versions can never lift an engine above the gate. | |
| 468 | + | func parseEngineVersionComponent(value string) (int, bool) { | |
| 469 | + | value = strings.TrimSpace(value) | |
| 470 | + | if value == "" { | |
| 471 | + | return 0, false | |
| 472 | + | } | |
| 473 | + | for _, r := range value { | |
| 474 | + | if r < '0' || r > '9' { | |
| 475 | + | return 0, false | |
| 476 | + | } | |
| 477 | + | } | |
| 478 | + | n, err := strconv.Atoi(value) | |
| 479 | + | if err != nil { | |
| 480 | + | return 0, false | |
| 481 | + | } | |
| 482 | + | return n, true | |
| 483 | + | } | |
| 484 | + | ||
| 485 | + | // buildDeviceRequests encodes ephemeral CDI device IDs as one native CDI | |
| 486 | + | // DeviceRequest. Capabilities and Options stay empty; Count stays 0 so the | |
| 487 | + | // explicit DeviceIDs are the only grant. CDI IDs are never logged or echoed | |
| 488 | + | // in errors: they are host inventory. | |
| 489 | + | func buildDeviceRequests(config *domain.ContainerConfig) []container.DeviceRequest { | |
| 490 | + | if len(config.CDIDevices) == 0 { | |
| 491 | + | return nil | |
| 492 | + | } | |
| 493 | + | ids := append([]string(nil), config.CDIDevices...) | |
| 494 | + | sort.Strings(ids) | |
| 495 | + | return []container.DeviceRequest{ | |
| 496 | + | { | |
| 497 | + | Driver: "cdi", | |
| 498 | + | DeviceIDs: ids, | |
| 499 | + | }, | |
| 500 | + | } | |
| 501 | + | } | |
| 502 | + | ||
| 337 | 503 | // StartContainer starts a container. | |
| 338 | 504 | func (r *Runtime) StartContainer(ctx context.Context, containerID string) error { | |
| 339 | 505 | ctx = zerowrap.CtxWithFields(ctx, map[string]any{ | |
| @@ -148,6 +148,15 @@ func (r *Runtime) CreateContainer(ctx context.Context, config *domain.ContainerC | |||||
| 148 | if err != nil { | 148 | if err != nil { | ||
| 149 | return nil, err | 149 | return nil, err | ||
| 150 | } | 150 | } | ||
| 151 | + | // Device-bearing creates require a CDI-capable engine. The gate runs | |||
| 152 | + | // only when devices are requested: ordinary apps never pay for it | |||
| 153 | + | // and never fail it. | |||
| 154 | + | if len(config.CDIDevices) > 0 { | |||
| 155 | + | if err := r.requireCDISupport(ctx); err != nil { | |||
| 156 | + | return nil, err | |||
| 157 | + | } | |||
| 158 | + | } | |||
| 159 | + | deviceRequests := buildDeviceRequests(config) | |||
| 151 | 160 | ||||
| 152 | // Create container configuration | 161 | // Create container configuration | ||
| 153 | containerConfig := &container.Config{ | 162 | containerConfig := &container.Config{ | ||
| @@ -168,6 +177,7 @@ func (r *Runtime) CreateContainer(ctx context.Context, config *domain.ContainerC | |||||
| 168 | Ulimits: []*units.Ulimit{ | 177 | Ulimits: []*units.Ulimit{ | ||
| 169 | {Name: "nofile", Soft: 65536, Hard: 65536}, | 178 | {Name: "nofile", Soft: 65536, Hard: 65536}, | ||
| 170 | }, | 179 | }, | ||
| 180 | + | DeviceRequests: deviceRequests, | |||
| 171 | } | 181 | } | ||
| 172 | if config.PidsLimit > 0 { | 182 | if config.PidsLimit > 0 { | ||
| 173 | resources.PidsLimit = &config.PidsLimit | 183 | resources.PidsLimit = &config.PidsLimit | ||
| @@ -334,6 +344,162 @@ func buildBindMounts(config *domain.ContainerConfig) ([]mount.Mount, error) { | |||||
| 334 | return mounts, nil | 344 | return mounts, nil | ||
| 335 | } | 345 | } | ||
| 336 | 346 | ||||
| 347 | + | // requireCDISupport fails closed when the connected engine cannot serve | |||
| 348 | + | // native CDI device requests. Family and version come from the daemon's | |||
| 349 | + | // own /version response; Gordon requires Podman 5.4+ or Docker 28.3+ for | |||
| 350 | + | // device-bearing creates. The version is a capability gate, not hardware | |||
| 351 | + | // proof: CDI specs, toolkit, and node permissions must still be correct | |||
| 352 | + | // on the host. | |||
| 353 | + | func (r *Runtime) requireCDISupport(ctx context.Context) error { | |||
| 354 | + | version, err := r.client.ServerVersion(ctx, client.ServerVersionOptions{}) | |||
| 355 | + | if err != nil { | |||
| 356 | + | return engineProbeError(ctx, err) | |||
| 357 | + | } | |||
| 358 | + | if err := checkCDISupport(version, r.runtimeName); err != nil { | |||
| 359 | + | return err | |||
| 360 | + | } | |||
| 361 | + | return nil | |||
| 362 | + | } | |||
| 363 | + | ||||
| 364 | + | // engineProbeError maps a failed /version probe onto the CDI capability | |||
| 365 | + | // gate. Cancellation and deadlines survive so callers can still tell an | |||
| 366 | + | // aborted probe from an incapable engine; every other cause collapses to | |||
| 367 | + | // ErrRuntimeUnsupported with the cause text dropped, because transport | |||
| 368 | + | // errors embed the daemon endpoint and the operator's home path. | |||
| 369 | + | func engineProbeError(ctx context.Context, err error) error { | |||
| 370 | + | var ctxErr error | |||
| 371 | + | switch { | |||
| 372 | + | case errors.Is(err, context.Canceled), errors.Is(ctx.Err(), context.Canceled): | |||
| 373 | + | ctxErr = context.Canceled | |||
| 374 | + | case errors.Is(err, context.DeadlineExceeded), errors.Is(ctx.Err(), context.DeadlineExceeded): | |||
| 375 | + | ctxErr = context.DeadlineExceeded | |||
| 376 | + | } | |||
| 377 | + | if ctxErr != nil { | |||
| 378 | + | return fmt.Errorf("%w: cannot verify engine version: %w", domain.ErrRuntimeUnsupported, ctxErr) | |||
| 379 | + | } | |||
| 380 | + | return fmt.Errorf("%w: cannot verify engine version", domain.ErrRuntimeUnsupported) | |||
| 381 | + | } | |||
| 382 | + | ||||
| 383 | + | // SupportsCDIDevices implements out.ContainerRuntime. Deployment calls it | |||
| 384 | + | // in preflight for device-bearing revisions so an unsupported engine | |||
| 385 | + | // fails before any workload mutation; CreateContainer keeps the same gate | |||
| 386 | + | // as defense in depth. | |||
| 387 | + | func (r *Runtime) SupportsCDIDevices(ctx context.Context) error { | |||
| 388 | + | return r.requireCDISupport(ctx) | |||
| 389 | + | } | |||
| 390 | + | ||||
| 391 | + | // checkCDISupport verifies one daemon version response against the CDI | |||
| 392 | + | // support matrix. Family detection prefers the daemon's own components | |||
| 393 | + | // (Podman Engine vs Engine) and falls back to the socket-path hint only | |||
| 394 | + | // when the response names nothing recognizable. | |||
| 395 | + | func checkCDISupport(version client.ServerVersionResult, socketHint string) error { | |||
| 396 | + | family, daemonVersion := cdiEngineIdentity(version, socketHint) | |||
| 397 | + | var minimum string | |||
| 398 | + | switch family { | |||
| 399 | + | case "podman": | |||
| 400 | + | minimum = "5.4" | |||
| 401 | + | case "docker": | |||
| 402 | + | minimum = "28.3" | |||
| 403 | + | default: | |||
| 404 | + | return fmt.Errorf("%w: unrecognized engine %q version %q: device requests require Podman 5.4+ or Docker 28.3+ with native CDI configured", domain.ErrRuntimeUnsupported, family, daemonVersion) | |||
| 405 | + | } | |||
| 406 | + | if compareEngineVersion(daemonVersion, minimum) < 0 { | |||
| 407 | + | return fmt.Errorf("%w: %s %s below minimum %s: device requests require Podman 5.4+ or Docker 28.3+ with native CDI configured (check engine upgrade, CDI specs, toolkit, and device permissions)", domain.ErrRuntimeUnsupported, family, daemonVersion, minimum) | |||
| 408 | + | } | |||
| 409 | + | return nil | |||
| 410 | + | } | |||
| 411 | + | ||||
| 412 | + | // cdiEngineIdentity names the daemon family and its version from the | |||
| 413 | + | // /version response. Podman answers with a "Podman Engine" component; | |||
| 414 | + | // Docker answers with an "Engine" component and its version at top | |||
| 415 | + | // level. Unknown responses keep the raw version with an "unknown" | |||
| 416 | + | // family so the caller fails closed. | |||
| 417 | + | func cdiEngineIdentity(version client.ServerVersionResult, socketHint string) (family, daemonVersion string) { | |||
| 418 | + | for _, component := range version.Components { | |||
| 419 | + | switch component.Name { | |||
| 420 | + | case "Podman Engine": | |||
| 421 | + | return "podman", component.Version | |||
| 422 | + | case "Engine": | |||
| 423 | + | return "docker", component.Version | |||
| 424 | + | } | |||
| 425 | + | } | |||
| 426 | + | if strings.Contains(strings.ToLower(version.Platform.Name), "podman") { | |||
| 427 | + | return "podman", version.Version | |||
| 428 | + | } | |||
| 429 | + | switch socketHint { | |||
| 430 | + | case "podman", "docker": | |||
| 431 | + | return socketHint, version.Version | |||
| 432 | + | } | |||
| 433 | + | return "unknown", version.Version | |||
| 434 | + | } | |||
| 435 | + | ||||
| 436 | + | // compareEngineVersion compares dotted major.minor versions. A missing | |||
| 437 | + | // or unparsable version compares below any minimum: the gate fails | |||
| 438 | + | // closed rather than trusting an engine it cannot identify. | |||
| 439 | + | func compareEngineVersion(version, minimum string) int { | |||
| 440 | + | parse := func(value string) (int, int) { | |||
| 441 | + | parts := strings.SplitN(value, ".", 3) | |||
| 442 | + | if len(parts) < 2 { | |||
| 443 | + | return -1, -1 | |||
| 444 | + | } | |||
| 445 | + | major, ok := parseEngineVersionComponent(parts[0]) | |||
| 446 | + | if !ok { | |||
| 447 | + | return -1, -1 | |||
| 448 | + | } | |||
| 449 | + | minor, ok := parseEngineVersionComponent(parts[1]) | |||
| 450 | + | if !ok { | |||
| 451 | + | return -1, -1 | |||
| 452 | + | } | |||
| 453 | + | return major, minor | |||
| 454 | + | } | |||
| 455 | + | major, minor := parse(version) | |||
| 456 | + | minMajor, minMinor := parse(minimum) | |||
| 457 | + | switch { | |||
| 458 | + | case major != minMajor: | |||
| 459 | + | return major - minMajor | |||
| 460 | + | default: | |||
| 461 | + | return minor - minMinor | |||
| 462 | + | } | |||
| 463 | + | } | |||
| 464 | + | ||||
| 465 | + | // parseEngineVersionComponent parses one numeric major/minor component. | |||
| 466 | + | // Anything but digits is refused, so prerelease suffixes such as "-rc1" | |||
| 467 | + | // and malformed versions can never lift an engine above the gate. | |||
| 468 | + | func parseEngineVersionComponent(value string) (int, bool) { | |||
| 469 | + | value = strings.TrimSpace(value) | |||
| 470 | + | if value == "" { | |||
| 471 | + | return 0, false | |||
| 472 | + | } | |||
| 473 | + | for _, r := range value { | |||
| 474 | + | if r < '0' || r > '9' { | |||
| 475 | + | return 0, false | |||
| 476 | + | } | |||
| 477 | + | } | |||
| 478 | + | n, err := strconv.Atoi(value) | |||
| 479 | + | if err != nil { | |||
| 480 | + | return 0, false | |||
| 481 | + | } | |||
| 482 | + | return n, true | |||
| 483 | + | } | |||
| 484 | + | ||||
| 485 | + | // buildDeviceRequests encodes ephemeral CDI device IDs as one native CDI | |||
| 486 | + | // DeviceRequest. Capabilities and Options stay empty; Count stays 0 so the | |||
| 487 | + | // explicit DeviceIDs are the only grant. CDI IDs are never logged or echoed | |||
| 488 | + | // in errors: they are host inventory. | |||
| 489 | + | func buildDeviceRequests(config *domain.ContainerConfig) []container.DeviceRequest { | |||
| 490 | + | if len(config.CDIDevices) == 0 { | |||
| 491 | + | return nil | |||
| 492 | + | } | |||
| 493 | + | ids := append([]string(nil), config.CDIDevices...) | |||
| 494 | + | sort.Strings(ids) | |||
| 495 | + | return []container.DeviceRequest{ | |||
| 496 | + | { | |||
| 497 | + | Driver: "cdi", | |||
| 498 | + | DeviceIDs: ids, | |||
| 499 | + | }, | |||
| 500 | + | } | |||
| 501 | + | } | |||
| 502 | + | ||||
| 337 | // StartContainer starts a container. | 503 | // StartContainer starts a container. | ||
| 338 | func (r *Runtime) StartContainer(ctx context.Context, containerID string) error { | 504 | func (r *Runtime) StartContainer(ctx context.Context, containerID string) error { | ||
| 339 | ctx = zerowrap.CtxWithFields(ctx, map[string]any{ | 505 | ctx = zerowrap.CtxWithFields(ctx, map[string]any{ | ||
A internal/adapters/out/docker/runtime_devices_test.go +124 -0
| @@ -0,0 +1,124 @@ | |||
| 1 | + | package docker | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "context" | |
| 5 | + | "errors" | |
| 6 | + | "fmt" | |
| 7 | + | "testing" | |
| 8 | + | ||
| 9 | + | "github.com/moby/moby/api/types/container" | |
| 10 | + | "github.com/moby/moby/api/types/system" | |
| 11 | + | "github.com/moby/moby/client" | |
| 12 | + | "github.com/stretchr/testify/assert" | |
| 13 | + | "github.com/stretchr/testify/require" | |
| 14 | + | ||
| 15 | + | "github.com/bnema/gordon/internal/domain" | |
| 16 | + | ) | |
| 17 | + | ||
| 18 | + | func podmanVersion(version string) client.ServerVersionResult { | |
| 19 | + | return client.ServerVersionResult{ | |
| 20 | + | Version: version, | |
| 21 | + | Components: []system.ComponentVersion{{Name: "Podman Engine", Version: version}}, | |
| 22 | + | } | |
| 23 | + | } | |
| 24 | + | ||
| 25 | + | func dockerVersion(version string) client.ServerVersionResult { | |
| 26 | + | return client.ServerVersionResult{ | |
| 27 | + | Version: version, | |
| 28 | + | Components: []system.ComponentVersion{{Name: "Engine", Version: version}}, | |
| 29 | + | } | |
| 30 | + | } | |
| 31 | + | ||
| 32 | + | func TestBuildDeviceRequests_EmptyIsNil(t *testing.T) { | |
| 33 | + | assert.Nil(t, buildDeviceRequests(&domain.ContainerConfig{})) | |
| 34 | + | assert.Nil(t, buildDeviceRequests(&domain.ContainerConfig{CDIDevices: nil})) | |
| 35 | + | } | |
| 36 | + | ||
| 37 | + | func TestBuildDeviceRequests_OneNativeCDIRequest(t *testing.T) { | |
| 38 | + | requests := buildDeviceRequests(&domain.ContainerConfig{ | |
| 39 | + | CDIDevices: []string{"example.com/gpu=GPU-b", "example.com/gpu=GPU-a"}, | |
| 40 | + | }) | |
| 41 | + | require.Len(t, requests, 1, "all CDI IDs travel in one native DeviceRequest") | |
| 42 | + | request := requests[0] | |
| 43 | + | assert.Equal(t, container.DeviceRequest{ | |
| 44 | + | Driver: "cdi", | |
| 45 | + | DeviceIDs: []string{"example.com/gpu=GPU-a", "example.com/gpu=GPU-b"}, | |
| 46 | + | }, request, "driver is cdi, capabilities/options stay empty, count stays 0, IDs sorted") | |
| 47 | + | } | |
| 48 | + | ||
| 49 | + | func TestBuildDeviceRequests_DoesNotMutateConfig(t *testing.T) { | |
| 50 | + | config := &domain.ContainerConfig{CDIDevices: []string{"example.com/gpu=GPU-b", "example.com/gpu=GPU-a"}} | |
| 51 | + | _ = buildDeviceRequests(config) | |
| 52 | + | assert.Equal(t, []string{"example.com/gpu=GPU-b", "example.com/gpu=GPU-a"}, config.CDIDevices) | |
| 53 | + | } | |
| 54 | + | ||
| 55 | + | func TestCheckCDISupportMatrix(t *testing.T) { | |
| 56 | + | cases := []struct { | |
| 57 | + | name string | |
| 58 | + | version client.ServerVersionResult | |
| 59 | + | hint string | |
| 60 | + | wantError bool | |
| 61 | + | }{ | |
| 62 | + | {"podman 6.1.1", podmanVersion("6.1.1"), "", false}, | |
| 63 | + | {"podman 5.4 exact", podmanVersion("5.4.0"), "", false}, | |
| 64 | + | {"podman 4.9 refused", podmanVersion("4.9.5"), "", true}, | |
| 65 | + | {"docker 28.3", dockerVersion("28.3.2"), "", false}, | |
| 66 | + | {"docker 27 refused", dockerVersion("27.5.1"), "", true}, | |
| 67 | + | {"unknown engine refused", client.ServerVersionResult{Version: "9.9.9"}, "unknown", true}, | |
| 68 | + | {"socket hint podman accepted", client.ServerVersionResult{Version: "6.1.1"}, "podman", false}, | |
| 69 | + | {"unparsable version refused", podmanVersion("not-a-version"), "", true}, | |
| 70 | + | {"prerelease minor refused", podmanVersion("5.4-rc1"), "", true}, | |
| 71 | + | } | |
| 72 | + | for _, tc := range cases { | |
| 73 | + | t.Run(tc.name, func(t *testing.T) { | |
| 74 | + | err := checkCDISupport(tc.version, tc.hint) | |
| 75 | + | if tc.wantError { | |
| 76 | + | require.Error(t, err) | |
| 77 | + | assert.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |
| 78 | + | } else { | |
| 79 | + | require.NoError(t, err) | |
| 80 | + | } | |
| 81 | + | }) | |
| 82 | + | } | |
| 83 | + | } | |
| 84 | + | ||
| 85 | + | func TestCompareEngineVersion(t *testing.T) { | |
| 86 | + | assert.Greater(t, compareEngineVersion("6.1.1", "5.4"), 0) | |
| 87 | + | assert.Less(t, compareEngineVersion("4.9.5", "5.4"), 0) | |
| 88 | + | assert.Equal(t, 0, compareEngineVersion("5.4.0", "5.4")) | |
| 89 | + | assert.Less(t, compareEngineVersion("not-a-version", "5.4"), 0) | |
| 90 | + | assert.Equal(t, 0, compareEngineVersion("28.3.2", "28.3"), "patch releases do not affect the gate") | |
| 91 | + | ||
| 92 | + | // A prerelease suffix must never lift an engine above the gate: the | |
| 93 | + | // minor component has to be numeric on its own. | |
| 94 | + | assert.Less(t, compareEngineVersion("5.4-rc1", "5.4"), 0) | |
| 95 | + | assert.Less(t, compareEngineVersion("28.3-rc1", "28.3"), 0) | |
| 96 | + | assert.Less(t, compareEngineVersion("5.", "5.4"), 0) | |
| 97 | + | } | |
| 98 | + | ||
| 99 | + | // TestEngineProbeError proves a failed /version probe keeps cancellation | |
| 100 | + | // recognizable while every other cause collapses to the unsupported | |
| 101 | + | // sentinel without the daemon endpoint in the text. | |
| 102 | + | func TestEngineProbeError(t *testing.T) { | |
| 103 | + | t.Run("canceled probe keeps the sentinel and drops the endpoint", func(t *testing.T) { | |
| 104 | + | ctx, cancel := context.WithCancel(context.Background()) | |
| 105 | + | cancel() | |
| 106 | + | ||
| 107 | + | err := engineProbeError(ctx, errors.New("dial unix /run/podman/podman.sock: i/o timeout")) | |
| 108 | + | require.ErrorIs(t, err, context.Canceled) | |
| 109 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |
| 110 | + | assert.NotContains(t, err.Error(), "podman.sock") | |
| 111 | + | }) | |
| 112 | + | ||
| 113 | + | t.Run("canceled adapter cause is preserved", func(t *testing.T) { | |
| 114 | + | err := engineProbeError(context.Background(), fmt.Errorf("probe: %w", context.Canceled)) | |
| 115 | + | require.ErrorIs(t, err, context.Canceled) | |
| 116 | + | }) | |
| 117 | + | ||
| 118 | + | t.Run("other probe failures stay redacted", func(t *testing.T) { | |
| 119 | + | err := engineProbeError(context.Background(), errors.New("dial unix /run/podman/podman.sock: connection refused")) | |
| 120 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |
| 121 | + | assert.False(t, errors.Is(err, context.Canceled)) | |
| 122 | + | assert.NotContains(t, err.Error(), "podman.sock") | |
| 123 | + | }) | |
| 124 | + | } | |
| @@ -0,0 +1,124 @@ | |||||
| 1 | + | package docker | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "context" | |||
| 5 | + | "errors" | |||
| 6 | + | "fmt" | |||
| 7 | + | "testing" | |||
| 8 | + | ||||
| 9 | + | "github.com/moby/moby/api/types/container" | |||
| 10 | + | "github.com/moby/moby/api/types/system" | |||
| 11 | + | "github.com/moby/moby/client" | |||
| 12 | + | "github.com/stretchr/testify/assert" | |||
| 13 | + | "github.com/stretchr/testify/require" | |||
| 14 | + | ||||
| 15 | + | "github.com/bnema/gordon/internal/domain" | |||
| 16 | + | ) | |||
| 17 | + | ||||
| 18 | + | func podmanVersion(version string) client.ServerVersionResult { | |||
| 19 | + | return client.ServerVersionResult{ | |||
| 20 | + | Version: version, | |||
| 21 | + | Components: []system.ComponentVersion{{Name: "Podman Engine", Version: version}}, | |||
| 22 | + | } | |||
| 23 | + | } | |||
| 24 | + | ||||
| 25 | + | func dockerVersion(version string) client.ServerVersionResult { | |||
| 26 | + | return client.ServerVersionResult{ | |||
| 27 | + | Version: version, | |||
| 28 | + | Components: []system.ComponentVersion{{Name: "Engine", Version: version}}, | |||
| 29 | + | } | |||
| 30 | + | } | |||
| 31 | + | ||||
| 32 | + | func TestBuildDeviceRequests_EmptyIsNil(t *testing.T) { | |||
| 33 | + | assert.Nil(t, buildDeviceRequests(&domain.ContainerConfig{})) | |||
| 34 | + | assert.Nil(t, buildDeviceRequests(&domain.ContainerConfig{CDIDevices: nil})) | |||
| 35 | + | } | |||
| 36 | + | ||||
| 37 | + | func TestBuildDeviceRequests_OneNativeCDIRequest(t *testing.T) { | |||
| 38 | + | requests := buildDeviceRequests(&domain.ContainerConfig{ | |||
| 39 | + | CDIDevices: []string{"example.com/gpu=GPU-b", "example.com/gpu=GPU-a"}, | |||
| 40 | + | }) | |||
| 41 | + | require.Len(t, requests, 1, "all CDI IDs travel in one native DeviceRequest") | |||
| 42 | + | request := requests[0] | |||
| 43 | + | assert.Equal(t, container.DeviceRequest{ | |||
| 44 | + | Driver: "cdi", | |||
| 45 | + | DeviceIDs: []string{"example.com/gpu=GPU-a", "example.com/gpu=GPU-b"}, | |||
| 46 | + | }, request, "driver is cdi, capabilities/options stay empty, count stays 0, IDs sorted") | |||
| 47 | + | } | |||
| 48 | + | ||||
| 49 | + | func TestBuildDeviceRequests_DoesNotMutateConfig(t *testing.T) { | |||
| 50 | + | config := &domain.ContainerConfig{CDIDevices: []string{"example.com/gpu=GPU-b", "example.com/gpu=GPU-a"}} | |||
| 51 | + | _ = buildDeviceRequests(config) | |||
| 52 | + | assert.Equal(t, []string{"example.com/gpu=GPU-b", "example.com/gpu=GPU-a"}, config.CDIDevices) | |||
| 53 | + | } | |||
| 54 | + | ||||
| 55 | + | func TestCheckCDISupportMatrix(t *testing.T) { | |||
| 56 | + | cases := []struct { | |||
| 57 | + | name string | |||
| 58 | + | version client.ServerVersionResult | |||
| 59 | + | hint string | |||
| 60 | + | wantError bool | |||
| 61 | + | }{ | |||
| 62 | + | {"podman 6.1.1", podmanVersion("6.1.1"), "", false}, | |||
| 63 | + | {"podman 5.4 exact", podmanVersion("5.4.0"), "", false}, | |||
| 64 | + | {"podman 4.9 refused", podmanVersion("4.9.5"), "", true}, | |||
| 65 | + | {"docker 28.3", dockerVersion("28.3.2"), "", false}, | |||
| 66 | + | {"docker 27 refused", dockerVersion("27.5.1"), "", true}, | |||
| 67 | + | {"unknown engine refused", client.ServerVersionResult{Version: "9.9.9"}, "unknown", true}, | |||
| 68 | + | {"socket hint podman accepted", client.ServerVersionResult{Version: "6.1.1"}, "podman", false}, | |||
| 69 | + | {"unparsable version refused", podmanVersion("not-a-version"), "", true}, | |||
| 70 | + | {"prerelease minor refused", podmanVersion("5.4-rc1"), "", true}, | |||
| 71 | + | } | |||
| 72 | + | for _, tc := range cases { | |||
| 73 | + | t.Run(tc.name, func(t *testing.T) { | |||
| 74 | + | err := checkCDISupport(tc.version, tc.hint) | |||
| 75 | + | if tc.wantError { | |||
| 76 | + | require.Error(t, err) | |||
| 77 | + | assert.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |||
| 78 | + | } else { | |||
| 79 | + | require.NoError(t, err) | |||
| 80 | + | } | |||
| 81 | + | }) | |||
| 82 | + | } | |||
| 83 | + | } | |||
| 84 | + | ||||
| 85 | + | func TestCompareEngineVersion(t *testing.T) { | |||
| 86 | + | assert.Greater(t, compareEngineVersion("6.1.1", "5.4"), 0) | |||
| 87 | + | assert.Less(t, compareEngineVersion("4.9.5", "5.4"), 0) | |||
| 88 | + | assert.Equal(t, 0, compareEngineVersion("5.4.0", "5.4")) | |||
| 89 | + | assert.Less(t, compareEngineVersion("not-a-version", "5.4"), 0) | |||
| 90 | + | assert.Equal(t, 0, compareEngineVersion("28.3.2", "28.3"), "patch releases do not affect the gate") | |||
| 91 | + | ||||
| 92 | + | // A prerelease suffix must never lift an engine above the gate: the | |||
| 93 | + | // minor component has to be numeric on its own. | |||
| 94 | + | assert.Less(t, compareEngineVersion("5.4-rc1", "5.4"), 0) | |||
| 95 | + | assert.Less(t, compareEngineVersion("28.3-rc1", "28.3"), 0) | |||
| 96 | + | assert.Less(t, compareEngineVersion("5.", "5.4"), 0) | |||
| 97 | + | } | |||
| 98 | + | ||||
| 99 | + | // TestEngineProbeError proves a failed /version probe keeps cancellation | |||
| 100 | + | // recognizable while every other cause collapses to the unsupported | |||
| 101 | + | // sentinel without the daemon endpoint in the text. | |||
| 102 | + | func TestEngineProbeError(t *testing.T) { | |||
| 103 | + | t.Run("canceled probe keeps the sentinel and drops the endpoint", func(t *testing.T) { | |||
| 104 | + | ctx, cancel := context.WithCancel(context.Background()) | |||
| 105 | + | cancel() | |||
| 106 | + | ||||
| 107 | + | err := engineProbeError(ctx, errors.New("dial unix /run/podman/podman.sock: i/o timeout")) | |||
| 108 | + | require.ErrorIs(t, err, context.Canceled) | |||
| 109 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |||
| 110 | + | assert.NotContains(t, err.Error(), "podman.sock") | |||
| 111 | + | }) | |||
| 112 | + | ||||
| 113 | + | t.Run("canceled adapter cause is preserved", func(t *testing.T) { | |||
| 114 | + | err := engineProbeError(context.Background(), fmt.Errorf("probe: %w", context.Canceled)) | |||
| 115 | + | require.ErrorIs(t, err, context.Canceled) | |||
| 116 | + | }) | |||
| 117 | + | ||||
| 118 | + | t.Run("other probe failures stay redacted", func(t *testing.T) { | |||
| 119 | + | err := engineProbeError(context.Background(), errors.New("dial unix /run/podman/podman.sock: connection refused")) | |||
| 120 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |||
| 121 | + | assert.False(t, errors.Is(err, context.Canceled)) | |||
| 122 | + | assert.NotContains(t, err.Error(), "podman.sock") | |||
| 123 | + | }) | |||
| 124 | + | } | |||
A internal/app/app_devices.go +107 -0
| @@ -0,0 +1,107 @@ | |||
| 1 | + | package app | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "fmt" | |
| 5 | + | "strings" | |
| 6 | + | ||
| 7 | + | "github.com/spf13/viper" | |
| 8 | + | ||
| 9 | + | "github.com/bnema/gordon/internal/domain" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | // AppDevicePolicy is one administrative device grant declared under | |
| 13 | + | // [app_devices.<name>]. The map key is the stable logical device name an | |
| 14 | + | // app manifest device references. CDI holds the explicit CDI device IDs | |
| 15 | + | // granted (aggregate "=all" selectors are rejected). AllowedApps and | |
| 16 | + | // AllowedServices are exact, non-empty allowlists. | |
| 17 | + | type AppDevicePolicy struct { | |
| 18 | + | CDI []string `mapstructure:"cdi"` | |
| 19 | + | AllowedApps []string `mapstructure:"allowed_apps"` | |
| 20 | + | AllowedServices []string `mapstructure:"allowed_services"` | |
| 21 | + | } | |
| 22 | + | ||
| 23 | + | // knownAppDeviceKeys is the allowlist of [app_devices.<name>] sub-keys. | |
| 24 | + | // Unknown keys (for example a typo like "cdis") fail closed instead of | |
| 25 | + | // silently dropping a grant. | |
| 26 | + | var knownAppDeviceKeys = map[string]struct{}{ | |
| 27 | + | "cdi": {}, | |
| 28 | + | "allowed_apps": {}, | |
| 29 | + | "allowed_services": {}, | |
| 30 | + | } | |
| 31 | + | ||
| 32 | + | // validateAppDeviceKeys rejects unknown sub-keys inside [app_devices.*]. | |
| 33 | + | // Viper unmarshals leniently, so a typo would otherwise drop the grant | |
| 34 | + | // silently. settings is the decoded app_devices subtree (viper lowercases | |
| 35 | + | // all keys) used only for key inspection: values still come from Config. | |
| 36 | + | func validateAppDeviceKeys(devices map[string]any) error { | |
| 37 | + | for name, raw := range devices { | |
| 38 | + | entry, ok := raw.(map[string]any) | |
| 39 | + | if !ok { | |
| 40 | + | return fmt.Errorf("invalid app_devices.%s: entry must be a table", name) | |
| 41 | + | } | |
| 42 | + | for key := range entry { | |
| 43 | + | if _, ok := knownAppDeviceKeys[key]; !ok { | |
| 44 | + | return fmt.Errorf("invalid app_devices.%s: unknown key %q (allowed: cdi, allowed_apps, allowed_services)", name, key) | |
| 45 | + | } | |
| 46 | + | } | |
| 47 | + | } | |
| 48 | + | return nil | |
| 49 | + | } | |
| 50 | + | ||
| 51 | + | // buildAppDevicePolicies converts configured app devices into validated | |
| 52 | + | // domain policies. Failures name only the device and the field-level | |
| 53 | + | // reason: host inventory never appears in config errors. | |
| 54 | + | func buildAppDevicePolicies(cfg Config) (map[string]domain.AppDevicePolicy, error) { | |
| 55 | + | policies := make(map[string]domain.AppDevicePolicy, len(cfg.AppDevices)) | |
| 56 | + | for name, device := range cfg.AppDevices { | |
| 57 | + | policy := domain.AppDevicePolicy{ | |
| 58 | + | Name: name, | |
| 59 | + | CDI: append([]string(nil), device.CDI...), | |
| 60 | + | AllowedApps: append([]string(nil), device.AllowedApps...), | |
| 61 | + | AllowedServices: append([]string(nil), device.AllowedServices...), | |
| 62 | + | } | |
| 63 | + | if err := policy.Validate(); err != nil { | |
| 64 | + | return nil, fmt.Errorf("invalid app_devices.%s: %w: %s", name, domain.ErrDevicePolicy, redactedDevicePolicyReason(err)) | |
| 65 | + | } | |
| 66 | + | policies[name] = policy | |
| 67 | + | } | |
| 68 | + | return policies, nil | |
| 69 | + | } | |
| 70 | + | ||
| 71 | + | // redactedDevicePolicyReason keeps the field-level reason from a domain | |
| 72 | + | // device policy validation error while stripping every quoted value so an | |
| 73 | + | // invalid config never echoes host inventory (CDI IDs, app names). | |
| 74 | + | func redactedDevicePolicyReason(err error) string { | |
| 75 | + | reason := policyQuotedValue.ReplaceAllString(err.Error(), "") | |
| 76 | + | reason = strings.Join(strings.Fields(reason), " ") | |
| 77 | + | // Keep only the innermost detail: the outer wrap repeats the generic | |
| 78 | + | // violation and the policy name, which is already part of the device key. | |
| 79 | + | if idx := strings.LastIndex(reason, ": "); idx >= 0 { | |
| 80 | + | reason = reason[idx+2:] | |
| 81 | + | } | |
| 82 | + | return reason | |
| 83 | + | } | |
| 84 | + | ||
| 85 | + | // validateDeviceConfig validates the [app_devices] subtree: unknown keys | |
| 86 | + | // fail closed, then the decoded policies validate. Split from | |
| 87 | + | // applyLoadedConfig to keep its complexity within budget. | |
| 88 | + | func validateDeviceConfig(v *viper.Viper, cfg Config) error { | |
| 89 | + | if err := validateAppDeviceKeys(v.GetStringMap("app_devices")); err != nil { | |
| 90 | + | return err | |
| 91 | + | } | |
| 92 | + | if _, err := buildAppDevicePolicies(cfg); err != nil { | |
| 93 | + | return err | |
| 94 | + | } | |
| 95 | + | return nil | |
| 96 | + | } | |
| 97 | + | ||
| 98 | + | // validateAppPolicies validates both administrative app policy subtrees | |
| 99 | + | // (bind mounts and device grants) before publication. Failed validation | |
| 100 | + | // keeps the previous policies live. Split from applyLoadedConfig to keep | |
| 101 | + | // its complexity within budget. | |
| 102 | + | func validateAppPolicies(v *viper.Viper, cfg Config) error { | |
| 103 | + | if _, err := buildAppMountPolicies(cfg); err != nil { | |
| 104 | + | return err | |
| 105 | + | } | |
| 106 | + | return validateDeviceConfig(v, cfg) | |
| 107 | + | } | |
| @@ -0,0 +1,107 @@ | |||||
| 1 | + | package app | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "fmt" | |||
| 5 | + | "strings" | |||
| 6 | + | ||||
| 7 | + | "github.com/spf13/viper" | |||
| 8 | + | ||||
| 9 | + | "github.com/bnema/gordon/internal/domain" | |||
| 10 | + | ) | |||
| 11 | + | ||||
| 12 | + | // AppDevicePolicy is one administrative device grant declared under | |||
| 13 | + | // [app_devices.<name>]. The map key is the stable logical device name an | |||
| 14 | + | // app manifest device references. CDI holds the explicit CDI device IDs | |||
| 15 | + | // granted (aggregate "=all" selectors are rejected). AllowedApps and | |||
| 16 | + | // AllowedServices are exact, non-empty allowlists. | |||
| 17 | + | type AppDevicePolicy struct { | |||
| 18 | + | CDI []string `mapstructure:"cdi"` | |||
| 19 | + | AllowedApps []string `mapstructure:"allowed_apps"` | |||
| 20 | + | AllowedServices []string `mapstructure:"allowed_services"` | |||
| 21 | + | } | |||
| 22 | + | ||||
| 23 | + | // knownAppDeviceKeys is the allowlist of [app_devices.<name>] sub-keys. | |||
| 24 | + | // Unknown keys (for example a typo like "cdis") fail closed instead of | |||
| 25 | + | // silently dropping a grant. | |||
| 26 | + | var knownAppDeviceKeys = map[string]struct{}{ | |||
| 27 | + | "cdi": {}, | |||
| 28 | + | "allowed_apps": {}, | |||
| 29 | + | "allowed_services": {}, | |||
| 30 | + | } | |||
| 31 | + | ||||
| 32 | + | // validateAppDeviceKeys rejects unknown sub-keys inside [app_devices.*]. | |||
| 33 | + | // Viper unmarshals leniently, so a typo would otherwise drop the grant | |||
| 34 | + | // silently. settings is the decoded app_devices subtree (viper lowercases | |||
| 35 | + | // all keys) used only for key inspection: values still come from Config. | |||
| 36 | + | func validateAppDeviceKeys(devices map[string]any) error { | |||
| 37 | + | for name, raw := range devices { | |||
| 38 | + | entry, ok := raw.(map[string]any) | |||
| 39 | + | if !ok { | |||
| 40 | + | return fmt.Errorf("invalid app_devices.%s: entry must be a table", name) | |||
| 41 | + | } | |||
| 42 | + | for key := range entry { | |||
| 43 | + | if _, ok := knownAppDeviceKeys[key]; !ok { | |||
| 44 | + | return fmt.Errorf("invalid app_devices.%s: unknown key %q (allowed: cdi, allowed_apps, allowed_services)", name, key) | |||
| 45 | + | } | |||
| 46 | + | } | |||
| 47 | + | } | |||
| 48 | + | return nil | |||
| 49 | + | } | |||
| 50 | + | ||||
| 51 | + | // buildAppDevicePolicies converts configured app devices into validated | |||
| 52 | + | // domain policies. Failures name only the device and the field-level | |||
| 53 | + | // reason: host inventory never appears in config errors. | |||
| 54 | + | func buildAppDevicePolicies(cfg Config) (map[string]domain.AppDevicePolicy, error) { | |||
| 55 | + | policies := make(map[string]domain.AppDevicePolicy, len(cfg.AppDevices)) | |||
| 56 | + | for name, device := range cfg.AppDevices { | |||
| 57 | + | policy := domain.AppDevicePolicy{ | |||
| 58 | + | Name: name, | |||
| 59 | + | CDI: append([]string(nil), device.CDI...), | |||
| 60 | + | AllowedApps: append([]string(nil), device.AllowedApps...), | |||
| 61 | + | AllowedServices: append([]string(nil), device.AllowedServices...), | |||
| 62 | + | } | |||
| 63 | + | if err := policy.Validate(); err != nil { | |||
| 64 | + | return nil, fmt.Errorf("invalid app_devices.%s: %w: %s", name, domain.ErrDevicePolicy, redactedDevicePolicyReason(err)) | |||
| 65 | + | } | |||
| 66 | + | policies[name] = policy | |||
| 67 | + | } | |||
| 68 | + | return policies, nil | |||
| 69 | + | } | |||
| 70 | + | ||||
| 71 | + | // redactedDevicePolicyReason keeps the field-level reason from a domain | |||
| 72 | + | // device policy validation error while stripping every quoted value so an | |||
| 73 | + | // invalid config never echoes host inventory (CDI IDs, app names). | |||
| 74 | + | func redactedDevicePolicyReason(err error) string { | |||
| 75 | + | reason := policyQuotedValue.ReplaceAllString(err.Error(), "") | |||
| 76 | + | reason = strings.Join(strings.Fields(reason), " ") | |||
| 77 | + | // Keep only the innermost detail: the outer wrap repeats the generic | |||
| 78 | + | // violation and the policy name, which is already part of the device key. | |||
| 79 | + | if idx := strings.LastIndex(reason, ": "); idx >= 0 { | |||
| 80 | + | reason = reason[idx+2:] | |||
| 81 | + | } | |||
| 82 | + | return reason | |||
| 83 | + | } | |||
| 84 | + | ||||
| 85 | + | // validateDeviceConfig validates the [app_devices] subtree: unknown keys | |||
| 86 | + | // fail closed, then the decoded policies validate. Split from | |||
| 87 | + | // applyLoadedConfig to keep its complexity within budget. | |||
| 88 | + | func validateDeviceConfig(v *viper.Viper, cfg Config) error { | |||
| 89 | + | if err := validateAppDeviceKeys(v.GetStringMap("app_devices")); err != nil { | |||
| 90 | + | return err | |||
| 91 | + | } | |||
| 92 | + | if _, err := buildAppDevicePolicies(cfg); err != nil { | |||
| 93 | + | return err | |||
| 94 | + | } | |||
| 95 | + | return nil | |||
| 96 | + | } | |||
| 97 | + | ||||
| 98 | + | // validateAppPolicies validates both administrative app policy subtrees | |||
| 99 | + | // (bind mounts and device grants) before publication. Failed validation | |||
| 100 | + | // keeps the previous policies live. Split from applyLoadedConfig to keep | |||
| 101 | + | // its complexity within budget. | |||
| 102 | + | func validateAppPolicies(v *viper.Viper, cfg Config) error { | |||
| 103 | + | if _, err := buildAppMountPolicies(cfg); err != nil { | |||
| 104 | + | return err | |||
| 105 | + | } | |||
| 106 | + | return validateDeviceConfig(v, cfg) | |||
| 107 | + | } | |||
A internal/app/app_devices_test.go +118 -0
| @@ -0,0 +1,118 @@ | |||
| 1 | + | package app | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "strings" | |
| 5 | + | "testing" | |
| 6 | + | ||
| 7 | + | "github.com/spf13/viper" | |
| 8 | + | "github.com/stretchr/testify/assert" | |
| 9 | + | "github.com/stretchr/testify/require" | |
| 10 | + | ||
| 11 | + | "github.com/bnema/gordon/internal/domain" | |
| 12 | + | ) | |
| 13 | + | ||
| 14 | + | func TestBuildAppDevicePolicies_ValidConfigUnmarshal(t *testing.T) { | |
| 15 | + | v := viper.New() | |
| 16 | + | v.SetConfigType("toml") | |
| 17 | + | require.NoError(t, v.ReadConfig(strings.NewReader(` | |
| 18 | + | [app_devices.test_gpu] | |
| 19 | + | cdi = ["example.com/gpu=GPU-test-uuid"] | |
| 20 | + | allowed_apps = ["demo"] | |
| 21 | + | allowed_services = ["worker"] | |
| 22 | + | `))) | |
| 23 | + | ||
| 24 | + | var cfg Config | |
| 25 | + | require.NoError(t, v.Unmarshal(&cfg)) | |
| 26 | + | require.NoError(t, validateAppDeviceKeys(v.GetStringMap("app_devices"))) | |
| 27 | + | ||
| 28 | + | policies, err := buildAppDevicePolicies(cfg) | |
| 29 | + | require.NoError(t, err) | |
| 30 | + | require.Contains(t, policies, "test_gpu") | |
| 31 | + | policy := policies["test_gpu"] | |
| 32 | + | assert.Equal(t, "test_gpu", policy.Name) | |
| 33 | + | assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, policy.CDI) | |
| 34 | + | assert.Equal(t, []string{"demo"}, policy.AllowedApps) | |
| 35 | + | assert.Equal(t, []string{"worker"}, policy.AllowedServices) | |
| 36 | + | } | |
| 37 | + | ||
| 38 | + | func TestBuildAppDevicePolicies_Invalid(t *testing.T) { | |
| 39 | + | cases := []struct { | |
| 40 | + | name string | |
| 41 | + | device AppDevicePolicy | |
| 42 | + | reason string | |
| 43 | + | }{ | |
| 44 | + | {"empty cdi", AppDevicePolicy{AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, "cdi must not be empty"}, | |
| 45 | + | {"raw path", AppDevicePolicy{CDI: []string{"/dev/card0"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, "not a host path"}, | |
| 46 | + | {"aggregate", AppDevicePolicy{CDI: []string{"example.com/gpu=all"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, "not allowed"}, | |
| 47 | + | {"empty allowed apps", AppDevicePolicy{CDI: []string{"example.com/gpu=0"}, AllowedServices: []string{"worker"}}, "allowed apps must not be empty"}, | |
| 48 | + | {"empty allowed services", AppDevicePolicy{CDI: []string{"example.com/gpu=0"}, AllowedApps: []string{"demo"}}, "allowed services must not be empty"}, | |
| 49 | + | {"duplicate cdi", AppDevicePolicy{CDI: []string{"example.com/gpu=0", "example.com/gpu=0"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, "duplicate cdi device"}, | |
| 50 | + | } | |
| 51 | + | for _, tc := range cases { | |
| 52 | + | t.Run(tc.name, func(t *testing.T) { | |
| 53 | + | cfg := Config{AppDevices: map[string]AppDevicePolicy{"test_gpu": tc.device}} | |
| 54 | + | _, err := buildAppDevicePolicies(cfg) | |
| 55 | + | require.Error(t, err) | |
| 56 | + | assert.Contains(t, err.Error(), "app_devices.test_gpu") | |
| 57 | + | assert.Contains(t, err.Error(), tc.reason, "the field-level reason must survive redaction") | |
| 58 | + | assert.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 59 | + | for _, id := range tc.device.CDI { | |
| 60 | + | assert.NotContains(t, err.Error(), id, "config errors must not echo CDI IDs") | |
| 61 | + | } | |
| 62 | + | }) | |
| 63 | + | } | |
| 64 | + | } | |
| 65 | + | ||
| 66 | + | func TestValidateAppDeviceKeys_UnknownKey(t *testing.T) { | |
| 67 | + | v := viper.New() | |
| 68 | + | v.SetConfigType("toml") | |
| 69 | + | require.NoError(t, v.ReadConfig(strings.NewReader(` | |
| 70 | + | [app_devices.test_gpu] | |
| 71 | + | cdis = ["example.com/gpu=0"] | |
| 72 | + | allowed_apps = ["demo"] | |
| 73 | + | allowed_services = ["worker"] | |
| 74 | + | `))) | |
| 75 | + | ||
| 76 | + | err := validateAppDeviceKeys(v.GetStringMap("app_devices")) | |
| 77 | + | require.Error(t, err) | |
| 78 | + | assert.Contains(t, err.Error(), "unknown key") | |
| 79 | + | assert.Contains(t, err.Error(), "app_devices.test_gpu") | |
| 80 | + | } | |
| 81 | + | ||
| 82 | + | func TestValidateAppDeviceKeys_Valid(t *testing.T) { | |
| 83 | + | v := viper.New() | |
| 84 | + | v.SetConfigType("toml") | |
| 85 | + | require.NoError(t, v.ReadConfig(strings.NewReader(` | |
| 86 | + | [app_devices.test_gpu] | |
| 87 | + | cdi = ["example.com/gpu=0"] | |
| 88 | + | allowed_apps = ["demo"] | |
| 89 | + | allowed_services = ["worker"] | |
| 90 | + | `))) | |
| 91 | + | ||
| 92 | + | require.NoError(t, validateAppDeviceKeys(v.GetStringMap("app_devices"))) | |
| 93 | + | require.NoError(t, validateAppDeviceKeys(nil)) | |
| 94 | + | } | |
| 95 | + | ||
| 96 | + | func TestBuildAppDevicePolicies_CopiesSlices(t *testing.T) { | |
| 97 | + | cfg := Config{AppDevices: map[string]AppDevicePolicy{ | |
| 98 | + | "test_gpu": { | |
| 99 | + | CDI: []string{"example.com/gpu=0"}, | |
| 100 | + | AllowedApps: []string{"demo"}, | |
| 101 | + | AllowedServices: []string{"worker"}, | |
| 102 | + | }, | |
| 103 | + | }} | |
| 104 | + | policies, err := buildAppDevicePolicies(cfg) | |
| 105 | + | require.NoError(t, err) | |
| 106 | + | // Mutating the published policy must not affect later builds from the | |
| 107 | + | // same config: reloads always rebuild from Config. | |
| 108 | + | policies["test_gpu"].CDI[0] = "mutated" | |
| 109 | + | again, err := buildAppDevicePolicies(cfg) | |
| 110 | + | require.NoError(t, err) | |
| 111 | + | assert.Equal(t, []string{"example.com/gpu=0"}, again["test_gpu"].CDI) | |
| 112 | + | } | |
| 113 | + | ||
| 114 | + | func TestConfig_AppDevicesDefaultsEmpty(t *testing.T) { | |
| 115 | + | policies, err := buildAppDevicePolicies(Config{}) | |
| 116 | + | require.NoError(t, err) | |
| 117 | + | assert.Empty(t, policies) | |
| 118 | + | } | |
| @@ -0,0 +1,118 @@ | |||||
| 1 | + | package app | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "strings" | |||
| 5 | + | "testing" | |||
| 6 | + | ||||
| 7 | + | "github.com/spf13/viper" | |||
| 8 | + | "github.com/stretchr/testify/assert" | |||
| 9 | + | "github.com/stretchr/testify/require" | |||
| 10 | + | ||||
| 11 | + | "github.com/bnema/gordon/internal/domain" | |||
| 12 | + | ) | |||
| 13 | + | ||||
| 14 | + | func TestBuildAppDevicePolicies_ValidConfigUnmarshal(t *testing.T) { | |||
| 15 | + | v := viper.New() | |||
| 16 | + | v.SetConfigType("toml") | |||
| 17 | + | require.NoError(t, v.ReadConfig(strings.NewReader(` | |||
| 18 | + | [app_devices.test_gpu] | |||
| 19 | + | cdi = ["example.com/gpu=GPU-test-uuid"] | |||
| 20 | + | allowed_apps = ["demo"] | |||
| 21 | + | allowed_services = ["worker"] | |||
| 22 | + | `))) | |||
| 23 | + | ||||
| 24 | + | var cfg Config | |||
| 25 | + | require.NoError(t, v.Unmarshal(&cfg)) | |||
| 26 | + | require.NoError(t, validateAppDeviceKeys(v.GetStringMap("app_devices"))) | |||
| 27 | + | ||||
| 28 | + | policies, err := buildAppDevicePolicies(cfg) | |||
| 29 | + | require.NoError(t, err) | |||
| 30 | + | require.Contains(t, policies, "test_gpu") | |||
| 31 | + | policy := policies["test_gpu"] | |||
| 32 | + | assert.Equal(t, "test_gpu", policy.Name) | |||
| 33 | + | assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, policy.CDI) | |||
| 34 | + | assert.Equal(t, []string{"demo"}, policy.AllowedApps) | |||
| 35 | + | assert.Equal(t, []string{"worker"}, policy.AllowedServices) | |||
| 36 | + | } | |||
| 37 | + | ||||
| 38 | + | func TestBuildAppDevicePolicies_Invalid(t *testing.T) { | |||
| 39 | + | cases := []struct { | |||
| 40 | + | name string | |||
| 41 | + | device AppDevicePolicy | |||
| 42 | + | reason string | |||
| 43 | + | }{ | |||
| 44 | + | {"empty cdi", AppDevicePolicy{AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, "cdi must not be empty"}, | |||
| 45 | + | {"raw path", AppDevicePolicy{CDI: []string{"/dev/card0"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, "not a host path"}, | |||
| 46 | + | {"aggregate", AppDevicePolicy{CDI: []string{"example.com/gpu=all"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, "not allowed"}, | |||
| 47 | + | {"empty allowed apps", AppDevicePolicy{CDI: []string{"example.com/gpu=0"}, AllowedServices: []string{"worker"}}, "allowed apps must not be empty"}, | |||
| 48 | + | {"empty allowed services", AppDevicePolicy{CDI: []string{"example.com/gpu=0"}, AllowedApps: []string{"demo"}}, "allowed services must not be empty"}, | |||
| 49 | + | {"duplicate cdi", AppDevicePolicy{CDI: []string{"example.com/gpu=0", "example.com/gpu=0"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, "duplicate cdi device"}, | |||
| 50 | + | } | |||
| 51 | + | for _, tc := range cases { | |||
| 52 | + | t.Run(tc.name, func(t *testing.T) { | |||
| 53 | + | cfg := Config{AppDevices: map[string]AppDevicePolicy{"test_gpu": tc.device}} | |||
| 54 | + | _, err := buildAppDevicePolicies(cfg) | |||
| 55 | + | require.Error(t, err) | |||
| 56 | + | assert.Contains(t, err.Error(), "app_devices.test_gpu") | |||
| 57 | + | assert.Contains(t, err.Error(), tc.reason, "the field-level reason must survive redaction") | |||
| 58 | + | assert.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 59 | + | for _, id := range tc.device.CDI { | |||
| 60 | + | assert.NotContains(t, err.Error(), id, "config errors must not echo CDI IDs") | |||
| 61 | + | } | |||
| 62 | + | }) | |||
| 63 | + | } | |||
| 64 | + | } | |||
| 65 | + | ||||
| 66 | + | func TestValidateAppDeviceKeys_UnknownKey(t *testing.T) { | |||
| 67 | + | v := viper.New() | |||
| 68 | + | v.SetConfigType("toml") | |||
| 69 | + | require.NoError(t, v.ReadConfig(strings.NewReader(` | |||
| 70 | + | [app_devices.test_gpu] | |||
| 71 | + | cdis = ["example.com/gpu=0"] | |||
| 72 | + | allowed_apps = ["demo"] | |||
| 73 | + | allowed_services = ["worker"] | |||
| 74 | + | `))) | |||
| 75 | + | ||||
| 76 | + | err := validateAppDeviceKeys(v.GetStringMap("app_devices")) | |||
| 77 | + | require.Error(t, err) | |||
| 78 | + | assert.Contains(t, err.Error(), "unknown key") | |||
| 79 | + | assert.Contains(t, err.Error(), "app_devices.test_gpu") | |||
| 80 | + | } | |||
| 81 | + | ||||
| 82 | + | func TestValidateAppDeviceKeys_Valid(t *testing.T) { | |||
| 83 | + | v := viper.New() | |||
| 84 | + | v.SetConfigType("toml") | |||
| 85 | + | require.NoError(t, v.ReadConfig(strings.NewReader(` | |||
| 86 | + | [app_devices.test_gpu] | |||
| 87 | + | cdi = ["example.com/gpu=0"] | |||
| 88 | + | allowed_apps = ["demo"] | |||
| 89 | + | allowed_services = ["worker"] | |||
| 90 | + | `))) | |||
| 91 | + | ||||
| 92 | + | require.NoError(t, validateAppDeviceKeys(v.GetStringMap("app_devices"))) | |||
| 93 | + | require.NoError(t, validateAppDeviceKeys(nil)) | |||
| 94 | + | } | |||
| 95 | + | ||||
| 96 | + | func TestBuildAppDevicePolicies_CopiesSlices(t *testing.T) { | |||
| 97 | + | cfg := Config{AppDevices: map[string]AppDevicePolicy{ | |||
| 98 | + | "test_gpu": { | |||
| 99 | + | CDI: []string{"example.com/gpu=0"}, | |||
| 100 | + | AllowedApps: []string{"demo"}, | |||
| 101 | + | AllowedServices: []string{"worker"}, | |||
| 102 | + | }, | |||
| 103 | + | }} | |||
| 104 | + | policies, err := buildAppDevicePolicies(cfg) | |||
| 105 | + | require.NoError(t, err) | |||
| 106 | + | // Mutating the published policy must not affect later builds from the | |||
| 107 | + | // same config: reloads always rebuild from Config. | |||
| 108 | + | policies["test_gpu"].CDI[0] = "mutated" | |||
| 109 | + | again, err := buildAppDevicePolicies(cfg) | |||
| 110 | + | require.NoError(t, err) | |||
| 111 | + | assert.Equal(t, []string{"example.com/gpu=0"}, again["test_gpu"].CDI) | |||
| 112 | + | } | |||
| 113 | + | ||||
| 114 | + | func TestConfig_AppDevicesDefaultsEmpty(t *testing.T) { | |||
| 115 | + | policies, err := buildAppDevicePolicies(Config{}) | |||
| 116 | + | require.NoError(t, err) | |||
| 117 | + | assert.Empty(t, policies) | |||
| 118 | + | } | |||
M internal/app/run.go +22 -2
| @@ -132,6 +132,11 @@ type Config struct { | |||
| 132 | 132 | // name, referenced by name from an app manifest's [[service.bind]]. | |
| 133 | 133 | AppMounts map[string]AppMountPolicy `mapstructure:"app_mounts"` | |
| 134 | 134 | ||
| 135 | + | // AppDevices declares administrative device grants keyed by stable | |
| 136 | + | // logical device name, referenced by name from an app manifest's | |
| 137 | + | // `devices` list. | |
| 138 | + | AppDevices map[string]AppDevicePolicy `mapstructure:"app_devices"` | |
| 139 | + | ||
| 135 | 140 | Logging struct { | |
| 136 | 141 | Level string `mapstructure:"level"` | |
| 137 | 142 | Format string `mapstructure:"format"` | |
| @@ -448,6 +453,9 @@ func initConfig(configPath string) (*viper.Viper, Config, error) { | |||
| 448 | 453 | if _, err := buildAppMountPolicies(cfg); err != nil { | |
| 449 | 454 | return nil, Config{}, err | |
| 450 | 455 | } | |
| 456 | + | if err := validateDeviceConfig(v, cfg); err != nil { | |
| 457 | + | return nil, Config{}, err | |
| 458 | + | } | |
| 451 | 459 | ||
| 452 | 460 | return v, cfg, nil | |
| 453 | 461 | } | |
| @@ -930,12 +938,18 @@ func (si *serviceInit) initApps() error { | |||
| 930 | 938 | if err != nil { | |
| 931 | 939 | return err | |
| 932 | 940 | } | |
| 941 | + | devicePolicies, err := buildAppDevicePolicies(si.cfg) | |
| 942 | + | if err != nil { | |
| 943 | + | return err | |
| 944 | + | } | |
| 933 | 945 | si.svc.appDeploySvc.SetBindPolicies(mountPolicies) | |
| 946 | + | si.svc.appDeploySvc.SetDevicePolicies(devicePolicies) | |
| 934 | 947 | appSvcImpl := newAppDaemonService(si.ctx, store, si.svc.appDeploySvc, appsecrets.NewStore(si.log), si.log). | |
| 935 | 948 | WithEntrypoints(appEntrypointListeners(si.cfg)). | |
| 936 | 949 | WithGCBarrier(si.svc.gcBarrier). | |
| 937 | 950 | WithImagePolicy(imagePolicy). | |
| 938 | - | WithBindPolicies(mountPolicies) | |
| 951 | + | WithBindPolicies(mountPolicies). | |
| 952 | + | WithDevicePolicies(devicePolicies) | |
| 939 | 953 | si.svc.appSvcImpl = appSvcImpl | |
| 940 | 954 | si.svc.appSvc = appSvcImpl | |
| 941 | 955 | // Health checks resolve from ACTIVE state (loopback backends). | |
| @@ -1085,9 +1099,15 @@ func (si *serviceInit) registerReloadCoordinatorHooks() { | |||
| 1085 | 1099 | if err != nil { | |
| 1086 | 1100 | return err | |
| 1087 | 1101 | } | |
| 1102 | + | devicePolicies, err := buildAppDevicePolicies(reloadCfg) | |
| 1103 | + | if err != nil { | |
| 1104 | + | return err | |
| 1105 | + | } | |
| 1088 | 1106 | si.svc.appSvcImpl.SetBindPolicies(policies) | |
| 1107 | + | si.svc.appSvcImpl.SetDevicePolicies(devicePolicies) | |
| 1089 | 1108 | if si.svc.appDeploySvc != nil { | |
| 1090 | 1109 | si.svc.appDeploySvc.SetBindPolicies(policies) | |
| 1110 | + | si.svc.appDeploySvc.SetDevicePolicies(devicePolicies) | |
| 1091 | 1111 | } | |
| 1092 | 1112 | return nil | |
| 1093 | 1113 | }) | |
| @@ -2062,7 +2082,7 @@ func (c *reloadCoordinator) applyLoadedConfig(ctx context.Context, now time.Time | |||
| 2062 | 2082 | if err := validateRetiredAppConfig(c.v); err != nil { | |
| 2063 | 2083 | return err | |
| 2064 | 2084 | } | |
| 2065 | - | if _, err := buildAppMountPolicies(reloadCfg); err != nil { | |
| 2085 | + | if err := validateAppPolicies(c.v, reloadCfg); err != nil { | |
| 2066 | 2086 | return err | |
| 2067 | 2087 | } | |
| 2068 | 2088 | ||
| @@ -132,6 +132,11 @@ type Config struct { | |||||
| 132 | // name, referenced by name from an app manifest's [[service.bind]]. | 132 | // name, referenced by name from an app manifest's [[service.bind]]. | ||
| 133 | AppMounts map[string]AppMountPolicy `mapstructure:"app_mounts"` | 133 | AppMounts map[string]AppMountPolicy `mapstructure:"app_mounts"` | ||
| 134 | 134 | ||||
| 135 | + | // AppDevices declares administrative device grants keyed by stable | |||
| 136 | + | // logical device name, referenced by name from an app manifest's | |||
| 137 | + | // `devices` list. | |||
| 138 | + | AppDevices map[string]AppDevicePolicy `mapstructure:"app_devices"` | |||
| 139 | + | ||||
| 135 | Logging struct { | 140 | Logging struct { | ||
| 136 | Level string `mapstructure:"level"` | 141 | Level string `mapstructure:"level"` | ||
| 137 | Format string `mapstructure:"format"` | 142 | Format string `mapstructure:"format"` | ||
| @@ -448,6 +453,9 @@ func initConfig(configPath string) (*viper.Viper, Config, error) { | |||||
| 448 | if _, err := buildAppMountPolicies(cfg); err != nil { | 453 | if _, err := buildAppMountPolicies(cfg); err != nil { | ||
| 449 | return nil, Config{}, err | 454 | return nil, Config{}, err | ||
| 450 | } | 455 | } | ||
| 456 | + | if err := validateDeviceConfig(v, cfg); err != nil { | |||
| 457 | + | return nil, Config{}, err | |||
| 458 | + | } | |||
| 451 | 459 | ||||
| 452 | return v, cfg, nil | 460 | return v, cfg, nil | ||
| 453 | } | 461 | } | ||
| @@ -930,12 +938,18 @@ func (si *serviceInit) initApps() error { | |||||
| 930 | if err != nil { | 938 | if err != nil { | ||
| 931 | return err | 939 | return err | ||
| 932 | } | 940 | } | ||
| 941 | + | devicePolicies, err := buildAppDevicePolicies(si.cfg) | |||
| 942 | + | if err != nil { | |||
| 943 | + | return err | |||
| 944 | + | } | |||
| 933 | si.svc.appDeploySvc.SetBindPolicies(mountPolicies) | 945 | si.svc.appDeploySvc.SetBindPolicies(mountPolicies) | ||
| 946 | + | si.svc.appDeploySvc.SetDevicePolicies(devicePolicies) | |||
| 934 | appSvcImpl := newAppDaemonService(si.ctx, store, si.svc.appDeploySvc, appsecrets.NewStore(si.log), si.log). | 947 | appSvcImpl := newAppDaemonService(si.ctx, store, si.svc.appDeploySvc, appsecrets.NewStore(si.log), si.log). | ||
| 935 | WithEntrypoints(appEntrypointListeners(si.cfg)). | 948 | WithEntrypoints(appEntrypointListeners(si.cfg)). | ||
| 936 | WithGCBarrier(si.svc.gcBarrier). | 949 | WithGCBarrier(si.svc.gcBarrier). | ||
| 937 | WithImagePolicy(imagePolicy). | 950 | WithImagePolicy(imagePolicy). | ||
| 938 | - | WithBindPolicies(mountPolicies) | 951 | + | WithBindPolicies(mountPolicies). |
| 952 | + | WithDevicePolicies(devicePolicies) | |||
| 939 | si.svc.appSvcImpl = appSvcImpl | 953 | si.svc.appSvcImpl = appSvcImpl | ||
| 940 | si.svc.appSvc = appSvcImpl | 954 | si.svc.appSvc = appSvcImpl | ||
| 941 | // Health checks resolve from ACTIVE state (loopback backends). | 955 | // Health checks resolve from ACTIVE state (loopback backends). | ||
| @@ -1085,9 +1099,15 @@ func (si *serviceInit) registerReloadCoordinatorHooks() { | |||||
| 1085 | if err != nil { | 1099 | if err != nil { | ||
| 1086 | return err | 1100 | return err | ||
| 1087 | } | 1101 | } | ||
| 1102 | + | devicePolicies, err := buildAppDevicePolicies(reloadCfg) | |||
| 1103 | + | if err != nil { | |||
| 1104 | + | return err | |||
| 1105 | + | } | |||
| 1088 | si.svc.appSvcImpl.SetBindPolicies(policies) | 1106 | si.svc.appSvcImpl.SetBindPolicies(policies) | ||
| 1107 | + | si.svc.appSvcImpl.SetDevicePolicies(devicePolicies) | |||
| 1089 | if si.svc.appDeploySvc != nil { | 1108 | if si.svc.appDeploySvc != nil { | ||
| 1090 | si.svc.appDeploySvc.SetBindPolicies(policies) | 1109 | si.svc.appDeploySvc.SetBindPolicies(policies) | ||
| 1110 | + | si.svc.appDeploySvc.SetDevicePolicies(devicePolicies) | |||
| 1091 | } | 1111 | } | ||
| 1092 | return nil | 1112 | return nil | ||
| 1093 | }) | 1113 | }) | ||
| @@ -2062,7 +2082,7 @@ func (c *reloadCoordinator) applyLoadedConfig(ctx context.Context, now time.Time | |||||
| 2062 | if err := validateRetiredAppConfig(c.v); err != nil { | 2082 | if err := validateRetiredAppConfig(c.v); err != nil { | ||
| 2063 | return err | 2083 | return err | ||
| 2064 | } | 2084 | } | ||
| 2065 | - | if _, err := buildAppMountPolicies(reloadCfg); err != nil { | 2085 | + | if err := validateAppPolicies(c.v, reloadCfg); err != nil { |
| 2066 | return err | 2086 | return err | ||
| 2067 | } | 2087 | } | ||
| 2068 | 2088 | ||||
M internal/boundaries/out/mocks/mock_container_runtime.go +51 -0
| @@ -2599,6 +2599,57 @@ func (_c *MockContainerRuntime_StopContainer_Call) RunAndReturn(run func(ctx con | |||
| 2599 | 2599 | return _c | |
| 2600 | 2600 | } | |
| 2601 | 2601 | ||
| 2602 | + | // SupportsCDIDevices provides a mock function for the type MockContainerRuntime | |
| 2603 | + | func (_mock *MockContainerRuntime) SupportsCDIDevices(ctx context.Context) error { | |
| 2604 | + | ret := _mock.Called(ctx) | |
| 2605 | + | ||
| 2606 | + | if len(ret) == 0 { | |
| 2607 | + | panic("no return value specified for SupportsCDIDevices") | |
| 2608 | + | } | |
| 2609 | + | ||
| 2610 | + | var r0 error | |
| 2611 | + | if returnFunc, ok := ret.Get(0).(func(context.Context) error); ok { | |
| 2612 | + | r0 = returnFunc(ctx) | |
| 2613 | + | } else { | |
| 2614 | + | r0 = ret.Error(0) | |
| 2615 | + | } | |
| 2616 | + | return r0 | |
| 2617 | + | } | |
| 2618 | + | ||
| 2619 | + | // MockContainerRuntime_SupportsCDIDevices_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'SupportsCDIDevices' | |
| 2620 | + | type MockContainerRuntime_SupportsCDIDevices_Call struct { | |
| 2621 | + | *mock.Call | |
| 2622 | + | } | |
| 2623 | + | ||
| 2624 | + | // SupportsCDIDevices is a helper method to define mock.On call | |
| 2625 | + | // - ctx context.Context | |
| 2626 | + | func (_e *MockContainerRuntime_Expecter) SupportsCDIDevices(ctx any) *MockContainerRuntime_SupportsCDIDevices_Call { | |
| 2627 | + | return &MockContainerRuntime_SupportsCDIDevices_Call{Call: _e.mock.On("SupportsCDIDevices", ctx)} | |
| 2628 | + | } | |
| 2629 | + | ||
| 2630 | + | func (_c *MockContainerRuntime_SupportsCDIDevices_Call) Run(run func(ctx context.Context)) *MockContainerRuntime_SupportsCDIDevices_Call { | |
| 2631 | + | _c.Call.Run(func(args mock.Arguments) { | |
| 2632 | + | var arg0 context.Context | |
| 2633 | + | if args[0] != nil { | |
| 2634 | + | arg0 = args[0].(context.Context) | |
| 2635 | + | } | |
| 2636 | + | run( | |
| 2637 | + | arg0, | |
| 2638 | + | ) | |
| 2639 | + | }) | |
| 2640 | + | return _c | |
| 2641 | + | } | |
| 2642 | + | ||
| 2643 | + | func (_c *MockContainerRuntime_SupportsCDIDevices_Call) Return(err error) *MockContainerRuntime_SupportsCDIDevices_Call { | |
| 2644 | + | _c.Call.Return(err) | |
| 2645 | + | return _c | |
| 2646 | + | } | |
| 2647 | + | ||
| 2648 | + | func (_c *MockContainerRuntime_SupportsCDIDevices_Call) RunAndReturn(run func(ctx context.Context) error) *MockContainerRuntime_SupportsCDIDevices_Call { | |
| 2649 | + | _c.Call.Return(run) | |
| 2650 | + | return _c | |
| 2651 | + | } | |
| 2652 | + | ||
| 2602 | 2653 | // TagImage provides a mock function for the type MockContainerRuntime | |
| 2603 | 2654 | func (_mock *MockContainerRuntime) TagImage(ctx context.Context, sourceRef string, targetRef string) error { | |
| 2604 | 2655 | ret := _mock.Called(ctx, sourceRef, targetRef) | |
| @@ -2599,6 +2599,57 @@ func (_c *MockContainerRuntime_StopContainer_Call) RunAndReturn(run func(ctx con | |||||
| 2599 | return _c | 2599 | return _c | ||
| 2600 | } | 2600 | } | ||
| 2601 | 2601 | ||||
| 2602 | + | // SupportsCDIDevices provides a mock function for the type MockContainerRuntime | |||
| 2603 | + | func (_mock *MockContainerRuntime) SupportsCDIDevices(ctx context.Context) error { | |||
| 2604 | + | ret := _mock.Called(ctx) | |||
| 2605 | + | ||||
| 2606 | + | if len(ret) == 0 { | |||
| 2607 | + | panic("no return value specified for SupportsCDIDevices") | |||
| 2608 | + | } | |||
| 2609 | + | ||||
| 2610 | + | var r0 error | |||
| 2611 | + | if returnFunc, ok := ret.Get(0).(func(context.Context) error); ok { | |||
| 2612 | + | r0 = returnFunc(ctx) | |||
| 2613 | + | } else { | |||
| 2614 | + | r0 = ret.Error(0) | |||
| 2615 | + | } | |||
| 2616 | + | return r0 | |||
| 2617 | + | } | |||
| 2618 | + | ||||
| 2619 | + | // MockContainerRuntime_SupportsCDIDevices_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'SupportsCDIDevices' | |||
| 2620 | + | type MockContainerRuntime_SupportsCDIDevices_Call struct { | |||
| 2621 | + | *mock.Call | |||
| 2622 | + | } | |||
| 2623 | + | ||||
| 2624 | + | // SupportsCDIDevices is a helper method to define mock.On call | |||
| 2625 | + | // - ctx context.Context | |||
| 2626 | + | func (_e *MockContainerRuntime_Expecter) SupportsCDIDevices(ctx any) *MockContainerRuntime_SupportsCDIDevices_Call { | |||
| 2627 | + | return &MockContainerRuntime_SupportsCDIDevices_Call{Call: _e.mock.On("SupportsCDIDevices", ctx)} | |||
| 2628 | + | } | |||
| 2629 | + | ||||
| 2630 | + | func (_c *MockContainerRuntime_SupportsCDIDevices_Call) Run(run func(ctx context.Context)) *MockContainerRuntime_SupportsCDIDevices_Call { | |||
| 2631 | + | _c.Call.Run(func(args mock.Arguments) { | |||
| 2632 | + | var arg0 context.Context | |||
| 2633 | + | if args[0] != nil { | |||
| 2634 | + | arg0 = args[0].(context.Context) | |||
| 2635 | + | } | |||
| 2636 | + | run( | |||
| 2637 | + | arg0, | |||
| 2638 | + | ) | |||
| 2639 | + | }) | |||
| 2640 | + | return _c | |||
| 2641 | + | } | |||
| 2642 | + | ||||
| 2643 | + | func (_c *MockContainerRuntime_SupportsCDIDevices_Call) Return(err error) *MockContainerRuntime_SupportsCDIDevices_Call { | |||
| 2644 | + | _c.Call.Return(err) | |||
| 2645 | + | return _c | |||
| 2646 | + | } | |||
| 2647 | + | ||||
| 2648 | + | func (_c *MockContainerRuntime_SupportsCDIDevices_Call) RunAndReturn(run func(ctx context.Context) error) *MockContainerRuntime_SupportsCDIDevices_Call { | |||
| 2649 | + | _c.Call.Return(run) | |||
| 2650 | + | return _c | |||
| 2651 | + | } | |||
| 2652 | + | ||||
| 2602 | // TagImage provides a mock function for the type MockContainerRuntime | 2653 | // TagImage provides a mock function for the type MockContainerRuntime | ||
| 2603 | func (_mock *MockContainerRuntime) TagImage(ctx context.Context, sourceRef string, targetRef string) error { | 2654 | func (_mock *MockContainerRuntime) TagImage(ctx context.Context, sourceRef string, targetRef string) error { | ||
| 2604 | ret := _mock.Called(ctx, sourceRef, targetRef) | 2655 | ret := _mock.Called(ctx, sourceRef, targetRef) | ||
M internal/boundaries/out/runtime.go +7 -0
| @@ -50,6 +50,13 @@ type ContainerRuntime interface { | |||
| 50 | 50 | // Runtime information | |
| 51 | 51 | Ping(ctx context.Context) error | |
| 52 | 52 | Version(ctx context.Context) (string, error) | |
| 53 | + | // SupportsCDIDevices reports whether the connected engine can serve | |
| 54 | + | // native CDI device requests. It fails closed with | |
| 55 | + | // domain.ErrRuntimeUnsupported on engines below the support matrix | |
| 56 | + | // (Podman 5.4+, Docker 28.3+). Callers invoke it before any workload | |
| 57 | + | // mutation so an unsupported engine fails in preflight, never after | |
| 58 | + | // withdrawing the serving generation. | |
| 59 | + | SupportsCDIDevices(ctx context.Context) error | |
| 53 | 60 | ||
| 54 | 61 | // Health and status | |
| 55 | 62 | IsContainerRunning(ctx context.Context, containerID string) (bool, error) | |
| @@ -50,6 +50,13 @@ type ContainerRuntime interface { | |||||
| 50 | // Runtime information | 50 | // Runtime information | ||
| 51 | Ping(ctx context.Context) error | 51 | Ping(ctx context.Context) error | ||
| 52 | Version(ctx context.Context) (string, error) | 52 | Version(ctx context.Context) (string, error) | ||
| 53 | + | // SupportsCDIDevices reports whether the connected engine can serve | |||
| 54 | + | // native CDI device requests. It fails closed with | |||
| 55 | + | // domain.ErrRuntimeUnsupported on engines below the support matrix | |||
| 56 | + | // (Podman 5.4+, Docker 28.3+). Callers invoke it before any workload | |||
| 57 | + | // mutation so an unsupported engine fails in preflight, never after | |||
| 58 | + | // withdrawing the serving generation. | |||
| 59 | + | SupportsCDIDevices(ctx context.Context) error | |||
| 53 | 60 | ||||
| 54 | // Health and status | 61 | // Health and status | ||
| 55 | IsContainerRunning(ctx context.Context, containerID string) (bool, error) | 62 | IsContainerRunning(ctx context.Context, containerID string) (bool, error) | ||
M internal/domain/app_bind_policy.go +1 -1
| @@ -88,7 +88,7 @@ func validateAllowlist(kind string, names []string, validate func(string) error) | |||
| 88 | 88 | seen := map[string]struct{}{} | |
| 89 | 89 | for _, name := range names { | |
| 90 | 90 | if err := validate(name); err != nil { | |
| 91 | - | return fmt.Errorf("allowed %s %q: %v", kind, name, err) | |
| 91 | + | return fmt.Errorf("allowed %s %q: %w", kind, name, err) | |
| 92 | 92 | } | |
| 93 | 93 | if _, ok := seen[name]; ok { | |
| 94 | 94 | return fmt.Errorf("allowed %s %q is duplicated", kind, name) | |
| @@ -88,7 +88,7 @@ func validateAllowlist(kind string, names []string, validate func(string) error) | |||||
| 88 | seen := map[string]struct{}{} | 88 | seen := map[string]struct{}{} | ||
| 89 | for _, name := range names { | 89 | for _, name := range names { | ||
| 90 | if err := validate(name); err != nil { | 90 | if err := validate(name); err != nil { | ||
| 91 | - | return fmt.Errorf("allowed %s %q: %v", kind, name, err) | 91 | + | return fmt.Errorf("allowed %s %q: %w", kind, name, err) |
| 92 | } | 92 | } | ||
| 93 | if _, ok := seen[name]; ok { | 93 | if _, ok := seen[name]; ok { | ||
| 94 | return fmt.Errorf("allowed %s %q is duplicated", kind, name) | 94 | return fmt.Errorf("allowed %s %q is duplicated", kind, name) | ||
A internal/domain/app_device_policy.go +165 -0
| @@ -0,0 +1,165 @@ | |||
| 1 | + | package domain | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "fmt" | |
| 5 | + | "regexp" | |
| 6 | + | "slices" | |
| 7 | + | "strings" | |
| 8 | + | ) | |
| 9 | + | ||
| 10 | + | // cdiDeviceNamePattern matches the CDI device-name grammar: an alphanumeric | |
| 11 | + | // start followed by alphanumerics, '_', '.', or '-'. Host paths and raw | |
| 12 | + | // device nodes never match. | |
| 13 | + | var cdiDeviceNamePattern = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.\-]*$`) | |
| 14 | + | ||
| 15 | + | // AppDevicePolicy is one named administrative device grant declared under | |
| 16 | + | // [app_devices.<name>]. The installation maps a logical device name to | |
| 17 | + | // explicit CDI device IDs and exact app/service allowlists. App manifests | |
| 18 | + | // request devices by logical name; Gordon resolves the CDI IDs at | |
| 19 | + | // activation time. Host CDI resolution is never persisted as desired | |
| 20 | + | // intent: revisions keep the logical names. | |
| 21 | + | type AppDevicePolicy struct { | |
| 22 | + | Name string | |
| 23 | + | // CDI holds the explicit CDI device IDs (for example | |
| 24 | + | // "nvidia.com/gpu=GPU-..."). Aggregate selectors such as "=all" | |
| 25 | + | // are rejected: every granted device must be named explicitly. | |
| 26 | + | CDI []string | |
| 27 | + | // AllowedApps and AllowedServices are exact, non-empty allowlists. | |
| 28 | + | AllowedApps []string | |
| 29 | + | AllowedServices []string | |
| 30 | + | } | |
| 31 | + | ||
| 32 | + | // ValidateDeviceName checks a logical device name (service charset plus -- ban). | |
| 33 | + | func ValidateDeviceName(name string) error { | |
| 34 | + | if !serviceNamePattern.MatchString(name) { | |
| 35 | + | return fmt.Errorf("%w: device name %q must match [a-z0-9_.-], max 63", ErrInvalidAppSpec, name) | |
| 36 | + | } | |
| 37 | + | if strings.Contains(name, "--") { | |
| 38 | + | return fmt.Errorf("%w: device name %q must not contain -- (reserved separator)", ErrInvalidAppSpec, name) | |
| 39 | + | } | |
| 40 | + | return nil | |
| 41 | + | } | |
| 42 | + | ||
| 43 | + | // Validate checks the policy shape without touching the runtime. | |
| 44 | + | func (p AppDevicePolicy) Validate() error { | |
| 45 | + | if err := ValidateDevicePolicyName(p.Name); err != nil { | |
| 46 | + | return err | |
| 47 | + | } | |
| 48 | + | if err := validateDeviceCDIIDs(p.Name, p.CDI); err != nil { | |
| 49 | + | return err | |
| 50 | + | } | |
| 51 | + | if err := validateAllowlist("app", p.AllowedApps, ValidateAppName); err != nil { | |
| 52 | + | return fmt.Errorf("%w: device policy %q: %w", ErrDevicePolicy, p.Name, err) | |
| 53 | + | } | |
| 54 | + | if err := validateAllowlist("service", p.AllowedServices, ValidateServiceName); err != nil { | |
| 55 | + | return fmt.Errorf("%w: device policy %q: %w", ErrDevicePolicy, p.Name, err) | |
| 56 | + | } | |
| 57 | + | return nil | |
| 58 | + | } | |
| 59 | + | ||
| 60 | + | // ValidateDevicePolicyName checks a stable policy name (service charset plus -- ban). | |
| 61 | + | func ValidateDevicePolicyName(name string) error { | |
| 62 | + | if !serviceNamePattern.MatchString(name) { | |
| 63 | + | return fmt.Errorf("%w: device policy name %q must match [a-z0-9_.-], max 63", ErrDevicePolicy, name) | |
| 64 | + | } | |
| 65 | + | if strings.Contains(name, "--") { | |
| 66 | + | return fmt.Errorf("%w: device policy name %q must not contain -- (reserved separator)", ErrDevicePolicy, name) | |
| 67 | + | } | |
| 68 | + | return nil | |
| 69 | + | } | |
| 70 | + | ||
| 71 | + | // validateDeviceCDIIDs requires a non-empty list of unique, well-formed CDI | |
| 72 | + | // device IDs. Raw /dev paths, empty entries, and aggregate "=all" | |
| 73 | + | // selectors are rejected. | |
| 74 | + | func validateDeviceCDIIDs(policy string, ids []string) error { | |
| 75 | + | if len(ids) == 0 { | |
| 76 | + | return fmt.Errorf("%w: device policy %q: cdi must not be empty", ErrDevicePolicy, policy) | |
| 77 | + | } | |
| 78 | + | seen := map[string]struct{}{} | |
| 79 | + | for _, id := range ids { | |
| 80 | + | if err := validateDeviceCDIID(id); err != nil { | |
| 81 | + | return fmt.Errorf("%w: device policy %q: %v", ErrDevicePolicy, policy, err) | |
| 82 | + | } | |
| 83 | + | if _, ok := seen[id]; ok { | |
| 84 | + | return fmt.Errorf("%w: device policy %q: duplicate cdi device %q", ErrDevicePolicy, policy, id) | |
| 85 | + | } | |
| 86 | + | seen[id] = struct{}{} | |
| 87 | + | } | |
| 88 | + | return nil | |
| 89 | + | } | |
| 90 | + | ||
| 91 | + | // validateDeviceCDIID checks one qualified CDI device ID of the form | |
| 92 | + | // "<domain>/<kind>=<name>" (for example "nvidia.com/gpu=GPU-..."). | |
| 93 | + | // The device name must not be empty, "all", or a host path. | |
| 94 | + | func validateDeviceCDIID(id string) error { | |
| 95 | + | if id == "" { | |
| 96 | + | return fmt.Errorf("cdi device must not be empty") | |
| 97 | + | } | |
| 98 | + | if strings.HasPrefix(id, "/") { | |
| 99 | + | return fmt.Errorf("cdi device %q must be a qualified CDI ID, not a host path", id) | |
| 100 | + | } | |
| 101 | + | kind, name, ok := strings.Cut(id, "=") | |
| 102 | + | if !ok || kind == "" || name == "" { | |
| 103 | + | return fmt.Errorf("cdi device %q must be a qualified CDI ID of the form \"<domain>/<kind>=<name>\"", id) | |
| 104 | + | } | |
| 105 | + | if !strings.Contains(kind, "/") { | |
| 106 | + | return fmt.Errorf("cdi device %q must be a qualified CDI ID of the form \"<domain>/<kind>=<name>\"", id) | |
| 107 | + | } | |
| 108 | + | if name == "all" { | |
| 109 | + | return fmt.Errorf("cdi device %q: aggregate \"=all\" selectors are not allowed, name each device explicitly", id) | |
| 110 | + | } | |
| 111 | + | if !cdiDeviceNamePattern.MatchString(name) { | |
| 112 | + | return fmt.Errorf("cdi device %q: name %q must match [A-Za-z0-9][A-Za-z0-9_.-]*, not a host path or raw device node", id, name) | |
| 113 | + | } | |
| 114 | + | return nil | |
| 115 | + | } | |
| 116 | + | ||
| 117 | + | // ResolveAppDevice resolves the logical device grant under this policy | |
| 118 | + | // for the given app and service. It validates the policy shape, enforces | |
| 119 | + | // the exact app/service allowlists, and returns the explicit CDI IDs. | |
| 120 | + | // The result is runtime-ready: callers encode the IDs as one native CDI | |
| 121 | + | // DeviceRequest. | |
| 122 | + | func (p AppDevicePolicy) ResolveAppDevice(app, service string) ([]string, error) { | |
| 123 | + | if err := p.Validate(); err != nil { | |
| 124 | + | return nil, err | |
| 125 | + | } | |
| 126 | + | if !allowlistContains(p.AllowedApps, app) { | |
| 127 | + | return nil, fmt.Errorf("%w: device policy %q does not allow app %q", ErrDevicePolicy, p.Name, app) | |
| 128 | + | } | |
| 129 | + | if !allowlistContains(p.AllowedServices, service) { | |
| 130 | + | return nil, fmt.Errorf("%w: device policy %q does not allow service %q", ErrDevicePolicy, p.Name, service) | |
| 131 | + | } | |
| 132 | + | return slices.Clone(p.CDI), nil | |
| 133 | + | } | |
| 134 | + | ||
| 135 | + | // ResolveAppDevices resolves every logical device in names against the | |
| 136 | + | // given policies for one app service. It returns the deterministic union | |
| 137 | + | // of CDI IDs (sorted) and rejects duplicate resolved IDs across logical | |
| 138 | + | // resources. Errors name only the app, service, and logical resource: | |
| 139 | + | // host inventory never appears. | |
| 140 | + | func ResolveAppDevices(app, service string, names []string, policies map[string]AppDevicePolicy) ([]string, error) { | |
| 141 | + | if len(names) == 0 { | |
| 142 | + | return nil, nil | |
| 143 | + | } | |
| 144 | + | var resolved []string | |
| 145 | + | seen := map[string]struct{}{} | |
| 146 | + | for _, name := range names { | |
| 147 | + | policy, ok := policies[name] | |
| 148 | + | if !ok { | |
| 149 | + | return nil, fmt.Errorf("%w: app %q service %q device %q: no administrative device policy configured", ErrDevicePolicy, app, service, name) | |
| 150 | + | } | |
| 151 | + | ids, err := policy.ResolveAppDevice(app, service) | |
| 152 | + | if err != nil { | |
| 153 | + | return nil, fmt.Errorf("%w: app %q service %q device %q: refused by administrative device policy", ErrDevicePolicy, app, service, name) | |
| 154 | + | } | |
| 155 | + | for _, id := range ids { | |
| 156 | + | if _, dup := seen[id]; dup { | |
| 157 | + | return nil, fmt.Errorf("%w: app %q service %q device %q: cdi device already granted by another device", ErrDevicePolicy, app, service, name) | |
| 158 | + | } | |
| 159 | + | seen[id] = struct{}{} | |
| 160 | + | resolved = append(resolved, id) | |
| 161 | + | } | |
| 162 | + | } | |
| 163 | + | slices.Sort(resolved) | |
| 164 | + | return resolved, nil | |
| 165 | + | } | |
| @@ -0,0 +1,165 @@ | |||||
| 1 | + | package domain | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "fmt" | |||
| 5 | + | "regexp" | |||
| 6 | + | "slices" | |||
| 7 | + | "strings" | |||
| 8 | + | ) | |||
| 9 | + | ||||
| 10 | + | // cdiDeviceNamePattern matches the CDI device-name grammar: an alphanumeric | |||
| 11 | + | // start followed by alphanumerics, '_', '.', or '-'. Host paths and raw | |||
| 12 | + | // device nodes never match. | |||
| 13 | + | var cdiDeviceNamePattern = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.\-]*$`) | |||
| 14 | + | ||||
| 15 | + | // AppDevicePolicy is one named administrative device grant declared under | |||
| 16 | + | // [app_devices.<name>]. The installation maps a logical device name to | |||
| 17 | + | // explicit CDI device IDs and exact app/service allowlists. App manifests | |||
| 18 | + | // request devices by logical name; Gordon resolves the CDI IDs at | |||
| 19 | + | // activation time. Host CDI resolution is never persisted as desired | |||
| 20 | + | // intent: revisions keep the logical names. | |||
| 21 | + | type AppDevicePolicy struct { | |||
| 22 | + | Name string | |||
| 23 | + | // CDI holds the explicit CDI device IDs (for example | |||
| 24 | + | // "nvidia.com/gpu=GPU-..."). Aggregate selectors such as "=all" | |||
| 25 | + | // are rejected: every granted device must be named explicitly. | |||
| 26 | + | CDI []string | |||
| 27 | + | // AllowedApps and AllowedServices are exact, non-empty allowlists. | |||
| 28 | + | AllowedApps []string | |||
| 29 | + | AllowedServices []string | |||
| 30 | + | } | |||
| 31 | + | ||||
| 32 | + | // ValidateDeviceName checks a logical device name (service charset plus -- ban). | |||
| 33 | + | func ValidateDeviceName(name string) error { | |||
| 34 | + | if !serviceNamePattern.MatchString(name) { | |||
| 35 | + | return fmt.Errorf("%w: device name %q must match [a-z0-9_.-], max 63", ErrInvalidAppSpec, name) | |||
| 36 | + | } | |||
| 37 | + | if strings.Contains(name, "--") { | |||
| 38 | + | return fmt.Errorf("%w: device name %q must not contain -- (reserved separator)", ErrInvalidAppSpec, name) | |||
| 39 | + | } | |||
| 40 | + | return nil | |||
| 41 | + | } | |||
| 42 | + | ||||
| 43 | + | // Validate checks the policy shape without touching the runtime. | |||
| 44 | + | func (p AppDevicePolicy) Validate() error { | |||
| 45 | + | if err := ValidateDevicePolicyName(p.Name); err != nil { | |||
| 46 | + | return err | |||
| 47 | + | } | |||
| 48 | + | if err := validateDeviceCDIIDs(p.Name, p.CDI); err != nil { | |||
| 49 | + | return err | |||
| 50 | + | } | |||
| 51 | + | if err := validateAllowlist("app", p.AllowedApps, ValidateAppName); err != nil { | |||
| 52 | + | return fmt.Errorf("%w: device policy %q: %w", ErrDevicePolicy, p.Name, err) | |||
| 53 | + | } | |||
| 54 | + | if err := validateAllowlist("service", p.AllowedServices, ValidateServiceName); err != nil { | |||
| 55 | + | return fmt.Errorf("%w: device policy %q: %w", ErrDevicePolicy, p.Name, err) | |||
| 56 | + | } | |||
| 57 | + | return nil | |||
| 58 | + | } | |||
| 59 | + | ||||
| 60 | + | // ValidateDevicePolicyName checks a stable policy name (service charset plus -- ban). | |||
| 61 | + | func ValidateDevicePolicyName(name string) error { | |||
| 62 | + | if !serviceNamePattern.MatchString(name) { | |||
| 63 | + | return fmt.Errorf("%w: device policy name %q must match [a-z0-9_.-], max 63", ErrDevicePolicy, name) | |||
| 64 | + | } | |||
| 65 | + | if strings.Contains(name, "--") { | |||
| 66 | + | return fmt.Errorf("%w: device policy name %q must not contain -- (reserved separator)", ErrDevicePolicy, name) | |||
| 67 | + | } | |||
| 68 | + | return nil | |||
| 69 | + | } | |||
| 70 | + | ||||
| 71 | + | // validateDeviceCDIIDs requires a non-empty list of unique, well-formed CDI | |||
| 72 | + | // device IDs. Raw /dev paths, empty entries, and aggregate "=all" | |||
| 73 | + | // selectors are rejected. | |||
| 74 | + | func validateDeviceCDIIDs(policy string, ids []string) error { | |||
| 75 | + | if len(ids) == 0 { | |||
| 76 | + | return fmt.Errorf("%w: device policy %q: cdi must not be empty", ErrDevicePolicy, policy) | |||
| 77 | + | } | |||
| 78 | + | seen := map[string]struct{}{} | |||
| 79 | + | for _, id := range ids { | |||
| 80 | + | if err := validateDeviceCDIID(id); err != nil { | |||
| 81 | + | return fmt.Errorf("%w: device policy %q: %v", ErrDevicePolicy, policy, err) | |||
| 82 | + | } | |||
| 83 | + | if _, ok := seen[id]; ok { | |||
| 84 | + | return fmt.Errorf("%w: device policy %q: duplicate cdi device %q", ErrDevicePolicy, policy, id) | |||
| 85 | + | } | |||
| 86 | + | seen[id] = struct{}{} | |||
| 87 | + | } | |||
| 88 | + | return nil | |||
| 89 | + | } | |||
| 90 | + | ||||
| 91 | + | // validateDeviceCDIID checks one qualified CDI device ID of the form | |||
| 92 | + | // "<domain>/<kind>=<name>" (for example "nvidia.com/gpu=GPU-..."). | |||
| 93 | + | // The device name must not be empty, "all", or a host path. | |||
| 94 | + | func validateDeviceCDIID(id string) error { | |||
| 95 | + | if id == "" { | |||
| 96 | + | return fmt.Errorf("cdi device must not be empty") | |||
| 97 | + | } | |||
| 98 | + | if strings.HasPrefix(id, "/") { | |||
| 99 | + | return fmt.Errorf("cdi device %q must be a qualified CDI ID, not a host path", id) | |||
| 100 | + | } | |||
| 101 | + | kind, name, ok := strings.Cut(id, "=") | |||
| 102 | + | if !ok || kind == "" || name == "" { | |||
| 103 | + | return fmt.Errorf("cdi device %q must be a qualified CDI ID of the form \"<domain>/<kind>=<name>\"", id) | |||
| 104 | + | } | |||
| 105 | + | if !strings.Contains(kind, "/") { | |||
| 106 | + | return fmt.Errorf("cdi device %q must be a qualified CDI ID of the form \"<domain>/<kind>=<name>\"", id) | |||
| 107 | + | } | |||
| 108 | + | if name == "all" { | |||
| 109 | + | return fmt.Errorf("cdi device %q: aggregate \"=all\" selectors are not allowed, name each device explicitly", id) | |||
| 110 | + | } | |||
| 111 | + | if !cdiDeviceNamePattern.MatchString(name) { | |||
| 112 | + | return fmt.Errorf("cdi device %q: name %q must match [A-Za-z0-9][A-Za-z0-9_.-]*, not a host path or raw device node", id, name) | |||
| 113 | + | } | |||
| 114 | + | return nil | |||
| 115 | + | } | |||
| 116 | + | ||||
| 117 | + | // ResolveAppDevice resolves the logical device grant under this policy | |||
| 118 | + | // for the given app and service. It validates the policy shape, enforces | |||
| 119 | + | // the exact app/service allowlists, and returns the explicit CDI IDs. | |||
| 120 | + | // The result is runtime-ready: callers encode the IDs as one native CDI | |||
| 121 | + | // DeviceRequest. | |||
| 122 | + | func (p AppDevicePolicy) ResolveAppDevice(app, service string) ([]string, error) { | |||
| 123 | + | if err := p.Validate(); err != nil { | |||
| 124 | + | return nil, err | |||
| 125 | + | } | |||
| 126 | + | if !allowlistContains(p.AllowedApps, app) { | |||
| 127 | + | return nil, fmt.Errorf("%w: device policy %q does not allow app %q", ErrDevicePolicy, p.Name, app) | |||
| 128 | + | } | |||
| 129 | + | if !allowlistContains(p.AllowedServices, service) { | |||
| 130 | + | return nil, fmt.Errorf("%w: device policy %q does not allow service %q", ErrDevicePolicy, p.Name, service) | |||
| 131 | + | } | |||
| 132 | + | return slices.Clone(p.CDI), nil | |||
| 133 | + | } | |||
| 134 | + | ||||
| 135 | + | // ResolveAppDevices resolves every logical device in names against the | |||
| 136 | + | // given policies for one app service. It returns the deterministic union | |||
| 137 | + | // of CDI IDs (sorted) and rejects duplicate resolved IDs across logical | |||
| 138 | + | // resources. Errors name only the app, service, and logical resource: | |||
| 139 | + | // host inventory never appears. | |||
| 140 | + | func ResolveAppDevices(app, service string, names []string, policies map[string]AppDevicePolicy) ([]string, error) { | |||
| 141 | + | if len(names) == 0 { | |||
| 142 | + | return nil, nil | |||
| 143 | + | } | |||
| 144 | + | var resolved []string | |||
| 145 | + | seen := map[string]struct{}{} | |||
| 146 | + | for _, name := range names { | |||
| 147 | + | policy, ok := policies[name] | |||
| 148 | + | if !ok { | |||
| 149 | + | return nil, fmt.Errorf("%w: app %q service %q device %q: no administrative device policy configured", ErrDevicePolicy, app, service, name) | |||
| 150 | + | } | |||
| 151 | + | ids, err := policy.ResolveAppDevice(app, service) | |||
| 152 | + | if err != nil { | |||
| 153 | + | return nil, fmt.Errorf("%w: app %q service %q device %q: refused by administrative device policy", ErrDevicePolicy, app, service, name) | |||
| 154 | + | } | |||
| 155 | + | for _, id := range ids { | |||
| 156 | + | if _, dup := seen[id]; dup { | |||
| 157 | + | return nil, fmt.Errorf("%w: app %q service %q device %q: cdi device already granted by another device", ErrDevicePolicy, app, service, name) | |||
| 158 | + | } | |||
| 159 | + | seen[id] = struct{}{} | |||
| 160 | + | resolved = append(resolved, id) | |||
| 161 | + | } | |||
| 162 | + | } | |||
| 163 | + | slices.Sort(resolved) | |||
| 164 | + | return resolved, nil | |||
| 165 | + | } | |||
A internal/domain/app_device_policy_test.go +188 -0
| @@ -0,0 +1,188 @@ | |||
| 1 | + | package domain | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "errors" | |
| 5 | + | "testing" | |
| 6 | + | "time" | |
| 7 | + | ||
| 8 | + | "github.com/stretchr/testify/assert" | |
| 9 | + | "github.com/stretchr/testify/require" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | func validDevicePolicy() AppDevicePolicy { | |
| 13 | + | return AppDevicePolicy{ | |
| 14 | + | Name: "test_gpu", | |
| 15 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |
| 16 | + | AllowedApps: []string{"demo"}, | |
| 17 | + | AllowedServices: []string{"worker"}, | |
| 18 | + | } | |
| 19 | + | } | |
| 20 | + | ||
| 21 | + | func TestDevicePolicyValid(t *testing.T) { | |
| 22 | + | p := validDevicePolicy() | |
| 23 | + | require.NoError(t, p.Validate()) | |
| 24 | + | } | |
| 25 | + | ||
| 26 | + | func TestDevicePolicyNameRejected(t *testing.T) { | |
| 27 | + | for _, name := range []string{"", "Bad", "a--b", "has space"} { | |
| 28 | + | p := validDevicePolicy() | |
| 29 | + | p.Name = name | |
| 30 | + | err := p.Validate() | |
| 31 | + | require.Error(t, err, "name %q", name) | |
| 32 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |
| 33 | + | } | |
| 34 | + | } | |
| 35 | + | ||
| 36 | + | func TestDevicePolicyCDIRejected(t *testing.T) { | |
| 37 | + | cases := map[string][]string{ | |
| 38 | + | "empty": {}, | |
| 39 | + | "empty entry": {""}, | |
| 40 | + | "raw path": {"/dev/card0"}, | |
| 41 | + | "unqualified": {"gpu0"}, | |
| 42 | + | "missing name": {"example.com/gpu="}, | |
| 43 | + | "missing kind": {"=gpu0"}, | |
| 44 | + | "aggregate": {"example.com/gpu=all"}, | |
| 45 | + | "duplicate": {"example.com/gpu=0", "example.com/gpu=0"}, | |
| 46 | + | "qualified host path": {"example.com/gpu=/dev/card0"}, | |
| 47 | + | } | |
| 48 | + | for name, cdi := range cases { | |
| 49 | + | p := validDevicePolicy() | |
| 50 | + | p.CDI = cdi | |
| 51 | + | err := p.Validate() | |
| 52 | + | require.Error(t, err, "case %s", name) | |
| 53 | + | assert.True(t, errors.Is(err, ErrDevicePolicy), "case %s", name) | |
| 54 | + | } | |
| 55 | + | } | |
| 56 | + | ||
| 57 | + | func TestDevicePolicyAllowlistRejected(t *testing.T) { | |
| 58 | + | p := validDevicePolicy() | |
| 59 | + | p.AllowedApps = nil | |
| 60 | + | err := p.Validate() | |
| 61 | + | require.Error(t, err) | |
| 62 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |
| 63 | + | ||
| 64 | + | p = validDevicePolicy() | |
| 65 | + | p.AllowedApps = []string{"demo", "demo"} | |
| 66 | + | require.Error(t, p.Validate()) | |
| 67 | + | ||
| 68 | + | p = validDevicePolicy() | |
| 69 | + | p.AllowedServices = []string{"worker", "worker"} | |
| 70 | + | require.Error(t, p.Validate()) | |
| 71 | + | ||
| 72 | + | // An invalid allowlist entry is both a policy violation and an invalid | |
| 73 | + | // app spec: callers must be able to detect either sentinel. | |
| 74 | + | p = validDevicePolicy() | |
| 75 | + | p.AllowedServices = []string{"Bad Name"} | |
| 76 | + | err = p.Validate() | |
| 77 | + | require.Error(t, err) | |
| 78 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |
| 79 | + | assert.True(t, errors.Is(err, ErrInvalidAppSpec), "the invalid-spec chain must survive allowlist validation") | |
| 80 | + | } | |
| 81 | + | ||
| 82 | + | func TestResolveAppDeviceDenyByDefault(t *testing.T) { | |
| 83 | + | p := validDevicePolicy() | |
| 84 | + | _, err := p.ResolveAppDevice("other", "worker") | |
| 85 | + | require.Error(t, err) | |
| 86 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |
| 87 | + | ||
| 88 | + | _, err = p.ResolveAppDevice("demo", "helper") | |
| 89 | + | require.Error(t, err) | |
| 90 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |
| 91 | + | } | |
| 92 | + | ||
| 93 | + | func TestResolveAppDeviceExactGrant(t *testing.T) { | |
| 94 | + | p := validDevicePolicy() | |
| 95 | + | ids, err := p.ResolveAppDevice("demo", "worker") | |
| 96 | + | require.NoError(t, err) | |
| 97 | + | assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, ids) | |
| 98 | + | // The returned slice must be a copy: mutating it must not affect | |
| 99 | + | // later resolutions. | |
| 100 | + | ids[0] = "mutated" | |
| 101 | + | again, err := p.ResolveAppDevice("demo", "worker") | |
| 102 | + | require.NoError(t, err) | |
| 103 | + | assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, again) | |
| 104 | + | } | |
| 105 | + | ||
| 106 | + | func TestResolveAppDevicesUnknownDenied(t *testing.T) { | |
| 107 | + | policies := map[string]AppDevicePolicy{"test_gpu": validDevicePolicy()} | |
| 108 | + | _, err := ResolveAppDevices("demo", "worker", []string{"unknown"}, policies) | |
| 109 | + | require.Error(t, err) | |
| 110 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |
| 111 | + | } | |
| 112 | + | ||
| 113 | + | func TestResolveAppDevicesDeterministicUnion(t *testing.T) { | |
| 114 | + | policies := map[string]AppDevicePolicy{ | |
| 115 | + | "b": {Name: "b", CDI: []string{"example.com/gpu=GPU-b"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, | |
| 116 | + | "a": {Name: "a", CDI: []string{"example.com/gpu=GPU-a"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, | |
| 117 | + | } | |
| 118 | + | ids, err := ResolveAppDevices("demo", "worker", []string{"b", "a"}, policies) | |
| 119 | + | require.NoError(t, err) | |
| 120 | + | assert.Equal(t, []string{"example.com/gpu=GPU-a", "example.com/gpu=GPU-b"}, ids) | |
| 121 | + | } | |
| 122 | + | ||
| 123 | + | func TestResolveAppDevicesDuplicateIDRejected(t *testing.T) { | |
| 124 | + | shared := "example.com/gpu=GPU-shared" | |
| 125 | + | policies := map[string]AppDevicePolicy{ | |
| 126 | + | "a": {Name: "a", CDI: []string{shared}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, | |
| 127 | + | "b": {Name: "b", CDI: []string{shared}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, | |
| 128 | + | } | |
| 129 | + | _, err := ResolveAppDevices("demo", "worker", []string{"a", "b"}, policies) | |
| 130 | + | require.Error(t, err) | |
| 131 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |
| 132 | + | assert.NotContains(t, err.Error(), shared, "duplicate-grant errors must not echo resolved CDI IDs") | |
| 133 | + | } | |
| 134 | + | ||
| 135 | + | func TestResolveAppDevicesEmptyIsNil(t *testing.T) { | |
| 136 | + | ids, err := ResolveAppDevices("demo", "worker", nil, nil) | |
| 137 | + | require.NoError(t, err) | |
| 138 | + | assert.Nil(t, ids) | |
| 139 | + | } | |
| 140 | + | ||
| 141 | + | func TestValidateDeviceName(t *testing.T) { | |
| 142 | + | require.NoError(t, ValidateDeviceName("test_gpu")) | |
| 143 | + | for _, name := range []string{"", "Bad", "a--b", "/dev/card0"} { | |
| 144 | + | require.Error(t, ValidateDeviceName(name), "name %q", name) | |
| 145 | + | } | |
| 146 | + | } | |
| 147 | + | ||
| 148 | + | func TestAppServiceDevicesValidation(t *testing.T) { | |
| 149 | + | base := AppSpec{Name: "demo", Services: []AppService{{Name: "worker", Image: "registry.example.com/demo/worker:1", StopGrace: 10 * time.Second, Readiness: AppReadiness{Type: "http", Path: "/healthz", Timeout: 30 * time.Second}}}} | |
| 150 | + | valid := base | |
| 151 | + | valid.Services[0].Devices = []string{"test_gpu"} | |
| 152 | + | require.NoError(t, valid.Validate()) | |
| 153 | + | ||
| 154 | + | dup := base | |
| 155 | + | dup.Services[0].Devices = []string{"test_gpu", "test_gpu"} | |
| 156 | + | err := dup.Validate() | |
| 157 | + | require.Error(t, err) | |
| 158 | + | assert.True(t, errors.Is(err, ErrInvalidAppSpec)) | |
| 159 | + | assert.Contains(t, err.Error(), "duplicate device") | |
| 160 | + | ||
| 161 | + | bad := base | |
| 162 | + | bad.Services[0].Devices = []string{"Bad!"} | |
| 163 | + | err = bad.Validate() | |
| 164 | + | require.Error(t, err) | |
| 165 | + | assert.True(t, errors.Is(err, ErrInvalidAppSpec)) | |
| 166 | + | } | |
| 167 | + | ||
| 168 | + | func TestDiffAppSpecDevices(t *testing.T) { | |
| 169 | + | svc := func(devices ...string) AppService { | |
| 170 | + | return AppService{Name: "worker", Image: "registry.example.com/demo/worker:1", StopGrace: 10 * time.Second, Readiness: AppReadiness{Type: "http", Path: "/healthz", Timeout: 30 * time.Second}, Devices: devices} | |
| 171 | + | } | |
| 172 | + | mk := func(services ...AppService) AppSpec { return AppSpec{Name: "demo", Services: services} } | |
| 173 | + | ||
| 174 | + | // Add/remove report service/worker/devices. | |
| 175 | + | oldSpec, nextSpec := mk(svc()), mk(svc("test_gpu")) | |
| 176 | + | diff := DiffAppSpec(oldSpec, nextSpec) | |
| 177 | + | assert.Contains(t, diff.Changed, "service/worker/devices") | |
| 178 | + | ||
| 179 | + | // Reorder-only is a no-op. | |
| 180 | + | oldSpec, nextSpec = mk(svc("a", "b")), mk(svc("b", "a")) | |
| 181 | + | diff = DiffAppSpec(oldSpec, nextSpec) | |
| 182 | + | assert.NotContains(t, diff.Changed, "service/worker/devices") | |
| 183 | + | ||
| 184 | + | // Nil vs empty stays a no-op. | |
| 185 | + | oldSpec, nextSpec = mk(svc()), mk(svc([]string{}...)) | |
| 186 | + | diff = DiffAppSpec(oldSpec, nextSpec) | |
| 187 | + | assert.NotContains(t, diff.Changed, "service/worker/devices") | |
| 188 | + | } | |
| @@ -0,0 +1,188 @@ | |||||
| 1 | + | package domain | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "errors" | |||
| 5 | + | "testing" | |||
| 6 | + | "time" | |||
| 7 | + | ||||
| 8 | + | "github.com/stretchr/testify/assert" | |||
| 9 | + | "github.com/stretchr/testify/require" | |||
| 10 | + | ) | |||
| 11 | + | ||||
| 12 | + | func validDevicePolicy() AppDevicePolicy { | |||
| 13 | + | return AppDevicePolicy{ | |||
| 14 | + | Name: "test_gpu", | |||
| 15 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |||
| 16 | + | AllowedApps: []string{"demo"}, | |||
| 17 | + | AllowedServices: []string{"worker"}, | |||
| 18 | + | } | |||
| 19 | + | } | |||
| 20 | + | ||||
| 21 | + | func TestDevicePolicyValid(t *testing.T) { | |||
| 22 | + | p := validDevicePolicy() | |||
| 23 | + | require.NoError(t, p.Validate()) | |||
| 24 | + | } | |||
| 25 | + | ||||
| 26 | + | func TestDevicePolicyNameRejected(t *testing.T) { | |||
| 27 | + | for _, name := range []string{"", "Bad", "a--b", "has space"} { | |||
| 28 | + | p := validDevicePolicy() | |||
| 29 | + | p.Name = name | |||
| 30 | + | err := p.Validate() | |||
| 31 | + | require.Error(t, err, "name %q", name) | |||
| 32 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |||
| 33 | + | } | |||
| 34 | + | } | |||
| 35 | + | ||||
| 36 | + | func TestDevicePolicyCDIRejected(t *testing.T) { | |||
| 37 | + | cases := map[string][]string{ | |||
| 38 | + | "empty": {}, | |||
| 39 | + | "empty entry": {""}, | |||
| 40 | + | "raw path": {"/dev/card0"}, | |||
| 41 | + | "unqualified": {"gpu0"}, | |||
| 42 | + | "missing name": {"example.com/gpu="}, | |||
| 43 | + | "missing kind": {"=gpu0"}, | |||
| 44 | + | "aggregate": {"example.com/gpu=all"}, | |||
| 45 | + | "duplicate": {"example.com/gpu=0", "example.com/gpu=0"}, | |||
| 46 | + | "qualified host path": {"example.com/gpu=/dev/card0"}, | |||
| 47 | + | } | |||
| 48 | + | for name, cdi := range cases { | |||
| 49 | + | p := validDevicePolicy() | |||
| 50 | + | p.CDI = cdi | |||
| 51 | + | err := p.Validate() | |||
| 52 | + | require.Error(t, err, "case %s", name) | |||
| 53 | + | assert.True(t, errors.Is(err, ErrDevicePolicy), "case %s", name) | |||
| 54 | + | } | |||
| 55 | + | } | |||
| 56 | + | ||||
| 57 | + | func TestDevicePolicyAllowlistRejected(t *testing.T) { | |||
| 58 | + | p := validDevicePolicy() | |||
| 59 | + | p.AllowedApps = nil | |||
| 60 | + | err := p.Validate() | |||
| 61 | + | require.Error(t, err) | |||
| 62 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |||
| 63 | + | ||||
| 64 | + | p = validDevicePolicy() | |||
| 65 | + | p.AllowedApps = []string{"demo", "demo"} | |||
| 66 | + | require.Error(t, p.Validate()) | |||
| 67 | + | ||||
| 68 | + | p = validDevicePolicy() | |||
| 69 | + | p.AllowedServices = []string{"worker", "worker"} | |||
| 70 | + | require.Error(t, p.Validate()) | |||
| 71 | + | ||||
| 72 | + | // An invalid allowlist entry is both a policy violation and an invalid | |||
| 73 | + | // app spec: callers must be able to detect either sentinel. | |||
| 74 | + | p = validDevicePolicy() | |||
| 75 | + | p.AllowedServices = []string{"Bad Name"} | |||
| 76 | + | err = p.Validate() | |||
| 77 | + | require.Error(t, err) | |||
| 78 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |||
| 79 | + | assert.True(t, errors.Is(err, ErrInvalidAppSpec), "the invalid-spec chain must survive allowlist validation") | |||
| 80 | + | } | |||
| 81 | + | ||||
| 82 | + | func TestResolveAppDeviceDenyByDefault(t *testing.T) { | |||
| 83 | + | p := validDevicePolicy() | |||
| 84 | + | _, err := p.ResolveAppDevice("other", "worker") | |||
| 85 | + | require.Error(t, err) | |||
| 86 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |||
| 87 | + | ||||
| 88 | + | _, err = p.ResolveAppDevice("demo", "helper") | |||
| 89 | + | require.Error(t, err) | |||
| 90 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |||
| 91 | + | } | |||
| 92 | + | ||||
| 93 | + | func TestResolveAppDeviceExactGrant(t *testing.T) { | |||
| 94 | + | p := validDevicePolicy() | |||
| 95 | + | ids, err := p.ResolveAppDevice("demo", "worker") | |||
| 96 | + | require.NoError(t, err) | |||
| 97 | + | assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, ids) | |||
| 98 | + | // The returned slice must be a copy: mutating it must not affect | |||
| 99 | + | // later resolutions. | |||
| 100 | + | ids[0] = "mutated" | |||
| 101 | + | again, err := p.ResolveAppDevice("demo", "worker") | |||
| 102 | + | require.NoError(t, err) | |||
| 103 | + | assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, again) | |||
| 104 | + | } | |||
| 105 | + | ||||
| 106 | + | func TestResolveAppDevicesUnknownDenied(t *testing.T) { | |||
| 107 | + | policies := map[string]AppDevicePolicy{"test_gpu": validDevicePolicy()} | |||
| 108 | + | _, err := ResolveAppDevices("demo", "worker", []string{"unknown"}, policies) | |||
| 109 | + | require.Error(t, err) | |||
| 110 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |||
| 111 | + | } | |||
| 112 | + | ||||
| 113 | + | func TestResolveAppDevicesDeterministicUnion(t *testing.T) { | |||
| 114 | + | policies := map[string]AppDevicePolicy{ | |||
| 115 | + | "b": {Name: "b", CDI: []string{"example.com/gpu=GPU-b"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, | |||
| 116 | + | "a": {Name: "a", CDI: []string{"example.com/gpu=GPU-a"}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, | |||
| 117 | + | } | |||
| 118 | + | ids, err := ResolveAppDevices("demo", "worker", []string{"b", "a"}, policies) | |||
| 119 | + | require.NoError(t, err) | |||
| 120 | + | assert.Equal(t, []string{"example.com/gpu=GPU-a", "example.com/gpu=GPU-b"}, ids) | |||
| 121 | + | } | |||
| 122 | + | ||||
| 123 | + | func TestResolveAppDevicesDuplicateIDRejected(t *testing.T) { | |||
| 124 | + | shared := "example.com/gpu=GPU-shared" | |||
| 125 | + | policies := map[string]AppDevicePolicy{ | |||
| 126 | + | "a": {Name: "a", CDI: []string{shared}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, | |||
| 127 | + | "b": {Name: "b", CDI: []string{shared}, AllowedApps: []string{"demo"}, AllowedServices: []string{"worker"}}, | |||
| 128 | + | } | |||
| 129 | + | _, err := ResolveAppDevices("demo", "worker", []string{"a", "b"}, policies) | |||
| 130 | + | require.Error(t, err) | |||
| 131 | + | assert.True(t, errors.Is(err, ErrDevicePolicy)) | |||
| 132 | + | assert.NotContains(t, err.Error(), shared, "duplicate-grant errors must not echo resolved CDI IDs") | |||
| 133 | + | } | |||
| 134 | + | ||||
| 135 | + | func TestResolveAppDevicesEmptyIsNil(t *testing.T) { | |||
| 136 | + | ids, err := ResolveAppDevices("demo", "worker", nil, nil) | |||
| 137 | + | require.NoError(t, err) | |||
| 138 | + | assert.Nil(t, ids) | |||
| 139 | + | } | |||
| 140 | + | ||||
| 141 | + | func TestValidateDeviceName(t *testing.T) { | |||
| 142 | + | require.NoError(t, ValidateDeviceName("test_gpu")) | |||
| 143 | + | for _, name := range []string{"", "Bad", "a--b", "/dev/card0"} { | |||
| 144 | + | require.Error(t, ValidateDeviceName(name), "name %q", name) | |||
| 145 | + | } | |||
| 146 | + | } | |||
| 147 | + | ||||
| 148 | + | func TestAppServiceDevicesValidation(t *testing.T) { | |||
| 149 | + | base := AppSpec{Name: "demo", Services: []AppService{{Name: "worker", Image: "registry.example.com/demo/worker:1", StopGrace: 10 * time.Second, Readiness: AppReadiness{Type: "http", Path: "/healthz", Timeout: 30 * time.Second}}}} | |||
| 150 | + | valid := base | |||
| 151 | + | valid.Services[0].Devices = []string{"test_gpu"} | |||
| 152 | + | require.NoError(t, valid.Validate()) | |||
| 153 | + | ||||
| 154 | + | dup := base | |||
| 155 | + | dup.Services[0].Devices = []string{"test_gpu", "test_gpu"} | |||
| 156 | + | err := dup.Validate() | |||
| 157 | + | require.Error(t, err) | |||
| 158 | + | assert.True(t, errors.Is(err, ErrInvalidAppSpec)) | |||
| 159 | + | assert.Contains(t, err.Error(), "duplicate device") | |||
| 160 | + | ||||
| 161 | + | bad := base | |||
| 162 | + | bad.Services[0].Devices = []string{"Bad!"} | |||
| 163 | + | err = bad.Validate() | |||
| 164 | + | require.Error(t, err) | |||
| 165 | + | assert.True(t, errors.Is(err, ErrInvalidAppSpec)) | |||
| 166 | + | } | |||
| 167 | + | ||||
| 168 | + | func TestDiffAppSpecDevices(t *testing.T) { | |||
| 169 | + | svc := func(devices ...string) AppService { | |||
| 170 | + | return AppService{Name: "worker", Image: "registry.example.com/demo/worker:1", StopGrace: 10 * time.Second, Readiness: AppReadiness{Type: "http", Path: "/healthz", Timeout: 30 * time.Second}, Devices: devices} | |||
| 171 | + | } | |||
| 172 | + | mk := func(services ...AppService) AppSpec { return AppSpec{Name: "demo", Services: services} } | |||
| 173 | + | ||||
| 174 | + | // Add/remove report service/worker/devices. | |||
| 175 | + | oldSpec, nextSpec := mk(svc()), mk(svc("test_gpu")) | |||
| 176 | + | diff := DiffAppSpec(oldSpec, nextSpec) | |||
| 177 | + | assert.Contains(t, diff.Changed, "service/worker/devices") | |||
| 178 | + | ||||
| 179 | + | // Reorder-only is a no-op. | |||
| 180 | + | oldSpec, nextSpec = mk(svc("a", "b")), mk(svc("b", "a")) | |||
| 181 | + | diff = DiffAppSpec(oldSpec, nextSpec) | |||
| 182 | + | assert.NotContains(t, diff.Changed, "service/worker/devices") | |||
| 183 | + | ||||
| 184 | + | // Nil vs empty stays a no-op. | |||
| 185 | + | oldSpec, nextSpec = mk(svc()), mk(svc([]string{}...)) | |||
| 186 | + | diff = DiffAppSpec(oldSpec, nextSpec) | |||
| 187 | + | assert.NotContains(t, diff.Changed, "service/worker/devices") | |||
| 188 | + | } | |||
M internal/domain/app_spec.go +47 -0
| @@ -6,6 +6,7 @@ import ( | |||
| 6 | 6 | "path" | |
| 7 | 7 | "reflect" | |
| 8 | 8 | "regexp" | |
| 9 | + | "slices" | |
| 9 | 10 | "sort" | |
| 10 | 11 | "strconv" | |
| 11 | 12 | "strings" | |
| @@ -104,6 +105,10 @@ type AppService struct { | |||
| 104 | 105 | Secrets map[string]string | |
| 105 | 106 | Volumes []AppVolume | |
| 106 | 107 | Binds []AppBind | |
| 108 | + | // Devices lists logical device names granted by administrative | |
| 109 | + | // [app_devices] policy. Names resolve to CDI device IDs at activation | |
| 110 | + | // time; the logical names (not host resolution) persist in revisions. | |
| 111 | + | Devices []string | |
| 107 | 112 | Databases []AppDatabase | |
| 108 | 113 | Backup AppBackup | |
| 109 | 114 | } | |
| @@ -525,6 +530,9 @@ func (s *AppService) validate() error { | |||
| 525 | 530 | if err := s.validateBinds(); err != nil { | |
| 526 | 531 | return err | |
| 527 | 532 | } | |
| 533 | + | if err := s.validateDevices(); err != nil { | |
| 534 | + | return err | |
| 535 | + | } | |
| 528 | 536 | if err := s.validateDatabases(volumes); err != nil { | |
| 529 | 537 | return err | |
| 530 | 538 | } | |
| @@ -645,6 +653,23 @@ func (s *AppService) validateBinds() error { | |||
| 645 | 653 | return nil | |
| 646 | 654 | } | |
| 647 | 655 | ||
| 656 | + | // validateDevices checks logical device names and duplicates. Authorization | |
| 657 | + | // against administrative policy happens at apply/deploy time, not here: | |
| 658 | + | // the manifest shape stays valid while policy may refuse to serve it. | |
| 659 | + | func (s *AppService) validateDevices() error { | |
| 660 | + | seen := map[string]struct{}{} | |
| 661 | + | for _, name := range s.Devices { | |
| 662 | + | if err := ValidateDeviceName(name); err != nil { | |
| 663 | + | return fmt.Errorf("%w: service %q: %v", ErrInvalidAppSpec, s.Name, err) | |
| 664 | + | } | |
| 665 | + | if _, ok := seen[name]; ok { | |
| 666 | + | return fmt.Errorf("%w: service %q duplicate device %q", ErrInvalidAppSpec, s.Name, name) | |
| 667 | + | } | |
| 668 | + | seen[name] = struct{}{} | |
| 669 | + | } | |
| 670 | + | return nil | |
| 671 | + | } | |
| 672 | + | ||
| 648 | 673 | // validateDatabases checks database declarations and backup references. | |
| 649 | 674 | func (s *AppService) validateDatabases(seenVolumes map[string]struct{}) error { | |
| 650 | 675 | seenDBs := map[string]struct{}{} | |
| @@ -1039,6 +1064,7 @@ func diffService(name string, desired, effective AppService) []string { | |||
| 1039 | 1064 | if !equalBinds(desired.Binds, effective.Binds) { | |
| 1040 | 1065 | changed = append(changed, "service/"+name+"/binds") | |
| 1041 | 1066 | } | |
| 1067 | + | changed = appendDeviceChanges(changed, name, desired.Devices, effective.Devices) | |
| 1042 | 1068 | if !reflect.DeepEqual(desired.Databases, effective.Databases) { | |
| 1043 | 1069 | changed = append(changed, "service/"+name+"/databases") | |
| 1044 | 1070 | } | |
| @@ -1134,3 +1160,24 @@ func equalBinds(a, b []AppBind) bool { | |||
| 1134 | 1160 | } | |
| 1135 | 1161 | return true | |
| 1136 | 1162 | } | |
| 1163 | + | ||
| 1164 | + | // equalDevices compares device slices as sets: reorder-only input is a | |
| 1165 | + | // no-op, while add/remove reshuffles the sorted comparison. | |
| 1166 | + | func equalDevices(a, b []string) bool { | |
| 1167 | + | if len(a) != len(b) { | |
| 1168 | + | return false | |
| 1169 | + | } | |
| 1170 | + | sortedA := slices.Sorted(slices.Values(a)) | |
| 1171 | + | sortedB := slices.Sorted(slices.Values(b)) | |
| 1172 | + | return slices.Equal(sortedA, sortedB) | |
| 1173 | + | } | |
| 1174 | + | ||
| 1175 | + | // appendDeviceChanges appends the service devices diff entry when the | |
| 1176 | + | // device sets differ. Split from diffService to keep its complexity | |
| 1177 | + | // within budget. | |
| 1178 | + | func appendDeviceChanges(changed []string, name string, desired, effective []string) []string { | |
| 1179 | + | if !equalDevices(desired, effective) { | |
| 1180 | + | changed = append(changed, "service/"+name+"/devices") | |
| 1181 | + | } | |
| 1182 | + | return changed | |
| 1183 | + | } | |
| @@ -6,6 +6,7 @@ import ( | |||||
| 6 | "path" | 6 | "path" | ||
| 7 | "reflect" | 7 | "reflect" | ||
| 8 | "regexp" | 8 | "regexp" | ||
| 9 | + | "slices" | |||
| 9 | "sort" | 10 | "sort" | ||
| 10 | "strconv" | 11 | "strconv" | ||
| 11 | "strings" | 12 | "strings" | ||
| @@ -104,6 +105,10 @@ type AppService struct { | |||||
| 104 | Secrets map[string]string | 105 | Secrets map[string]string | ||
| 105 | Volumes []AppVolume | 106 | Volumes []AppVolume | ||
| 106 | Binds []AppBind | 107 | Binds []AppBind | ||
| 108 | + | // Devices lists logical device names granted by administrative | |||
| 109 | + | // [app_devices] policy. Names resolve to CDI device IDs at activation | |||
| 110 | + | // time; the logical names (not host resolution) persist in revisions. | |||
| 111 | + | Devices []string | |||
| 107 | Databases []AppDatabase | 112 | Databases []AppDatabase | ||
| 108 | Backup AppBackup | 113 | Backup AppBackup | ||
| 109 | } | 114 | } | ||
| @@ -525,6 +530,9 @@ func (s *AppService) validate() error { | |||||
| 525 | if err := s.validateBinds(); err != nil { | 530 | if err := s.validateBinds(); err != nil { | ||
| 526 | return err | 531 | return err | ||
| 527 | } | 532 | } | ||
| 533 | + | if err := s.validateDevices(); err != nil { | |||
| 534 | + | return err | |||
| 535 | + | } | |||
| 528 | if err := s.validateDatabases(volumes); err != nil { | 536 | if err := s.validateDatabases(volumes); err != nil { | ||
| 529 | return err | 537 | return err | ||
| 530 | } | 538 | } | ||
| @@ -645,6 +653,23 @@ func (s *AppService) validateBinds() error { | |||||
| 645 | return nil | 653 | return nil | ||
| 646 | } | 654 | } | ||
| 647 | 655 | ||||
| 656 | + | // validateDevices checks logical device names and duplicates. Authorization | |||
| 657 | + | // against administrative policy happens at apply/deploy time, not here: | |||
| 658 | + | // the manifest shape stays valid while policy may refuse to serve it. | |||
| 659 | + | func (s *AppService) validateDevices() error { | |||
| 660 | + | seen := map[string]struct{}{} | |||
| 661 | + | for _, name := range s.Devices { | |||
| 662 | + | if err := ValidateDeviceName(name); err != nil { | |||
| 663 | + | return fmt.Errorf("%w: service %q: %v", ErrInvalidAppSpec, s.Name, err) | |||
| 664 | + | } | |||
| 665 | + | if _, ok := seen[name]; ok { | |||
| 666 | + | return fmt.Errorf("%w: service %q duplicate device %q", ErrInvalidAppSpec, s.Name, name) | |||
| 667 | + | } | |||
| 668 | + | seen[name] = struct{}{} | |||
| 669 | + | } | |||
| 670 | + | return nil | |||
| 671 | + | } | |||
| 672 | + | ||||
| 648 | // validateDatabases checks database declarations and backup references. | 673 | // validateDatabases checks database declarations and backup references. | ||
| 649 | func (s *AppService) validateDatabases(seenVolumes map[string]struct{}) error { | 674 | func (s *AppService) validateDatabases(seenVolumes map[string]struct{}) error { | ||
| 650 | seenDBs := map[string]struct{}{} | 675 | seenDBs := map[string]struct{}{} | ||
| @@ -1039,6 +1064,7 @@ func diffService(name string, desired, effective AppService) []string { | |||||
| 1039 | if !equalBinds(desired.Binds, effective.Binds) { | 1064 | if !equalBinds(desired.Binds, effective.Binds) { | ||
| 1040 | changed = append(changed, "service/"+name+"/binds") | 1065 | changed = append(changed, "service/"+name+"/binds") | ||
| 1041 | } | 1066 | } | ||
| 1067 | + | changed = appendDeviceChanges(changed, name, desired.Devices, effective.Devices) | |||
| 1042 | if !reflect.DeepEqual(desired.Databases, effective.Databases) { | 1068 | if !reflect.DeepEqual(desired.Databases, effective.Databases) { | ||
| 1043 | changed = append(changed, "service/"+name+"/databases") | 1069 | changed = append(changed, "service/"+name+"/databases") | ||
| 1044 | } | 1070 | } | ||
| @@ -1134,3 +1160,24 @@ func equalBinds(a, b []AppBind) bool { | |||||
| 1134 | } | 1160 | } | ||
| 1135 | return true | 1161 | return true | ||
| 1136 | } | 1162 | } | ||
| 1163 | + | ||||
| 1164 | + | // equalDevices compares device slices as sets: reorder-only input is a | |||
| 1165 | + | // no-op, while add/remove reshuffles the sorted comparison. | |||
| 1166 | + | func equalDevices(a, b []string) bool { | |||
| 1167 | + | if len(a) != len(b) { | |||
| 1168 | + | return false | |||
| 1169 | + | } | |||
| 1170 | + | sortedA := slices.Sorted(slices.Values(a)) | |||
| 1171 | + | sortedB := slices.Sorted(slices.Values(b)) | |||
| 1172 | + | return slices.Equal(sortedA, sortedB) | |||
| 1173 | + | } | |||
| 1174 | + | ||||
| 1175 | + | // appendDeviceChanges appends the service devices diff entry when the | |||
| 1176 | + | // device sets differ. Split from diffService to keep its complexity | |||
| 1177 | + | // within budget. | |||
| 1178 | + | func appendDeviceChanges(changed []string, name string, desired, effective []string) []string { | |||
| 1179 | + | if !equalDevices(desired, effective) { | |||
| 1180 | + | changed = append(changed, "service/"+name+"/devices") | |||
| 1181 | + | } | |||
| 1182 | + | return changed | |||
| 1183 | + | } | |||
M internal/domain/container.go +14 -10
| @@ -116,16 +116,20 @@ type ContainerConfig struct { | |||
| 116 | 116 | Volumes map[string]string // map[containerPath]volumeName | |
| 117 | 117 | ReadOnlyVolumes map[string]string // containerPath -> volumeName (mounted read-only) | |
| 118 | 118 | Binds []ContainerBind // ephemeral resolved host binds, keyed by destination | |
| 119 | - | NetworkMode string // Network to join | |
| 120 | - | Hostname string // Container hostname for DNS | |
| 121 | - | Aliases []string // Additional network aliases | |
| 122 | - | MemoryLimit int64 // Memory limit in bytes (0 = no limit) | |
| 123 | - | NanoCPUs int64 // CPU quota in nanoseconds (1e9 = 1 core, 0 = no limit) | |
| 124 | - | PidsLimit int64 // Max number of PIDs (0 = no limit) | |
| 125 | - | ReadOnlyRootFS bool // Mount container root filesystem read-only | |
| 126 | - | User string // User to run as | |
| 127 | - | CapDrop []string // Linux capabilities to drop; nil uses runtime compat defaults | |
| 128 | - | CapAdd []string // Linux capabilities to add; nil uses runtime compat defaults | |
| 119 | + | // CDIDevices holds explicit CDI device IDs resolved from administrative | |
| 120 | + | // device policy at activation time. It is ephemeral like Binds: never | |
| 121 | + | // persisted, never logged by the runtime adapter. | |
| 122 | + | CDIDevices []string // ephemeral resolved CDI IDs, encoded as one native CDI DeviceRequest | |
| 123 | + | NetworkMode string // Network to join | |
| 124 | + | Hostname string // Container hostname for DNS | |
| 125 | + | Aliases []string // Additional network aliases | |
| 126 | + | MemoryLimit int64 // Memory limit in bytes (0 = no limit) | |
| 127 | + | NanoCPUs int64 // CPU quota in nanoseconds (1e9 = 1 core, 0 = no limit) | |
| 128 | + | PidsLimit int64 // Max number of PIDs (0 = no limit) | |
| 129 | + | ReadOnlyRootFS bool // Mount container root filesystem read-only | |
| 130 | + | User string // User to run as | |
| 131 | + | CapDrop []string // Linux capabilities to drop; nil uses runtime compat defaults | |
| 132 | + | CapAdd []string // Linux capabilities to add; nil uses runtime compat defaults | |
| 129 | 133 | } | |
| 130 | 134 | ||
| 131 | 135 | // ContainerStatus represents the current state of a container. | |
| @@ -116,16 +116,20 @@ type ContainerConfig struct { | |||||
| 116 | Volumes map[string]string // map[containerPath]volumeName | 116 | Volumes map[string]string // map[containerPath]volumeName | ||
| 117 | ReadOnlyVolumes map[string]string // containerPath -> volumeName (mounted read-only) | 117 | ReadOnlyVolumes map[string]string // containerPath -> volumeName (mounted read-only) | ||
| 118 | Binds []ContainerBind // ephemeral resolved host binds, keyed by destination | 118 | Binds []ContainerBind // ephemeral resolved host binds, keyed by destination | ||
| 119 | - | NetworkMode string // Network to join | 119 | + | // CDIDevices holds explicit CDI device IDs resolved from administrative |
| 120 | - | Hostname string // Container hostname for DNS | 120 | + | // device policy at activation time. It is ephemeral like Binds: never |
| 121 | - | Aliases []string // Additional network aliases | 121 | + | // persisted, never logged by the runtime adapter. |
| 122 | - | MemoryLimit int64 // Memory limit in bytes (0 = no limit) | 122 | + | CDIDevices []string // ephemeral resolved CDI IDs, encoded as one native CDI DeviceRequest |
| 123 | - | NanoCPUs int64 // CPU quota in nanoseconds (1e9 = 1 core, 0 = no limit) | 123 | + | NetworkMode string // Network to join |
| 124 | - | PidsLimit int64 // Max number of PIDs (0 = no limit) | 124 | + | Hostname string // Container hostname for DNS |
| 125 | - | ReadOnlyRootFS bool // Mount container root filesystem read-only | 125 | + | Aliases []string // Additional network aliases |
| 126 | - | User string // User to run as | 126 | + | MemoryLimit int64 // Memory limit in bytes (0 = no limit) |
| 127 | - | CapDrop []string // Linux capabilities to drop; nil uses runtime compat defaults | 127 | + | NanoCPUs int64 // CPU quota in nanoseconds (1e9 = 1 core, 0 = no limit) |
| 128 | - | CapAdd []string // Linux capabilities to add; nil uses runtime compat defaults | 128 | + | PidsLimit int64 // Max number of PIDs (0 = no limit) |
| 129 | + | ReadOnlyRootFS bool // Mount container root filesystem read-only | |||
| 130 | + | User string // User to run as | |||
| 131 | + | CapDrop []string // Linux capabilities to drop; nil uses runtime compat defaults | |||
| 132 | + | CapAdd []string // Linux capabilities to add; nil uses runtime compat defaults | |||
| 129 | } | 133 | } | ||
| 130 | 134 | ||||
| 131 | // ContainerStatus represents the current state of a container. | 135 | // ContainerStatus represents the current state of a container. | ||
M internal/domain/errors.go +8 -0
| @@ -11,6 +11,10 @@ var ( | |||
| 11 | 11 | ErrContainerNotRunning = errors.New("container is not running") | |
| 12 | 12 | ErrContainerRunning = errors.New("container is already running") | |
| 13 | 13 | ErrContainerExited = errors.New("container exited") | |
| 14 | + | // ErrRuntimeUnsupported wraps device-bearing create failures on | |
| 15 | + | // engines that cannot serve CDI device requests. It fails closed: | |
| 16 | + | // Gordon never retries without devices. | |
| 17 | + | ErrRuntimeUnsupported = errors.New("runtime does not support CDI device requests") | |
| 14 | 18 | ||
| 15 | 19 | // Image errors | |
| 16 | 20 | ErrImageNotFound = errors.New("image not found") | |
| @@ -75,6 +79,10 @@ var ( | |||
| 75 | 79 | // distinct from ErrInvalidAppSpec: the manifest may be valid while the | |
| 76 | 80 | // installation policy refuses to serve it. | |
| 77 | 81 | ErrBindPolicy = errors.New("bind policy violation") | |
| 82 | + | // ErrDevicePolicy wraps administrative device policy violations. It is | |
| 83 | + | // distinct from ErrInvalidAppSpec: the manifest may be valid while the | |
| 84 | + | // installation policy refuses to serve it. | |
| 85 | + | ErrDevicePolicy = errors.New("device policy violation") | |
| 78 | 86 | ||
| 79 | 87 | // App state errors | |
| 80 | 88 | ErrAppStateIO = errors.New("app state storage failure") | |
| @@ -11,6 +11,10 @@ var ( | |||||
| 11 | ErrContainerNotRunning = errors.New("container is not running") | 11 | ErrContainerNotRunning = errors.New("container is not running") | ||
| 12 | ErrContainerRunning = errors.New("container is already running") | 12 | ErrContainerRunning = errors.New("container is already running") | ||
| 13 | ErrContainerExited = errors.New("container exited") | 13 | ErrContainerExited = errors.New("container exited") | ||
| 14 | + | // ErrRuntimeUnsupported wraps device-bearing create failures on | |||
| 15 | + | // engines that cannot serve CDI device requests. It fails closed: | |||
| 16 | + | // Gordon never retries without devices. | |||
| 17 | + | ErrRuntimeUnsupported = errors.New("runtime does not support CDI device requests") | |||
| 14 | 18 | ||||
| 15 | // Image errors | 19 | // Image errors | ||
| 16 | ErrImageNotFound = errors.New("image not found") | 20 | ErrImageNotFound = errors.New("image not found") | ||
| @@ -75,6 +79,10 @@ var ( | |||||
| 75 | // distinct from ErrInvalidAppSpec: the manifest may be valid while the | 79 | // distinct from ErrInvalidAppSpec: the manifest may be valid while the | ||
| 76 | // installation policy refuses to serve it. | 80 | // installation policy refuses to serve it. | ||
| 77 | ErrBindPolicy = errors.New("bind policy violation") | 81 | ErrBindPolicy = errors.New("bind policy violation") | ||
| 82 | + | // ErrDevicePolicy wraps administrative device policy violations. It is | |||
| 83 | + | // distinct from ErrInvalidAppSpec: the manifest may be valid while the | |||
| 84 | + | // installation policy refuses to serve it. | |||
| 85 | + | ErrDevicePolicy = errors.New("device policy violation") | |||
| 78 | 86 | ||||
| 79 | // App state errors | 87 | // App state errors | ||
| 80 | ErrAppStateIO = errors.New("app state storage failure") | 88 | ErrAppStateIO = errors.New("app state storage failure") | ||
M internal/usecase/apps/app_service.go +12 -0
| @@ -187,6 +187,18 @@ func (s *AppServiceImpl) SetBindPolicies(policies map[string]domain.AppBindPolic | |||
| 187 | 187 | s.core.SetBindPolicies(policies) | |
| 188 | 188 | } | |
| 189 | 189 | ||
| 190 | + | // WithDevicePolicies supplies the administrative device policies used at apply time. | |
| 191 | + | func (s *AppServiceImpl) WithDevicePolicies(policies map[string]domain.AppDevicePolicy) *AppServiceImpl { | |
| 192 | + | s.core.SetDevicePolicies(policies) | |
| 193 | + | return s | |
| 194 | + | } | |
| 195 | + | ||
| 196 | + | // SetDevicePolicies atomically replaces the administrative device policies. It is | |
| 197 | + | // safe to call from a config reload while applies are in flight. | |
| 198 | + | func (s *AppServiceImpl) SetDevicePolicies(policies map[string]domain.AppDevicePolicy) { | |
| 199 | + | s.core.SetDevicePolicies(policies) | |
| 200 | + | } | |
| 201 | + | ||
| 190 | 202 | var _ in.AppService = (*AppServiceImpl)(nil) | |
| 191 | 203 | ||
| 192 | 204 | // Apply implements in.AppService. | |
| @@ -187,6 +187,18 @@ func (s *AppServiceImpl) SetBindPolicies(policies map[string]domain.AppBindPolic | |||||
| 187 | s.core.SetBindPolicies(policies) | 187 | s.core.SetBindPolicies(policies) | ||
| 188 | } | 188 | } | ||
| 189 | 189 | ||||
| 190 | + | // WithDevicePolicies supplies the administrative device policies used at apply time. | |||
| 191 | + | func (s *AppServiceImpl) WithDevicePolicies(policies map[string]domain.AppDevicePolicy) *AppServiceImpl { | |||
| 192 | + | s.core.SetDevicePolicies(policies) | |||
| 193 | + | return s | |||
| 194 | + | } | |||
| 195 | + | ||||
| 196 | + | // SetDevicePolicies atomically replaces the administrative device policies. It is | |||
| 197 | + | // safe to call from a config reload while applies are in flight. | |||
| 198 | + | func (s *AppServiceImpl) SetDevicePolicies(policies map[string]domain.AppDevicePolicy) { | |||
| 199 | + | s.core.SetDevicePolicies(policies) | |||
| 200 | + | } | |||
| 201 | + | ||||
| 190 | var _ in.AppService = (*AppServiceImpl)(nil) | 202 | var _ in.AppService = (*AppServiceImpl)(nil) | ||
| 191 | 203 | ||||
| 192 | // Apply implements in.AppService. | 204 | // Apply implements in.AppService. | ||
A internal/usecase/apps/device_policy_test.go +129 -0
| @@ -0,0 +1,129 @@ | |||
| 1 | + | package apps_test | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "context" | |
| 5 | + | "testing" | |
| 6 | + | ||
| 7 | + | "github.com/bnema/zerowrap" | |
| 8 | + | "github.com/stretchr/testify/assert" | |
| 9 | + | "github.com/stretchr/testify/require" | |
| 10 | + | ||
| 11 | + | outmocks "github.com/bnema/gordon/internal/boundaries/out/mocks" | |
| 12 | + | "github.com/bnema/gordon/internal/domain" | |
| 13 | + | "github.com/bnema/gordon/internal/usecase/apps" | |
| 14 | + | ) | |
| 15 | + | ||
| 16 | + | func devicePolicy(apps, services []string) domain.AppDevicePolicy { | |
| 17 | + | return domain.AppDevicePolicy{ | |
| 18 | + | Name: "test_gpu", | |
| 19 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |
| 20 | + | AllowedApps: apps, | |
| 21 | + | AllowedServices: services, | |
| 22 | + | } | |
| 23 | + | } | |
| 24 | + | ||
| 25 | + | func TestApply_AcceptsDevicesUnderAdministrativePolicy(t *testing.T) { | |
| 26 | + | store := outmocks.NewMockAppState(t) | |
| 27 | + | spec := testSpec("blog") | |
| 28 | + | spec.Services[0].Devices = []string{"test_gpu"} | |
| 29 | + | applySuccess(store, spec) | |
| 30 | + | ||
| 31 | + | svc := apps.NewService(store, zerowrap.Default()). | |
| 32 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 33 | + | "test_gpu": devicePolicy([]string{"blog"}, []string{"web"}), | |
| 34 | + | }) | |
| 35 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |
| 36 | + | require.NoError(t, err) | |
| 37 | + | } | |
| 38 | + | ||
| 39 | + | func TestApply_RejectsDeviceWithoutPolicyBeforeStoreAccess(t *testing.T) { | |
| 40 | + | store := outmocks.NewMockAppState(t) | |
| 41 | + | spec := testSpec("blog") | |
| 42 | + | spec.Services[0].Devices = []string{"test_gpu"} | |
| 43 | + | svc := apps.NewService(store, zerowrap.Default()) | |
| 44 | + | ||
| 45 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |
| 46 | + | ||
| 47 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 48 | + | assert.Contains(t, err.Error(), "blog") | |
| 49 | + | assert.Contains(t, err.Error(), "web") | |
| 50 | + | assert.Contains(t, err.Error(), "test_gpu") | |
| 51 | + | } | |
| 52 | + | ||
| 53 | + | func TestApply_RejectsDeviceOutsideAllowlists(t *testing.T) { | |
| 54 | + | spec := testSpec("blog") | |
| 55 | + | spec.Services[0].Devices = []string{"test_gpu"} | |
| 56 | + | ||
| 57 | + | t.Run("app not allowed", func(t *testing.T) { | |
| 58 | + | store := outmocks.NewMockAppState(t) | |
| 59 | + | svc := apps.NewService(store, zerowrap.Default()). | |
| 60 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 61 | + | "test_gpu": devicePolicy([]string{"other"}, []string{"web"}), | |
| 62 | + | }) | |
| 63 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |
| 64 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 65 | + | assert.NotContains(t, err.Error(), "GPU-test-uuid") | |
| 66 | + | }) | |
| 67 | + | ||
| 68 | + | t.Run("service not allowed", func(t *testing.T) { | |
| 69 | + | store := outmocks.NewMockAppState(t) | |
| 70 | + | svc := apps.NewService(store, zerowrap.Default()). | |
| 71 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 72 | + | "test_gpu": devicePolicy([]string{"blog"}, []string{"other"}), | |
| 73 | + | }) | |
| 74 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |
| 75 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 76 | + | assert.NotContains(t, err.Error(), "GPU-test-uuid") | |
| 77 | + | }) | |
| 78 | + | } | |
| 79 | + | ||
| 80 | + | // TestApply_DevicePoliciesReplaceOnReload proves a reload-style replacement | |
| 81 | + | // takes effect immediately: the second apply is refused before any store | |
| 82 | + | // access. | |
| 83 | + | func TestApply_DevicePoliciesReplaceOnReload(t *testing.T) { | |
| 84 | + | spec := testSpec("blog") | |
| 85 | + | spec.Services[0].Devices = []string{"test_gpu"} | |
| 86 | + | ||
| 87 | + | store := outmocks.NewMockAppState(t) | |
| 88 | + | applySuccess(store, spec) | |
| 89 | + | svc := apps.NewService(store, zerowrap.Default()). | |
| 90 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 91 | + | "test_gpu": devicePolicy([]string{"blog"}, []string{"web"}), | |
| 92 | + | }) | |
| 93 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |
| 94 | + | require.NoError(t, err) | |
| 95 | + | ||
| 96 | + | svc.SetDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 97 | + | "test_gpu": devicePolicy([]string{"other"}, []string{"web"}), | |
| 98 | + | }) | |
| 99 | + | _, _, err = svc.Apply(context.Background(), spec, []byte("manifest"), false) | |
| 100 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 101 | + | } | |
| 102 | + | ||
| 103 | + | func TestApply_DeviceDryRunRefusedWithoutMutation(t *testing.T) { | |
| 104 | + | store := outmocks.NewMockAppState(t) | |
| 105 | + | spec := testSpec("blog") | |
| 106 | + | spec.Services[0].Devices = []string{"test_gpu"} | |
| 107 | + | svc := apps.NewService(store, zerowrap.Default()) | |
| 108 | + | ||
| 109 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), true) | |
| 110 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 111 | + | store.AssertNotCalled(t, "StageApply", context.Background(), spec.Name, uint64(0)) | |
| 112 | + | } | |
| 113 | + | ||
| 114 | + | func TestApply_RejectsDuplicateCDIIDsAcrossDevices(t *testing.T) { | |
| 115 | + | shared := "example.com/gpu=GPU-shared" | |
| 116 | + | store := outmocks.NewMockAppState(t) | |
| 117 | + | spec := testSpec("blog") | |
| 118 | + | spec.Services[0].Devices = []string{"gpu-a", "gpu-b"} | |
| 119 | + | svc := apps.NewService(store, zerowrap.Default()). | |
| 120 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 121 | + | "gpu-a": {Name: "gpu-a", CDI: []string{shared}, AllowedApps: []string{"blog"}, AllowedServices: []string{"web"}}, | |
| 122 | + | "gpu-b": {Name: "gpu-b", CDI: []string{shared}, AllowedApps: []string{"blog"}, AllowedServices: []string{"web"}}, | |
| 123 | + | }) | |
| 124 | + | ||
| 125 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |
| 126 | + | ||
| 127 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 128 | + | assert.NotContains(t, err.Error(), shared, "apply errors must never leak CDI IDs") | |
| 129 | + | } | |
| @@ -0,0 +1,129 @@ | |||||
| 1 | + | package apps_test | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "context" | |||
| 5 | + | "testing" | |||
| 6 | + | ||||
| 7 | + | "github.com/bnema/zerowrap" | |||
| 8 | + | "github.com/stretchr/testify/assert" | |||
| 9 | + | "github.com/stretchr/testify/require" | |||
| 10 | + | ||||
| 11 | + | outmocks "github.com/bnema/gordon/internal/boundaries/out/mocks" | |||
| 12 | + | "github.com/bnema/gordon/internal/domain" | |||
| 13 | + | "github.com/bnema/gordon/internal/usecase/apps" | |||
| 14 | + | ) | |||
| 15 | + | ||||
| 16 | + | func devicePolicy(apps, services []string) domain.AppDevicePolicy { | |||
| 17 | + | return domain.AppDevicePolicy{ | |||
| 18 | + | Name: "test_gpu", | |||
| 19 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |||
| 20 | + | AllowedApps: apps, | |||
| 21 | + | AllowedServices: services, | |||
| 22 | + | } | |||
| 23 | + | } | |||
| 24 | + | ||||
| 25 | + | func TestApply_AcceptsDevicesUnderAdministrativePolicy(t *testing.T) { | |||
| 26 | + | store := outmocks.NewMockAppState(t) | |||
| 27 | + | spec := testSpec("blog") | |||
| 28 | + | spec.Services[0].Devices = []string{"test_gpu"} | |||
| 29 | + | applySuccess(store, spec) | |||
| 30 | + | ||||
| 31 | + | svc := apps.NewService(store, zerowrap.Default()). | |||
| 32 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 33 | + | "test_gpu": devicePolicy([]string{"blog"}, []string{"web"}), | |||
| 34 | + | }) | |||
| 35 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |||
| 36 | + | require.NoError(t, err) | |||
| 37 | + | } | |||
| 38 | + | ||||
| 39 | + | func TestApply_RejectsDeviceWithoutPolicyBeforeStoreAccess(t *testing.T) { | |||
| 40 | + | store := outmocks.NewMockAppState(t) | |||
| 41 | + | spec := testSpec("blog") | |||
| 42 | + | spec.Services[0].Devices = []string{"test_gpu"} | |||
| 43 | + | svc := apps.NewService(store, zerowrap.Default()) | |||
| 44 | + | ||||
| 45 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |||
| 46 | + | ||||
| 47 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 48 | + | assert.Contains(t, err.Error(), "blog") | |||
| 49 | + | assert.Contains(t, err.Error(), "web") | |||
| 50 | + | assert.Contains(t, err.Error(), "test_gpu") | |||
| 51 | + | } | |||
| 52 | + | ||||
| 53 | + | func TestApply_RejectsDeviceOutsideAllowlists(t *testing.T) { | |||
| 54 | + | spec := testSpec("blog") | |||
| 55 | + | spec.Services[0].Devices = []string{"test_gpu"} | |||
| 56 | + | ||||
| 57 | + | t.Run("app not allowed", func(t *testing.T) { | |||
| 58 | + | store := outmocks.NewMockAppState(t) | |||
| 59 | + | svc := apps.NewService(store, zerowrap.Default()). | |||
| 60 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 61 | + | "test_gpu": devicePolicy([]string{"other"}, []string{"web"}), | |||
| 62 | + | }) | |||
| 63 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |||
| 64 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 65 | + | assert.NotContains(t, err.Error(), "GPU-test-uuid") | |||
| 66 | + | }) | |||
| 67 | + | ||||
| 68 | + | t.Run("service not allowed", func(t *testing.T) { | |||
| 69 | + | store := outmocks.NewMockAppState(t) | |||
| 70 | + | svc := apps.NewService(store, zerowrap.Default()). | |||
| 71 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 72 | + | "test_gpu": devicePolicy([]string{"blog"}, []string{"other"}), | |||
| 73 | + | }) | |||
| 74 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |||
| 75 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 76 | + | assert.NotContains(t, err.Error(), "GPU-test-uuid") | |||
| 77 | + | }) | |||
| 78 | + | } | |||
| 79 | + | ||||
| 80 | + | // TestApply_DevicePoliciesReplaceOnReload proves a reload-style replacement | |||
| 81 | + | // takes effect immediately: the second apply is refused before any store | |||
| 82 | + | // access. | |||
| 83 | + | func TestApply_DevicePoliciesReplaceOnReload(t *testing.T) { | |||
| 84 | + | spec := testSpec("blog") | |||
| 85 | + | spec.Services[0].Devices = []string{"test_gpu"} | |||
| 86 | + | ||||
| 87 | + | store := outmocks.NewMockAppState(t) | |||
| 88 | + | applySuccess(store, spec) | |||
| 89 | + | svc := apps.NewService(store, zerowrap.Default()). | |||
| 90 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 91 | + | "test_gpu": devicePolicy([]string{"blog"}, []string{"web"}), | |||
| 92 | + | }) | |||
| 93 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |||
| 94 | + | require.NoError(t, err) | |||
| 95 | + | ||||
| 96 | + | svc.SetDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 97 | + | "test_gpu": devicePolicy([]string{"other"}, []string{"web"}), | |||
| 98 | + | }) | |||
| 99 | + | _, _, err = svc.Apply(context.Background(), spec, []byte("manifest"), false) | |||
| 100 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 101 | + | } | |||
| 102 | + | ||||
| 103 | + | func TestApply_DeviceDryRunRefusedWithoutMutation(t *testing.T) { | |||
| 104 | + | store := outmocks.NewMockAppState(t) | |||
| 105 | + | spec := testSpec("blog") | |||
| 106 | + | spec.Services[0].Devices = []string{"test_gpu"} | |||
| 107 | + | svc := apps.NewService(store, zerowrap.Default()) | |||
| 108 | + | ||||
| 109 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), true) | |||
| 110 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 111 | + | store.AssertNotCalled(t, "StageApply", context.Background(), spec.Name, uint64(0)) | |||
| 112 | + | } | |||
| 113 | + | ||||
| 114 | + | func TestApply_RejectsDuplicateCDIIDsAcrossDevices(t *testing.T) { | |||
| 115 | + | shared := "example.com/gpu=GPU-shared" | |||
| 116 | + | store := outmocks.NewMockAppState(t) | |||
| 117 | + | spec := testSpec("blog") | |||
| 118 | + | spec.Services[0].Devices = []string{"gpu-a", "gpu-b"} | |||
| 119 | + | svc := apps.NewService(store, zerowrap.Default()). | |||
| 120 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 121 | + | "gpu-a": {Name: "gpu-a", CDI: []string{shared}, AllowedApps: []string{"blog"}, AllowedServices: []string{"web"}}, | |||
| 122 | + | "gpu-b": {Name: "gpu-b", CDI: []string{shared}, AllowedApps: []string{"blog"}, AllowedServices: []string{"web"}}, | |||
| 123 | + | }) | |||
| 124 | + | ||||
| 125 | + | _, _, err := svc.Apply(context.Background(), spec, []byte("manifest"), false) | |||
| 126 | + | ||||
| 127 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 128 | + | assert.NotContains(t, err.Error(), shared, "apply errors must never leak CDI IDs") | |||
| 129 | + | } | |||
M internal/usecase/apps/service.go +66 -0
| @@ -33,6 +33,10 @@ type Service struct { | |||
| 33 | 33 | // atomically; applies read the current map without holding the lock. | |
| 34 | 34 | bindMu sync.RWMutex | |
| 35 | 35 | bindPolicies map[string]domain.AppBindPolicy | |
| 36 | + | // deviceMu guards devicePolicies. Reload replaces the whole map | |
| 37 | + | // atomically; applies read the current map without holding the lock. | |
| 38 | + | deviceMu sync.RWMutex | |
| 39 | + | devicePolicies map[string]domain.AppDevicePolicy | |
| 36 | 40 | } | |
| 37 | 41 | ||
| 38 | 42 | // NewService creates the apps use case over an AppState store. | |
| @@ -81,6 +85,37 @@ func (s *Service) snapshotBindPolicies() map[string]domain.AppBindPolicy { | |||
| 81 | 85 | return s.bindPolicies | |
| 82 | 86 | } | |
| 83 | 87 | ||
| 88 | + | // WithDevicePolicies supplies the initial administrative device policies. | |
| 89 | + | func (s *Service) WithDevicePolicies(policies map[string]domain.AppDevicePolicy) *Service { | |
| 90 | + | s.SetDevicePolicies(policies) | |
| 91 | + | return s | |
| 92 | + | } | |
| 93 | + | ||
| 94 | + | // SetDevicePolicies atomically replaces the administrative device policies | |
| 95 | + | // used to authorize manifest devices. The map and its slices are copied so | |
| 96 | + | // later caller mutation cannot race an in-flight apply. | |
| 97 | + | func (s *Service) SetDevicePolicies(policies map[string]domain.AppDevicePolicy) { | |
| 98 | + | copied := make(map[string]domain.AppDevicePolicy, len(policies)) | |
| 99 | + | for name, policy := range policies { | |
| 100 | + | policy.CDI = append([]string(nil), policy.CDI...) | |
| 101 | + | policy.AllowedApps = append([]string(nil), policy.AllowedApps...) | |
| 102 | + | policy.AllowedServices = append([]string(nil), policy.AllowedServices...) | |
| 103 | + | copied[name] = policy | |
| 104 | + | } | |
| 105 | + | s.deviceMu.Lock() | |
| 106 | + | s.devicePolicies = copied | |
| 107 | + | s.deviceMu.Unlock() | |
| 108 | + | } | |
| 109 | + | ||
| 110 | + | // snapshotDevicePolicies returns the current policy map. SetDevicePolicies | |
| 111 | + | // never mutates a published map, so the snapshot stays safe after the | |
| 112 | + | // lock drops. | |
| 113 | + | func (s *Service) snapshotDevicePolicies() map[string]domain.AppDevicePolicy { | |
| 114 | + | s.deviceMu.RLock() | |
| 115 | + | defer s.deviceMu.RUnlock() | |
| 116 | + | return s.devicePolicies | |
| 117 | + | } | |
| 118 | + | ||
| 84 | 119 | // noopLease is a GC lease for an unwired barrier. | |
| 85 | 120 | type noopLease struct{} | |
| 86 | 121 | ||
| @@ -174,6 +209,9 @@ func (s *Service) Apply(ctx context.Context, spec domain.AppSpec, source []byte, | |||
| 174 | 209 | if err := s.validateBindPolicies(spec); err != nil { | |
| 175 | 210 | return nil, nil, err | |
| 176 | 211 | } | |
| 212 | + | if err := s.validateDevicePolicies(spec); err != nil { | |
| 213 | + | return nil, nil, err | |
| 214 | + | } | |
| 177 | 215 | if err := s.validateEntrypointCompatibility(spec); err != nil { | |
| 178 | 216 | return nil, nil, err | |
| 179 | 217 | } | |
| @@ -228,6 +266,34 @@ func (s *Service) validateBindPolicies(spec domain.AppSpec) error { | |||
| 228 | 266 | return nil | |
| 229 | 267 | } | |
| 230 | 268 | ||
| 269 | + | // validateDevicePolicies resolves every declared device against the | |
| 270 | + | // administrative device policies before any desired state is persisted. | |
| 271 | + | // It runs the same union resolution as activation: duplicate CDI IDs | |
| 272 | + | // across logical resources fail here, not later at preflight. Errors | |
| 273 | + | // name only the device, app, and service with an actionable hint: | |
| 274 | + | // host inventory never appears in apply errors. | |
| 275 | + | func (s *Service) validateDevicePolicies(spec domain.AppSpec) error { | |
| 276 | + | policies := s.snapshotDevicePolicies() | |
| 277 | + | for _, svc := range spec.Services { | |
| 278 | + | for _, device := range svc.Devices { | |
| 279 | + | policy, ok := policies[device] | |
| 280 | + | if !ok { | |
| 281 | + | return fmt.Errorf("%w: app %q service %q device %q: no administrative device policy configured (declare [app_devices.%s] with allowed_apps/allowed_services)", domain.ErrDevicePolicy, spec.Name, svc.Name, device, device) | |
| 282 | + | } | |
| 283 | + | if _, err := policy.ResolveAppDevice(spec.Name, svc.Name); err != nil { | |
| 284 | + | return fmt.Errorf("%w: app %q service %q device %q: refused by administrative device policy (check allowed_apps/allowed_services)", domain.ErrDevicePolicy, spec.Name, svc.Name, device) | |
| 285 | + | } | |
| 286 | + | } | |
| 287 | + | // The union check catches duplicate CDI IDs granted under two | |
| 288 | + | // logical names; single-device errors above already named the | |
| 289 | + | // device, so this wraps only the cross-device conflict. | |
| 290 | + | if _, err := domain.ResolveAppDevices(spec.Name, svc.Name, svc.Devices, policies); err != nil { | |
| 291 | + | return fmt.Errorf("apps: %w", err) | |
| 292 | + | } | |
| 293 | + | } | |
| 294 | + | return nil | |
| 295 | + | } | |
| 296 | + | ||
| 231 | 297 | // validateEntrypointCompatibility checks every declared TCP/UDP interface | |
| 232 | 298 | // against the installation entrypoint listeners before any persistence or | |
| 233 | 299 | // workload effect. The runtime binds the entrypoint address, so a publish | |
| @@ -33,6 +33,10 @@ type Service struct { | |||||
| 33 | // atomically; applies read the current map without holding the lock. | 33 | // atomically; applies read the current map without holding the lock. | ||
| 34 | bindMu sync.RWMutex | 34 | bindMu sync.RWMutex | ||
| 35 | bindPolicies map[string]domain.AppBindPolicy | 35 | bindPolicies map[string]domain.AppBindPolicy | ||
| 36 | + | // deviceMu guards devicePolicies. Reload replaces the whole map | |||
| 37 | + | // atomically; applies read the current map without holding the lock. | |||
| 38 | + | deviceMu sync.RWMutex | |||
| 39 | + | devicePolicies map[string]domain.AppDevicePolicy | |||
| 36 | } | 40 | } | ||
| 37 | 41 | ||||
| 38 | // NewService creates the apps use case over an AppState store. | 42 | // NewService creates the apps use case over an AppState store. | ||
| @@ -81,6 +85,37 @@ func (s *Service) snapshotBindPolicies() map[string]domain.AppBindPolicy { | |||||
| 81 | return s.bindPolicies | 85 | return s.bindPolicies | ||
| 82 | } | 86 | } | ||
| 83 | 87 | ||||
| 88 | + | // WithDevicePolicies supplies the initial administrative device policies. | |||
| 89 | + | func (s *Service) WithDevicePolicies(policies map[string]domain.AppDevicePolicy) *Service { | |||
| 90 | + | s.SetDevicePolicies(policies) | |||
| 91 | + | return s | |||
| 92 | + | } | |||
| 93 | + | ||||
| 94 | + | // SetDevicePolicies atomically replaces the administrative device policies | |||
| 95 | + | // used to authorize manifest devices. The map and its slices are copied so | |||
| 96 | + | // later caller mutation cannot race an in-flight apply. | |||
| 97 | + | func (s *Service) SetDevicePolicies(policies map[string]domain.AppDevicePolicy) { | |||
| 98 | + | copied := make(map[string]domain.AppDevicePolicy, len(policies)) | |||
| 99 | + | for name, policy := range policies { | |||
| 100 | + | policy.CDI = append([]string(nil), policy.CDI...) | |||
| 101 | + | policy.AllowedApps = append([]string(nil), policy.AllowedApps...) | |||
| 102 | + | policy.AllowedServices = append([]string(nil), policy.AllowedServices...) | |||
| 103 | + | copied[name] = policy | |||
| 104 | + | } | |||
| 105 | + | s.deviceMu.Lock() | |||
| 106 | + | s.devicePolicies = copied | |||
| 107 | + | s.deviceMu.Unlock() | |||
| 108 | + | } | |||
| 109 | + | ||||
| 110 | + | // snapshotDevicePolicies returns the current policy map. SetDevicePolicies | |||
| 111 | + | // never mutates a published map, so the snapshot stays safe after the | |||
| 112 | + | // lock drops. | |||
| 113 | + | func (s *Service) snapshotDevicePolicies() map[string]domain.AppDevicePolicy { | |||
| 114 | + | s.deviceMu.RLock() | |||
| 115 | + | defer s.deviceMu.RUnlock() | |||
| 116 | + | return s.devicePolicies | |||
| 117 | + | } | |||
| 118 | + | ||||
| 84 | // noopLease is a GC lease for an unwired barrier. | 119 | // noopLease is a GC lease for an unwired barrier. | ||
| 85 | type noopLease struct{} | 120 | type noopLease struct{} | ||
| 86 | 121 | ||||
| @@ -174,6 +209,9 @@ func (s *Service) Apply(ctx context.Context, spec domain.AppSpec, source []byte, | |||||
| 174 | if err := s.validateBindPolicies(spec); err != nil { | 209 | if err := s.validateBindPolicies(spec); err != nil { | ||
| 175 | return nil, nil, err | 210 | return nil, nil, err | ||
| 176 | } | 211 | } | ||
| 212 | + | if err := s.validateDevicePolicies(spec); err != nil { | |||
| 213 | + | return nil, nil, err | |||
| 214 | + | } | |||
| 177 | if err := s.validateEntrypointCompatibility(spec); err != nil { | 215 | if err := s.validateEntrypointCompatibility(spec); err != nil { | ||
| 178 | return nil, nil, err | 216 | return nil, nil, err | ||
| 179 | } | 217 | } | ||
| @@ -228,6 +266,34 @@ func (s *Service) validateBindPolicies(spec domain.AppSpec) error { | |||||
| 228 | return nil | 266 | return nil | ||
| 229 | } | 267 | } | ||
| 230 | 268 | ||||
| 269 | + | // validateDevicePolicies resolves every declared device against the | |||
| 270 | + | // administrative device policies before any desired state is persisted. | |||
| 271 | + | // It runs the same union resolution as activation: duplicate CDI IDs | |||
| 272 | + | // across logical resources fail here, not later at preflight. Errors | |||
| 273 | + | // name only the device, app, and service with an actionable hint: | |||
| 274 | + | // host inventory never appears in apply errors. | |||
| 275 | + | func (s *Service) validateDevicePolicies(spec domain.AppSpec) error { | |||
| 276 | + | policies := s.snapshotDevicePolicies() | |||
| 277 | + | for _, svc := range spec.Services { | |||
| 278 | + | for _, device := range svc.Devices { | |||
| 279 | + | policy, ok := policies[device] | |||
| 280 | + | if !ok { | |||
| 281 | + | return fmt.Errorf("%w: app %q service %q device %q: no administrative device policy configured (declare [app_devices.%s] with allowed_apps/allowed_services)", domain.ErrDevicePolicy, spec.Name, svc.Name, device, device) | |||
| 282 | + | } | |||
| 283 | + | if _, err := policy.ResolveAppDevice(spec.Name, svc.Name); err != nil { | |||
| 284 | + | return fmt.Errorf("%w: app %q service %q device %q: refused by administrative device policy (check allowed_apps/allowed_services)", domain.ErrDevicePolicy, spec.Name, svc.Name, device) | |||
| 285 | + | } | |||
| 286 | + | } | |||
| 287 | + | // The union check catches duplicate CDI IDs granted under two | |||
| 288 | + | // logical names; single-device errors above already named the | |||
| 289 | + | // device, so this wraps only the cross-device conflict. | |||
| 290 | + | if _, err := domain.ResolveAppDevices(spec.Name, svc.Name, svc.Devices, policies); err != nil { | |||
| 291 | + | return fmt.Errorf("apps: %w", err) | |||
| 292 | + | } | |||
| 293 | + | } | |||
| 294 | + | return nil | |||
| 295 | + | } | |||
| 296 | + | ||||
| 231 | // validateEntrypointCompatibility checks every declared TCP/UDP interface | 297 | // validateEntrypointCompatibility checks every declared TCP/UDP interface | ||
| 232 | // against the installation entrypoint listeners before any persistence or | 298 | // against the installation entrypoint listeners before any persistence or | ||
| 233 | // workload effect. The runtime binds the entrypoint address, so a publish | 299 | // workload effect. The runtime binds the entrypoint address, so a publish | ||
A internal/usecase/deployment/device_policy_internal_test.go +304 -0
| @@ -0,0 +1,304 @@ | |||
| 1 | + | package deployment | |
| 2 | + | ||
| 3 | + | import ( | |
| 4 | + | "context" | |
| 5 | + | "errors" | |
| 6 | + | "fmt" | |
| 7 | + | "testing" | |
| 8 | + | ||
| 9 | + | "github.com/bnema/zerowrap" | |
| 10 | + | "github.com/stretchr/testify/assert" | |
| 11 | + | "github.com/stretchr/testify/mock" | |
| 12 | + | "github.com/stretchr/testify/require" | |
| 13 | + | ||
| 14 | + | outmocks "github.com/bnema/gordon/internal/boundaries/out/mocks" | |
| 15 | + | "github.com/bnema/gordon/internal/domain" | |
| 16 | + | ) | |
| 17 | + | ||
| 18 | + | func devicePolicyFor(apps, services []string) domain.AppDevicePolicy { | |
| 19 | + | return domain.AppDevicePolicy{ | |
| 20 | + | Name: "test_gpu", | |
| 21 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |
| 22 | + | AllowedApps: apps, | |
| 23 | + | AllowedServices: services, | |
| 24 | + | } | |
| 25 | + | } | |
| 26 | + | ||
| 27 | + | func TestResolveServiceDevices_ExactResolution(t *testing.T) { | |
| 28 | + | svc := NewService(Deps{}, zerowrap.Default()). | |
| 29 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 30 | + | "test_gpu": devicePolicyFor([]string{"blog"}, []string{"web"}), | |
| 31 | + | }) | |
| 32 | + | ||
| 33 | + | got, err := svc.resolveServiceDevices("blog", domain.AppService{ | |
| 34 | + | Name: "web", | |
| 35 | + | Devices: []string{"test_gpu"}, | |
| 36 | + | }) | |
| 37 | + | require.NoError(t, err) | |
| 38 | + | assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, got) | |
| 39 | + | } | |
| 40 | + | ||
| 41 | + | func TestResolveServiceDevices_EmptyIsNil(t *testing.T) { | |
| 42 | + | svc := NewService(Deps{}, zerowrap.Default()) | |
| 43 | + | got, err := svc.resolveServiceDevices("blog", domain.AppService{Name: "web"}) | |
| 44 | + | require.NoError(t, err) | |
| 45 | + | assert.Nil(t, got) | |
| 46 | + | } | |
| 47 | + | ||
| 48 | + | func TestResolveServiceDevices_UnknownAndRevokedPolicy(t *testing.T) { | |
| 49 | + | spec := domain.AppService{ | |
| 50 | + | Name: "web", | |
| 51 | + | Devices: []string{"test_gpu"}, | |
| 52 | + | } | |
| 53 | + | ||
| 54 | + | t.Run("unknown policy", func(t *testing.T) { | |
| 55 | + | svc := NewService(Deps{}, zerowrap.Default()) | |
| 56 | + | _, err := svc.resolveServiceDevices("blog", spec) | |
| 57 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 58 | + | assert.Contains(t, err.Error(), "blog") | |
| 59 | + | assert.Contains(t, err.Error(), "web") | |
| 60 | + | assert.Contains(t, err.Error(), "test_gpu") | |
| 61 | + | }) | |
| 62 | + | ||
| 63 | + | t.Run("revoked policy", func(t *testing.T) { | |
| 64 | + | svc := NewService(Deps{}, zerowrap.Default()). | |
| 65 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 66 | + | "test_gpu": devicePolicyFor([]string{"blog"}, []string{"web"}), | |
| 67 | + | }) | |
| 68 | + | _, err := svc.resolveServiceDevices("blog", spec) | |
| 69 | + | require.NoError(t, err) | |
| 70 | + | ||
| 71 | + | svc.SetDevicePolicies(nil) | |
| 72 | + | _, err = svc.resolveServiceDevices("blog", spec) | |
| 73 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 74 | + | }) | |
| 75 | + | ||
| 76 | + | t.Run("refused policy omits device IDs", func(t *testing.T) { | |
| 77 | + | svc := NewService(Deps{}, zerowrap.Default()). | |
| 78 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 79 | + | "test_gpu": devicePolicyFor([]string{"other"}, []string{"web"}), | |
| 80 | + | }) | |
| 81 | + | _, err := svc.resolveServiceDevices("blog", spec) | |
| 82 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 83 | + | assert.NotContains(t, err.Error(), "GPU-test-uuid", "errors must never leak CDI IDs") | |
| 84 | + | }) | |
| 85 | + | } | |
| 86 | + | ||
| 87 | + | // TestCreateAndStart_RefusesRevokedDeviceBeforeRuntimeMutation proves the | |
| 88 | + | // immediate-pre-create re-resolution fails before any runtime call when the | |
| 89 | + | // device policy is missing or revoked. | |
| 90 | + | func TestCreateAndStart_RefusesRevokedDeviceBeforeRuntimeMutation(t *testing.T) { | |
| 91 | + | runtime := outmocks.NewMockContainerRuntime(t) | |
| 92 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |
| 93 | + | p := pinnedService{ | |
| 94 | + | name: "web", | |
| 95 | + | spec: domain.AppService{ | |
| 96 | + | Name: "web", | |
| 97 | + | Devices: []string{"test_gpu"}, | |
| 98 | + | }, | |
| 99 | + | } | |
| 100 | + | ||
| 101 | + | _, _, _, err := svc.createAndStart(context.Background(), "blog", "rev-1", p, "op-1", nil) | |
| 102 | + | ||
| 103 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |
| 104 | + | assert.Contains(t, err.Error(), "test_gpu") | |
| 105 | + | runtime.AssertNotCalled(t, "CreateContainer") | |
| 106 | + | runtime.AssertNotCalled(t, "CreateVolume") | |
| 107 | + | runtime.AssertNotCalled(t, "StartContainer") | |
| 108 | + | runtime.AssertNotCalled(t, "ConnectContainerToNetwork") | |
| 109 | + | runtime.AssertExpectations(t) | |
| 110 | + | } | |
| 111 | + | ||
| 112 | + | // TestCreateAndStart_PassesResolvedCDIIDsToRuntime proves the deploy path | |
| 113 | + | // carries authorized policy resolution into ContainerConfig.CDIDevices: | |
| 114 | + | // only the granted service receives IDs, helpers and ordinary services | |
| 115 | + | // receive none. | |
| 116 | + | func TestCreateAndStart_PassesResolvedCDIIDsToRuntime(t *testing.T) { | |
| 117 | + | ctx := context.Background() | |
| 118 | + | state := outmocks.NewMockAppState(t) | |
| 119 | + | runtime := outmocks.NewMockContainerRuntime(t) | |
| 120 | + | ||
| 121 | + | private := domain.AppPrivateNetworkName("gordon", "app-1") | |
| 122 | + | state.EXPECT().LoadOwnership(mock.Anything, "blog"). | |
| 123 | + | Return(domain.AppOwnership{App: "blog", ID: "app-1"}, nil).Once() | |
| 124 | + | runtime.EXPECT().ListNetworks(mock.Anything).Return([]*domain.NetworkInfo{}, nil).Once() | |
| 125 | + | runtime.EXPECT().CreateNetwork(mock.Anything, private, mock.Anything).Return(nil).Once() | |
| 126 | + | runtime.EXPECT().CreateContainer(mock.Anything, mock.MatchedBy(func(cfg *domain.ContainerConfig) bool { | |
| 127 | + | return assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, cfg.CDIDevices) | |
| 128 | + | })).Return(&domain.Container{ID: "c-1", Name: "web"}, nil).Once() | |
| 129 | + | runtime.EXPECT().StartContainer(mock.Anything, "c-1").Return(nil).Once() | |
| 130 | + | ||
| 131 | + | svc := NewService(Deps{State: state, Runtime: runtime, Networks: NetworkConfig{Prefix: "gordon"}}, zerowrap.Default()). | |
| 132 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 133 | + | "test_gpu": devicePolicyFor([]string{"blog"}, []string{"web"}), | |
| 134 | + | }) | |
| 135 | + | created, _, _, err := svc.createAndStart(ctx, "blog", "rev-1", pinnedService{ | |
| 136 | + | name: "web", | |
| 137 | + | spec: domain.AppService{Name: "web", Devices: []string{"test_gpu"}}, | |
| 138 | + | }, "op-1", nil) | |
| 139 | + | require.NoError(t, err) | |
| 140 | + | require.NotNil(t, created) | |
| 141 | + | runtime.AssertExpectations(t) | |
| 142 | + | } | |
| 143 | + | ||
| 144 | + | // TestCreateContainer_DeviceBearingRuntimeErrorIsRedactedNotDevicePolicy | |
| 145 | + | // proves a CreateContainer failure on a device-bearing service is reported | |
| 146 | + | // as a generic redacted runtime error, never as a device policy violation, | |
| 147 | + | // and never leaks the resolved CDI IDs embedded in the runtime error. | |
| 148 | + | func TestCreateContainer_DeviceBearingRuntimeErrorIsRedactedNotDevicePolicy(t *testing.T) { | |
| 149 | + | const deviceID = "example.com/gpu=GPU-test-uuid" | |
| 150 | + | runtime := outmocks.NewMockContainerRuntime(t) | |
| 151 | + | runtime.EXPECT().CreateContainer(mock.Anything, mock.Anything). | |
| 152 | + | Return(nil, fmt.Errorf("device %s: permission denied", deviceID)).Once() | |
| 153 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |
| 154 | + | ||
| 155 | + | _, err := svc.createContainer(context.Background(), "web", &domain.ContainerConfig{ | |
| 156 | + | CDIDevices: []string{deviceID}, | |
| 157 | + | }) | |
| 158 | + | ||
| 159 | + | require.Error(t, err) | |
| 160 | + | require.NotErrorIs(t, err, domain.ErrDevicePolicy, "a runtime failure is not a policy refusal") | |
| 161 | + | assert.NotContains(t, err.Error(), deviceID, "errors must never leak resolved CDI IDs") | |
| 162 | + | assert.NotContains(t, err.Error(), "permission denied", "no runtime text may reach the caller") | |
| 163 | + | runtime.AssertExpectations(t) | |
| 164 | + | } | |
| 165 | + | ||
| 166 | + | // TestRestartOneService_RevokedDeviceFailsWithoutRuntimeMutation proves an | |
| 167 | + | // in-place restart fails closed when the device policy is missing/revoked, | |
| 168 | + | // before touching the runtime. | |
| 169 | + | func TestRestartOneService_RevokedDeviceFailsWithoutRuntimeMutation(t *testing.T) { | |
| 170 | + | runtime := outmocks.NewMockContainerRuntime(t) | |
| 171 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |
| 172 | + | eff := domain.AppEffectiveService{ | |
| 173 | + | Container: "c1", | |
| 174 | + | Spec: domain.AppService{ | |
| 175 | + | Name: "web", | |
| 176 | + | Devices: []string{"test_gpu"}, | |
| 177 | + | }, | |
| 178 | + | } | |
| 179 | + | ||
| 180 | + | op := &domain.AppOperation{Op: "op-1", Steps: []domain.AppOperationStep{{ | |
| 181 | + | ID: "service.web.restart", State: domain.AppStepPending, Before: eff.Container, | |
| 182 | + | }}} | |
| 183 | + | result := svc.restartOneService(context.Background(), "blog", "op-1", "web", eff, op, 0) | |
| 184 | + | ||
| 185 | + | assert.Equal(t, domain.AppStepFailed, op.Steps[0].State) | |
| 186 | + | assert.Contains(t, op.Steps[0].Error, "test_gpu") | |
| 187 | + | assert.Equal(t, "failed", result.Result) | |
| 188 | + | runtime.AssertNotCalled(t, "RestartContainer") | |
| 189 | + | runtime.AssertExpectations(t) | |
| 190 | + | } | |
| 191 | + | ||
| 192 | + | // TestEnsureServiceRunning_RevokedDeviceFailsWithoutRuntimeMutation proves | |
| 193 | + | // the boot/start recovery path fails closed before starting a container | |
| 194 | + | // when the device policy is missing/revoked. | |
| 195 | + | func TestEnsureServiceRunning_RevokedDeviceFailsWithoutRuntimeMutation(t *testing.T) { | |
| 196 | + | runtime := outmocks.NewMockContainerRuntime(t) | |
| 197 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |
| 198 | + | eff := domain.AppEffectiveService{ | |
| 199 | + | Container: "c1", | |
| 200 | + | Spec: domain.AppService{ | |
| 201 | + | Name: "web", | |
| 202 | + | Devices: []string{"test_gpu"}, | |
| 203 | + | }, | |
| 204 | + | } | |
| 205 | + | op := &domain.AppOperation{Op: "op-1", Steps: []domain.AppOperationStep{{ | |
| 206 | + | ID: "service.web.start", State: domain.AppStepPending, Before: eff.Container, | |
| 207 | + | }}} | |
| 208 | + | result := &LifecycleResult{Services: map[string]ServiceResult{}} | |
| 209 | + | ||
| 210 | + | svc.ensureServiceRunning(context.Background(), "blog", "op-1", "web", eff, op, 0, result) | |
| 211 | + | ||
| 212 | + | assert.Equal(t, domain.AppStepFailed, op.Steps[0].State) | |
| 213 | + | assert.Contains(t, op.Steps[0].Error, "test_gpu") | |
| 214 | + | runtime.AssertNotCalled(t, "StartContainer") | |
| 215 | + | runtime.AssertExpectations(t) | |
| 216 | + | } | |
| 217 | + | ||
| 218 | + | // TestPreflightServices_ProbesEngineOncePerRevision proves preflight calls | |
| 219 | + | // SupportsCDIDevices exactly once for a multi-service device-bearing | |
| 220 | + | // revision, and that pinned services carry their logical device requests. | |
| 221 | + | func TestPreflightServices_ProbesEngineOncePerRevision(t *testing.T) { | |
| 222 | + | ctx := context.Background() | |
| 223 | + | state := outmocks.NewMockAppState(t) | |
| 224 | + | runtime := outmocks.NewMockContainerRuntime(t) | |
| 225 | + | images := outmocks.NewMockImageResolver(t) | |
| 226 | + | secrets := outmocks.NewMockSecretProvider(t) | |
| 227 | + | ||
| 228 | + | rev := domain.AppDesiredRevision{ | |
| 229 | + | Revision: "rev-1", | |
| 230 | + | App: "blog", | |
| 231 | + | Spec: domain.AppSpec{ | |
| 232 | + | Name: "blog", | |
| 233 | + | Env: map[string]string{}, | |
| 234 | + | Services: []domain.AppService{ | |
| 235 | + | {Name: "web", Image: "img:1", Devices: []string{"test_gpu"}}, | |
| 236 | + | {Name: "worker", Image: "img:1", Devices: []string{"test_gpu"}}, | |
| 237 | + | }, | |
| 238 | + | }, | |
| 239 | + | } | |
| 240 | + | state.EXPECT().LoadOwnership(mock.Anything, "blog"). | |
| 241 | + | Return(domain.AppOwnership{App: "blog", ID: "app-1"}, nil).Once() | |
| 242 | + | images.EXPECT().ResolveDigest(mock.Anything, "img:1").Return("sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", nil).Twice() | |
| 243 | + | runtime.EXPECT().InspectImageVolumes(mock.Anything, "img:1").Return(nil, nil).Twice() | |
| 244 | + | runtime.EXPECT().SupportsCDIDevices(mock.Anything).Return(nil).Once() | |
| 245 | + | state.EXPECT().LoadCheckpoint(mock.Anything).Return(domain.AppStoreCheckpoint{}, nil).Once() | |
| 246 | + | ||
| 247 | + | svc := NewService(Deps{State: state, Runtime: runtime, Images: images, Secrets: secrets}, zerowrap.Default()). | |
| 248 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |
| 249 | + | "test_gpu": { | |
| 250 | + | Name: "test_gpu", | |
| 251 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |
| 252 | + | AllowedApps: []string{"blog"}, | |
| 253 | + | AllowedServices: []string{"web", "worker"}, | |
| 254 | + | }, | |
| 255 | + | }) | |
| 256 | + | pinned, err := svc.preflightServices(ctx, "blog", rev, "") | |
| 257 | + | require.NoError(t, err) | |
| 258 | + | require.Len(t, pinned, 2) | |
| 259 | + | runtime.AssertExpectations(t) | |
| 260 | + | } | |
| 261 | + | ||
| 262 | + | // TestCreateContainer_RuntimeUnsupportedSurvivesRedaction proves the | |
| 263 | + | // engine-unsupported sentinel from a device-bearing create survives the | |
| 264 | + | // host-inventory redaction so callers map it to the structured | |
| 265 | + | // runtime-unsupported envelope. | |
| 266 | + | func TestCreateContainer_RuntimeUnsupportedSurvivesRedaction(t *testing.T) { | |
| 267 | + | runtime := outmocks.NewMockContainerRuntime(t) | |
| 268 | + | runtime.EXPECT().CreateContainer(mock.Anything, mock.Anything). | |
| 269 | + | Return(nil, domain.ErrRuntimeUnsupported).Once() | |
| 270 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |
| 271 | + | ||
| 272 | + | _, err := svc.createContainer(context.Background(), "web", &domain.ContainerConfig{ | |
| 273 | + | CDIDevices: []string{"example.com/gpu=GPU-test-uuid"}, | |
| 274 | + | }) | |
| 275 | + | ||
| 276 | + | require.Error(t, err) | |
| 277 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |
| 278 | + | assert.NotContains(t, err.Error(), "GPU-test-uuid", "the sanitized error must not echo CDI IDs") | |
| 279 | + | runtime.AssertExpectations(t) | |
| 280 | + | } | |
| 281 | + | ||
| 282 | + | // TestSanitizedEngineProbeError proves the capability-probe error keeps | |
| 283 | + | // cancellation meaningful, collapses every other cause to the unsupported | |
| 284 | + | // sentinel alone, and never echoes the adapter text. | |
| 285 | + | func TestSanitizedEngineProbeError(t *testing.T) { | |
| 286 | + | t.Run("cancellation is preserved and reported as cancellation", func(t *testing.T) { | |
| 287 | + | err := sanitizedEngineProbeError("web", fmt.Errorf("dial unix /run/podman/podman.sock: %w", context.Canceled)) | |
| 288 | + | require.ErrorIs(t, err, context.Canceled) | |
| 289 | + | assert.False(t, errors.Is(err, domain.ErrRuntimeUnsupported), "an aborted probe is not an incapable engine") | |
| 290 | + | assert.NotContains(t, err.Error(), "podman.sock") | |
| 291 | + | }) | |
| 292 | + | ||
| 293 | + | t.Run("deadline is preserved", func(t *testing.T) { | |
| 294 | + | err := sanitizedEngineProbeError("web", fmt.Errorf("probe: %w", context.DeadlineExceeded)) | |
| 295 | + | require.ErrorIs(t, err, context.DeadlineExceeded) | |
| 296 | + | assert.NotContains(t, err.Error(), "probe") | |
| 297 | + | }) | |
| 298 | + | ||
| 299 | + | t.Run("other causes collapse to the unsupported sentinel", func(t *testing.T) { | |
| 300 | + | err := sanitizedEngineProbeError("web", errors.New("dial unix /run/podman/podman.sock: connection refused")) | |
| 301 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |
| 302 | + | assert.NotContains(t, err.Error(), "podman.sock") | |
| 303 | + | }) | |
| 304 | + | } | |
| @@ -0,0 +1,304 @@ | |||||
| 1 | + | package deployment | |||
| 2 | + | ||||
| 3 | + | import ( | |||
| 4 | + | "context" | |||
| 5 | + | "errors" | |||
| 6 | + | "fmt" | |||
| 7 | + | "testing" | |||
| 8 | + | ||||
| 9 | + | "github.com/bnema/zerowrap" | |||
| 10 | + | "github.com/stretchr/testify/assert" | |||
| 11 | + | "github.com/stretchr/testify/mock" | |||
| 12 | + | "github.com/stretchr/testify/require" | |||
| 13 | + | ||||
| 14 | + | outmocks "github.com/bnema/gordon/internal/boundaries/out/mocks" | |||
| 15 | + | "github.com/bnema/gordon/internal/domain" | |||
| 16 | + | ) | |||
| 17 | + | ||||
| 18 | + | func devicePolicyFor(apps, services []string) domain.AppDevicePolicy { | |||
| 19 | + | return domain.AppDevicePolicy{ | |||
| 20 | + | Name: "test_gpu", | |||
| 21 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |||
| 22 | + | AllowedApps: apps, | |||
| 23 | + | AllowedServices: services, | |||
| 24 | + | } | |||
| 25 | + | } | |||
| 26 | + | ||||
| 27 | + | func TestResolveServiceDevices_ExactResolution(t *testing.T) { | |||
| 28 | + | svc := NewService(Deps{}, zerowrap.Default()). | |||
| 29 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 30 | + | "test_gpu": devicePolicyFor([]string{"blog"}, []string{"web"}), | |||
| 31 | + | }) | |||
| 32 | + | ||||
| 33 | + | got, err := svc.resolveServiceDevices("blog", domain.AppService{ | |||
| 34 | + | Name: "web", | |||
| 35 | + | Devices: []string{"test_gpu"}, | |||
| 36 | + | }) | |||
| 37 | + | require.NoError(t, err) | |||
| 38 | + | assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, got) | |||
| 39 | + | } | |||
| 40 | + | ||||
| 41 | + | func TestResolveServiceDevices_EmptyIsNil(t *testing.T) { | |||
| 42 | + | svc := NewService(Deps{}, zerowrap.Default()) | |||
| 43 | + | got, err := svc.resolveServiceDevices("blog", domain.AppService{Name: "web"}) | |||
| 44 | + | require.NoError(t, err) | |||
| 45 | + | assert.Nil(t, got) | |||
| 46 | + | } | |||
| 47 | + | ||||
| 48 | + | func TestResolveServiceDevices_UnknownAndRevokedPolicy(t *testing.T) { | |||
| 49 | + | spec := domain.AppService{ | |||
| 50 | + | Name: "web", | |||
| 51 | + | Devices: []string{"test_gpu"}, | |||
| 52 | + | } | |||
| 53 | + | ||||
| 54 | + | t.Run("unknown policy", func(t *testing.T) { | |||
| 55 | + | svc := NewService(Deps{}, zerowrap.Default()) | |||
| 56 | + | _, err := svc.resolveServiceDevices("blog", spec) | |||
| 57 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 58 | + | assert.Contains(t, err.Error(), "blog") | |||
| 59 | + | assert.Contains(t, err.Error(), "web") | |||
| 60 | + | assert.Contains(t, err.Error(), "test_gpu") | |||
| 61 | + | }) | |||
| 62 | + | ||||
| 63 | + | t.Run("revoked policy", func(t *testing.T) { | |||
| 64 | + | svc := NewService(Deps{}, zerowrap.Default()). | |||
| 65 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 66 | + | "test_gpu": devicePolicyFor([]string{"blog"}, []string{"web"}), | |||
| 67 | + | }) | |||
| 68 | + | _, err := svc.resolveServiceDevices("blog", spec) | |||
| 69 | + | require.NoError(t, err) | |||
| 70 | + | ||||
| 71 | + | svc.SetDevicePolicies(nil) | |||
| 72 | + | _, err = svc.resolveServiceDevices("blog", spec) | |||
| 73 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 74 | + | }) | |||
| 75 | + | ||||
| 76 | + | t.Run("refused policy omits device IDs", func(t *testing.T) { | |||
| 77 | + | svc := NewService(Deps{}, zerowrap.Default()). | |||
| 78 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 79 | + | "test_gpu": devicePolicyFor([]string{"other"}, []string{"web"}), | |||
| 80 | + | }) | |||
| 81 | + | _, err := svc.resolveServiceDevices("blog", spec) | |||
| 82 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 83 | + | assert.NotContains(t, err.Error(), "GPU-test-uuid", "errors must never leak CDI IDs") | |||
| 84 | + | }) | |||
| 85 | + | } | |||
| 86 | + | ||||
| 87 | + | // TestCreateAndStart_RefusesRevokedDeviceBeforeRuntimeMutation proves the | |||
| 88 | + | // immediate-pre-create re-resolution fails before any runtime call when the | |||
| 89 | + | // device policy is missing or revoked. | |||
| 90 | + | func TestCreateAndStart_RefusesRevokedDeviceBeforeRuntimeMutation(t *testing.T) { | |||
| 91 | + | runtime := outmocks.NewMockContainerRuntime(t) | |||
| 92 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |||
| 93 | + | p := pinnedService{ | |||
| 94 | + | name: "web", | |||
| 95 | + | spec: domain.AppService{ | |||
| 96 | + | Name: "web", | |||
| 97 | + | Devices: []string{"test_gpu"}, | |||
| 98 | + | }, | |||
| 99 | + | } | |||
| 100 | + | ||||
| 101 | + | _, _, _, err := svc.createAndStart(context.Background(), "blog", "rev-1", p, "op-1", nil) | |||
| 102 | + | ||||
| 103 | + | require.ErrorIs(t, err, domain.ErrDevicePolicy) | |||
| 104 | + | assert.Contains(t, err.Error(), "test_gpu") | |||
| 105 | + | runtime.AssertNotCalled(t, "CreateContainer") | |||
| 106 | + | runtime.AssertNotCalled(t, "CreateVolume") | |||
| 107 | + | runtime.AssertNotCalled(t, "StartContainer") | |||
| 108 | + | runtime.AssertNotCalled(t, "ConnectContainerToNetwork") | |||
| 109 | + | runtime.AssertExpectations(t) | |||
| 110 | + | } | |||
| 111 | + | ||||
| 112 | + | // TestCreateAndStart_PassesResolvedCDIIDsToRuntime proves the deploy path | |||
| 113 | + | // carries authorized policy resolution into ContainerConfig.CDIDevices: | |||
| 114 | + | // only the granted service receives IDs, helpers and ordinary services | |||
| 115 | + | // receive none. | |||
| 116 | + | func TestCreateAndStart_PassesResolvedCDIIDsToRuntime(t *testing.T) { | |||
| 117 | + | ctx := context.Background() | |||
| 118 | + | state := outmocks.NewMockAppState(t) | |||
| 119 | + | runtime := outmocks.NewMockContainerRuntime(t) | |||
| 120 | + | ||||
| 121 | + | private := domain.AppPrivateNetworkName("gordon", "app-1") | |||
| 122 | + | state.EXPECT().LoadOwnership(mock.Anything, "blog"). | |||
| 123 | + | Return(domain.AppOwnership{App: "blog", ID: "app-1"}, nil).Once() | |||
| 124 | + | runtime.EXPECT().ListNetworks(mock.Anything).Return([]*domain.NetworkInfo{}, nil).Once() | |||
| 125 | + | runtime.EXPECT().CreateNetwork(mock.Anything, private, mock.Anything).Return(nil).Once() | |||
| 126 | + | runtime.EXPECT().CreateContainer(mock.Anything, mock.MatchedBy(func(cfg *domain.ContainerConfig) bool { | |||
| 127 | + | return assert.Equal(t, []string{"example.com/gpu=GPU-test-uuid"}, cfg.CDIDevices) | |||
| 128 | + | })).Return(&domain.Container{ID: "c-1", Name: "web"}, nil).Once() | |||
| 129 | + | runtime.EXPECT().StartContainer(mock.Anything, "c-1").Return(nil).Once() | |||
| 130 | + | ||||
| 131 | + | svc := NewService(Deps{State: state, Runtime: runtime, Networks: NetworkConfig{Prefix: "gordon"}}, zerowrap.Default()). | |||
| 132 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 133 | + | "test_gpu": devicePolicyFor([]string{"blog"}, []string{"web"}), | |||
| 134 | + | }) | |||
| 135 | + | created, _, _, err := svc.createAndStart(ctx, "blog", "rev-1", pinnedService{ | |||
| 136 | + | name: "web", | |||
| 137 | + | spec: domain.AppService{Name: "web", Devices: []string{"test_gpu"}}, | |||
| 138 | + | }, "op-1", nil) | |||
| 139 | + | require.NoError(t, err) | |||
| 140 | + | require.NotNil(t, created) | |||
| 141 | + | runtime.AssertExpectations(t) | |||
| 142 | + | } | |||
| 143 | + | ||||
| 144 | + | // TestCreateContainer_DeviceBearingRuntimeErrorIsRedactedNotDevicePolicy | |||
| 145 | + | // proves a CreateContainer failure on a device-bearing service is reported | |||
| 146 | + | // as a generic redacted runtime error, never as a device policy violation, | |||
| 147 | + | // and never leaks the resolved CDI IDs embedded in the runtime error. | |||
| 148 | + | func TestCreateContainer_DeviceBearingRuntimeErrorIsRedactedNotDevicePolicy(t *testing.T) { | |||
| 149 | + | const deviceID = "example.com/gpu=GPU-test-uuid" | |||
| 150 | + | runtime := outmocks.NewMockContainerRuntime(t) | |||
| 151 | + | runtime.EXPECT().CreateContainer(mock.Anything, mock.Anything). | |||
| 152 | + | Return(nil, fmt.Errorf("device %s: permission denied", deviceID)).Once() | |||
| 153 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |||
| 154 | + | ||||
| 155 | + | _, err := svc.createContainer(context.Background(), "web", &domain.ContainerConfig{ | |||
| 156 | + | CDIDevices: []string{deviceID}, | |||
| 157 | + | }) | |||
| 158 | + | ||||
| 159 | + | require.Error(t, err) | |||
| 160 | + | require.NotErrorIs(t, err, domain.ErrDevicePolicy, "a runtime failure is not a policy refusal") | |||
| 161 | + | assert.NotContains(t, err.Error(), deviceID, "errors must never leak resolved CDI IDs") | |||
| 162 | + | assert.NotContains(t, err.Error(), "permission denied", "no runtime text may reach the caller") | |||
| 163 | + | runtime.AssertExpectations(t) | |||
| 164 | + | } | |||
| 165 | + | ||||
| 166 | + | // TestRestartOneService_RevokedDeviceFailsWithoutRuntimeMutation proves an | |||
| 167 | + | // in-place restart fails closed when the device policy is missing/revoked, | |||
| 168 | + | // before touching the runtime. | |||
| 169 | + | func TestRestartOneService_RevokedDeviceFailsWithoutRuntimeMutation(t *testing.T) { | |||
| 170 | + | runtime := outmocks.NewMockContainerRuntime(t) | |||
| 171 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |||
| 172 | + | eff := domain.AppEffectiveService{ | |||
| 173 | + | Container: "c1", | |||
| 174 | + | Spec: domain.AppService{ | |||
| 175 | + | Name: "web", | |||
| 176 | + | Devices: []string{"test_gpu"}, | |||
| 177 | + | }, | |||
| 178 | + | } | |||
| 179 | + | ||||
| 180 | + | op := &domain.AppOperation{Op: "op-1", Steps: []domain.AppOperationStep{{ | |||
| 181 | + | ID: "service.web.restart", State: domain.AppStepPending, Before: eff.Container, | |||
| 182 | + | }}} | |||
| 183 | + | result := svc.restartOneService(context.Background(), "blog", "op-1", "web", eff, op, 0) | |||
| 184 | + | ||||
| 185 | + | assert.Equal(t, domain.AppStepFailed, op.Steps[0].State) | |||
| 186 | + | assert.Contains(t, op.Steps[0].Error, "test_gpu") | |||
| 187 | + | assert.Equal(t, "failed", result.Result) | |||
| 188 | + | runtime.AssertNotCalled(t, "RestartContainer") | |||
| 189 | + | runtime.AssertExpectations(t) | |||
| 190 | + | } | |||
| 191 | + | ||||
| 192 | + | // TestEnsureServiceRunning_RevokedDeviceFailsWithoutRuntimeMutation proves | |||
| 193 | + | // the boot/start recovery path fails closed before starting a container | |||
| 194 | + | // when the device policy is missing/revoked. | |||
| 195 | + | func TestEnsureServiceRunning_RevokedDeviceFailsWithoutRuntimeMutation(t *testing.T) { | |||
| 196 | + | runtime := outmocks.NewMockContainerRuntime(t) | |||
| 197 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |||
| 198 | + | eff := domain.AppEffectiveService{ | |||
| 199 | + | Container: "c1", | |||
| 200 | + | Spec: domain.AppService{ | |||
| 201 | + | Name: "web", | |||
| 202 | + | Devices: []string{"test_gpu"}, | |||
| 203 | + | }, | |||
| 204 | + | } | |||
| 205 | + | op := &domain.AppOperation{Op: "op-1", Steps: []domain.AppOperationStep{{ | |||
| 206 | + | ID: "service.web.start", State: domain.AppStepPending, Before: eff.Container, | |||
| 207 | + | }}} | |||
| 208 | + | result := &LifecycleResult{Services: map[string]ServiceResult{}} | |||
| 209 | + | ||||
| 210 | + | svc.ensureServiceRunning(context.Background(), "blog", "op-1", "web", eff, op, 0, result) | |||
| 211 | + | ||||
| 212 | + | assert.Equal(t, domain.AppStepFailed, op.Steps[0].State) | |||
| 213 | + | assert.Contains(t, op.Steps[0].Error, "test_gpu") | |||
| 214 | + | runtime.AssertNotCalled(t, "StartContainer") | |||
| 215 | + | runtime.AssertExpectations(t) | |||
| 216 | + | } | |||
| 217 | + | ||||
| 218 | + | // TestPreflightServices_ProbesEngineOncePerRevision proves preflight calls | |||
| 219 | + | // SupportsCDIDevices exactly once for a multi-service device-bearing | |||
| 220 | + | // revision, and that pinned services carry their logical device requests. | |||
| 221 | + | func TestPreflightServices_ProbesEngineOncePerRevision(t *testing.T) { | |||
| 222 | + | ctx := context.Background() | |||
| 223 | + | state := outmocks.NewMockAppState(t) | |||
| 224 | + | runtime := outmocks.NewMockContainerRuntime(t) | |||
| 225 | + | images := outmocks.NewMockImageResolver(t) | |||
| 226 | + | secrets := outmocks.NewMockSecretProvider(t) | |||
| 227 | + | ||||
| 228 | + | rev := domain.AppDesiredRevision{ | |||
| 229 | + | Revision: "rev-1", | |||
| 230 | + | App: "blog", | |||
| 231 | + | Spec: domain.AppSpec{ | |||
| 232 | + | Name: "blog", | |||
| 233 | + | Env: map[string]string{}, | |||
| 234 | + | Services: []domain.AppService{ | |||
| 235 | + | {Name: "web", Image: "img:1", Devices: []string{"test_gpu"}}, | |||
| 236 | + | {Name: "worker", Image: "img:1", Devices: []string{"test_gpu"}}, | |||
| 237 | + | }, | |||
| 238 | + | }, | |||
| 239 | + | } | |||
| 240 | + | state.EXPECT().LoadOwnership(mock.Anything, "blog"). | |||
| 241 | + | Return(domain.AppOwnership{App: "blog", ID: "app-1"}, nil).Once() | |||
| 242 | + | images.EXPECT().ResolveDigest(mock.Anything, "img:1").Return("sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", nil).Twice() | |||
| 243 | + | runtime.EXPECT().InspectImageVolumes(mock.Anything, "img:1").Return(nil, nil).Twice() | |||
| 244 | + | runtime.EXPECT().SupportsCDIDevices(mock.Anything).Return(nil).Once() | |||
| 245 | + | state.EXPECT().LoadCheckpoint(mock.Anything).Return(domain.AppStoreCheckpoint{}, nil).Once() | |||
| 246 | + | ||||
| 247 | + | svc := NewService(Deps{State: state, Runtime: runtime, Images: images, Secrets: secrets}, zerowrap.Default()). | |||
| 248 | + | WithDevicePolicies(map[string]domain.AppDevicePolicy{ | |||
| 249 | + | "test_gpu": { | |||
| 250 | + | Name: "test_gpu", | |||
| 251 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |||
| 252 | + | AllowedApps: []string{"blog"}, | |||
| 253 | + | AllowedServices: []string{"web", "worker"}, | |||
| 254 | + | }, | |||
| 255 | + | }) | |||
| 256 | + | pinned, err := svc.preflightServices(ctx, "blog", rev, "") | |||
| 257 | + | require.NoError(t, err) | |||
| 258 | + | require.Len(t, pinned, 2) | |||
| 259 | + | runtime.AssertExpectations(t) | |||
| 260 | + | } | |||
| 261 | + | ||||
| 262 | + | // TestCreateContainer_RuntimeUnsupportedSurvivesRedaction proves the | |||
| 263 | + | // engine-unsupported sentinel from a device-bearing create survives the | |||
| 264 | + | // host-inventory redaction so callers map it to the structured | |||
| 265 | + | // runtime-unsupported envelope. | |||
| 266 | + | func TestCreateContainer_RuntimeUnsupportedSurvivesRedaction(t *testing.T) { | |||
| 267 | + | runtime := outmocks.NewMockContainerRuntime(t) | |||
| 268 | + | runtime.EXPECT().CreateContainer(mock.Anything, mock.Anything). | |||
| 269 | + | Return(nil, domain.ErrRuntimeUnsupported).Once() | |||
| 270 | + | svc := NewService(Deps{Runtime: runtime}, zerowrap.Default()) | |||
| 271 | + | ||||
| 272 | + | _, err := svc.createContainer(context.Background(), "web", &domain.ContainerConfig{ | |||
| 273 | + | CDIDevices: []string{"example.com/gpu=GPU-test-uuid"}, | |||
| 274 | + | }) | |||
| 275 | + | ||||
| 276 | + | require.Error(t, err) | |||
| 277 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |||
| 278 | + | assert.NotContains(t, err.Error(), "GPU-test-uuid", "the sanitized error must not echo CDI IDs") | |||
| 279 | + | runtime.AssertExpectations(t) | |||
| 280 | + | } | |||
| 281 | + | ||||
| 282 | + | // TestSanitizedEngineProbeError proves the capability-probe error keeps | |||
| 283 | + | // cancellation meaningful, collapses every other cause to the unsupported | |||
| 284 | + | // sentinel alone, and never echoes the adapter text. | |||
| 285 | + | func TestSanitizedEngineProbeError(t *testing.T) { | |||
| 286 | + | t.Run("cancellation is preserved and reported as cancellation", func(t *testing.T) { | |||
| 287 | + | err := sanitizedEngineProbeError("web", fmt.Errorf("dial unix /run/podman/podman.sock: %w", context.Canceled)) | |||
| 288 | + | require.ErrorIs(t, err, context.Canceled) | |||
| 289 | + | assert.False(t, errors.Is(err, domain.ErrRuntimeUnsupported), "an aborted probe is not an incapable engine") | |||
| 290 | + | assert.NotContains(t, err.Error(), "podman.sock") | |||
| 291 | + | }) | |||
| 292 | + | ||||
| 293 | + | t.Run("deadline is preserved", func(t *testing.T) { | |||
| 294 | + | err := sanitizedEngineProbeError("web", fmt.Errorf("probe: %w", context.DeadlineExceeded)) | |||
| 295 | + | require.ErrorIs(t, err, context.DeadlineExceeded) | |||
| 296 | + | assert.NotContains(t, err.Error(), "probe") | |||
| 297 | + | }) | |||
| 298 | + | ||||
| 299 | + | t.Run("other causes collapse to the unsupported sentinel", func(t *testing.T) { | |||
| 300 | + | err := sanitizedEngineProbeError("web", errors.New("dial unix /run/podman/podman.sock: connection refused")) | |||
| 301 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |||
| 302 | + | assert.NotContains(t, err.Error(), "podman.sock") | |||
| 303 | + | }) | |||
| 304 | + | } | |||
M internal/usecase/deployment/execute.go +25 -0
| @@ -3,6 +3,7 @@ package deployment | |||
| 3 | 3 | import ( | |
| 4 | 4 | "bufio" | |
| 5 | 5 | "context" | |
| 6 | + | "errors" | |
| 6 | 7 | "fmt" | |
| 7 | 8 | "io" | |
| 8 | 9 | "sort" | |
| @@ -804,6 +805,9 @@ func (s *Service) deployService(ctx context.Context, app, revision string, p pin | |||
| 804 | 805 | if _, err := s.resolveServiceBinds(app, p.spec); err != nil { | |
| 805 | 806 | return s.failResult(revision, before, "", err) | |
| 806 | 807 | } | |
| 808 | + | if _, err := s.resolveServiceDevices(app, p.spec); err != nil { | |
| 809 | + | return s.failResult(revision, before, "", err) | |
| 810 | + | } | |
| 807 | 811 | // Withdraw the service from traffic first and confirm it: a withdrawal | |
| 808 | 812 | // that cannot be applied must block every container mutation, so no | |
| 809 | 813 | // unverified generation stays routable. A first deployment has no | |
| @@ -977,6 +981,12 @@ func (s *Service) createAndStart(ctx context.Context, app, revision string, p pi | |||
| 977 | 981 | if err != nil { | |
| 978 | 982 | return nil, nil, nil, err | |
| 979 | 983 | } | |
| 984 | + | // Re-resolve devices from the current policy under the same | |
| 985 | + | // fail-closed rule: revoked grants never reach the runtime. | |
| 986 | + | resolvedDevices, err := s.resolveServiceDevices(app, p.spec) | |
| 987 | + | if err != nil { | |
| 988 | + | return nil, nil, nil, err | |
| 989 | + | } | |
| 980 | 990 | env, err := s.serviceEnv(ctx, app, p) | |
| 981 | 991 | if err != nil { | |
| 982 | 992 | return nil, nil, nil, err | |
| @@ -1026,6 +1036,7 @@ func (s *Service) createAndStart(ctx context.Context, app, revision string, p pi | |||
| 1026 | 1036 | Volumes: volumes, | |
| 1027 | 1037 | ReadOnlyVolumes: readOnlyVolumes, | |
| 1028 | 1038 | Binds: resolvedBinds, | |
| 1039 | + | CDIDevices: resolvedDevices, | |
| 1029 | 1040 | Labels: appLabels(app, p.spec.Name, revision), | |
| 1030 | 1041 | AutoRemove: false, | |
| 1031 | 1042 | RestartPolicy: domain.RestartPolicyAlways, | |
| @@ -1098,6 +1109,14 @@ func (s *Service) createContainer(ctx context.Context, service string, config *d | |||
| 1098 | 1109 | if err == nil { | |
| 1099 | 1110 | return created, nil | |
| 1100 | 1111 | } | |
| 1112 | + | // The engine-unsupported sentinel survives redaction so callers can | |
| 1113 | + | // map it to the structured runtime-unsupported envelope. It carries | |
| 1114 | + | // no host inventory (engine family/version only), and the device | |
| 1115 | + | // gate runs only for device-bearing creates, so this branch also | |
| 1116 | + | // covers services that declare both binds and devices. | |
| 1117 | + | if errors.Is(err, domain.ErrRuntimeUnsupported) { | |
| 1118 | + | return nil, fmt.Errorf("deployment: create container for service %q with administrative devices: %w", service, domain.ErrRuntimeUnsupported) | |
| 1119 | + | } | |
| 1101 | 1120 | if len(config.Binds) > 0 { | |
| 1102 | 1121 | // A CreateContainer failure is a runtime error, not a bind policy | |
| 1103 | 1122 | // violation: policy was already enforced by resolveServiceBinds | |
| @@ -1106,6 +1125,12 @@ func (s *Service) createContainer(ctx context.Context, service string, config *d | |||
| 1106 | 1125 | // keep it out of operation journals and API/CLI responses. | |
| 1107 | 1126 | return nil, fmt.Errorf("deployment: create container for service %q with administrative mounts: runtime error redacted", service) | |
| 1108 | 1127 | } | |
| 1128 | + | if len(config.CDIDevices) > 0 { | |
| 1129 | + | // Same redaction rule for device-bearing creates: runtime errors | |
| 1130 | + | // may embed resolved CDI IDs, which are host inventory. Policy | |
| 1131 | + | // was already enforced by resolveServiceDevices before this call. | |
| 1132 | + | return nil, fmt.Errorf("deployment: create container for service %q with administrative devices: runtime error redacted", service) | |
| 1133 | + | } | |
| 1109 | 1134 | return nil, fmt.Errorf("deployment: create container: %w", err) | |
| 1110 | 1135 | } | |
| 1111 | 1136 | ||
| @@ -3,6 +3,7 @@ package deployment | |||||
| 3 | import ( | 3 | import ( | ||
| 4 | "bufio" | 4 | "bufio" | ||
| 5 | "context" | 5 | "context" | ||
| 6 | + | "errors" | |||
| 6 | "fmt" | 7 | "fmt" | ||
| 7 | "io" | 8 | "io" | ||
| 8 | "sort" | 9 | "sort" | ||
| @@ -804,6 +805,9 @@ func (s *Service) deployService(ctx context.Context, app, revision string, p pin | |||||
| 804 | if _, err := s.resolveServiceBinds(app, p.spec); err != nil { | 805 | if _, err := s.resolveServiceBinds(app, p.spec); err != nil { | ||
| 805 | return s.failResult(revision, before, "", err) | 806 | return s.failResult(revision, before, "", err) | ||
| 806 | } | 807 | } | ||
| 808 | + | if _, err := s.resolveServiceDevices(app, p.spec); err != nil { | |||
| 809 | + | return s.failResult(revision, before, "", err) | |||
| 810 | + | } | |||
| 807 | // Withdraw the service from traffic first and confirm it: a withdrawal | 811 | // Withdraw the service from traffic first and confirm it: a withdrawal | ||
| 808 | // that cannot be applied must block every container mutation, so no | 812 | // that cannot be applied must block every container mutation, so no | ||
| 809 | // unverified generation stays routable. A first deployment has no | 813 | // unverified generation stays routable. A first deployment has no | ||
| @@ -977,6 +981,12 @@ func (s *Service) createAndStart(ctx context.Context, app, revision string, p pi | |||||
| 977 | if err != nil { | 981 | if err != nil { | ||
| 978 | return nil, nil, nil, err | 982 | return nil, nil, nil, err | ||
| 979 | } | 983 | } | ||
| 984 | + | // Re-resolve devices from the current policy under the same | |||
| 985 | + | // fail-closed rule: revoked grants never reach the runtime. | |||
| 986 | + | resolvedDevices, err := s.resolveServiceDevices(app, p.spec) | |||
| 987 | + | if err != nil { | |||
| 988 | + | return nil, nil, nil, err | |||
| 989 | + | } | |||
| 980 | env, err := s.serviceEnv(ctx, app, p) | 990 | env, err := s.serviceEnv(ctx, app, p) | ||
| 981 | if err != nil { | 991 | if err != nil { | ||
| 982 | return nil, nil, nil, err | 992 | return nil, nil, nil, err | ||
| @@ -1026,6 +1036,7 @@ func (s *Service) createAndStart(ctx context.Context, app, revision string, p pi | |||||
| 1026 | Volumes: volumes, | 1036 | Volumes: volumes, | ||
| 1027 | ReadOnlyVolumes: readOnlyVolumes, | 1037 | ReadOnlyVolumes: readOnlyVolumes, | ||
| 1028 | Binds: resolvedBinds, | 1038 | Binds: resolvedBinds, | ||
| 1039 | + | CDIDevices: resolvedDevices, | |||
| 1029 | Labels: appLabels(app, p.spec.Name, revision), | 1040 | Labels: appLabels(app, p.spec.Name, revision), | ||
| 1030 | AutoRemove: false, | 1041 | AutoRemove: false, | ||
| 1031 | RestartPolicy: domain.RestartPolicyAlways, | 1042 | RestartPolicy: domain.RestartPolicyAlways, | ||
| @@ -1098,6 +1109,14 @@ func (s *Service) createContainer(ctx context.Context, service string, config *d | |||||
| 1098 | if err == nil { | 1109 | if err == nil { | ||
| 1099 | return created, nil | 1110 | return created, nil | ||
| 1100 | } | 1111 | } | ||
| 1112 | + | // The engine-unsupported sentinel survives redaction so callers can | |||
| 1113 | + | // map it to the structured runtime-unsupported envelope. It carries | |||
| 1114 | + | // no host inventory (engine family/version only), and the device | |||
| 1115 | + | // gate runs only for device-bearing creates, so this branch also | |||
| 1116 | + | // covers services that declare both binds and devices. | |||
| 1117 | + | if errors.Is(err, domain.ErrRuntimeUnsupported) { | |||
| 1118 | + | return nil, fmt.Errorf("deployment: create container for service %q with administrative devices: %w", service, domain.ErrRuntimeUnsupported) | |||
| 1119 | + | } | |||
| 1101 | if len(config.Binds) > 0 { | 1120 | if len(config.Binds) > 0 { | ||
| 1102 | // A CreateContainer failure is a runtime error, not a bind policy | 1121 | // A CreateContainer failure is a runtime error, not a bind policy | ||
| 1103 | // violation: policy was already enforced by resolveServiceBinds | 1122 | // violation: policy was already enforced by resolveServiceBinds | ||
| @@ -1106,6 +1125,12 @@ func (s *Service) createContainer(ctx context.Context, service string, config *d | |||||
| 1106 | // keep it out of operation journals and API/CLI responses. | 1125 | // keep it out of operation journals and API/CLI responses. | ||
| 1107 | return nil, fmt.Errorf("deployment: create container for service %q with administrative mounts: runtime error redacted", service) | 1126 | return nil, fmt.Errorf("deployment: create container for service %q with administrative mounts: runtime error redacted", service) | ||
| 1108 | } | 1127 | } | ||
| 1128 | + | if len(config.CDIDevices) > 0 { | |||
| 1129 | + | // Same redaction rule for device-bearing creates: runtime errors | |||
| 1130 | + | // may embed resolved CDI IDs, which are host inventory. Policy | |||
| 1131 | + | // was already enforced by resolveServiceDevices before this call. | |||
| 1132 | + | return nil, fmt.Errorf("deployment: create container for service %q with administrative devices: runtime error redacted", service) | |||
| 1133 | + | } | |||
| 1109 | return nil, fmt.Errorf("deployment: create container: %w", err) | 1134 | return nil, fmt.Errorf("deployment: create container: %w", err) | ||
| 1110 | } | 1135 | } | ||
| 1111 | 1136 | ||||
M internal/usecase/deployment/lifecycle.go +15 -0
| @@ -304,6 +304,12 @@ func (s *Service) restartOneService(ctx context.Context, app, opID, name string, | |||
| 304 | 304 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | |
| 305 | 305 | return fail(err.Error()) | |
| 306 | 306 | } | |
| 307 | + | // A device grant revoked since ACTIVE was published must fail | |
| 308 | + | // before any runtime mutation, under the same rule as binds. An | |
| 309 | + | // in-place restart never rewrites device configuration. | |
| 310 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |
| 311 | + | return fail(err.Error()) | |
| 312 | + | } | |
| 307 | 313 | if err := s.refuseInhibitedRestart(ctx, app, name, eff.Container); err != nil { | |
| 308 | 314 | _ = s.withdrawForRecovery(ctx, app, name) | |
| 309 | 315 | return fail(err.Error()) | |
| @@ -606,6 +612,11 @@ func (s *Service) ensureServiceRunning(ctx context.Context, app, opID, name stri | |||
| 606 | 612 | s.failServiceStep(ctx, app, name, eff, step, result, err.Error()) | |
| 607 | 613 | return | |
| 608 | 614 | } | |
| 615 | + | // Revoked device grants fail before any runtime mutation. | |
| 616 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |
| 617 | + | s.failServiceStep(ctx, app, name, eff, step, result, err.Error()) | |
| 618 | + | return | |
| 619 | + | } | |
| 609 | 620 | // Boot recovery refuses a generation whose recovery is durably | |
| 610 | 621 | // inhibited: a replacement may already have written to its | |
| 611 | 622 | // volume, so reviving this ID could corrupt the newer data. | |
| @@ -660,6 +671,10 @@ func (s *Service) redeployPinned(ctx context.Context, app, opID, name string, ef | |||
| 660 | 671 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | |
| 661 | 672 | return failedServiceResult(eff.Container, err), err | |
| 662 | 673 | } | |
| 674 | + | // Revoked device grants fail before any runtime mutation. | |
| 675 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |
| 676 | + | return failedServiceResult(eff.Container, err), err | |
| 677 | + | } | |
| 663 | 678 | pinned := pinnedService{ | |
| 664 | 679 | name: name, spec: eff.Spec, digest: eff.Digest, runtimeImage: runtimeImage, appEnv: maps.Clone(rev.Spec.Env), | |
| 665 | 680 | appNetworks: append([]domain.AppSharedNetwork(nil), rev.Spec.Networks...), | |
| @@ -304,6 +304,12 @@ func (s *Service) restartOneService(ctx context.Context, app, opID, name string, | |||||
| 304 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | 304 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | ||
| 305 | return fail(err.Error()) | 305 | return fail(err.Error()) | ||
| 306 | } | 306 | } | ||
| 307 | + | // A device grant revoked since ACTIVE was published must fail | |||
| 308 | + | // before any runtime mutation, under the same rule as binds. An | |||
| 309 | + | // in-place restart never rewrites device configuration. | |||
| 310 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |||
| 311 | + | return fail(err.Error()) | |||
| 312 | + | } | |||
| 307 | if err := s.refuseInhibitedRestart(ctx, app, name, eff.Container); err != nil { | 313 | if err := s.refuseInhibitedRestart(ctx, app, name, eff.Container); err != nil { | ||
| 308 | _ = s.withdrawForRecovery(ctx, app, name) | 314 | _ = s.withdrawForRecovery(ctx, app, name) | ||
| 309 | return fail(err.Error()) | 315 | return fail(err.Error()) | ||
| @@ -606,6 +612,11 @@ func (s *Service) ensureServiceRunning(ctx context.Context, app, opID, name stri | |||||
| 606 | s.failServiceStep(ctx, app, name, eff, step, result, err.Error()) | 612 | s.failServiceStep(ctx, app, name, eff, step, result, err.Error()) | ||
| 607 | return | 613 | return | ||
| 608 | } | 614 | } | ||
| 615 | + | // Revoked device grants fail before any runtime mutation. | |||
| 616 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |||
| 617 | + | s.failServiceStep(ctx, app, name, eff, step, result, err.Error()) | |||
| 618 | + | return | |||
| 619 | + | } | |||
| 609 | // Boot recovery refuses a generation whose recovery is durably | 620 | // Boot recovery refuses a generation whose recovery is durably | ||
| 610 | // inhibited: a replacement may already have written to its | 621 | // inhibited: a replacement may already have written to its | ||
| 611 | // volume, so reviving this ID could corrupt the newer data. | 622 | // volume, so reviving this ID could corrupt the newer data. | ||
| @@ -660,6 +671,10 @@ func (s *Service) redeployPinned(ctx context.Context, app, opID, name string, ef | |||||
| 660 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | 671 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | ||
| 661 | return failedServiceResult(eff.Container, err), err | 672 | return failedServiceResult(eff.Container, err), err | ||
| 662 | } | 673 | } | ||
| 674 | + | // Revoked device grants fail before any runtime mutation. | |||
| 675 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |||
| 676 | + | return failedServiceResult(eff.Container, err), err | |||
| 677 | + | } | |||
| 663 | pinned := pinnedService{ | 678 | pinned := pinnedService{ | ||
| 664 | name: name, spec: eff.Spec, digest: eff.Digest, runtimeImage: runtimeImage, appEnv: maps.Clone(rev.Spec.Env), | 679 | name: name, spec: eff.Spec, digest: eff.Digest, runtimeImage: runtimeImage, appEnv: maps.Clone(rev.Spec.Env), | ||
| 665 | appNetworks: append([]domain.AppSharedNetwork(nil), rev.Spec.Networks...), | 680 | appNetworks: append([]domain.AppSharedNetwork(nil), rev.Spec.Networks...), | ||
M internal/usecase/deployment/monitor.go +8 -0
| @@ -638,6 +638,10 @@ func (s *Service) startRecovered(serviceCtx context.Context, app, name string, e | |||
| 638 | 638 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | |
| 639 | 639 | return false, err | |
| 640 | 640 | } | |
| 641 | + | // Revoked device grants fail before any runtime mutation. | |
| 642 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |
| 643 | + | return false, err | |
| 644 | + | } | |
| 641 | 645 | if err := s.deps.Runtime.StartContainer(serviceCtx, eff.Container); err != nil { | |
| 642 | 646 | if errors.Is(err, domain.ErrContainerNotFound) { | |
| 643 | 647 | s.backoff.forget(key) | |
| @@ -690,6 +694,10 @@ func (s *Service) recoverHealth(serviceCtx context.Context, app, name string, ef | |||
| 690 | 694 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | |
| 691 | 695 | return false, false, err | |
| 692 | 696 | } | |
| 697 | + | // Revoked device grants fail before any runtime mutation. | |
| 698 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |
| 699 | + | return false, false, err | |
| 700 | + | } | |
| 693 | 701 | if err := s.deps.Runtime.RestartContainer(serviceCtx, eff.Container, serviceStopGrace(eff)); err != nil { | |
| 694 | 702 | s.backoff.recordFailure(key) | |
| 695 | 703 | return false, false, fmt.Errorf("deployment: app %q service %q restart unhealthy %s: %w", app, name, eff.Container, err) | |
| @@ -638,6 +638,10 @@ func (s *Service) startRecovered(serviceCtx context.Context, app, name string, e | |||||
| 638 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | 638 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | ||
| 639 | return false, err | 639 | return false, err | ||
| 640 | } | 640 | } | ||
| 641 | + | // Revoked device grants fail before any runtime mutation. | |||
| 642 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |||
| 643 | + | return false, err | |||
| 644 | + | } | |||
| 641 | if err := s.deps.Runtime.StartContainer(serviceCtx, eff.Container); err != nil { | 645 | if err := s.deps.Runtime.StartContainer(serviceCtx, eff.Container); err != nil { | ||
| 642 | if errors.Is(err, domain.ErrContainerNotFound) { | 646 | if errors.Is(err, domain.ErrContainerNotFound) { | ||
| 643 | s.backoff.forget(key) | 647 | s.backoff.forget(key) | ||
| @@ -690,6 +694,10 @@ func (s *Service) recoverHealth(serviceCtx context.Context, app, name string, ef | |||||
| 690 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | 694 | if _, err := s.resolveServiceBinds(app, eff.Spec); err != nil { | ||
| 691 | return false, false, err | 695 | return false, false, err | ||
| 692 | } | 696 | } | ||
| 697 | + | // Revoked device grants fail before any runtime mutation. | |||
| 698 | + | if _, err := s.resolveServiceDevices(app, eff.Spec); err != nil { | |||
| 699 | + | return false, false, err | |||
| 700 | + | } | |||
| 693 | if err := s.deps.Runtime.RestartContainer(serviceCtx, eff.Container, serviceStopGrace(eff)); err != nil { | 701 | if err := s.deps.Runtime.RestartContainer(serviceCtx, eff.Container, serviceStopGrace(eff)); err != nil { | ||
| 694 | s.backoff.recordFailure(key) | 702 | s.backoff.recordFailure(key) | ||
| 695 | return false, false, fmt.Errorf("deployment: app %q service %q restart unhealthy %s: %w", app, name, eff.Container, err) | 703 | return false, false, fmt.Errorf("deployment: app %q service %q restart unhealthy %s: %w", app, name, eff.Container, err) | ||
M internal/usecase/deployment/preflight_gate_test.go +46 -0
| @@ -287,3 +287,49 @@ func TestDeploy_HoldsSharedGCBarrierAcrossResourceSelection(t *testing.T) { | |||
| 287 | 287 | assert.Equal(t, "deployed", result.Services["web"].Result) | |
| 288 | 288 | assert.False(t, held, "the shared lease must be released when the operation ends") | |
| 289 | 289 | } | |
| 290 | + | ||
| 291 | + | // deviceRevision returns a device-bearing revision over the standard | |
| 292 | + | // mockRevision shape with device policies authorizing it. | |
| 293 | + | func deviceRevision() domain.AppDesiredRevision { | |
| 294 | + | rev := mockRevision() | |
| 295 | + | rev.Spec.Services[0].Devices = []string{"test_gpu"} | |
| 296 | + | return rev | |
| 297 | + | } | |
| 298 | + | ||
| 299 | + | func devicePolicies() map[string]domain.AppDevicePolicy { | |
| 300 | + | return map[string]domain.AppDevicePolicy{ | |
| 301 | + | "test_gpu": { | |
| 302 | + | Name: "test_gpu", | |
| 303 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |
| 304 | + | AllowedApps: []string{"blog"}, | |
| 305 | + | AllowedServices: []string{"web"}, | |
| 306 | + | }, | |
| 307 | + | } | |
| 308 | + | } | |
| 309 | + | ||
| 310 | + | // TestDeploy_UnsupportedEngineFailsBeforeMutation proves a device-bearing | |
| 311 | + | // revision on an engine that cannot serve CDI fails in preflight with | |
| 312 | + | // ErrRuntimeUnsupported: the serving generation is never withdrawn and no | |
| 313 | + | // container is created, started, stopped, or removed. | |
| 314 | + | func TestDeploy_UnsupportedEngineFailsBeforeMutation(t *testing.T) { | |
| 315 | + | ctx := context.Background() | |
| 316 | + | state, runtime, images, secrets := mockDeps(t) | |
| 317 | + | rev := deviceRevision() | |
| 318 | + | ||
| 319 | + | state.EXPECT().Recover(mock.Anything).Return(nil) | |
| 320 | + | state.EXPECT().LoadDesired(mock.Anything, "blog").Return(rev, true, nil) | |
| 321 | + | state.EXPECT().LoadOwnership(mock.Anything, "blog").Return(domain.AppOwnership{App: "blog", ID: "app-blog"}, nil) | |
| 322 | + | images.EXPECT().ResolveDigest(mock.Anything, rev.Spec.Services[0].Image).Return(restartTestDigest, nil).Once() | |
| 323 | + | secrets.EXPECT().GetSecret(mock.Anything, "gordon/apps/app-blog/web/database-url").Return("x", nil) | |
| 324 | + | runtime.EXPECT().InspectImageVolumes(mock.Anything, rev.Spec.Services[0].Image).Return(nil, nil) | |
| 325 | + | runtime.EXPECT().SupportsCDIDevices(mock.Anything).Return(domain.ErrRuntimeUnsupported).Once() | |
| 326 | + | saved := recordSavedOperations(state) | |
| 327 | + | ||
| 328 | + | svc := keyedService(t, state, runtime, images, secrets).WithDevicePolicies(devicePolicies()) | |
| 329 | + | _, err := svc.Deploy(ctx, deployment.DeployInput{App: "blog"}) | |
| 330 | + | ||
| 331 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |
| 332 | + | runtime.AssertNotCalled(t, "CreateContainer", mock.Anything, mock.Anything) | |
| 333 | + | assertNoDeployMutation(t, runtime, state) | |
| 334 | + | assertPreflightGateJournal(t, *saved) | |
| 335 | + | } | |
| @@ -287,3 +287,49 @@ func TestDeploy_HoldsSharedGCBarrierAcrossResourceSelection(t *testing.T) { | |||||
| 287 | assert.Equal(t, "deployed", result.Services["web"].Result) | 287 | assert.Equal(t, "deployed", result.Services["web"].Result) | ||
| 288 | assert.False(t, held, "the shared lease must be released when the operation ends") | 288 | assert.False(t, held, "the shared lease must be released when the operation ends") | ||
| 289 | } | 289 | } | ||
| 290 | + | ||||
| 291 | + | // deviceRevision returns a device-bearing revision over the standard | |||
| 292 | + | // mockRevision shape with device policies authorizing it. | |||
| 293 | + | func deviceRevision() domain.AppDesiredRevision { | |||
| 294 | + | rev := mockRevision() | |||
| 295 | + | rev.Spec.Services[0].Devices = []string{"test_gpu"} | |||
| 296 | + | return rev | |||
| 297 | + | } | |||
| 298 | + | ||||
| 299 | + | func devicePolicies() map[string]domain.AppDevicePolicy { | |||
| 300 | + | return map[string]domain.AppDevicePolicy{ | |||
| 301 | + | "test_gpu": { | |||
| 302 | + | Name: "test_gpu", | |||
| 303 | + | CDI: []string{"example.com/gpu=GPU-test-uuid"}, | |||
| 304 | + | AllowedApps: []string{"blog"}, | |||
| 305 | + | AllowedServices: []string{"web"}, | |||
| 306 | + | }, | |||
| 307 | + | } | |||
| 308 | + | } | |||
| 309 | + | ||||
| 310 | + | // TestDeploy_UnsupportedEngineFailsBeforeMutation proves a device-bearing | |||
| 311 | + | // revision on an engine that cannot serve CDI fails in preflight with | |||
| 312 | + | // ErrRuntimeUnsupported: the serving generation is never withdrawn and no | |||
| 313 | + | // container is created, started, stopped, or removed. | |||
| 314 | + | func TestDeploy_UnsupportedEngineFailsBeforeMutation(t *testing.T) { | |||
| 315 | + | ctx := context.Background() | |||
| 316 | + | state, runtime, images, secrets := mockDeps(t) | |||
| 317 | + | rev := deviceRevision() | |||
| 318 | + | ||||
| 319 | + | state.EXPECT().Recover(mock.Anything).Return(nil) | |||
| 320 | + | state.EXPECT().LoadDesired(mock.Anything, "blog").Return(rev, true, nil) | |||
| 321 | + | state.EXPECT().LoadOwnership(mock.Anything, "blog").Return(domain.AppOwnership{App: "blog", ID: "app-blog"}, nil) | |||
| 322 | + | images.EXPECT().ResolveDigest(mock.Anything, rev.Spec.Services[0].Image).Return(restartTestDigest, nil).Once() | |||
| 323 | + | secrets.EXPECT().GetSecret(mock.Anything, "gordon/apps/app-blog/web/database-url").Return("x", nil) | |||
| 324 | + | runtime.EXPECT().InspectImageVolumes(mock.Anything, rev.Spec.Services[0].Image).Return(nil, nil) | |||
| 325 | + | runtime.EXPECT().SupportsCDIDevices(mock.Anything).Return(domain.ErrRuntimeUnsupported).Once() | |||
| 326 | + | saved := recordSavedOperations(state) | |||
| 327 | + | ||||
| 328 | + | svc := keyedService(t, state, runtime, images, secrets).WithDevicePolicies(devicePolicies()) | |||
| 329 | + | _, err := svc.Deploy(ctx, deployment.DeployInput{App: "blog"}) | |||
| 330 | + | ||||
| 331 | + | require.ErrorIs(t, err, domain.ErrRuntimeUnsupported) | |||
| 332 | + | runtime.AssertNotCalled(t, "CreateContainer", mock.Anything, mock.Anything) | |||
| 333 | + | assertNoDeployMutation(t, runtime, state) | |||
| 334 | + | assertPreflightGateJournal(t, *saved) | |||
| 335 | + | } | |||
M internal/usecase/deployment/service.go +109 -1
| @@ -103,6 +103,11 @@ type Service struct { | |||
| 103 | 103 | // the lock. | |
| 104 | 104 | bindMu sync.RWMutex | |
| 105 | 105 | bindPolicies map[string]domain.AppBindPolicy | |
| 106 | + | // deviceMu guards devicePolicies. A config reload replaces the whole | |
| 107 | + | // map atomically; deploy paths read the current map without holding | |
| 108 | + | // the lock. | |
| 109 | + | deviceMu sync.RWMutex | |
| 110 | + | devicePolicies map[string]domain.AppDevicePolicy | |
| 106 | 111 | // liveMu guards liveOps. liveOps holds the operations this process | |
| 107 | 112 | // owns between the StartDeploy claim and the end of ExecuteDeploy. It | |
| 108 | 113 | // is in-memory only: a fresh Service (boot) starts empty and may | |
| @@ -201,6 +206,51 @@ func (s *Service) snapshotBindPolicies() map[string]domain.AppBindPolicy { | |||
| 201 | 206 | return s.bindPolicies | |
| 202 | 207 | } | |
| 203 | 208 | ||
| 209 | + | // WithDevicePolicies supplies the initial administrative device policies. | |
| 210 | + | func (s *Service) WithDevicePolicies(policies map[string]domain.AppDevicePolicy) *Service { | |
| 211 | + | s.SetDevicePolicies(policies) | |
| 212 | + | return s | |
| 213 | + | } | |
| 214 | + | ||
| 215 | + | // SetDevicePolicies atomically replaces the administrative device policies | |
| 216 | + | // used to authorize service devices. The map and its slices are copied so | |
| 217 | + | // later caller mutation cannot race an in-flight deploy. | |
| 218 | + | func (s *Service) SetDevicePolicies(policies map[string]domain.AppDevicePolicy) { | |
| 219 | + | copied := make(map[string]domain.AppDevicePolicy, len(policies)) | |
| 220 | + | for name, policy := range policies { | |
| 221 | + | policy.CDI = append([]string(nil), policy.CDI...) | |
| 222 | + | policy.AllowedApps = append([]string(nil), policy.AllowedApps...) | |
| 223 | + | policy.AllowedServices = append([]string(nil), policy.AllowedServices...) | |
| 224 | + | copied[name] = policy | |
| 225 | + | } | |
| 226 | + | s.deviceMu.Lock() | |
| 227 | + | s.devicePolicies = copied | |
| 228 | + | s.deviceMu.Unlock() | |
| 229 | + | } | |
| 230 | + | ||
| 231 | + | // snapshotDevicePolicies returns the current policy map. SetDevicePolicies | |
| 232 | + | // never mutates a published map, so the snapshot stays safe after the | |
| 233 | + | // lock drops. | |
| 234 | + | func (s *Service) snapshotDevicePolicies() map[string]domain.AppDevicePolicy { | |
| 235 | + | s.deviceMu.RLock() | |
| 236 | + | defer s.deviceMu.RUnlock() | |
| 237 | + | return s.devicePolicies | |
| 238 | + | } | |
| 239 | + | ||
| 240 | + | // resolveServiceDevices resolves every declared logical device against the | |
| 241 | + | // current administrative policies into runtime CDI IDs. Errors name only | |
| 242 | + | // the app, service, and device: host inventory never appears. | |
| 243 | + | func (s *Service) resolveServiceDevices(app string, svc domain.AppService) ([]string, error) { | |
| 244 | + | if len(svc.Devices) == 0 { | |
| 245 | + | return nil, nil | |
| 246 | + | } | |
| 247 | + | ids, err := domain.ResolveAppDevices(app, svc.Name, svc.Devices, s.snapshotDevicePolicies()) | |
| 248 | + | if err != nil { | |
| 249 | + | return nil, fmt.Errorf("deployment: %w", err) | |
| 250 | + | } | |
| 251 | + | return ids, nil | |
| 252 | + | } | |
| 253 | + | ||
| 204 | 254 | // resolveServiceBinds resolves every declared bind against the current | |
| 205 | 255 | // administrative policies into ephemeral runtime binds. Errors name only the | |
| 206 | 256 | // app, service, and mount: host source paths never appear. | |
| @@ -617,7 +667,7 @@ func selectPreflightServices(rev domain.AppDesiredRevision, onlyService string) | |||
| 617 | 667 | return nil, fmt.Errorf("deployment: service %q not in revision %s: %w", onlyService, rev.Revision, domain.ErrAppStateConflict) | |
| 618 | 668 | } | |
| 619 | 669 | ||
| 620 | - | // preflightServices runs the five preflight gates in order. No mutation. | |
| 670 | + | // preflightServices runs the preflight gates in order. No mutation. | |
| 621 | 671 | func (s *Service) preflightServices(ctx context.Context, app string, rev domain.AppDesiredRevision, onlyService string) ([]pinnedService, error) { | |
| 622 | 672 | services, err := selectPreflightServices(rev, onlyService) | |
| 623 | 673 | if err != nil { | |
| @@ -661,6 +711,9 @@ func (s *Service) preflightServices(ctx context.Context, app string, rev domain. | |||
| 661 | 711 | if _, err := s.resolveServiceBinds(app, svc); err != nil { | |
| 662 | 712 | return nil, err | |
| 663 | 713 | } | |
| 714 | + | if err := s.preflightServiceDevices(ctx, app, svc, pinned); err != nil { | |
| 715 | + | return nil, err | |
| 716 | + | } | |
| 664 | 717 | pinned = append(pinned, pinnedService{ | |
| 665 | 718 | name: svc.Name, spec: svc, digest: digest, runtimeImage: runtimeImage, appEnv: appEnv, | |
| 666 | 719 | appNetworks: append([]domain.AppSharedNetwork(nil), rev.Spec.Networks...), | |
| @@ -676,6 +729,61 @@ func (s *Service) preflightServices(ctx context.Context, app string, rev domain. | |||
| 676 | 729 | return pinned, nil | |
| 677 | 730 | } | |
| 678 | 731 | ||
| 732 | + | // preflightServiceDevices runs the device authorization and engine | |
| 733 | + | // capability gates for one service. Split from preflightServices to keep | |
| 734 | + | // its complexity within budget. | |
| 735 | + | func (s *Service) preflightServiceDevices(ctx context.Context, app string, svc domain.AppService, pinned []pinnedService) error { | |
| 736 | + | if _, err := s.resolveServiceDevices(app, svc); err != nil { | |
| 737 | + | return err | |
| 738 | + | } | |
| 739 | + | // Engine capability is a knowable-before-mutation failure: a | |
| 740 | + | // device-bearing revision on an unsupported engine must fail here, | |
| 741 | + | // before withdrawal retires the serving generation. The check runs | |
| 742 | + | // once per revision, on the first device-bearing service; | |
| 743 | + | // CreateContainer keeps the same gate as defense. | |
| 744 | + | if len(svc.Devices) > 0 && !devicesEngineChecked(pinned) { | |
| 745 | + | if s.deps.Runtime == nil { | |
| 746 | + | return fmt.Errorf("deployment: runtime unavailable: %w", domain.ErrRuntimeUnsupported) | |
| 747 | + | } | |
| 748 | + | // Sanitize the adapter cause: version-probe failures may embed | |
| 749 | + | // the daemon endpoint, which must not reach journals or API | |
| 750 | + | // responses. Cancellation and deadlines are the exception: they | |
| 751 | + | // stay recognizable so aborted deploys are not misread as | |
| 752 | + | // incapable engines. | |
| 753 | + | if err := s.deps.Runtime.SupportsCDIDevices(ctx); err != nil { | |
| 754 | + | return sanitizedEngineProbeError(svc.Name, err) | |
| 755 | + | } | |
| 756 | + | } | |
| 757 | + | return nil | |
| 758 | + | } | |
| 759 | + | ||
| 760 | + | // sanitizedEngineProbeError maps an engine device-capability probe failure | |
| 761 | + | // onto the caller-visible error. Cancellation and deadlines survive so | |
| 762 | + | // shutdown and timeout handling keep working; every other cause collapses | |
| 763 | + | // to ErrRuntimeUnsupported alone, dropping the adapter text that may embed | |
| 764 | + | // the daemon endpoint. | |
| 765 | + | func sanitizedEngineProbeError(service string, err error) error { | |
| 766 | + | switch { | |
| 767 | + | case errors.Is(err, context.Canceled): | |
| 768 | + | return fmt.Errorf("deployment: engine device capability check for service %q: %w", service, context.Canceled) | |
| 769 | + | case errors.Is(err, context.DeadlineExceeded): | |
| 770 | + | return fmt.Errorf("deployment: engine device capability check for service %q: %w", service, context.DeadlineExceeded) | |
| 771 | + | default: | |
| 772 | + | return fmt.Errorf("deployment: engine device capability check for service %q: %w", service, domain.ErrRuntimeUnsupported) | |
| 773 | + | } | |
| 774 | + | } | |
| 775 | + | ||
| 776 | + | // devicesEngineChecked reports whether an earlier pinned service already | |
| 777 | + | // triggered the engine capability probe for this revision. | |
| 778 | + | func devicesEngineChecked(pinned []pinnedService) bool { | |
| 779 | + | for _, p := range pinned { | |
| 780 | + | if len(p.spec.Devices) > 0 { | |
| 781 | + | return true | |
| 782 | + | } | |
| 783 | + | } | |
| 784 | + | return false | |
| 785 | + | } | |
| 786 | + | ||
| 679 | 787 | // checkSecrets reads every required secret path once. Values stay in | |
| 680 | 788 | // memory for container creation only; nothing is written anywhere. | |
| 681 | 789 | // Paths are UUID-keyed so a removed app's secrets are never adopted | |
| @@ -103,6 +103,11 @@ type Service struct { | |||||
| 103 | // the lock. | 103 | // the lock. | ||
| 104 | bindMu sync.RWMutex | 104 | bindMu sync.RWMutex | ||
| 105 | bindPolicies map[string]domain.AppBindPolicy | 105 | bindPolicies map[string]domain.AppBindPolicy | ||
| 106 | + | // deviceMu guards devicePolicies. A config reload replaces the whole | |||
| 107 | + | // map atomically; deploy paths read the current map without holding | |||
| 108 | + | // the lock. | |||
| 109 | + | deviceMu sync.RWMutex | |||
| 110 | + | devicePolicies map[string]domain.AppDevicePolicy | |||
| 106 | // liveMu guards liveOps. liveOps holds the operations this process | 111 | // liveMu guards liveOps. liveOps holds the operations this process | ||
| 107 | // owns between the StartDeploy claim and the end of ExecuteDeploy. It | 112 | // owns between the StartDeploy claim and the end of ExecuteDeploy. It | ||
| 108 | // is in-memory only: a fresh Service (boot) starts empty and may | 113 | // is in-memory only: a fresh Service (boot) starts empty and may | ||
| @@ -201,6 +206,51 @@ func (s *Service) snapshotBindPolicies() map[string]domain.AppBindPolicy { | |||||
| 201 | return s.bindPolicies | 206 | return s.bindPolicies | ||
| 202 | } | 207 | } | ||
| 203 | 208 | ||||
| 209 | + | // WithDevicePolicies supplies the initial administrative device policies. | |||
| 210 | + | func (s *Service) WithDevicePolicies(policies map[string]domain.AppDevicePolicy) *Service { | |||
| 211 | + | s.SetDevicePolicies(policies) | |||
| 212 | + | return s | |||
| 213 | + | } | |||
| 214 | + | ||||
| 215 | + | // SetDevicePolicies atomically replaces the administrative device policies | |||
| 216 | + | // used to authorize service devices. The map and its slices are copied so | |||
| 217 | + | // later caller mutation cannot race an in-flight deploy. | |||
| 218 | + | func (s *Service) SetDevicePolicies(policies map[string]domain.AppDevicePolicy) { | |||
| 219 | + | copied := make(map[string]domain.AppDevicePolicy, len(policies)) | |||
| 220 | + | for name, policy := range policies { | |||
| 221 | + | policy.CDI = append([]string(nil), policy.CDI...) | |||
| 222 | + | policy.AllowedApps = append([]string(nil), policy.AllowedApps...) | |||
| 223 | + | policy.AllowedServices = append([]string(nil), policy.AllowedServices...) | |||
| 224 | + | copied[name] = policy | |||
| 225 | + | } | |||
| 226 | + | s.deviceMu.Lock() | |||
| 227 | + | s.devicePolicies = copied | |||
| 228 | + | s.deviceMu.Unlock() | |||
| 229 | + | } | |||
| 230 | + | ||||
| 231 | + | // snapshotDevicePolicies returns the current policy map. SetDevicePolicies | |||
| 232 | + | // never mutates a published map, so the snapshot stays safe after the | |||
| 233 | + | // lock drops. | |||
| 234 | + | func (s *Service) snapshotDevicePolicies() map[string]domain.AppDevicePolicy { | |||
| 235 | + | s.deviceMu.RLock() | |||
| 236 | + | defer s.deviceMu.RUnlock() | |||
| 237 | + | return s.devicePolicies | |||
| 238 | + | } | |||
| 239 | + | ||||
| 240 | + | // resolveServiceDevices resolves every declared logical device against the | |||
| 241 | + | // current administrative policies into runtime CDI IDs. Errors name only | |||
| 242 | + | // the app, service, and device: host inventory never appears. | |||
| 243 | + | func (s *Service) resolveServiceDevices(app string, svc domain.AppService) ([]string, error) { | |||
| 244 | + | if len(svc.Devices) == 0 { | |||
| 245 | + | return nil, nil | |||
| 246 | + | } | |||
| 247 | + | ids, err := domain.ResolveAppDevices(app, svc.Name, svc.Devices, s.snapshotDevicePolicies()) | |||
| 248 | + | if err != nil { | |||
| 249 | + | return nil, fmt.Errorf("deployment: %w", err) | |||
| 250 | + | } | |||
| 251 | + | return ids, nil | |||
| 252 | + | } | |||
| 253 | + | ||||
| 204 | // resolveServiceBinds resolves every declared bind against the current | 254 | // resolveServiceBinds resolves every declared bind against the current | ||
| 205 | // administrative policies into ephemeral runtime binds. Errors name only the | 255 | // administrative policies into ephemeral runtime binds. Errors name only the | ||
| 206 | // app, service, and mount: host source paths never appear. | 256 | // app, service, and mount: host source paths never appear. | ||
| @@ -617,7 +667,7 @@ func selectPreflightServices(rev domain.AppDesiredRevision, onlyService string) | |||||
| 617 | return nil, fmt.Errorf("deployment: service %q not in revision %s: %w", onlyService, rev.Revision, domain.ErrAppStateConflict) | 667 | return nil, fmt.Errorf("deployment: service %q not in revision %s: %w", onlyService, rev.Revision, domain.ErrAppStateConflict) | ||
| 618 | } | 668 | } | ||
| 619 | 669 | ||||
| 620 | - | // preflightServices runs the five preflight gates in order. No mutation. | 670 | + | // preflightServices runs the preflight gates in order. No mutation. |
| 621 | func (s *Service) preflightServices(ctx context.Context, app string, rev domain.AppDesiredRevision, onlyService string) ([]pinnedService, error) { | 671 | func (s *Service) preflightServices(ctx context.Context, app string, rev domain.AppDesiredRevision, onlyService string) ([]pinnedService, error) { | ||
| 622 | services, err := selectPreflightServices(rev, onlyService) | 672 | services, err := selectPreflightServices(rev, onlyService) | ||
| 623 | if err != nil { | 673 | if err != nil { | ||
| @@ -661,6 +711,9 @@ func (s *Service) preflightServices(ctx context.Context, app string, rev domain. | |||||
| 661 | if _, err := s.resolveServiceBinds(app, svc); err != nil { | 711 | if _, err := s.resolveServiceBinds(app, svc); err != nil { | ||
| 662 | return nil, err | 712 | return nil, err | ||
| 663 | } | 713 | } | ||
| 714 | + | if err := s.preflightServiceDevices(ctx, app, svc, pinned); err != nil { | |||
| 715 | + | return nil, err | |||
| 716 | + | } | |||
| 664 | pinned = append(pinned, pinnedService{ | 717 | pinned = append(pinned, pinnedService{ | ||
| 665 | name: svc.Name, spec: svc, digest: digest, runtimeImage: runtimeImage, appEnv: appEnv, | 718 | name: svc.Name, spec: svc, digest: digest, runtimeImage: runtimeImage, appEnv: appEnv, | ||
| 666 | appNetworks: append([]domain.AppSharedNetwork(nil), rev.Spec.Networks...), | 719 | appNetworks: append([]domain.AppSharedNetwork(nil), rev.Spec.Networks...), | ||
| @@ -676,6 +729,61 @@ func (s *Service) preflightServices(ctx context.Context, app string, rev domain. | |||||
| 676 | return pinned, nil | 729 | return pinned, nil | ||
| 677 | } | 730 | } | ||
| 678 | 731 | ||||
| 732 | + | // preflightServiceDevices runs the device authorization and engine | |||
| 733 | + | // capability gates for one service. Split from preflightServices to keep | |||
| 734 | + | // its complexity within budget. | |||
| 735 | + | func (s *Service) preflightServiceDevices(ctx context.Context, app string, svc domain.AppService, pinned []pinnedService) error { | |||
| 736 | + | if _, err := s.resolveServiceDevices(app, svc); err != nil { | |||
| 737 | + | return err | |||
| 738 | + | } | |||
| 739 | + | // Engine capability is a knowable-before-mutation failure: a | |||
| 740 | + | // device-bearing revision on an unsupported engine must fail here, | |||
| 741 | + | // before withdrawal retires the serving generation. The check runs | |||
| 742 | + | // once per revision, on the first device-bearing service; | |||
| 743 | + | // CreateContainer keeps the same gate as defense. | |||
| 744 | + | if len(svc.Devices) > 0 && !devicesEngineChecked(pinned) { | |||
| 745 | + | if s.deps.Runtime == nil { | |||
| 746 | + | return fmt.Errorf("deployment: runtime unavailable: %w", domain.ErrRuntimeUnsupported) | |||
| 747 | + | } | |||
| 748 | + | // Sanitize the adapter cause: version-probe failures may embed | |||
| 749 | + | // the daemon endpoint, which must not reach journals or API | |||
| 750 | + | // responses. Cancellation and deadlines are the exception: they | |||
| 751 | + | // stay recognizable so aborted deploys are not misread as | |||
| 752 | + | // incapable engines. | |||
| 753 | + | if err := s.deps.Runtime.SupportsCDIDevices(ctx); err != nil { | |||
| 754 | + | return sanitizedEngineProbeError(svc.Name, err) | |||
| 755 | + | } | |||
| 756 | + | } | |||
| 757 | + | return nil | |||
| 758 | + | } | |||
| 759 | + | ||||
| 760 | + | // sanitizedEngineProbeError maps an engine device-capability probe failure | |||
| 761 | + | // onto the caller-visible error. Cancellation and deadlines survive so | |||
| 762 | + | // shutdown and timeout handling keep working; every other cause collapses | |||
| 763 | + | // to ErrRuntimeUnsupported alone, dropping the adapter text that may embed | |||
| 764 | + | // the daemon endpoint. | |||
| 765 | + | func sanitizedEngineProbeError(service string, err error) error { | |||
| 766 | + | switch { | |||
| 767 | + | case errors.Is(err, context.Canceled): | |||
| 768 | + | return fmt.Errorf("deployment: engine device capability check for service %q: %w", service, context.Canceled) | |||
| 769 | + | case errors.Is(err, context.DeadlineExceeded): | |||
| 770 | + | return fmt.Errorf("deployment: engine device capability check for service %q: %w", service, context.DeadlineExceeded) | |||
| 771 | + | default: | |||
| 772 | + | return fmt.Errorf("deployment: engine device capability check for service %q: %w", service, domain.ErrRuntimeUnsupported) | |||
| 773 | + | } | |||
| 774 | + | } | |||
| 775 | + | ||||
| 776 | + | // devicesEngineChecked reports whether an earlier pinned service already | |||
| 777 | + | // triggered the engine capability probe for this revision. | |||
| 778 | + | func devicesEngineChecked(pinned []pinnedService) bool { | |||
| 779 | + | for _, p := range pinned { | |||
| 780 | + | if len(p.spec.Devices) > 0 { | |||
| 781 | + | return true | |||
| 782 | + | } | |||
| 783 | + | } | |||
| 784 | + | return false | |||
| 785 | + | } | |||
| 786 | + | ||||
| 679 | // checkSecrets reads every required secret path once. Values stay in | 787 | // checkSecrets reads every required secret path once. Values stay in | ||
| 680 | // memory for container creation only; nothing is written anywhere. | 788 | // memory for container creation only; nothing is written anywhere. | ||
| 681 | // Paths are UUID-keyed so a removed app's secrets are never adopted | 789 | // Paths are UUID-keyed so a removed app's secrets are never adopted | ||