feat(ci): workflow_dispatch, schedule and workflow_run triggers (#34)
- commit
- 8a6cfecd74244a0a5fbccc67073ce8dbfc21f848
- author
- brice · · 2026-10-15 10:13 UTC
- committer
- bnema · · 2026-10-15 10:13 UTC
- parent
- 815fb98
- signature
- SSH Unverified
SHA256:VOCA5lYARHcbKDZ7GhTV45VmcjWnJ6ZQIBazj17LivAThe key is unknown - refs
- tag: v0.15.0
48 files · +4384 -805
M README.md +9 -1
| @@ -115,7 +115,7 @@ A merge request proposes a branch for its target (the default branch unless set) | |||
| 115 | 115 | ||
| 116 | 116 | ## CI | |
| 117 | 117 | ||
| 118 | - | ermite runs the `.github/workflows/*.yml` workflows of a repository on `push` (a branch or a tag, and the merge of a merge request) and on `pull_request` (the merge requests, below) and shows each job as a check on the commit, the branch list and the merge request. CI is on once a runner is registered; there is no switch. Before that a push makes no run and no check. | |
| 118 | + | ermite runs the `.github/workflows/*.yml` workflows of a repository on `push` (a branch or a tag, and the merge of a merge request), on `pull_request` (the merge requests, below), `workflow_dispatch`, `schedule` and `workflow_run` (below) and shows each job as a check on the commit, the branch list and the merge request. CI is on once a runner is registered; there is no switch. Before that a push makes no run and no check. | |
| 119 | 119 | ||
| 120 | 120 | Runs execute on [ermite-runner](https://git.bnema.dev/bnema/ermite-runner) hosts, which call ermite over its API. Create a token for the runner host with `ermite token create <label> --preset runner` (the `runner:register` scope, which also gives access to the code and the secrets of every repository: give it only to runners you operate), then register the runner with it. `ermite runner list` shows the runners and whether they are online; `ermite runner remove <id>` stops one. A job goes to a runner that has every label of its `runs-on`; a job no runner takes within 24 hours fails. | |
| 121 | 121 | ||
| @@ -125,6 +125,14 @@ An action of `ERMITE_OWNER/<repo>` that exists on this server wins over the same | |||
| 125 | 125 | ||
| 126 | 126 | **`pull_request`.** A merge request is a pull request for the workflows. ermite keeps `refs/pull/<n>/head` at the head of its source branch and, while that head merges cleanly into the target, `refs/pull/<n>/merge` at a test merge commit (parents: the target tip, then the head; message `Merge <head> into <base>`). Both are served to `git fetch`, never listed as branches or tags, and refused on every push (for any token, a job's included). The activities are `opened` (the request is created), `synchronize` (its source branch is pushed; each push runs for the commit it pushed, with the rights of whoever pushed it) and `closed` (closed or merged, with `merged` true or false; it uses the last test merge commit, or the head when there is none, and is never computed again, even when the source branch is gone); `reopened` is accepted in `types:` but merge requests cannot be reopened. Without `types:` a workflow gets `opened`, `synchronize` and `reopened`. The run has `github.ref` `refs/pull/<n>/merge`, `github.sha` the merge commit (the workflow file is read from it, so `actions/checkout` gets the merged tree), `github.head_ref` and `github.base_ref` set, and a payload shaped like GitHub's `pull_request` webhook with the fields ermite has. `branches` and `paths` (and their `-ignore` forms) apply to the target branch and to the files changed between the merge base and the head. A head that conflicts with the target makes no run for that update. The checks of the run are on the head commit. The run is trusted (secrets, write token) only when whoever caused the event is the owner, like a push; a job token starts no run. | |
| 127 | 127 | ||
| 128 | + | **`workflow_dispatch`.** `ermite workflow run <file> --repo <repo> [--ref <branch or tag>] [-f name=value …]` starts a run of a workflow that declares `workflow_dispatch` (the API is `RunService.DispatchWorkflow`, feature `ci.dispatch`; it needs `ci:write`). The file is its path or its name under `.github/workflows`, and it is read at the ref (the default branch when `--ref` is left out). The inputs are checked against the workflow: an undeclared input, a missing `required` input without a default, a `boolean` other than `true` or `false`, a `number` that is not a decimal number, a `choice` outside its `options`, and an `environment` that is not a valid environment name are refused; a `choice` without default takes its first option. A workflow has at most 25 inputs and their values at most 65,535 characters. In the run, `github.event.inputs` holds the values as strings and the `inputs` context as their declared types (`boolean`, `number`, else string). The actor is the caller. The run is trusted (secrets, write token) only when **both** the caller is the owner **and** the ref it runs on was moved to its current commit by the owner (see `schedule`): the owner dispatching an agent's branch gets an untrusted run, like the owner re-running an agent's run, and a ref with no record (pushed before ermite recorded pushers) is untrusted too. This holds for a tag as for a branch, and for a job token as for the owner. A token limited to branch prefixes can dispatch only on a branch under them and its run is never trusted. A job can dispatch with its repository token when its `permissions:` give `actions: write` (GitHub's exception to "events made with `GITHUB_TOKEN` start no run"), even without `contents: read`; a token that does not hold the permission is refused. | |
| 129 | + | ||
| 130 | + | **`schedule`.** `on.schedule[].cron` (five fields, in UTC; `timezone` is ignored) is read on the **default branch** only. Every minute the scheduler starts the workflows whose cron matches that minute, on the latest commit of the default branch; the payload is `{"schedule": "<cron>"}`. A minute is handled once: ticks missed while ermite was down or busy are not made up, and a workflow with several matching entries starts one run. The actor is **whoever last moved the default branch to its current commit** (a push or a merge), and the run is trusted only if that was the owner. ermite records who moves each branch and tag **before** it moves (in the pre-receive decision of a push, and in the merge under its lock; a record that cannot be written refuses the push), together with the commit it is for: the record counts only while the ref still points to that commit, so a late write cannot leave the owner's record on a tip an agent moved there (owner A, agent B, agent A). A tip with no record, a record for another commit (a push refused after its record was written) or an unreadable record runs untrusted, without secrets. If the owner and a token push the very same commit at the same time, the owner's record may remain: the content is the same and the owner pushed it. At most 64 repositories are looked at per second, in rotation, so a pass over more takes several seconds and never starves one. The schedules of a repository are read again when its default branch moves, and go with the repository. | |
| 131 | + | ||
| 132 | + | **`workflow_run`.** A workflow that lists `workflow_run: workflows: [<name>]` runs when a run of a workflow with that `name` is `requested` (created) or `completed` (`types:` picks one; `in_progress` never happens). `branches` and `branches-ignore` apply to the head branch of the triggering run (the source branch of a merge request). The workflow is read from the **default branch** and runs on its latest commit; the payload has `workflow_run` (`id`, `name`, `head_branch`, `head_sha`, `event`, `status`, `conclusion`, `run_number`, `run_attempt`, `path`, `actor`) and `workflow`. The run inherits the actor and the trust of the triggering run and never gains any: a workflow_run of an untrusted run has no secrets. A chain stops after three levels of `workflow_run`. | |
| 133 | + | ||
| 134 | + | The event of a run is `push`, `pull_request`, `workflow_dispatch`, `schedule` or `workflow_run`: `ermite run list` and `view` and the Actions pages show it. | |
| 135 | + | ||
| 128 | 136 | The Actions that run here call a small part of GitHub's REST API, served at `github.api_url` (`/-/github/api`): `GET /repos/{owner}/{repo}`, `/repos/{owner}/{repo}/hash-algorithm`, `/repos/{owner}/{repo}/tarball[/{ref}]` (streamed with 200, not redirected; a `HEAD` answers the headers without building the archive) and `GET /repos/{owner}/{repo}/pulls/{number}/files` (what `dorny/paths-filter` reads on a `pull_request`: up to 3000 files, `page` and `per_page` of at most 100, 30 by default, with the `Link` header; a job token needs `pull-requests: read`, else `403 Resource not accessible by integration`). The owner in these paths and in the Git URLs is matched without regard to case, as on GitHub. Any other path is GitHub's `404 {"message":"Not Found",…}`, so an action that needs more fails as on a GitHub instance without that feature. | |
| 129 | 137 | ||
| 130 | 138 | Actions from github.com are resolved with `git ls-remote https://github.com/<owner>/<repo>` (no credentials, 20 s, 16 MiB of output, at most four at once, 60 s for a whole call). ermite hands git only the proxy settings of its own environment (`HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`, `ALL_PROXY` and their lower-case forms), so a server that reaches the Internet through a proxy sets them for `ermite serve`; nothing else of its environment, no credential, reaches git. | |
| @@ -115,7 +115,7 @@ A merge request proposes a branch for its target (the default branch unless set) | |||||
| 115 | 115 | ||||
| 116 | ## CI | 116 | ## CI | ||
| 117 | 117 | ||||
| 118 | - | ermite runs the `.github/workflows/*.yml` workflows of a repository on `push` (a branch or a tag, and the merge of a merge request) and on `pull_request` (the merge requests, below) and shows each job as a check on the commit, the branch list and the merge request. CI is on once a runner is registered; there is no switch. Before that a push makes no run and no check. | 118 | + | ermite runs the `.github/workflows/*.yml` workflows of a repository on `push` (a branch or a tag, and the merge of a merge request), on `pull_request` (the merge requests, below), `workflow_dispatch`, `schedule` and `workflow_run` (below) and shows each job as a check on the commit, the branch list and the merge request. CI is on once a runner is registered; there is no switch. Before that a push makes no run and no check. |
| 119 | 119 | ||||
| 120 | Runs execute on [ermite-runner](https://git.bnema.dev/bnema/ermite-runner) hosts, which call ermite over its API. Create a token for the runner host with `ermite token create <label> --preset runner` (the `runner:register` scope, which also gives access to the code and the secrets of every repository: give it only to runners you operate), then register the runner with it. `ermite runner list` shows the runners and whether they are online; `ermite runner remove <id>` stops one. A job goes to a runner that has every label of its `runs-on`; a job no runner takes within 24 hours fails. | 120 | Runs execute on [ermite-runner](https://git.bnema.dev/bnema/ermite-runner) hosts, which call ermite over its API. Create a token for the runner host with `ermite token create <label> --preset runner` (the `runner:register` scope, which also gives access to the code and the secrets of every repository: give it only to runners you operate), then register the runner with it. `ermite runner list` shows the runners and whether they are online; `ermite runner remove <id>` stops one. A job goes to a runner that has every label of its `runs-on`; a job no runner takes within 24 hours fails. | ||
| 121 | 121 | ||||
| @@ -125,6 +125,14 @@ An action of `ERMITE_OWNER/<repo>` that exists on this server wins over the same | |||||
| 125 | 125 | ||||
| 126 | **`pull_request`.** A merge request is a pull request for the workflows. ermite keeps `refs/pull/<n>/head` at the head of its source branch and, while that head merges cleanly into the target, `refs/pull/<n>/merge` at a test merge commit (parents: the target tip, then the head; message `Merge <head> into <base>`). Both are served to `git fetch`, never listed as branches or tags, and refused on every push (for any token, a job's included). The activities are `opened` (the request is created), `synchronize` (its source branch is pushed; each push runs for the commit it pushed, with the rights of whoever pushed it) and `closed` (closed or merged, with `merged` true or false; it uses the last test merge commit, or the head when there is none, and is never computed again, even when the source branch is gone); `reopened` is accepted in `types:` but merge requests cannot be reopened. Without `types:` a workflow gets `opened`, `synchronize` and `reopened`. The run has `github.ref` `refs/pull/<n>/merge`, `github.sha` the merge commit (the workflow file is read from it, so `actions/checkout` gets the merged tree), `github.head_ref` and `github.base_ref` set, and a payload shaped like GitHub's `pull_request` webhook with the fields ermite has. `branches` and `paths` (and their `-ignore` forms) apply to the target branch and to the files changed between the merge base and the head. A head that conflicts with the target makes no run for that update. The checks of the run are on the head commit. The run is trusted (secrets, write token) only when whoever caused the event is the owner, like a push; a job token starts no run. | 126 | **`pull_request`.** A merge request is a pull request for the workflows. ermite keeps `refs/pull/<n>/head` at the head of its source branch and, while that head merges cleanly into the target, `refs/pull/<n>/merge` at a test merge commit (parents: the target tip, then the head; message `Merge <head> into <base>`). Both are served to `git fetch`, never listed as branches or tags, and refused on every push (for any token, a job's included). The activities are `opened` (the request is created), `synchronize` (its source branch is pushed; each push runs for the commit it pushed, with the rights of whoever pushed it) and `closed` (closed or merged, with `merged` true or false; it uses the last test merge commit, or the head when there is none, and is never computed again, even when the source branch is gone); `reopened` is accepted in `types:` but merge requests cannot be reopened. Without `types:` a workflow gets `opened`, `synchronize` and `reopened`. The run has `github.ref` `refs/pull/<n>/merge`, `github.sha` the merge commit (the workflow file is read from it, so `actions/checkout` gets the merged tree), `github.head_ref` and `github.base_ref` set, and a payload shaped like GitHub's `pull_request` webhook with the fields ermite has. `branches` and `paths` (and their `-ignore` forms) apply to the target branch and to the files changed between the merge base and the head. A head that conflicts with the target makes no run for that update. The checks of the run are on the head commit. The run is trusted (secrets, write token) only when whoever caused the event is the owner, like a push; a job token starts no run. | ||
| 127 | 127 | ||||
| 128 | + | **`workflow_dispatch`.** `ermite workflow run <file> --repo <repo> [--ref <branch or tag>] [-f name=value …]` starts a run of a workflow that declares `workflow_dispatch` (the API is `RunService.DispatchWorkflow`, feature `ci.dispatch`; it needs `ci:write`). The file is its path or its name under `.github/workflows`, and it is read at the ref (the default branch when `--ref` is left out). The inputs are checked against the workflow: an undeclared input, a missing `required` input without a default, a `boolean` other than `true` or `false`, a `number` that is not a decimal number, a `choice` outside its `options`, and an `environment` that is not a valid environment name are refused; a `choice` without default takes its first option. A workflow has at most 25 inputs and their values at most 65,535 characters. In the run, `github.event.inputs` holds the values as strings and the `inputs` context as their declared types (`boolean`, `number`, else string). The actor is the caller. The run is trusted (secrets, write token) only when **both** the caller is the owner **and** the ref it runs on was moved to its current commit by the owner (see `schedule`): the owner dispatching an agent's branch gets an untrusted run, like the owner re-running an agent's run, and a ref with no record (pushed before ermite recorded pushers) is untrusted too. This holds for a tag as for a branch, and for a job token as for the owner. A token limited to branch prefixes can dispatch only on a branch under them and its run is never trusted. A job can dispatch with its repository token when its `permissions:` give `actions: write` (GitHub's exception to "events made with `GITHUB_TOKEN` start no run"), even without `contents: read`; a token that does not hold the permission is refused. | |||
| 129 | + | ||||
| 130 | + | **`schedule`.** `on.schedule[].cron` (five fields, in UTC; `timezone` is ignored) is read on the **default branch** only. Every minute the scheduler starts the workflows whose cron matches that minute, on the latest commit of the default branch; the payload is `{"schedule": "<cron>"}`. A minute is handled once: ticks missed while ermite was down or busy are not made up, and a workflow with several matching entries starts one run. The actor is **whoever last moved the default branch to its current commit** (a push or a merge), and the run is trusted only if that was the owner. ermite records who moves each branch and tag **before** it moves (in the pre-receive decision of a push, and in the merge under its lock; a record that cannot be written refuses the push), together with the commit it is for: the record counts only while the ref still points to that commit, so a late write cannot leave the owner's record on a tip an agent moved there (owner A, agent B, agent A). A tip with no record, a record for another commit (a push refused after its record was written) or an unreadable record runs untrusted, without secrets. If the owner and a token push the very same commit at the same time, the owner's record may remain: the content is the same and the owner pushed it. At most 64 repositories are looked at per second, in rotation, so a pass over more takes several seconds and never starves one. The schedules of a repository are read again when its default branch moves, and go with the repository. | |||
| 131 | + | ||||
| 132 | + | **`workflow_run`.** A workflow that lists `workflow_run: workflows: [<name>]` runs when a run of a workflow with that `name` is `requested` (created) or `completed` (`types:` picks one; `in_progress` never happens). `branches` and `branches-ignore` apply to the head branch of the triggering run (the source branch of a merge request). The workflow is read from the **default branch** and runs on its latest commit; the payload has `workflow_run` (`id`, `name`, `head_branch`, `head_sha`, `event`, `status`, `conclusion`, `run_number`, `run_attempt`, `path`, `actor`) and `workflow`. The run inherits the actor and the trust of the triggering run and never gains any: a workflow_run of an untrusted run has no secrets. A chain stops after three levels of `workflow_run`. | |||
| 133 | + | ||||
| 134 | + | The event of a run is `push`, `pull_request`, `workflow_dispatch`, `schedule` or `workflow_run`: `ermite run list` and `view` and the Actions pages show it. | |||
| 135 | + | ||||
| 128 | The Actions that run here call a small part of GitHub's REST API, served at `github.api_url` (`/-/github/api`): `GET /repos/{owner}/{repo}`, `/repos/{owner}/{repo}/hash-algorithm`, `/repos/{owner}/{repo}/tarball[/{ref}]` (streamed with 200, not redirected; a `HEAD` answers the headers without building the archive) and `GET /repos/{owner}/{repo}/pulls/{number}/files` (what `dorny/paths-filter` reads on a `pull_request`: up to 3000 files, `page` and `per_page` of at most 100, 30 by default, with the `Link` header; a job token needs `pull-requests: read`, else `403 Resource not accessible by integration`). The owner in these paths and in the Git URLs is matched without regard to case, as on GitHub. Any other path is GitHub's `404 {"message":"Not Found",…}`, so an action that needs more fails as on a GitHub instance without that feature. | 136 | The Actions that run here call a small part of GitHub's REST API, served at `github.api_url` (`/-/github/api`): `GET /repos/{owner}/{repo}`, `/repos/{owner}/{repo}/hash-algorithm`, `/repos/{owner}/{repo}/tarball[/{ref}]` (streamed with 200, not redirected; a `HEAD` answers the headers without building the archive) and `GET /repos/{owner}/{repo}/pulls/{number}/files` (what `dorny/paths-filter` reads on a `pull_request`: up to 3000 files, `page` and `per_page` of at most 100, 30 by default, with the `Link` header; a job token needs `pull-requests: read`, else `403 Resource not accessible by integration`). The owner in these paths and in the Git URLs is matched without regard to case, as on GitHub. Any other path is GitHub's `404 {"message":"Not Found",…}`, so an action that needs more fails as on a GitHub instance without that feature. | ||
| 129 | 137 | ||||
| 130 | Actions from github.com are resolved with `git ls-remote https://github.com/<owner>/<repo>` (no credentials, 20 s, 16 MiB of output, at most four at once, 60 s for a whole call). ermite hands git only the proxy settings of its own environment (`HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`, `ALL_PROXY` and their lower-case forms), so a server that reaches the Internet through a proxy sets them for `ermite serve`; nothing else of its environment, no credential, reaches git. | 138 | Actions from github.com are resolved with `git ls-remote https://github.com/<owner>/<repo>` (no credentials, 20 s, 16 MiB of output, at most four at once, 60 s for a whole call). ermite hands git only the proxy settings of its own environment (`HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`, `ALL_PROXY` and their lower-case forms), so a server that reaches the Internet through a proxy sets them for `ermite serve`; nothing else of its environment, no credential, reaches git. | ||
M gen/ermite/storage/ci.pb.go +127 -36 generated file: collapsed
M internal/adapters/client/client_test.go +3 -3
| @@ -139,16 +139,16 @@ func TestDescribeBoundsLongMessages(t *testing.T) { | |||
| 139 | 139 | func TestMissingFeatures(t *testing.T) { | |
| 140 | 140 | assert.Empty(t, client.MissingFeatures(client.UsedFeatures())) | |
| 141 | 141 | assert.Equal(t, | |
| 142 | - | []string{client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSigningKey, client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets}, | |
| 142 | + | []string{client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSigningKey, client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, client.FeatureCIDispatch}, | |
| 143 | 143 | client.MissingFeatures([]string{client.FeatureRepoUpdate, client.FeatureSession, client.FeaturePasskeys, "other"})) | |
| 144 | 144 | assert.Equal(t, []string{client.FeatureSigningKey}, client.MissingFeatures([]string{ | |
| 145 | 145 | client.FeatureRepoUpdate, client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSession, | |
| 146 | - | client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, | |
| 146 | + | client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, client.FeatureCIDispatch, | |
| 147 | 147 | client.FeaturePasskeys, | |
| 148 | 148 | })) | |
| 149 | 149 | assert.Equal(t, []string{client.FeaturePasskeys}, client.MissingFeatures([]string{ | |
| 150 | 150 | client.FeatureRepoUpdate, client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSigningKey, client.FeatureSession, | |
| 151 | - | client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, | |
| 151 | + | client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, client.FeatureCIDispatch, | |
| 152 | 152 | })) | |
| 153 | 153 | assert.Equal(t, client.UsedFeatures(), client.MissingFeatures(nil)) | |
| 154 | 154 | } | |
| @@ -139,16 +139,16 @@ func TestDescribeBoundsLongMessages(t *testing.T) { | |||||
| 139 | func TestMissingFeatures(t *testing.T) { | 139 | func TestMissingFeatures(t *testing.T) { | ||
| 140 | assert.Empty(t, client.MissingFeatures(client.UsedFeatures())) | 140 | assert.Empty(t, client.MissingFeatures(client.UsedFeatures())) | ||
| 141 | assert.Equal(t, | 141 | assert.Equal(t, | ||
| 142 | - | []string{client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSigningKey, client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets}, | 142 | + | []string{client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSigningKey, client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, client.FeatureCIDispatch}, |
| 143 | client.MissingFeatures([]string{client.FeatureRepoUpdate, client.FeatureSession, client.FeaturePasskeys, "other"})) | 143 | client.MissingFeatures([]string{client.FeatureRepoUpdate, client.FeatureSession, client.FeaturePasskeys, "other"})) | ||
| 144 | assert.Equal(t, []string{client.FeatureSigningKey}, client.MissingFeatures([]string{ | 144 | assert.Equal(t, []string{client.FeatureSigningKey}, client.MissingFeatures([]string{ | ||
| 145 | client.FeatureRepoUpdate, client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSession, | 145 | client.FeatureRepoUpdate, client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSession, | ||
| 146 | - | client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, | 146 | + | client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, client.FeatureCIDispatch, |
| 147 | client.FeaturePasskeys, | 147 | client.FeaturePasskeys, | ||
| 148 | })) | 148 | })) | ||
| 149 | assert.Equal(t, []string{client.FeaturePasskeys}, client.MissingFeatures([]string{ | 149 | assert.Equal(t, []string{client.FeaturePasskeys}, client.MissingFeatures([]string{ | ||
| 150 | client.FeatureRepoUpdate, client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSigningKey, client.FeatureSession, | 150 | client.FeatureRepoUpdate, client.FeatureEventsResume, client.FeatureMergeRequests, client.FeatureMergeDeleteBranch, client.FeatureSigningKey, client.FeatureSession, | ||
| 151 | - | client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, | 151 | + | client.FeatureTokenBranchPrefixes, client.FeatureCIRuns, client.FeatureCIRunners, client.FeatureCISecrets, client.FeatureCIDispatch, |
| 152 | })) | 152 | })) | ||
| 153 | assert.Equal(t, client.UsedFeatures(), client.MissingFeatures(nil)) | 153 | assert.Equal(t, client.UsedFeatures(), client.MissingFeatures(nil)) | ||
| 154 | } | 154 | } | ||
M internal/adapters/client/features.go +3 -1
| @@ -40,11 +40,13 @@ const ( | |||
| 40 | 40 | // FeatureCISecrets is SecretService, VariableService and EnvironmentService | |
| 41 | 41 | // (`ermite secret`, `variable`, `environment`). | |
| 42 | 42 | FeatureCISecrets = "ci.secrets" | |
| 43 | + | // FeatureCIDispatch is RunService.DispatchWorkflow (`ermite workflow run`). | |
| 44 | + | FeatureCIDispatch = "ci.dispatch" | |
| 43 | 45 | ) | |
| 44 | 46 | ||
| 45 | 47 | // UsedFeatures lists the features the CLI uses, in the order they are reported. | |
| 46 | 48 | func UsedFeatures() []string { | |
| 47 | - | return []string{FeatureRepoUpdate, FeatureEventsResume, FeatureMergeRequests, FeatureMergeDeleteBranch, FeatureSigningKey, FeatureSession, FeatureTokenBranchPrefixes, FeatureCIRuns, FeatureCIRunners, FeatureCISecrets, FeaturePasskeys} | |
| 49 | + | return []string{FeatureRepoUpdate, FeatureEventsResume, FeatureMergeRequests, FeatureMergeDeleteBranch, FeatureSigningKey, FeatureSession, FeatureTokenBranchPrefixes, FeatureCIRuns, FeatureCIRunners, FeatureCISecrets, FeatureCIDispatch, FeaturePasskeys} | |
| 48 | 50 | } | |
| 49 | 51 | ||
| 50 | 52 | // MissingFeatures returns the features the CLI uses that have is lacking. | |
| @@ -40,11 +40,13 @@ const ( | |||||
| 40 | // FeatureCISecrets is SecretService, VariableService and EnvironmentService | 40 | // FeatureCISecrets is SecretService, VariableService and EnvironmentService | ||
| 41 | // (`ermite secret`, `variable`, `environment`). | 41 | // (`ermite secret`, `variable`, `environment`). | ||
| 42 | FeatureCISecrets = "ci.secrets" | 42 | FeatureCISecrets = "ci.secrets" | ||
| 43 | + | // FeatureCIDispatch is RunService.DispatchWorkflow (`ermite workflow run`). | |||
| 44 | + | FeatureCIDispatch = "ci.dispatch" | |||
| 43 | ) | 45 | ) | ||
| 44 | 46 | ||||
| 45 | // UsedFeatures lists the features the CLI uses, in the order they are reported. | 47 | // UsedFeatures lists the features the CLI uses, in the order they are reported. | ||
| 46 | func UsedFeatures() []string { | 48 | func UsedFeatures() []string { | ||
| 47 | - | return []string{FeatureRepoUpdate, FeatureEventsResume, FeatureMergeRequests, FeatureMergeDeleteBranch, FeatureSigningKey, FeatureSession, FeatureTokenBranchPrefixes, FeatureCIRuns, FeatureCIRunners, FeatureCISecrets, FeaturePasskeys} | 49 | + | return []string{FeatureRepoUpdate, FeatureEventsResume, FeatureMergeRequests, FeatureMergeDeleteBranch, FeatureSigningKey, FeatureSession, FeatureTokenBranchPrefixes, FeatureCIRuns, FeatureCIRunners, FeatureCISecrets, FeatureCIDispatch, FeaturePasskeys} |
| 48 | } | 50 | } | ||
| 49 | 51 | ||||
| 50 | // MissingFeatures returns the features the CLI uses that have is lacking. | 52 | // MissingFeatures returns the features the CLI uses that have is lacking. | ||
M internal/adapters/kvstore/ci.go +2 -1
| @@ -38,6 +38,7 @@ import ( | |||
| 38 | 38 | // ciactive/<job key> job assigned or running | |
| 39 | 39 | // jobtoken/<hash> job token (value: job key) | |
| 40 | 40 | // concur/<repo>/<key(group)> concurrency group (ci_concurrency.go) | |
| 41 | + | // cireftip/<repo>/<ref> who moves a branch or tag to its object (ci_tip.go) | |
| 41 | 42 | // | |
| 42 | 43 | // The job repository tokens and the action resolutions are in ci_token.go, the | |
| 43 | 44 | // annotations in ci_annotation.go. | |
| @@ -885,7 +886,7 @@ func ciKeys(ctx context.Context, snap *kv.Snapshot, repo domain.RepoName) [][]by | |||
| 885 | 886 | keys = append(keys, slices.Clone(key)) | |
| 886 | 887 | } | |
| 887 | 888 | keys = append(keys, repoReaderKeys(ctx, snap, repo)...) | |
| 888 | - | for _, p := range []string{checkPrefix + repo.String() + "/", runNumPrefix + repo.String() + "/", concurPrefix + repo.String() + "/", concurMarkPrefixAll + repo.String() + "/"} { | |
| 889 | + | for _, p := range []string{checkPrefix + repo.String() + "/", runNumPrefix + repo.String() + "/", concurPrefix + repo.String() + "/", concurMarkPrefixAll + repo.String() + "/", refTipsPrefix(repo)} { | |
| 889 | 890 | for k := range snap.Prefix(ctx, []byte(p)) { | |
| 890 | 891 | keys = append(keys, slices.Clone(k)) | |
| 891 | 892 | } | |
| @@ -38,6 +38,7 @@ import ( | |||||
| 38 | // ciactive/<job key> job assigned or running | 38 | // ciactive/<job key> job assigned or running | ||
| 39 | // jobtoken/<hash> job token (value: job key) | 39 | // jobtoken/<hash> job token (value: job key) | ||
| 40 | // concur/<repo>/<key(group)> concurrency group (ci_concurrency.go) | 40 | // concur/<repo>/<key(group)> concurrency group (ci_concurrency.go) | ||
| 41 | + | // cireftip/<repo>/<ref> who moves a branch or tag to its object (ci_tip.go) | |||
| 41 | // | 42 | // | ||
| 42 | // The job repository tokens and the action resolutions are in ci_token.go, the | 43 | // The job repository tokens and the action resolutions are in ci_token.go, the | ||
| 43 | // annotations in ci_annotation.go. | 44 | // annotations in ci_annotation.go. | ||
| @@ -885,7 +886,7 @@ func ciKeys(ctx context.Context, snap *kv.Snapshot, repo domain.RepoName) [][]by | |||||
| 885 | keys = append(keys, slices.Clone(key)) | 886 | keys = append(keys, slices.Clone(key)) | ||
| 886 | } | 887 | } | ||
| 887 | keys = append(keys, repoReaderKeys(ctx, snap, repo)...) | 888 | keys = append(keys, repoReaderKeys(ctx, snap, repo)...) | ||
| 888 | - | for _, p := range []string{checkPrefix + repo.String() + "/", runNumPrefix + repo.String() + "/", concurPrefix + repo.String() + "/", concurMarkPrefixAll + repo.String() + "/"} { | 889 | + | for _, p := range []string{checkPrefix + repo.String() + "/", runNumPrefix + repo.String() + "/", concurPrefix + repo.String() + "/", concurMarkPrefixAll + repo.String() + "/", refTipsPrefix(repo)} { |
| 889 | for k := range snap.Prefix(ctx, []byte(p)) { | 890 | for k := range snap.Prefix(ctx, []byte(p)) { | ||
| 890 | keys = append(keys, slices.Clone(k)) | 891 | keys = append(keys, slices.Clone(k)) | ||
| 891 | } | 892 | } | ||
A internal/adapters/kvstore/ci_tip.go +91 -0
| @@ -0,0 +1,91 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package kvstore | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "errors" | |
| 8 | + | "fmt" | |
| 9 | + | ||
| 10 | + | kv "bnema.dev/ermite-kv" | |
| 11 | + | "google.golang.org/protobuf/proto" | |
| 12 | + | ||
| 13 | + | storagepb "bnema.dev/ermite/gen/ermite/storage" | |
| 14 | + | "bnema.dev/ermite/internal/domain" | |
| 15 | + | "bnema.dev/ermite/internal/ports" | |
| 16 | + | ) | |
| 17 | + | ||
| 18 | + | // Who moves a branch or a tag to its object (CI-3, D1; domain.CIRefTip): | |
| 19 | + | // | |
| 20 | + | // cireftip/<repo>/<full ref name> CIRefTip | |
| 21 | + | // | |
| 22 | + | // Written under the guard of the repository key, deleted with the repository | |
| 23 | + | // (ciKeys). Ref names hold '/', so the key is split at the first '/' after the | |
| 24 | + | // repository, whose name holds none. | |
| 25 | + | const refTipPrefix = "cireftip/" | |
| 26 | + | ||
| 27 | + | var _ ports.CIRefTipStore = (*CI)(nil) | |
| 28 | + | ||
| 29 | + | func refTipKey(repo domain.RepoName, ref string) []byte { | |
| 30 | + | return []byte(refTipPrefix + repo.String() + "/" + ref) | |
| 31 | + | } | |
| 32 | + | ||
| 33 | + | func refTipsPrefix(repo domain.RepoName) string { return refTipPrefix + repo.String() + "/" } | |
| 34 | + | ||
| 35 | + | // updateAttemptsTip bounds the retries of a write that conflicts with another | |
| 36 | + | // write of the repository. | |
| 37 | + | const updateAttemptsTip = 8 | |
| 38 | + | ||
| 39 | + | // PutRefTip records who moves the ref to its object. A repository that does not | |
| 40 | + | // exist is domain.ErrNotFound. | |
| 41 | + | func (c *CI) PutRefTip(ctx context.Context, t domain.CIRefTip) error { | |
| 42 | + | val, err := proto.Marshal(&storagepb.CIRefTip{ | |
| 43 | + | Sha: t.Sha, Actor: t.Actor, Trusted: t.Trusted, | |
| 44 | + | PusherKind: string(t.Pusher.Kind), PusherTokenId: t.Pusher.TokenID, | |
| 45 | + | }) | |
| 46 | + | if err != nil { | |
| 47 | + | return fmt.Errorf("encode ref tip: %w", err) | |
| 48 | + | } | |
| 49 | + | for range updateAttemptsTip { | |
| 50 | + | var b kv.Batch | |
| 51 | + | if err := c.guardRepo(ctx, &b, t.Repo); err != nil { | |
| 52 | + | return err | |
| 53 | + | } | |
| 54 | + | b.Put(refTipKey(t.Repo, t.Ref), val) | |
| 55 | + | _, err := c.db.Apply(ctx, &b) | |
| 56 | + | if errors.Is(err, kv.ErrConflict) { | |
| 57 | + | continue | |
| 58 | + | } | |
| 59 | + | if err != nil { | |
| 60 | + | return fmt.Errorf("store ref tip: %w", err) | |
| 61 | + | } | |
| 62 | + | return nil | |
| 63 | + | } | |
| 64 | + | return fmt.Errorf("store ref tip: %w", domain.ErrConflict) | |
| 65 | + | } | |
| 66 | + | ||
| 67 | + | // GetRefTip reads the record of a ref, or domain.ErrNotFound. | |
| 68 | + | func (c *CI) GetRefTip(ctx context.Context, repo domain.RepoName, ref string) (domain.CIRefTip, error) { | |
| 69 | + | it, ok := c.db.Get(ctx, refTipKey(repo, ref)) | |
| 70 | + | if !ok { | |
| 71 | + | return domain.CIRefTip{}, domain.ErrNotFound | |
| 72 | + | } | |
| 73 | + | var pb storagepb.CIRefTip | |
| 74 | + | if err := proto.Unmarshal(it.Value, &pb); err != nil { | |
| 75 | + | return domain.CIRefTip{}, fmt.Errorf("decode ref tip: %w", err) | |
| 76 | + | } | |
| 77 | + | return domain.CIRefTip{ | |
| 78 | + | Repo: repo, Ref: ref, Sha: pb.GetSha(), Actor: pb.GetActor(), Trusted: pb.GetTrusted(), | |
| 79 | + | Pusher: domain.CIPusher{Kind: domain.CIPusherKind(pb.GetPusherKind()), TokenID: pb.GetPusherTokenId()}, | |
| 80 | + | }, nil | |
| 81 | + | } | |
| 82 | + | ||
| 83 | + | // DeleteRefTip removes the record of a ref (the ref is deleted). | |
| 84 | + | func (c *CI) DeleteRefTip(ctx context.Context, repo domain.RepoName, ref string) error { | |
| 85 | + | var b kv.Batch | |
| 86 | + | b.Delete(refTipKey(repo, ref)) | |
| 87 | + | if _, err := c.db.Apply(ctx, &b); err != nil { | |
| 88 | + | return fmt.Errorf("delete ref tip: %w", err) | |
| 89 | + | } | |
| 90 | + | return nil | |
| 91 | + | } | |
| @@ -0,0 +1,91 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package kvstore | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "context" | |||
| 7 | + | "errors" | |||
| 8 | + | "fmt" | |||
| 9 | + | ||||
| 10 | + | kv "bnema.dev/ermite-kv" | |||
| 11 | + | "google.golang.org/protobuf/proto" | |||
| 12 | + | ||||
| 13 | + | storagepb "bnema.dev/ermite/gen/ermite/storage" | |||
| 14 | + | "bnema.dev/ermite/internal/domain" | |||
| 15 | + | "bnema.dev/ermite/internal/ports" | |||
| 16 | + | ) | |||
| 17 | + | ||||
| 18 | + | // Who moves a branch or a tag to its object (CI-3, D1; domain.CIRefTip): | |||
| 19 | + | // | |||
| 20 | + | // cireftip/<repo>/<full ref name> CIRefTip | |||
| 21 | + | // | |||
| 22 | + | // Written under the guard of the repository key, deleted with the repository | |||
| 23 | + | // (ciKeys). Ref names hold '/', so the key is split at the first '/' after the | |||
| 24 | + | // repository, whose name holds none. | |||
| 25 | + | const refTipPrefix = "cireftip/" | |||
| 26 | + | ||||
| 27 | + | var _ ports.CIRefTipStore = (*CI)(nil) | |||
| 28 | + | ||||
| 29 | + | func refTipKey(repo domain.RepoName, ref string) []byte { | |||
| 30 | + | return []byte(refTipPrefix + repo.String() + "/" + ref) | |||
| 31 | + | } | |||
| 32 | + | ||||
| 33 | + | func refTipsPrefix(repo domain.RepoName) string { return refTipPrefix + repo.String() + "/" } | |||
| 34 | + | ||||
| 35 | + | // updateAttemptsTip bounds the retries of a write that conflicts with another | |||
| 36 | + | // write of the repository. | |||
| 37 | + | const updateAttemptsTip = 8 | |||
| 38 | + | ||||
| 39 | + | // PutRefTip records who moves the ref to its object. A repository that does not | |||
| 40 | + | // exist is domain.ErrNotFound. | |||
| 41 | + | func (c *CI) PutRefTip(ctx context.Context, t domain.CIRefTip) error { | |||
| 42 | + | val, err := proto.Marshal(&storagepb.CIRefTip{ | |||
| 43 | + | Sha: t.Sha, Actor: t.Actor, Trusted: t.Trusted, | |||
| 44 | + | PusherKind: string(t.Pusher.Kind), PusherTokenId: t.Pusher.TokenID, | |||
| 45 | + | }) | |||
| 46 | + | if err != nil { | |||
| 47 | + | return fmt.Errorf("encode ref tip: %w", err) | |||
| 48 | + | } | |||
| 49 | + | for range updateAttemptsTip { | |||
| 50 | + | var b kv.Batch | |||
| 51 | + | if err := c.guardRepo(ctx, &b, t.Repo); err != nil { | |||
| 52 | + | return err | |||
| 53 | + | } | |||
| 54 | + | b.Put(refTipKey(t.Repo, t.Ref), val) | |||
| 55 | + | _, err := c.db.Apply(ctx, &b) | |||
| 56 | + | if errors.Is(err, kv.ErrConflict) { | |||
| 57 | + | continue | |||
| 58 | + | } | |||
| 59 | + | if err != nil { | |||
| 60 | + | return fmt.Errorf("store ref tip: %w", err) | |||
| 61 | + | } | |||
| 62 | + | return nil | |||
| 63 | + | } | |||
| 64 | + | return fmt.Errorf("store ref tip: %w", domain.ErrConflict) | |||
| 65 | + | } | |||
| 66 | + | ||||
| 67 | + | // GetRefTip reads the record of a ref, or domain.ErrNotFound. | |||
| 68 | + | func (c *CI) GetRefTip(ctx context.Context, repo domain.RepoName, ref string) (domain.CIRefTip, error) { | |||
| 69 | + | it, ok := c.db.Get(ctx, refTipKey(repo, ref)) | |||
| 70 | + | if !ok { | |||
| 71 | + | return domain.CIRefTip{}, domain.ErrNotFound | |||
| 72 | + | } | |||
| 73 | + | var pb storagepb.CIRefTip | |||
| 74 | + | if err := proto.Unmarshal(it.Value, &pb); err != nil { | |||
| 75 | + | return domain.CIRefTip{}, fmt.Errorf("decode ref tip: %w", err) | |||
| 76 | + | } | |||
| 77 | + | return domain.CIRefTip{ | |||
| 78 | + | Repo: repo, Ref: ref, Sha: pb.GetSha(), Actor: pb.GetActor(), Trusted: pb.GetTrusted(), | |||
| 79 | + | Pusher: domain.CIPusher{Kind: domain.CIPusherKind(pb.GetPusherKind()), TokenID: pb.GetPusherTokenId()}, | |||
| 80 | + | }, nil | |||
| 81 | + | } | |||
| 82 | + | ||||
| 83 | + | // DeleteRefTip removes the record of a ref (the ref is deleted). | |||
| 84 | + | func (c *CI) DeleteRefTip(ctx context.Context, repo domain.RepoName, ref string) error { | |||
| 85 | + | var b kv.Batch | |||
| 86 | + | b.Delete(refTipKey(repo, ref)) | |||
| 87 | + | if _, err := c.db.Apply(ctx, &b); err != nil { | |||
| 88 | + | return fmt.Errorf("delete ref tip: %w", err) | |||
| 89 | + | } | |||
| 90 | + | return nil | |||
| 91 | + | } | |||
M internal/adapters/rpc/api.go +1 -1
| @@ -34,7 +34,7 @@ const RunnerMaxRequestBytes = MaxRequestBytes + 64<<10 | |||
| 34 | 34 | // always implements, as GetVersion lists them. FeaturePasskeys comes on top | |
| 35 | 35 | // when Config.Passkeys is set, and FeatureCISecrets when the settings services | |
| 36 | 36 | // are. | |
| 37 | - | var Features = []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency"} | |
| 37 | + | var Features = []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.dispatch"} | |
| 38 | 38 | ||
| 39 | 39 | // FeaturePasskeys is listed by GetVersion when passkeys are available. | |
| 40 | 40 | const FeaturePasskeys = "passkeys" | |
| @@ -34,7 +34,7 @@ const RunnerMaxRequestBytes = MaxRequestBytes + 64<<10 | |||||
| 34 | // always implements, as GetVersion lists them. FeaturePasskeys comes on top | 34 | // always implements, as GetVersion lists them. FeaturePasskeys comes on top | ||
| 35 | // when Config.Passkeys is set, and FeatureCISecrets when the settings services | 35 | // when Config.Passkeys is set, and FeatureCISecrets when the settings services | ||
| 36 | // are. | 36 | // are. | ||
| 37 | - | var Features = []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency"} | 37 | + | var Features = []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.dispatch"} |
| 38 | 38 | ||||
| 39 | // FeaturePasskeys is listed by GetVersion when passkeys are available. | 39 | // FeaturePasskeys is listed by GetVersion when passkeys are available. | ||
| 40 | const FeaturePasskeys = "passkeys" | 40 | const FeaturePasskeys = "passkeys" | ||
M internal/adapters/rpc/ci.go +14 -2
| @@ -208,8 +208,6 @@ func (s *runnerService) UploadSummary(ctx context.Context, req *connect.Request[ | |||
| 208 | 208 | ||
| 209 | 209 | // runService implements ermiteconnect.RunServiceHandler. | |
| 210 | 210 | type runService struct { | |
| 211 | - | // DispatchWorkflow is not implemented yet: the feature "ci.dispatch" is not | |
| 212 | - | // listed and the RPC answers unimplemented. | |
| 213 | 211 | ermiteconnect.UnimplementedRunServiceHandler | |
| 214 | 212 | runs ports.RunService | |
| 215 | 213 | } | |
| @@ -314,6 +312,20 @@ func (s *runService) RerunRun(ctx context.Context, req *connect.Request[pb.Rerun | |||
| 314 | 312 | return connect.NewResponse(&pb.RerunRunResponse{Run: runToPB(run)}), nil | |
| 315 | 313 | } | |
| 316 | 314 | ||
| 315 | + | func (s *runService) DispatchWorkflow(ctx context.Context, req *connect.Request[pb.DispatchWorkflowRequest]) (*connect.Response[pb.DispatchWorkflowResponse], error) { | |
| 316 | + | name, err := repoOf(req.Msg.GetRepo()) | |
| 317 | + | if err != nil { | |
| 318 | + | return nil, ciError(ctx, err) | |
| 319 | + | } | |
| 320 | + | run, err := s.runs.DispatchWorkflow(ctx, caller(ctx), name, ports.DispatchParams{ | |
| 321 | + | WorkflowPath: req.Msg.GetWorkflowPath(), Ref: req.Msg.GetRef(), Inputs: req.Msg.GetInputs(), | |
| 322 | + | }) | |
| 323 | + | if err != nil { | |
| 324 | + | return nil, ciError(ctx, err) | |
| 325 | + | } | |
| 326 | + | return connect.NewResponse(&pb.DispatchWorkflowResponse{Run: runToPB(run)}), nil | |
| 327 | + | } | |
| 328 | + | ||
| 317 | 329 | func (s *runService) ListChecks(ctx context.Context, req *connect.Request[pb.ListChecksRequest]) (*connect.Response[pb.ListChecksResponse], error) { | |
| 318 | 330 | name, err := repoOf(req.Msg.GetRepo()) | |
| 319 | 331 | if err != nil { | |
| @@ -208,8 +208,6 @@ func (s *runnerService) UploadSummary(ctx context.Context, req *connect.Request[ | |||||
| 208 | 208 | ||||
| 209 | // runService implements ermiteconnect.RunServiceHandler. | 209 | // runService implements ermiteconnect.RunServiceHandler. | ||
| 210 | type runService struct { | 210 | type runService struct { | ||
| 211 | - | // DispatchWorkflow is not implemented yet: the feature "ci.dispatch" is not | |||
| 212 | - | // listed and the RPC answers unimplemented. | |||
| 213 | ermiteconnect.UnimplementedRunServiceHandler | 211 | ermiteconnect.UnimplementedRunServiceHandler | ||
| 214 | runs ports.RunService | 212 | runs ports.RunService | ||
| 215 | } | 213 | } | ||
| @@ -314,6 +312,20 @@ func (s *runService) RerunRun(ctx context.Context, req *connect.Request[pb.Rerun | |||||
| 314 | return connect.NewResponse(&pb.RerunRunResponse{Run: runToPB(run)}), nil | 312 | return connect.NewResponse(&pb.RerunRunResponse{Run: runToPB(run)}), nil | ||
| 315 | } | 313 | } | ||
| 316 | 314 | ||||
| 315 | + | func (s *runService) DispatchWorkflow(ctx context.Context, req *connect.Request[pb.DispatchWorkflowRequest]) (*connect.Response[pb.DispatchWorkflowResponse], error) { | |||
| 316 | + | name, err := repoOf(req.Msg.GetRepo()) | |||
| 317 | + | if err != nil { | |||
| 318 | + | return nil, ciError(ctx, err) | |||
| 319 | + | } | |||
| 320 | + | run, err := s.runs.DispatchWorkflow(ctx, caller(ctx), name, ports.DispatchParams{ | |||
| 321 | + | WorkflowPath: req.Msg.GetWorkflowPath(), Ref: req.Msg.GetRef(), Inputs: req.Msg.GetInputs(), | |||
| 322 | + | }) | |||
| 323 | + | if err != nil { | |||
| 324 | + | return nil, ciError(ctx, err) | |||
| 325 | + | } | |||
| 326 | + | return connect.NewResponse(&pb.DispatchWorkflowResponse{Run: runToPB(run)}), nil | |||
| 327 | + | } | |||
| 328 | + | ||||
| 317 | func (s *runService) ListChecks(ctx context.Context, req *connect.Request[pb.ListChecksRequest]) (*connect.Response[pb.ListChecksResponse], error) { | 329 | func (s *runService) ListChecks(ctx context.Context, req *connect.Request[pb.ListChecksRequest]) (*connect.Response[pb.ListChecksResponse], error) { | ||
| 318 | name, err := repoOf(req.Msg.GetRepo()) | 330 | name, err := repoOf(req.Msg.GetRepo()) | ||
| 319 | if err != nil { | 331 | if err != nil { | ||
M internal/adapters/web/actions.templ +5 -0
| @@ -40,6 +40,9 @@ templ RunListRow(r RunRow) { | |||
| 40 | 40 | <td class="c-name"> | |
| 41 | 41 | <a href={ templ.SafeURL(r.Href) }>{ r.Workflow }</a> | |
| 42 | 42 | <span class="mute">{ r.Number }</span> | |
| 43 | + | if r.Event != "" { | |
| 44 | + | <span class="pill">{ r.Event }</span> | |
| 45 | + | } | |
| 43 | 46 | if r.Attempt > 1 { | |
| 44 | 47 | <span class="pill">attempt { itoa(int(r.Attempt)) }</span> | |
| 45 | 48 | } | |
| @@ -239,6 +242,8 @@ templ runRail(p RunPage) { | |||
| 239 | 242 | <dl class="kv"> | |
| 240 | 243 | <dt>result</dt> | |
| 241 | 244 | <dd>{ p.State }</dd> | |
| 245 | + | <dt>event</dt> | |
| 246 | + | <dd>{ p.Event }</dd> | |
| 242 | 247 | <dt>commit</dt> | |
| 243 | 248 | <dd><a href={ templ.SafeURL(p.CHref) }>{ p.Short }</a></dd> | |
| 244 | 249 | <dt>ref</dt> | |
| @@ -40,6 +40,9 @@ templ RunListRow(r RunRow) { | |||||
| 40 | <td class="c-name"> | 40 | <td class="c-name"> | ||
| 41 | <a href={ templ.SafeURL(r.Href) }>{ r.Workflow }</a> | 41 | <a href={ templ.SafeURL(r.Href) }>{ r.Workflow }</a> | ||
| 42 | <span class="mute">{ r.Number }</span> | 42 | <span class="mute">{ r.Number }</span> | ||
| 43 | + | if r.Event != "" { | |||
| 44 | + | <span class="pill">{ r.Event }</span> | |||
| 45 | + | } | |||
| 43 | if r.Attempt > 1 { | 46 | if r.Attempt > 1 { | ||
| 44 | <span class="pill">attempt { itoa(int(r.Attempt)) }</span> | 47 | <span class="pill">attempt { itoa(int(r.Attempt)) }</span> | ||
| 45 | } | 48 | } | ||
| @@ -239,6 +242,8 @@ templ runRail(p RunPage) { | |||||
| 239 | <dl class="kv"> | 242 | <dl class="kv"> | ||
| 240 | <dt>result</dt> | 243 | <dt>result</dt> | ||
| 241 | <dd>{ p.State }</dd> | 244 | <dd>{ p.State }</dd> | ||
| 245 | + | <dt>event</dt> | |||
| 246 | + | <dd>{ p.Event }</dd> | |||
| 242 | <dt>commit</dt> | 247 | <dt>commit</dt> | ||
| 243 | <dd><a href={ templ.SafeURL(p.CHref) }>{ p.Short }</a></dd> | 248 | <dd><a href={ templ.SafeURL(p.CHref) }>{ p.Short }</a></dd> | ||
| 244 | <dt>ref</dt> | 249 | <dt>ref</dt> | ||
M internal/adapters/web/actions_pages.go +4 -1
| @@ -119,6 +119,9 @@ type RunRow struct { | |||
| 119 | 119 | Kind string // ciKind | |
| 120 | 120 | State string // ciState | |
| 121 | 121 | Workflow string | |
| 122 | + | // Event is the event that triggered the run (push, pull_request, | |
| 123 | + | // workflow_dispatch, schedule, workflow_run). | |
| 124 | + | Event string | |
| 122 | 125 | Title string | |
| 123 | 126 | Ref string | |
| 124 | 127 | RefHref string | |
| @@ -228,7 +231,7 @@ func (s *Server) runRow(repo string, run domain.CIRun, now time.Time) RunRow { | |||
| 228 | 231 | row := RunRow{ | |
| 229 | 232 | ID: strconv.FormatInt(run.ID, 10), Href: runHref(repo, run.ID), | |
| 230 | 233 | Kind: ciKind(run.Status, run.Conclusion), State: ciState(run.Status, run.Conclusion), | |
| 231 | - | Workflow: visibleText(run.WorkflowName), Title: visibleText(run.Title), | |
| 234 | + | Workflow: visibleText(run.WorkflowName), Event: visibleText(run.Event), Title: visibleText(run.Title), | |
| 232 | 235 | Ref: visibleText(strings.TrimPrefix(strings.TrimPrefix(run.Ref, "refs/heads/"), "refs/tags/")), | |
| 233 | 236 | Short: short(run.Sha), CHref: repoHref(repo, "commit", run.Sha), Actor: visibleText(run.Actor), | |
| 234 | 237 | Duration: duration(run.StartedAt, run.CompletedAt, now), Number: "#" + strconv.FormatInt(run.Number, 10), Attempt: run.Attempt, | |
| @@ -119,6 +119,9 @@ type RunRow struct { | |||||
| 119 | Kind string // ciKind | 119 | Kind string // ciKind | ||
| 120 | State string // ciState | 120 | State string // ciState | ||
| 121 | Workflow string | 121 | Workflow string | ||
| 122 | + | // Event is the event that triggered the run (push, pull_request, | |||
| 123 | + | // workflow_dispatch, schedule, workflow_run). | |||
| 124 | + | Event string | |||
| 122 | Title string | 125 | Title string | ||
| 123 | Ref string | 126 | Ref string | ||
| 124 | RefHref string | 127 | RefHref string | ||
| @@ -228,7 +231,7 @@ func (s *Server) runRow(repo string, run domain.CIRun, now time.Time) RunRow { | |||||
| 228 | row := RunRow{ | 231 | row := RunRow{ | ||
| 229 | ID: strconv.FormatInt(run.ID, 10), Href: runHref(repo, run.ID), | 232 | ID: strconv.FormatInt(run.ID, 10), Href: runHref(repo, run.ID), | ||
| 230 | Kind: ciKind(run.Status, run.Conclusion), State: ciState(run.Status, run.Conclusion), | 233 | Kind: ciKind(run.Status, run.Conclusion), State: ciState(run.Status, run.Conclusion), | ||
| 231 | - | Workflow: visibleText(run.WorkflowName), Title: visibleText(run.Title), | 234 | + | Workflow: visibleText(run.WorkflowName), Event: visibleText(run.Event), Title: visibleText(run.Title), |
| 232 | Ref: visibleText(strings.TrimPrefix(strings.TrimPrefix(run.Ref, "refs/heads/"), "refs/tags/")), | 235 | Ref: visibleText(strings.TrimPrefix(strings.TrimPrefix(run.Ref, "refs/heads/"), "refs/tags/")), | ||
| 233 | Short: short(run.Sha), CHref: repoHref(repo, "commit", run.Sha), Actor: visibleText(run.Actor), | 236 | Short: short(run.Sha), CHref: repoHref(repo, "commit", run.Sha), Actor: visibleText(run.Actor), | ||
| 234 | Duration: duration(run.StartedAt, run.CompletedAt, now), Number: "#" + strconv.FormatInt(run.Number, 10), Attempt: run.Attempt, | 237 | Duration: duration(run.StartedAt, run.CompletedAt, now), Number: "#" + strconv.FormatInt(run.Number, 10), Attempt: run.Attempt, | ||
M internal/adapters/web/actions_templ.go +760 -728 generated file: collapsed
M internal/app/api_test.go +1 -1
| @@ -196,7 +196,7 @@ func TestAC2ReadAccess(t *testing.T) { | |||
| 196 | 196 | for _, who := range allPrincipals { | |
| 197 | 197 | resp, err := r.who[who].server.GetVersion(ctx, connect.NewRequest(&pb.GetVersionRequest{})) | |
| 198 | 198 | require.NoError(t, err, who) | |
| 199 | - | assert.ElementsMatch(t, []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.secrets"}, resp.Msg.GetFeatures(), who) | |
| 199 | + | assert.ElementsMatch(t, []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.dispatch", "ci.secrets"}, resp.Msg.GetFeatures(), who) | |
| 200 | 200 | assert.NotEmpty(t, resp.Msg.GetVersion()) | |
| 201 | 201 | } | |
| 202 | 202 | }) | |
| @@ -196,7 +196,7 @@ func TestAC2ReadAccess(t *testing.T) { | |||||
| 196 | for _, who := range allPrincipals { | 196 | for _, who := range allPrincipals { | ||
| 197 | resp, err := r.who[who].server.GetVersion(ctx, connect.NewRequest(&pb.GetVersionRequest{})) | 197 | resp, err := r.who[who].server.GetVersion(ctx, connect.NewRequest(&pb.GetVersionRequest{})) | ||
| 198 | require.NoError(t, err, who) | 198 | require.NoError(t, err, who) | ||
| 199 | - | assert.ElementsMatch(t, []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.secrets"}, resp.Msg.GetFeatures(), who) | 199 | + | assert.ElementsMatch(t, []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.dispatch", "ci.secrets"}, resp.Msg.GetFeatures(), who) |
| 200 | assert.NotEmpty(t, resp.Msg.GetVersion()) | 200 | assert.NotEmpty(t, resp.Msg.GetVersion()) | ||
| 201 | } | 201 | } | ||
| 202 | }) | 202 | }) | ||
A internal/app/ci_dispatch_test.go +239 -0
| @@ -0,0 +1,239 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package app | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "encoding/json" | |
| 7 | + | "testing" | |
| 8 | + | ||
| 9 | + | "connectrpc.com/connect" | |
| 10 | + | "github.com/stretchr/testify/assert" | |
| 11 | + | "github.com/stretchr/testify/require" | |
| 12 | + | ||
| 13 | + | pb "bnema.dev/ermite-api/gen/ermite" | |
| 14 | + | "bnema.dev/ermite/internal/domain" | |
| 15 | + | ) | |
| 16 | + | ||
| 17 | + | // CI-3 D5, AC5: workflow_dispatch. | |
| 18 | + | ||
| 19 | + | const dispatchWorkflow = `name: deploy | |
| 20 | + | on: | |
| 21 | + | workflow_dispatch: | |
| 22 | + | inputs: | |
| 23 | + | version: | |
| 24 | + | type: string | |
| 25 | + | required: true | |
| 26 | + | dry: | |
| 27 | + | type: boolean | |
| 28 | + | default: false | |
| 29 | + | count: | |
| 30 | + | type: number | |
| 31 | + | default: 3 | |
| 32 | + | level: | |
| 33 | + | type: choice | |
| 34 | + | options: [low, high] | |
| 35 | + | target: | |
| 36 | + | type: environment | |
| 37 | + | required: false | |
| 38 | + | run-name: deploy ${{ inputs.version }} dry=${{ inputs.dry }} | |
| 39 | + | jobs: | |
| 40 | + | j: | |
| 41 | + | runs-on: ubuntu-latest | |
| 42 | + | steps: | |
| 43 | + | - run: echo hi | |
| 44 | + | ` | |
| 45 | + | ||
| 46 | + | const pushOnlyWorkflow = "name: ci\non: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n" | |
| 47 | + | ||
| 48 | + | const jobTokenDispatcher = `name: dispatcher | |
| 49 | + | on: push | |
| 50 | + | permissions: | |
| 51 | + | actions: write | |
| 52 | + | contents: read | |
| 53 | + | jobs: | |
| 54 | + | with: | |
| 55 | + | runs-on: ubuntu-latest | |
| 56 | + | steps: | |
| 57 | + | - run: echo hi | |
| 58 | + | without: | |
| 59 | + | runs-on: ubuntu-latest | |
| 60 | + | permissions: | |
| 61 | + | contents: read | |
| 62 | + | steps: | |
| 63 | + | - run: echo hi | |
| 64 | + | actionsonly: | |
| 65 | + | runs-on: ubuntu-latest | |
| 66 | + | permissions: | |
| 67 | + | actions: write | |
| 68 | + | steps: | |
| 69 | + | - run: echo hi | |
| 70 | + | readonly: | |
| 71 | + | runs-on: ubuntu-latest | |
| 72 | + | permissions: {} | |
| 73 | + | steps: | |
| 74 | + | - run: echo hi | |
| 75 | + | ` | |
| 76 | + | ||
| 77 | + | func (c *ciHarness) dispatch(api *clients, path, ref string, inputs map[string]string) (*pb.Run, error) { | |
| 78 | + | c.t.Helper() | |
| 79 | + | res, err := api.run.DispatchWorkflow(c.ctx, connect.NewRequest(&pb.DispatchWorkflowRequest{ | |
| 80 | + | Repo: c.repo, WorkflowPath: path, Ref: ref, Inputs: inputs, | |
| 81 | + | })) | |
| 82 | + | if err != nil { | |
| 83 | + | return nil, err | |
| 84 | + | } | |
| 85 | + | return res.Msg.GetRun(), nil | |
| 86 | + | } | |
| 87 | + | ||
| 88 | + | func TestDispatchInputsAreValidatedAndTyped(t *testing.T) { | |
| 89 | + | c := newCIHarness(t, nil) | |
| 90 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 91 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow, ".github/workflows/ci.yml": pushOnlyWorkflow}, "") | |
| 92 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "push run") | |
| 93 | + | ||
| 94 | + | _, err := c.dispatch(c.root, "deploy.yml", "", nil) | |
| 95 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "a required input is missing") | |
| 96 | + | assert.Contains(t, err.Error(), "Required input 'version'") | |
| 97 | + | _, err = c.dispatch(c.root, "deploy.yml", "", map[string]string{"version": "1", "nope": "x"}) | |
| 98 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "an undeclared input") | |
| 99 | + | _, err = c.dispatch(c.root, "deploy.yml", "", map[string]string{"version": "1", "dry": "maybe"}) | |
| 100 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "a boolean") | |
| 101 | + | _, err = c.dispatch(c.root, "deploy.yml", "", map[string]string{"version": "1", "count": "0x10"}) | |
| 102 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "a number") | |
| 103 | + | _, err = c.dispatch(c.root, "deploy.yml", "", map[string]string{"version": "1", "level": "medium"}) | |
| 104 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "a choice outside its options") | |
| 105 | + | _, err = c.dispatch(c.root, "ci.yml", "", nil) | |
| 106 | + | assert.Equal(t, connect.CodeFailedPrecondition, ciErrCode(err), "no workflow_dispatch trigger") | |
| 107 | + | _, err = c.dispatch(c.root, "missing.yml", "", nil) | |
| 108 | + | assert.Equal(t, connect.CodeNotFound, ciErrCode(err)) | |
| 109 | + | _, err = c.dispatch(c.root, "deploy.yml", "nobranch", map[string]string{"version": "1"}) | |
| 110 | + | assert.Equal(t, connect.CodeNotFound, ciErrCode(err), "a ref that does not exist") | |
| 111 | + | assert.Len(t, c.runs(), 1, "no refused dispatch made a run") | |
| 112 | + | ||
| 113 | + | run, err := c.dispatch(c.root, ".github/workflows/deploy.yml", "main", map[string]string{"version": "1.2", "dry": "true"}) | |
| 114 | + | require.NoError(t, err) | |
| 115 | + | assert.Equal(t, "workflow_dispatch", run.GetEvent()) | |
| 116 | + | assert.Equal(t, "refs/heads/main", run.GetRef()) | |
| 117 | + | assert.True(t, run.GetTrusted()) | |
| 118 | + | assert.Equal(t, "root", run.GetActor(), "the caller, as for a push") | |
| 119 | + | assert.Equal(t, "deploy 1.2 dry=true", run.GetName(), "run-name sees the typed inputs") | |
| 120 | + | assert.Equal(t, pb.CiStatus_CI_STATUS_QUEUED, run.GetStatus()) | |
| 121 | + | ||
| 122 | + | job := runner.await() | |
| 123 | + | gh := c.ghContext(job) | |
| 124 | + | assert.Equal(t, "workflow_dispatch", gh["event_name"]) | |
| 125 | + | assert.Equal(t, "refs/heads/main", gh["ref"]) | |
| 126 | + | ev := gh["event"].(map[string]any) | |
| 127 | + | assert.Equal(t, map[string]any{"version": "1.2", "dry": "true", "count": "3", "level": "low"}, ev["inputs"], "github.event.inputs are strings, defaults applied") | |
| 128 | + | assert.Equal(t, ".github/workflows/deploy.yml", ev["workflow"]) | |
| 129 | + | assert.Equal(t, "refs/heads/main", ev["ref"]) | |
| 130 | + | ||
| 131 | + | var ctxs map[string]json.RawMessage | |
| 132 | + | require.NoError(t, json.Unmarshal([]byte(job.GetContexts()), &ctxs)) | |
| 133 | + | var inputs map[string]any | |
| 134 | + | require.NoError(t, json.Unmarshal(ctxs["inputs"], &inputs)) | |
| 135 | + | assert.Equal(t, map[string]any{"version": "1.2", "dry": true, "count": float64(3), "level": "low", "target": ""}, inputs, "the inputs context is typed") | |
| 136 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 137 | + | } | |
| 138 | + | ||
| 139 | + | func TestDispatchReadsTheWorkflowAtTheRef(t *testing.T) { | |
| 140 | + | c := newCIHarness(t, nil) | |
| 141 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 142 | + | c.push("main", map[string]string{".github/workflows/ci.yml": pushOnlyWorkflow}, "") | |
| 143 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "push run") | |
| 144 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "feat") | |
| 145 | + | c.push("feat", map[string]string{".github/workflows/ci.yml": "name: ci\non: [push, workflow_dispatch]\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"}, "") | |
| 146 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "push run") | |
| 147 | + | ||
| 148 | + | _, err := c.dispatch(c.root, "ci.yml", "", nil) | |
| 149 | + | assert.Equal(t, connect.CodeFailedPrecondition, ciErrCode(err), "main has no workflow_dispatch") | |
| 150 | + | run, err := c.dispatch(c.root, "ci.yml", "feat", nil) | |
| 151 | + | require.NoError(t, err) | |
| 152 | + | assert.Equal(t, "refs/heads/feat", run.GetRef()) | |
| 153 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 154 | + | } | |
| 155 | + | ||
| 156 | + | func TestDispatchAccess(t *testing.T) { | |
| 157 | + | c := newCIHarness(t, nil) | |
| 158 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 159 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow}, "") | |
| 160 | + | in := map[string]string{"version": "1"} | |
| 161 | + | ||
| 162 | + | readOnly := c.api(c.token(scopes(domain.ScopeRepoRead), "proj")) | |
| 163 | + | _, err := c.dispatch(readOnly, "deploy.yml", "", in) | |
| 164 | + | assert.Equal(t, connect.CodePermissionDenied, ciErrCode(err), "ci:write is needed") | |
| 165 | + | ||
| 166 | + | // A token with ci:write but limited to branch prefixes: its run is not trusted, | |
| 167 | + | // and it may only dispatch on a branch under its prefixes. | |
| 168 | + | secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeCIWrite), []string{"agent/claude/"}, "proj") | |
| 169 | + | limited := c.api(secret) | |
| 170 | + | _, err = c.dispatch(limited, "deploy.yml", "main", in) | |
| 171 | + | assert.Equal(t, connect.CodePermissionDenied, ciErrCode(err), "main is outside the prefixes") | |
| 172 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "agent/claude/x") | |
| 173 | + | c.push("agent/claude/x", nil, "") | |
| 174 | + | run, err := c.dispatch(limited, "deploy.yml", "agent/claude/x", in) | |
| 175 | + | require.NoError(t, err) | |
| 176 | + | assert.False(t, run.GetTrusted(), "a branch-limited caller gets an untrusted run") | |
| 177 | + | job := runner.await() | |
| 178 | + | assert.False(t, job.GetTrusted()) | |
| 179 | + | assert.Empty(t, job.GetSecrets()) | |
| 180 | + | } | |
| 181 | + | ||
| 182 | + | func TestDispatchWithAJobToken(t *testing.T) { | |
| 183 | + | c := newCIHarness(t, nil) | |
| 184 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 185 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow, ".github/workflows/d.yml": jobTokenDispatcher}, "") | |
| 186 | + | jobs := runner.jobsByKey(4) | |
| 187 | + | with, without := jobs["with"], jobs["without"] | |
| 188 | + | require.NotNil(t, with) | |
| 189 | + | require.NotNil(t, without) | |
| 190 | + | onlyActions, none := jobs["actionsonly"], jobs["readonly"] | |
| 191 | + | ||
| 192 | + | in := map[string]string{"version": "9"} | |
| 193 | + | _, err := c.dispatch(c.api(without.GetGithubToken()), "deploy.yml", "", in) | |
| 194 | + | assert.Equal(t, connect.CodePermissionDenied, ciErrCode(err), "no actions: write") | |
| 195 | + | run, err := c.dispatch(c.api(with.GetGithubToken()), "deploy.yml", "", in) | |
| 196 | + | require.NoError(t, err, "actions: write lets a job token dispatch (GitHub's exception)") | |
| 197 | + | assert.Equal(t, "workflow_dispatch", run.GetEvent()) | |
| 198 | + | assert.Equal(t, "github-actions[bot]", run.GetActor()) | |
| 199 | + | assert.True(t, run.GetTrusted(), "the dispatching run was trusted") | |
| 200 | + | ||
| 201 | + | // The repository is private and this token has no contents permission at all: | |
| 202 | + | // actions: write alone is enough, and a token without it is told nothing. | |
| 203 | + | _, err = c.dispatch(c.api(none.GetGithubToken()), "deploy.yml", "", in) | |
| 204 | + | assert.Equal(t, connect.CodeNotFound, ciErrCode(err), "no permission at all on a private repository: concealed") | |
| 205 | + | run, err = c.dispatch(c.api(onlyActions.GetGithubToken()), "deploy.yml", "", in) | |
| 206 | + | require.NoError(t, err, "actions: write without contents: read") | |
| 207 | + | assert.Equal(t, "workflow_dispatch", run.GetEvent()) | |
| 208 | + | ||
| 209 | + | for _, j := range []*pb.Job{with, without, onlyActions, none} { | |
| 210 | + | runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 211 | + | } | |
| 212 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "dispatched") | |
| 213 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "dispatched") | |
| 214 | + | } | |
| 215 | + | ||
| 216 | + | func TestWorkflowRunCommand(t *testing.T) { | |
| 217 | + | c := newCIHarness(t, nil) | |
| 218 | + | h := &cliHarness{harness: c.harness, url: c.srv.URL} | |
| 219 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 220 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow}, "") | |
| 221 | + | ||
| 222 | + | bad := h.cli(t, adminToken, "workflow", "run", "deploy.yml", "--repo", "proj", "-f", "oops") | |
| 223 | + | require.Error(t, bad.err) | |
| 224 | + | missing := h.cli(t, adminToken, "workflow", "run", "deploy.yml", "--repo", "proj") | |
| 225 | + | require.Error(t, missing.err) | |
| 226 | + | assert.Contains(t, missing.err.Error(), "Required input 'version'") | |
| 227 | + | ||
| 228 | + | ok := h.cli(t, adminToken, "workflow", "run", "deploy.yml", "--repo", "proj", "--ref", "main", "-f", "version=a=b", "-f", "dry=true") | |
| 229 | + | require.NoError(t, ok.err, ok.stderr) | |
| 230 | + | assert.Contains(t, ok.stderr, "started on main") | |
| 231 | + | job := runner.await() | |
| 232 | + | ev := c.ghContext(job)["event"].(map[string]any) | |
| 233 | + | assert.Equal(t, "a=b", ev["inputs"].(map[string]any)["version"]) | |
| 234 | + | ||
| 235 | + | list := h.cli(t, adminToken, "run", "list", "proj") | |
| 236 | + | require.NoError(t, list.err) | |
| 237 | + | assert.Contains(t, list.stdout, "EVENT") | |
| 238 | + | assert.Contains(t, list.stdout, "workflow_dispatch") | |
| 239 | + | } | |
| @@ -0,0 +1,239 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package app | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "encoding/json" | |||
| 7 | + | "testing" | |||
| 8 | + | ||||
| 9 | + | "connectrpc.com/connect" | |||
| 10 | + | "github.com/stretchr/testify/assert" | |||
| 11 | + | "github.com/stretchr/testify/require" | |||
| 12 | + | ||||
| 13 | + | pb "bnema.dev/ermite-api/gen/ermite" | |||
| 14 | + | "bnema.dev/ermite/internal/domain" | |||
| 15 | + | ) | |||
| 16 | + | ||||
| 17 | + | // CI-3 D5, AC5: workflow_dispatch. | |||
| 18 | + | ||||
| 19 | + | const dispatchWorkflow = `name: deploy | |||
| 20 | + | on: | |||
| 21 | + | workflow_dispatch: | |||
| 22 | + | inputs: | |||
| 23 | + | version: | |||
| 24 | + | type: string | |||
| 25 | + | required: true | |||
| 26 | + | dry: | |||
| 27 | + | type: boolean | |||
| 28 | + | default: false | |||
| 29 | + | count: | |||
| 30 | + | type: number | |||
| 31 | + | default: 3 | |||
| 32 | + | level: | |||
| 33 | + | type: choice | |||
| 34 | + | options: [low, high] | |||
| 35 | + | target: | |||
| 36 | + | type: environment | |||
| 37 | + | required: false | |||
| 38 | + | run-name: deploy ${{ inputs.version }} dry=${{ inputs.dry }} | |||
| 39 | + | jobs: | |||
| 40 | + | j: | |||
| 41 | + | runs-on: ubuntu-latest | |||
| 42 | + | steps: | |||
| 43 | + | - run: echo hi | |||
| 44 | + | ` | |||
| 45 | + | ||||
| 46 | + | const pushOnlyWorkflow = "name: ci\non: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n" | |||
| 47 | + | ||||
| 48 | + | const jobTokenDispatcher = `name: dispatcher | |||
| 49 | + | on: push | |||
| 50 | + | permissions: | |||
| 51 | + | actions: write | |||
| 52 | + | contents: read | |||
| 53 | + | jobs: | |||
| 54 | + | with: | |||
| 55 | + | runs-on: ubuntu-latest | |||
| 56 | + | steps: | |||
| 57 | + | - run: echo hi | |||
| 58 | + | without: | |||
| 59 | + | runs-on: ubuntu-latest | |||
| 60 | + | permissions: | |||
| 61 | + | contents: read | |||
| 62 | + | steps: | |||
| 63 | + | - run: echo hi | |||
| 64 | + | actionsonly: | |||
| 65 | + | runs-on: ubuntu-latest | |||
| 66 | + | permissions: | |||
| 67 | + | actions: write | |||
| 68 | + | steps: | |||
| 69 | + | - run: echo hi | |||
| 70 | + | readonly: | |||
| 71 | + | runs-on: ubuntu-latest | |||
| 72 | + | permissions: {} | |||
| 73 | + | steps: | |||
| 74 | + | - run: echo hi | |||
| 75 | + | ` | |||
| 76 | + | ||||
| 77 | + | func (c *ciHarness) dispatch(api *clients, path, ref string, inputs map[string]string) (*pb.Run, error) { | |||
| 78 | + | c.t.Helper() | |||
| 79 | + | res, err := api.run.DispatchWorkflow(c.ctx, connect.NewRequest(&pb.DispatchWorkflowRequest{ | |||
| 80 | + | Repo: c.repo, WorkflowPath: path, Ref: ref, Inputs: inputs, | |||
| 81 | + | })) | |||
| 82 | + | if err != nil { | |||
| 83 | + | return nil, err | |||
| 84 | + | } | |||
| 85 | + | return res.Msg.GetRun(), nil | |||
| 86 | + | } | |||
| 87 | + | ||||
| 88 | + | func TestDispatchInputsAreValidatedAndTyped(t *testing.T) { | |||
| 89 | + | c := newCIHarness(t, nil) | |||
| 90 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 91 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow, ".github/workflows/ci.yml": pushOnlyWorkflow}, "") | |||
| 92 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "push run") | |||
| 93 | + | ||||
| 94 | + | _, err := c.dispatch(c.root, "deploy.yml", "", nil) | |||
| 95 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "a required input is missing") | |||
| 96 | + | assert.Contains(t, err.Error(), "Required input 'version'") | |||
| 97 | + | _, err = c.dispatch(c.root, "deploy.yml", "", map[string]string{"version": "1", "nope": "x"}) | |||
| 98 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "an undeclared input") | |||
| 99 | + | _, err = c.dispatch(c.root, "deploy.yml", "", map[string]string{"version": "1", "dry": "maybe"}) | |||
| 100 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "a boolean") | |||
| 101 | + | _, err = c.dispatch(c.root, "deploy.yml", "", map[string]string{"version": "1", "count": "0x10"}) | |||
| 102 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "a number") | |||
| 103 | + | _, err = c.dispatch(c.root, "deploy.yml", "", map[string]string{"version": "1", "level": "medium"}) | |||
| 104 | + | assert.Equal(t, connect.CodeInvalidArgument, ciErrCode(err), "a choice outside its options") | |||
| 105 | + | _, err = c.dispatch(c.root, "ci.yml", "", nil) | |||
| 106 | + | assert.Equal(t, connect.CodeFailedPrecondition, ciErrCode(err), "no workflow_dispatch trigger") | |||
| 107 | + | _, err = c.dispatch(c.root, "missing.yml", "", nil) | |||
| 108 | + | assert.Equal(t, connect.CodeNotFound, ciErrCode(err)) | |||
| 109 | + | _, err = c.dispatch(c.root, "deploy.yml", "nobranch", map[string]string{"version": "1"}) | |||
| 110 | + | assert.Equal(t, connect.CodeNotFound, ciErrCode(err), "a ref that does not exist") | |||
| 111 | + | assert.Len(t, c.runs(), 1, "no refused dispatch made a run") | |||
| 112 | + | ||||
| 113 | + | run, err := c.dispatch(c.root, ".github/workflows/deploy.yml", "main", map[string]string{"version": "1.2", "dry": "true"}) | |||
| 114 | + | require.NoError(t, err) | |||
| 115 | + | assert.Equal(t, "workflow_dispatch", run.GetEvent()) | |||
| 116 | + | assert.Equal(t, "refs/heads/main", run.GetRef()) | |||
| 117 | + | assert.True(t, run.GetTrusted()) | |||
| 118 | + | assert.Equal(t, "root", run.GetActor(), "the caller, as for a push") | |||
| 119 | + | assert.Equal(t, "deploy 1.2 dry=true", run.GetName(), "run-name sees the typed inputs") | |||
| 120 | + | assert.Equal(t, pb.CiStatus_CI_STATUS_QUEUED, run.GetStatus()) | |||
| 121 | + | ||||
| 122 | + | job := runner.await() | |||
| 123 | + | gh := c.ghContext(job) | |||
| 124 | + | assert.Equal(t, "workflow_dispatch", gh["event_name"]) | |||
| 125 | + | assert.Equal(t, "refs/heads/main", gh["ref"]) | |||
| 126 | + | ev := gh["event"].(map[string]any) | |||
| 127 | + | assert.Equal(t, map[string]any{"version": "1.2", "dry": "true", "count": "3", "level": "low"}, ev["inputs"], "github.event.inputs are strings, defaults applied") | |||
| 128 | + | assert.Equal(t, ".github/workflows/deploy.yml", ev["workflow"]) | |||
| 129 | + | assert.Equal(t, "refs/heads/main", ev["ref"]) | |||
| 130 | + | ||||
| 131 | + | var ctxs map[string]json.RawMessage | |||
| 132 | + | require.NoError(t, json.Unmarshal([]byte(job.GetContexts()), &ctxs)) | |||
| 133 | + | var inputs map[string]any | |||
| 134 | + | require.NoError(t, json.Unmarshal(ctxs["inputs"], &inputs)) | |||
| 135 | + | assert.Equal(t, map[string]any{"version": "1.2", "dry": true, "count": float64(3), "level": "low", "target": ""}, inputs, "the inputs context is typed") | |||
| 136 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 137 | + | } | |||
| 138 | + | ||||
| 139 | + | func TestDispatchReadsTheWorkflowAtTheRef(t *testing.T) { | |||
| 140 | + | c := newCIHarness(t, nil) | |||
| 141 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 142 | + | c.push("main", map[string]string{".github/workflows/ci.yml": pushOnlyWorkflow}, "") | |||
| 143 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "push run") | |||
| 144 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "feat") | |||
| 145 | + | c.push("feat", map[string]string{".github/workflows/ci.yml": "name: ci\non: [push, workflow_dispatch]\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"}, "") | |||
| 146 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "push run") | |||
| 147 | + | ||||
| 148 | + | _, err := c.dispatch(c.root, "ci.yml", "", nil) | |||
| 149 | + | assert.Equal(t, connect.CodeFailedPrecondition, ciErrCode(err), "main has no workflow_dispatch") | |||
| 150 | + | run, err := c.dispatch(c.root, "ci.yml", "feat", nil) | |||
| 151 | + | require.NoError(t, err) | |||
| 152 | + | assert.Equal(t, "refs/heads/feat", run.GetRef()) | |||
| 153 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 154 | + | } | |||
| 155 | + | ||||
| 156 | + | func TestDispatchAccess(t *testing.T) { | |||
| 157 | + | c := newCIHarness(t, nil) | |||
| 158 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 159 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow}, "") | |||
| 160 | + | in := map[string]string{"version": "1"} | |||
| 161 | + | ||||
| 162 | + | readOnly := c.api(c.token(scopes(domain.ScopeRepoRead), "proj")) | |||
| 163 | + | _, err := c.dispatch(readOnly, "deploy.yml", "", in) | |||
| 164 | + | assert.Equal(t, connect.CodePermissionDenied, ciErrCode(err), "ci:write is needed") | |||
| 165 | + | ||||
| 166 | + | // A token with ci:write but limited to branch prefixes: its run is not trusted, | |||
| 167 | + | // and it may only dispatch on a branch under its prefixes. | |||
| 168 | + | secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeCIWrite), []string{"agent/claude/"}, "proj") | |||
| 169 | + | limited := c.api(secret) | |||
| 170 | + | _, err = c.dispatch(limited, "deploy.yml", "main", in) | |||
| 171 | + | assert.Equal(t, connect.CodePermissionDenied, ciErrCode(err), "main is outside the prefixes") | |||
| 172 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "agent/claude/x") | |||
| 173 | + | c.push("agent/claude/x", nil, "") | |||
| 174 | + | run, err := c.dispatch(limited, "deploy.yml", "agent/claude/x", in) | |||
| 175 | + | require.NoError(t, err) | |||
| 176 | + | assert.False(t, run.GetTrusted(), "a branch-limited caller gets an untrusted run") | |||
| 177 | + | job := runner.await() | |||
| 178 | + | assert.False(t, job.GetTrusted()) | |||
| 179 | + | assert.Empty(t, job.GetSecrets()) | |||
| 180 | + | } | |||
| 181 | + | ||||
| 182 | + | func TestDispatchWithAJobToken(t *testing.T) { | |||
| 183 | + | c := newCIHarness(t, nil) | |||
| 184 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 185 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow, ".github/workflows/d.yml": jobTokenDispatcher}, "") | |||
| 186 | + | jobs := runner.jobsByKey(4) | |||
| 187 | + | with, without := jobs["with"], jobs["without"] | |||
| 188 | + | require.NotNil(t, with) | |||
| 189 | + | require.NotNil(t, without) | |||
| 190 | + | onlyActions, none := jobs["actionsonly"], jobs["readonly"] | |||
| 191 | + | ||||
| 192 | + | in := map[string]string{"version": "9"} | |||
| 193 | + | _, err := c.dispatch(c.api(without.GetGithubToken()), "deploy.yml", "", in) | |||
| 194 | + | assert.Equal(t, connect.CodePermissionDenied, ciErrCode(err), "no actions: write") | |||
| 195 | + | run, err := c.dispatch(c.api(with.GetGithubToken()), "deploy.yml", "", in) | |||
| 196 | + | require.NoError(t, err, "actions: write lets a job token dispatch (GitHub's exception)") | |||
| 197 | + | assert.Equal(t, "workflow_dispatch", run.GetEvent()) | |||
| 198 | + | assert.Equal(t, "github-actions[bot]", run.GetActor()) | |||
| 199 | + | assert.True(t, run.GetTrusted(), "the dispatching run was trusted") | |||
| 200 | + | ||||
| 201 | + | // The repository is private and this token has no contents permission at all: | |||
| 202 | + | // actions: write alone is enough, and a token without it is told nothing. | |||
| 203 | + | _, err = c.dispatch(c.api(none.GetGithubToken()), "deploy.yml", "", in) | |||
| 204 | + | assert.Equal(t, connect.CodeNotFound, ciErrCode(err), "no permission at all on a private repository: concealed") | |||
| 205 | + | run, err = c.dispatch(c.api(onlyActions.GetGithubToken()), "deploy.yml", "", in) | |||
| 206 | + | require.NoError(t, err, "actions: write without contents: read") | |||
| 207 | + | assert.Equal(t, "workflow_dispatch", run.GetEvent()) | |||
| 208 | + | ||||
| 209 | + | for _, j := range []*pb.Job{with, without, onlyActions, none} { | |||
| 210 | + | runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 211 | + | } | |||
| 212 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "dispatched") | |||
| 213 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "dispatched") | |||
| 214 | + | } | |||
| 215 | + | ||||
| 216 | + | func TestWorkflowRunCommand(t *testing.T) { | |||
| 217 | + | c := newCIHarness(t, nil) | |||
| 218 | + | h := &cliHarness{harness: c.harness, url: c.srv.URL} | |||
| 219 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 220 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow}, "") | |||
| 221 | + | ||||
| 222 | + | bad := h.cli(t, adminToken, "workflow", "run", "deploy.yml", "--repo", "proj", "-f", "oops") | |||
| 223 | + | require.Error(t, bad.err) | |||
| 224 | + | missing := h.cli(t, adminToken, "workflow", "run", "deploy.yml", "--repo", "proj") | |||
| 225 | + | require.Error(t, missing.err) | |||
| 226 | + | assert.Contains(t, missing.err.Error(), "Required input 'version'") | |||
| 227 | + | ||||
| 228 | + | ok := h.cli(t, adminToken, "workflow", "run", "deploy.yml", "--repo", "proj", "--ref", "main", "-f", "version=a=b", "-f", "dry=true") | |||
| 229 | + | require.NoError(t, ok.err, ok.stderr) | |||
| 230 | + | assert.Contains(t, ok.stderr, "started on main") | |||
| 231 | + | job := runner.await() | |||
| 232 | + | ev := c.ghContext(job)["event"].(map[string]any) | |||
| 233 | + | assert.Equal(t, "a=b", ev["inputs"].(map[string]any)["version"]) | |||
| 234 | + | ||||
| 235 | + | list := h.cli(t, adminToken, "run", "list", "proj") | |||
| 236 | + | require.NoError(t, list.err) | |||
| 237 | + | assert.Contains(t, list.stdout, "EVENT") | |||
| 238 | + | assert.Contains(t, list.stdout, "workflow_dispatch") | |||
| 239 | + | } | |||
A internal/app/ci_schedule_test.go +257 -0
| @@ -0,0 +1,257 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package app | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "sync" | |
| 7 | + | "testing" | |
| 8 | + | "time" | |
| 9 | + | ||
| 10 | + | "connectrpc.com/connect" | |
| 11 | + | "github.com/stretchr/testify/assert" | |
| 12 | + | "github.com/stretchr/testify/require" | |
| 13 | + | ||
| 14 | + | pb "bnema.dev/ermite-api/gen/ermite" | |
| 15 | + | "bnema.dev/ermite/internal/domain" | |
| 16 | + | "bnema.dev/ermite/internal/ports" | |
| 17 | + | "bnema.dev/ermite/internal/usecase" | |
| 18 | + | ) | |
| 19 | + | ||
| 20 | + | // CI-3 D5, AC5: schedule, with an injected clock. | |
| 21 | + | ||
| 22 | + | // testClock is a clock the test moves. | |
| 23 | + | type testClock struct { | |
| 24 | + | mu sync.Mutex | |
| 25 | + | t time.Time | |
| 26 | + | } | |
| 27 | + | ||
| 28 | + | func (c *testClock) Now() time.Time { | |
| 29 | + | c.mu.Lock() | |
| 30 | + | defer c.mu.Unlock() | |
| 31 | + | return c.t | |
| 32 | + | } | |
| 33 | + | ||
| 34 | + | func (c *testClock) set(s string) { | |
| 35 | + | c.mu.Lock() | |
| 36 | + | defer c.mu.Unlock() | |
| 37 | + | t, err := time.Parse("2006-01-02 15:04:05", s) | |
| 38 | + | if err != nil { | |
| 39 | + | panic(err) | |
| 40 | + | } | |
| 41 | + | c.t = t | |
| 42 | + | } | |
| 43 | + | ||
| 44 | + | func scheduleWorkflow(crons ...string) map[string]string { | |
| 45 | + | on := "on:\n schedule:\n" | |
| 46 | + | for _, c := range crons { | |
| 47 | + | on += " - cron: '" + c + "'\n" | |
| 48 | + | } | |
| 49 | + | return map[string]string{".github/workflows/nightly.yml": "name: nightly\n" + on + "jobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"} | |
| 50 | + | } | |
| 51 | + | ||
| 52 | + | func newScheduleHarness(t *testing.T, start string) (*ciHarness, *testClock, *fakeRunner) { | |
| 53 | + | t.Helper() | |
| 54 | + | clk := &testClock{} | |
| 55 | + | clk.set(start) | |
| 56 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |
| 57 | + | cfg.Clock = clk | |
| 58 | + | cfg.ScheduleTick = time.Hour // the tests call ScheduleNow | |
| 59 | + | }) | |
| 60 | + | return c, clk, c.registerRunner("alpha", "ubuntu-latest") | |
| 61 | + | } | |
| 62 | + | ||
| 63 | + | func createRepoParams(name domain.RepoName) ports.CreateRepoParams { | |
| 64 | + | return ports.CreateRepoParams{Name: name, Visibility: domain.VisibilityPrivate} | |
| 65 | + | } | |
| 66 | + | ||
| 67 | + | func (c *ciHarness) scheduled() []*pb.Run { return scheduleRuns(c.runsOf("nightly")) } | |
| 68 | + | ||
| 69 | + | func scheduleRuns(runs []*pb.Run) []*pb.Run { | |
| 70 | + | var out []*pb.Run | |
| 71 | + | for _, r := range runs { | |
| 72 | + | if r.GetEvent() == "schedule" { | |
| 73 | + | out = append(out, r) | |
| 74 | + | } | |
| 75 | + | } | |
| 76 | + | return out | |
| 77 | + | } | |
| 78 | + | ||
| 79 | + | func TestScheduleFiresOncePerMatchingMinuteWithoutCatchUp(t *testing.T) { | |
| 80 | + | c, clk, runner := newScheduleHarness(t, "2026-03-04 10:00:30") | |
| 81 | + | c.push("main", scheduleWorkflow("*/5 * * * *"), "") | |
| 82 | + | ci := c.forge.CI | |
| 83 | + | tick := func(at string) { | |
| 84 | + | clk.set(at) | |
| 85 | + | ci.ScheduleNow(c.ctx) | |
| 86 | + | } | |
| 87 | + | ||
| 88 | + | tick("2026-03-04 10:00:40") | |
| 89 | + | assert.Empty(t, c.scheduled(), "the minute ermite started in belongs to the process before it") | |
| 90 | + | ||
| 91 | + | tick("2026-03-04 10:05:10") | |
| 92 | + | require.Len(t, c.scheduled(), 1) | |
| 93 | + | run := c.scheduled()[0] | |
| 94 | + | assert.Equal(t, "schedule", run.GetEvent()) | |
| 95 | + | assert.Equal(t, "refs/heads/main", run.GetRef()) | |
| 96 | + | assert.Equal(t, c.ref("refs/heads/main"), run.GetSha(), "the latest commit of the default branch") | |
| 97 | + | assert.Equal(t, "root", run.GetActor(), "whoever pushed the tip (the root token here)") | |
| 98 | + | assert.True(t, run.GetTrusted()) | |
| 99 | + | ||
| 100 | + | tick("2026-03-04 10:05:50") | |
| 101 | + | tick("2026-03-04 10:05:59") | |
| 102 | + | assert.Len(t, c.scheduled(), 1, "a minute fires once, however many ticks it holds") | |
| 103 | + | tick("2026-03-04 10:06:00") | |
| 104 | + | tick("2026-03-04 10:09:00") | |
| 105 | + | assert.Len(t, c.scheduled(), 1, "minutes that do not match fire nothing") | |
| 106 | + | ||
| 107 | + | // ermite was busy or down for the minutes 10:10 and 10:15: they are not made up. | |
| 108 | + | tick("2026-03-04 10:20:00") | |
| 109 | + | assert.Len(t, c.scheduled(), 2, "only the minute the clock reached fires, not the ones skipped") | |
| 110 | + | assert.Equal(t, time.Date(2026, 3, 4, 10, 20, 0, 0, time.UTC), ci.SchedulePassed()) | |
| 111 | + | ||
| 112 | + | job := runner.await() | |
| 113 | + | gh := c.ghContext(job) | |
| 114 | + | assert.Equal(t, "schedule", gh["event_name"]) | |
| 115 | + | assert.Equal(t, "refs/heads/main", gh["ref"]) | |
| 116 | + | assert.Equal(t, "*/5 * * * *", gh["event"].(map[string]any)["schedule"], "the payload is the cron entry") | |
| 117 | + | assert.True(t, job.GetTrusted()) | |
| 118 | + | } | |
| 119 | + | ||
| 120 | + | // Several cron entries of one workflow that match the same minute start one | |
| 121 | + | // run, with the first matching entry as payload; an entry that matches another | |
| 122 | + | // minute starts its own. | |
| 123 | + | func TestScheduleSeveralMatchingCronsOfOneWorkflowStartOneRun(t *testing.T) { | |
| 124 | + | c, clk, runner := newScheduleHarness(t, "2026-03-04 10:00:30") | |
| 125 | + | c.push("main", scheduleWorkflow("*/2 * * * *", "* * * * *", "7 10 * * *"), "") | |
| 126 | + | ci := c.forge.CI | |
| 127 | + | ||
| 128 | + | clk.set("2026-03-04 10:02:00") // the first and the second entries match | |
| 129 | + | ci.ScheduleNow(c.ctx) | |
| 130 | + | require.Len(t, c.scheduled(), 1, "one run for the workflow, not one per entry") | |
| 131 | + | job := runner.await() | |
| 132 | + | assert.Equal(t, "*/2 * * * *", c.ghContext(job)["event"].(map[string]any)["schedule"], "the first entry that matches") | |
| 133 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 134 | + | ||
| 135 | + | clk.set("2026-03-04 10:03:00") // only the second | |
| 136 | + | ci.ScheduleNow(c.ctx) | |
| 137 | + | require.Len(t, c.scheduled(), 2) | |
| 138 | + | job = runner.await() | |
| 139 | + | assert.Equal(t, "* * * * *", c.ghContext(job)["event"].(map[string]any)["schedule"]) | |
| 140 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 141 | + | } | |
| 142 | + | ||
| 143 | + | func TestScheduleReadsAgainAfterAPushToTheDefaultBranch(t *testing.T) { | |
| 144 | + | c, clk, _ := newScheduleHarness(t, "2026-03-04 10:00:30") | |
| 145 | + | c.push("main", scheduleWorkflow("*/5 * * * *"), "") | |
| 146 | + | ci := c.forge.CI | |
| 147 | + | tick := func(at string) { | |
| 148 | + | clk.set(at) | |
| 149 | + | ci.ScheduleNow(c.ctx) | |
| 150 | + | } | |
| 151 | + | tick("2026-03-04 10:05:00") | |
| 152 | + | require.Len(t, c.scheduled(), 1) | |
| 153 | + | assert.Equal(t, 1, ci.CachedSchedules(c.name("proj")), "the schedules are cached") | |
| 154 | + | ||
| 155 | + | // 10:07 matches no */5 entry; the new file fires every minute. | |
| 156 | + | c.push("main", scheduleWorkflow("* * * * *"), "") | |
| 157 | + | tick("2026-03-04 10:06:00") | |
| 158 | + | require.Len(t, c.scheduled(), 2, "the pushed workflow is the one scheduled") | |
| 159 | + | assert.Equal(t, 1, ci.CachedSchedules(c.name("proj"))) | |
| 160 | + | ||
| 161 | + | // A push to another branch changes nothing; a push that removes the schedule | |
| 162 | + | // ends it. | |
| 163 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "side") | |
| 164 | + | c.push("side", scheduleWorkflow("0 0 1 1 *"), "") | |
| 165 | + | tick("2026-03-04 10:07:00") | |
| 166 | + | require.Len(t, c.scheduled(), 3, "the default branch's workflow still fires every minute") | |
| 167 | + | ||
| 168 | + | c.mustGit(c.wt, "checkout", "-q", "main") | |
| 169 | + | c.push("main", map[string]string{".github/workflows/nightly.yml": "name: nightly\non: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"}, "") | |
| 170 | + | tick("2026-03-04 10:08:00") | |
| 171 | + | c.waitFor("the push run of the new file", func() bool { return len(c.runs()) == 4 }) | |
| 172 | + | assert.Len(t, scheduleRuns(c.runs()), 3, "no schedule left on the default branch") | |
| 173 | + | assert.Equal(t, 0, ci.CachedSchedules(c.name("proj"))) | |
| 174 | + | } | |
| 175 | + | ||
| 176 | + | func TestScheduleIsRemovedWithTheRepository(t *testing.T) { | |
| 177 | + | c, clk, _ := newScheduleHarness(t, "2026-03-04 10:00:30") | |
| 178 | + | c.push("main", scheduleWorkflow("* * * * *"), "") | |
| 179 | + | ci := c.forge.CI | |
| 180 | + | clk.set("2026-03-04 10:01:00") | |
| 181 | + | ci.ScheduleNow(c.ctx) | |
| 182 | + | require.Len(t, c.scheduled(), 1) | |
| 183 | + | require.Equal(t, 1, ci.CachedSchedules(c.name("proj"))) | |
| 184 | + | ||
| 185 | + | _, err := c.root.repo.DeleteRepo(c.ctx, connect.NewRequest(&pb.DeleteRepoRequest{Repo: "proj"})) | |
| 186 | + | require.NoError(t, err) | |
| 187 | + | assert.Equal(t, -1, ci.CachedSchedules(c.name("proj")), "the cached schedules go with the repository") | |
| 188 | + | ||
| 189 | + | for _, at := range []string{"2026-03-04 10:02:00", "2026-03-04 10:03:00"} { | |
| 190 | + | clk.set(at) | |
| 191 | + | ci.ScheduleNow(c.ctx) | |
| 192 | + | } | |
| 193 | + | assert.Equal(t, -1, ci.CachedSchedules(c.name("proj")), "never scheduled again") | |
| 194 | + | ||
| 195 | + | // The same name created again starts empty: no schedule until it has a workflow. | |
| 196 | + | _, err = c.forge.Repos.Create(c.ctx, domain.Root(), createRepoParams(c.name("proj"))) | |
| 197 | + | require.NoError(t, err) | |
| 198 | + | clk.set("2026-03-04 10:04:00") | |
| 199 | + | ci.ScheduleNow(c.ctx) | |
| 200 | + | assert.Empty(t, c.scheduled()) | |
| 201 | + | } | |
| 202 | + | ||
| 203 | + | // More repositories than one tick can handle, and a new minute at every tick: | |
| 204 | + | // each pass is dropped after its first repository, and the cursor makes the next | |
| 205 | + | // one start after it, so that none is starved. | |
| 206 | + | func TestScheduleDoesNotStarveRepositoriesBeyondTheBudget(t *testing.T) { | |
| 207 | + | clk := &testClock{} | |
| 208 | + | clk.set("2026-03-04 10:00:30") | |
| 209 | + | visited := map[domain.RepoName]bool{} | |
| 210 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |
| 211 | + | cfg.Clock = clk | |
| 212 | + | cfg.ScheduleTick = time.Hour | |
| 213 | + | cfg.ScheduleBudget = 1 | |
| 214 | + | cfg.ScheduleVisit = func(r domain.RepoName) { visited[r] = true } | |
| 215 | + | }) | |
| 216 | + | c.registerRunner("alpha", "ubuntu-latest") | |
| 217 | + | names := []domain.RepoName{c.name("proj")} | |
| 218 | + | for _, n := range []string{"aa", "zz", "mm", "bb"} { | |
| 219 | + | names = append(names, c.name(n)) | |
| 220 | + | _, err := c.forge.Repos.Create(c.ctx, domain.Root(), createRepoParams(c.name(n))) | |
| 221 | + | require.NoError(t, err) | |
| 222 | + | } | |
| 223 | + | ci := c.forge.CI | |
| 224 | + | seen := func() int { return len(visited) } | |
| 225 | + | minutes := []string{"10:01:00", "10:02:00", "10:03:00", "10:04:00", "10:05:00"} | |
| 226 | + | for i, m := range minutes { | |
| 227 | + | clk.set("2026-03-04 " + m) | |
| 228 | + | ci.ScheduleNow(c.ctx) // one repository, then the next minute drops the pass | |
| 229 | + | assert.Equal(t, i+1, seen(), "each minute reaches a repository no earlier minute reached") | |
| 230 | + | } | |
| 231 | + | assert.Equal(t, len(names), seen(), "every repository was processed") | |
| 232 | + | } | |
| 233 | + | ||
| 234 | + | func TestScheduleBoundsTheWorkPerTick(t *testing.T) { | |
| 235 | + | clk := &testClock{} | |
| 236 | + | clk.set("2026-03-04 10:00:30") | |
| 237 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |
| 238 | + | cfg.Clock = clk | |
| 239 | + | cfg.ScheduleTick = time.Hour | |
| 240 | + | cfg.ScheduleBudget = 1 | |
| 241 | + | }) | |
| 242 | + | c.registerRunner("alpha", "ubuntu-latest") | |
| 243 | + | c.push("main", scheduleWorkflow("* * * * *"), "") | |
| 244 | + | for _, name := range []string{"two", "three"} { | |
| 245 | + | _, err := c.forge.Repos.Create(c.ctx, domain.Root(), createRepoParams(c.name(name))) | |
| 246 | + | require.NoError(t, err) | |
| 247 | + | } | |
| 248 | + | ci := c.forge.CI | |
| 249 | + | clk.set("2026-03-04 10:01:00") | |
| 250 | + | ci.ScheduleNow(c.ctx) | |
| 251 | + | assert.True(t, ci.SchedulePassed().IsZero(), "one repository per tick: the pass is not over") | |
| 252 | + | ci.ScheduleNow(c.ctx) | |
| 253 | + | assert.True(t, ci.SchedulePassed().IsZero()) | |
| 254 | + | ci.ScheduleNow(c.ctx) | |
| 255 | + | assert.Equal(t, time.Date(2026, 3, 4, 10, 1, 0, 0, time.UTC), ci.SchedulePassed(), "three ticks go through three repositories") | |
| 256 | + | assert.Len(t, c.scheduled(), 1) | |
| 257 | + | } | |
| @@ -0,0 +1,257 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package app | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "sync" | |||
| 7 | + | "testing" | |||
| 8 | + | "time" | |||
| 9 | + | ||||
| 10 | + | "connectrpc.com/connect" | |||
| 11 | + | "github.com/stretchr/testify/assert" | |||
| 12 | + | "github.com/stretchr/testify/require" | |||
| 13 | + | ||||
| 14 | + | pb "bnema.dev/ermite-api/gen/ermite" | |||
| 15 | + | "bnema.dev/ermite/internal/domain" | |||
| 16 | + | "bnema.dev/ermite/internal/ports" | |||
| 17 | + | "bnema.dev/ermite/internal/usecase" | |||
| 18 | + | ) | |||
| 19 | + | ||||
| 20 | + | // CI-3 D5, AC5: schedule, with an injected clock. | |||
| 21 | + | ||||
| 22 | + | // testClock is a clock the test moves. | |||
| 23 | + | type testClock struct { | |||
| 24 | + | mu sync.Mutex | |||
| 25 | + | t time.Time | |||
| 26 | + | } | |||
| 27 | + | ||||
| 28 | + | func (c *testClock) Now() time.Time { | |||
| 29 | + | c.mu.Lock() | |||
| 30 | + | defer c.mu.Unlock() | |||
| 31 | + | return c.t | |||
| 32 | + | } | |||
| 33 | + | ||||
| 34 | + | func (c *testClock) set(s string) { | |||
| 35 | + | c.mu.Lock() | |||
| 36 | + | defer c.mu.Unlock() | |||
| 37 | + | t, err := time.Parse("2006-01-02 15:04:05", s) | |||
| 38 | + | if err != nil { | |||
| 39 | + | panic(err) | |||
| 40 | + | } | |||
| 41 | + | c.t = t | |||
| 42 | + | } | |||
| 43 | + | ||||
| 44 | + | func scheduleWorkflow(crons ...string) map[string]string { | |||
| 45 | + | on := "on:\n schedule:\n" | |||
| 46 | + | for _, c := range crons { | |||
| 47 | + | on += " - cron: '" + c + "'\n" | |||
| 48 | + | } | |||
| 49 | + | return map[string]string{".github/workflows/nightly.yml": "name: nightly\n" + on + "jobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"} | |||
| 50 | + | } | |||
| 51 | + | ||||
| 52 | + | func newScheduleHarness(t *testing.T, start string) (*ciHarness, *testClock, *fakeRunner) { | |||
| 53 | + | t.Helper() | |||
| 54 | + | clk := &testClock{} | |||
| 55 | + | clk.set(start) | |||
| 56 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |||
| 57 | + | cfg.Clock = clk | |||
| 58 | + | cfg.ScheduleTick = time.Hour // the tests call ScheduleNow | |||
| 59 | + | }) | |||
| 60 | + | return c, clk, c.registerRunner("alpha", "ubuntu-latest") | |||
| 61 | + | } | |||
| 62 | + | ||||
| 63 | + | func createRepoParams(name domain.RepoName) ports.CreateRepoParams { | |||
| 64 | + | return ports.CreateRepoParams{Name: name, Visibility: domain.VisibilityPrivate} | |||
| 65 | + | } | |||
| 66 | + | ||||
| 67 | + | func (c *ciHarness) scheduled() []*pb.Run { return scheduleRuns(c.runsOf("nightly")) } | |||
| 68 | + | ||||
| 69 | + | func scheduleRuns(runs []*pb.Run) []*pb.Run { | |||
| 70 | + | var out []*pb.Run | |||
| 71 | + | for _, r := range runs { | |||
| 72 | + | if r.GetEvent() == "schedule" { | |||
| 73 | + | out = append(out, r) | |||
| 74 | + | } | |||
| 75 | + | } | |||
| 76 | + | return out | |||
| 77 | + | } | |||
| 78 | + | ||||
| 79 | + | func TestScheduleFiresOncePerMatchingMinuteWithoutCatchUp(t *testing.T) { | |||
| 80 | + | c, clk, runner := newScheduleHarness(t, "2026-03-04 10:00:30") | |||
| 81 | + | c.push("main", scheduleWorkflow("*/5 * * * *"), "") | |||
| 82 | + | ci := c.forge.CI | |||
| 83 | + | tick := func(at string) { | |||
| 84 | + | clk.set(at) | |||
| 85 | + | ci.ScheduleNow(c.ctx) | |||
| 86 | + | } | |||
| 87 | + | ||||
| 88 | + | tick("2026-03-04 10:00:40") | |||
| 89 | + | assert.Empty(t, c.scheduled(), "the minute ermite started in belongs to the process before it") | |||
| 90 | + | ||||
| 91 | + | tick("2026-03-04 10:05:10") | |||
| 92 | + | require.Len(t, c.scheduled(), 1) | |||
| 93 | + | run := c.scheduled()[0] | |||
| 94 | + | assert.Equal(t, "schedule", run.GetEvent()) | |||
| 95 | + | assert.Equal(t, "refs/heads/main", run.GetRef()) | |||
| 96 | + | assert.Equal(t, c.ref("refs/heads/main"), run.GetSha(), "the latest commit of the default branch") | |||
| 97 | + | assert.Equal(t, "root", run.GetActor(), "whoever pushed the tip (the root token here)") | |||
| 98 | + | assert.True(t, run.GetTrusted()) | |||
| 99 | + | ||||
| 100 | + | tick("2026-03-04 10:05:50") | |||
| 101 | + | tick("2026-03-04 10:05:59") | |||
| 102 | + | assert.Len(t, c.scheduled(), 1, "a minute fires once, however many ticks it holds") | |||
| 103 | + | tick("2026-03-04 10:06:00") | |||
| 104 | + | tick("2026-03-04 10:09:00") | |||
| 105 | + | assert.Len(t, c.scheduled(), 1, "minutes that do not match fire nothing") | |||
| 106 | + | ||||
| 107 | + | // ermite was busy or down for the minutes 10:10 and 10:15: they are not made up. | |||
| 108 | + | tick("2026-03-04 10:20:00") | |||
| 109 | + | assert.Len(t, c.scheduled(), 2, "only the minute the clock reached fires, not the ones skipped") | |||
| 110 | + | assert.Equal(t, time.Date(2026, 3, 4, 10, 20, 0, 0, time.UTC), ci.SchedulePassed()) | |||
| 111 | + | ||||
| 112 | + | job := runner.await() | |||
| 113 | + | gh := c.ghContext(job) | |||
| 114 | + | assert.Equal(t, "schedule", gh["event_name"]) | |||
| 115 | + | assert.Equal(t, "refs/heads/main", gh["ref"]) | |||
| 116 | + | assert.Equal(t, "*/5 * * * *", gh["event"].(map[string]any)["schedule"], "the payload is the cron entry") | |||
| 117 | + | assert.True(t, job.GetTrusted()) | |||
| 118 | + | } | |||
| 119 | + | ||||
| 120 | + | // Several cron entries of one workflow that match the same minute start one | |||
| 121 | + | // run, with the first matching entry as payload; an entry that matches another | |||
| 122 | + | // minute starts its own. | |||
| 123 | + | func TestScheduleSeveralMatchingCronsOfOneWorkflowStartOneRun(t *testing.T) { | |||
| 124 | + | c, clk, runner := newScheduleHarness(t, "2026-03-04 10:00:30") | |||
| 125 | + | c.push("main", scheduleWorkflow("*/2 * * * *", "* * * * *", "7 10 * * *"), "") | |||
| 126 | + | ci := c.forge.CI | |||
| 127 | + | ||||
| 128 | + | clk.set("2026-03-04 10:02:00") // the first and the second entries match | |||
| 129 | + | ci.ScheduleNow(c.ctx) | |||
| 130 | + | require.Len(t, c.scheduled(), 1, "one run for the workflow, not one per entry") | |||
| 131 | + | job := runner.await() | |||
| 132 | + | assert.Equal(t, "*/2 * * * *", c.ghContext(job)["event"].(map[string]any)["schedule"], "the first entry that matches") | |||
| 133 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 134 | + | ||||
| 135 | + | clk.set("2026-03-04 10:03:00") // only the second | |||
| 136 | + | ci.ScheduleNow(c.ctx) | |||
| 137 | + | require.Len(t, c.scheduled(), 2) | |||
| 138 | + | job = runner.await() | |||
| 139 | + | assert.Equal(t, "* * * * *", c.ghContext(job)["event"].(map[string]any)["schedule"]) | |||
| 140 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 141 | + | } | |||
| 142 | + | ||||
| 143 | + | func TestScheduleReadsAgainAfterAPushToTheDefaultBranch(t *testing.T) { | |||
| 144 | + | c, clk, _ := newScheduleHarness(t, "2026-03-04 10:00:30") | |||
| 145 | + | c.push("main", scheduleWorkflow("*/5 * * * *"), "") | |||
| 146 | + | ci := c.forge.CI | |||
| 147 | + | tick := func(at string) { | |||
| 148 | + | clk.set(at) | |||
| 149 | + | ci.ScheduleNow(c.ctx) | |||
| 150 | + | } | |||
| 151 | + | tick("2026-03-04 10:05:00") | |||
| 152 | + | require.Len(t, c.scheduled(), 1) | |||
| 153 | + | assert.Equal(t, 1, ci.CachedSchedules(c.name("proj")), "the schedules are cached") | |||
| 154 | + | ||||
| 155 | + | // 10:07 matches no */5 entry; the new file fires every minute. | |||
| 156 | + | c.push("main", scheduleWorkflow("* * * * *"), "") | |||
| 157 | + | tick("2026-03-04 10:06:00") | |||
| 158 | + | require.Len(t, c.scheduled(), 2, "the pushed workflow is the one scheduled") | |||
| 159 | + | assert.Equal(t, 1, ci.CachedSchedules(c.name("proj"))) | |||
| 160 | + | ||||
| 161 | + | // A push to another branch changes nothing; a push that removes the schedule | |||
| 162 | + | // ends it. | |||
| 163 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "side") | |||
| 164 | + | c.push("side", scheduleWorkflow("0 0 1 1 *"), "") | |||
| 165 | + | tick("2026-03-04 10:07:00") | |||
| 166 | + | require.Len(t, c.scheduled(), 3, "the default branch's workflow still fires every minute") | |||
| 167 | + | ||||
| 168 | + | c.mustGit(c.wt, "checkout", "-q", "main") | |||
| 169 | + | c.push("main", map[string]string{".github/workflows/nightly.yml": "name: nightly\non: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"}, "") | |||
| 170 | + | tick("2026-03-04 10:08:00") | |||
| 171 | + | c.waitFor("the push run of the new file", func() bool { return len(c.runs()) == 4 }) | |||
| 172 | + | assert.Len(t, scheduleRuns(c.runs()), 3, "no schedule left on the default branch") | |||
| 173 | + | assert.Equal(t, 0, ci.CachedSchedules(c.name("proj"))) | |||
| 174 | + | } | |||
| 175 | + | ||||
| 176 | + | func TestScheduleIsRemovedWithTheRepository(t *testing.T) { | |||
| 177 | + | c, clk, _ := newScheduleHarness(t, "2026-03-04 10:00:30") | |||
| 178 | + | c.push("main", scheduleWorkflow("* * * * *"), "") | |||
| 179 | + | ci := c.forge.CI | |||
| 180 | + | clk.set("2026-03-04 10:01:00") | |||
| 181 | + | ci.ScheduleNow(c.ctx) | |||
| 182 | + | require.Len(t, c.scheduled(), 1) | |||
| 183 | + | require.Equal(t, 1, ci.CachedSchedules(c.name("proj"))) | |||
| 184 | + | ||||
| 185 | + | _, err := c.root.repo.DeleteRepo(c.ctx, connect.NewRequest(&pb.DeleteRepoRequest{Repo: "proj"})) | |||
| 186 | + | require.NoError(t, err) | |||
| 187 | + | assert.Equal(t, -1, ci.CachedSchedules(c.name("proj")), "the cached schedules go with the repository") | |||
| 188 | + | ||||
| 189 | + | for _, at := range []string{"2026-03-04 10:02:00", "2026-03-04 10:03:00"} { | |||
| 190 | + | clk.set(at) | |||
| 191 | + | ci.ScheduleNow(c.ctx) | |||
| 192 | + | } | |||
| 193 | + | assert.Equal(t, -1, ci.CachedSchedules(c.name("proj")), "never scheduled again") | |||
| 194 | + | ||||
| 195 | + | // The same name created again starts empty: no schedule until it has a workflow. | |||
| 196 | + | _, err = c.forge.Repos.Create(c.ctx, domain.Root(), createRepoParams(c.name("proj"))) | |||
| 197 | + | require.NoError(t, err) | |||
| 198 | + | clk.set("2026-03-04 10:04:00") | |||
| 199 | + | ci.ScheduleNow(c.ctx) | |||
| 200 | + | assert.Empty(t, c.scheduled()) | |||
| 201 | + | } | |||
| 202 | + | ||||
| 203 | + | // More repositories than one tick can handle, and a new minute at every tick: | |||
| 204 | + | // each pass is dropped after its first repository, and the cursor makes the next | |||
| 205 | + | // one start after it, so that none is starved. | |||
| 206 | + | func TestScheduleDoesNotStarveRepositoriesBeyondTheBudget(t *testing.T) { | |||
| 207 | + | clk := &testClock{} | |||
| 208 | + | clk.set("2026-03-04 10:00:30") | |||
| 209 | + | visited := map[domain.RepoName]bool{} | |||
| 210 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |||
| 211 | + | cfg.Clock = clk | |||
| 212 | + | cfg.ScheduleTick = time.Hour | |||
| 213 | + | cfg.ScheduleBudget = 1 | |||
| 214 | + | cfg.ScheduleVisit = func(r domain.RepoName) { visited[r] = true } | |||
| 215 | + | }) | |||
| 216 | + | c.registerRunner("alpha", "ubuntu-latest") | |||
| 217 | + | names := []domain.RepoName{c.name("proj")} | |||
| 218 | + | for _, n := range []string{"aa", "zz", "mm", "bb"} { | |||
| 219 | + | names = append(names, c.name(n)) | |||
| 220 | + | _, err := c.forge.Repos.Create(c.ctx, domain.Root(), createRepoParams(c.name(n))) | |||
| 221 | + | require.NoError(t, err) | |||
| 222 | + | } | |||
| 223 | + | ci := c.forge.CI | |||
| 224 | + | seen := func() int { return len(visited) } | |||
| 225 | + | minutes := []string{"10:01:00", "10:02:00", "10:03:00", "10:04:00", "10:05:00"} | |||
| 226 | + | for i, m := range minutes { | |||
| 227 | + | clk.set("2026-03-04 " + m) | |||
| 228 | + | ci.ScheduleNow(c.ctx) // one repository, then the next minute drops the pass | |||
| 229 | + | assert.Equal(t, i+1, seen(), "each minute reaches a repository no earlier minute reached") | |||
| 230 | + | } | |||
| 231 | + | assert.Equal(t, len(names), seen(), "every repository was processed") | |||
| 232 | + | } | |||
| 233 | + | ||||
| 234 | + | func TestScheduleBoundsTheWorkPerTick(t *testing.T) { | |||
| 235 | + | clk := &testClock{} | |||
| 236 | + | clk.set("2026-03-04 10:00:30") | |||
| 237 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |||
| 238 | + | cfg.Clock = clk | |||
| 239 | + | cfg.ScheduleTick = time.Hour | |||
| 240 | + | cfg.ScheduleBudget = 1 | |||
| 241 | + | }) | |||
| 242 | + | c.registerRunner("alpha", "ubuntu-latest") | |||
| 243 | + | c.push("main", scheduleWorkflow("* * * * *"), "") | |||
| 244 | + | for _, name := range []string{"two", "three"} { | |||
| 245 | + | _, err := c.forge.Repos.Create(c.ctx, domain.Root(), createRepoParams(c.name(name))) | |||
| 246 | + | require.NoError(t, err) | |||
| 247 | + | } | |||
| 248 | + | ci := c.forge.CI | |||
| 249 | + | clk.set("2026-03-04 10:01:00") | |||
| 250 | + | ci.ScheduleNow(c.ctx) | |||
| 251 | + | assert.True(t, ci.SchedulePassed().IsZero(), "one repository per tick: the pass is not over") | |||
| 252 | + | ci.ScheduleNow(c.ctx) | |||
| 253 | + | assert.True(t, ci.SchedulePassed().IsZero()) | |||
| 254 | + | ci.ScheduleNow(c.ctx) | |||
| 255 | + | assert.Equal(t, time.Date(2026, 3, 4, 10, 1, 0, 0, time.UTC), ci.SchedulePassed(), "three ticks go through three repositories") | |||
| 256 | + | assert.Len(t, c.scheduled(), 1) | |||
| 257 | + | } | |||
A internal/app/ci_schedule_trust_test.go +366 -0
| @@ -0,0 +1,366 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package app | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "strings" | |
| 7 | + | "testing" | |
| 8 | + | "time" | |
| 9 | + | ||
| 10 | + | "connectrpc.com/connect" | |
| 11 | + | "github.com/stretchr/testify/assert" | |
| 12 | + | "github.com/stretchr/testify/require" | |
| 13 | + | ||
| 14 | + | pb "bnema.dev/ermite-api/gen/ermite" | |
| 15 | + | "bnema.dev/ermite/internal/config" | |
| 16 | + | "bnema.dev/ermite/internal/domain" | |
| 17 | + | "bnema.dev/ermite/internal/ports" | |
| 18 | + | "bnema.dev/ermite/internal/usecase" | |
| 19 | + | ) | |
| 20 | + | ||
| 21 | + | // CI-3 D1, S1: a scheduled run is run as whoever moved the default branch to | |
| 22 | + | // its tip, with their trust. | |
| 23 | + | ||
| 24 | + | const scheduleSecretWorkflow = `name: nightly | |
| 25 | + | on: | |
| 26 | + | schedule: | |
| 27 | + | - cron: '* * * * *' | |
| 28 | + | jobs: | |
| 29 | + | j: | |
| 30 | + | runs-on: ubuntu-latest | |
| 31 | + | steps: | |
| 32 | + | - run: echo ${{ secrets.NIGHTLY_TOKEN }} | |
| 33 | + | ` | |
| 34 | + | ||
| 35 | + | func newScheduleSecretsHarness(t *testing.T) (*ciHarness, *testClock, *fakeRunner) { | |
| 36 | + | t.Helper() | |
| 37 | + | clk := &testClock{} | |
| 38 | + | clk.set("2026-03-04 10:00:30") | |
| 39 | + | h := newHarnessWith(t, harnessOpts{env: map[string]string{config.EnvSecretsKey: secretsKeyA}}, func(o *options) { | |
| 40 | + | o.ci = func(cfg *usecase.CIConfig) { | |
| 41 | + | cfg.Clock = clk | |
| 42 | + | cfg.ScheduleTick = time.Hour | |
| 43 | + | } | |
| 44 | + | }) | |
| 45 | + | c := &ciHarness{harness: h, root: h.api(adminToken), repo: "proj"} | |
| 46 | + | _, err := h.forge.Repos.Create(h.ctx, domain.Root(), ports.CreateRepoParams{Name: h.name("proj"), Visibility: domain.VisibilityPrivate}) | |
| 47 | + | require.NoError(t, err) | |
| 48 | + | c.wt = h.workTree("main") | |
| 49 | + | c.setSecret(repoScope("proj"), "NIGHTLY_TOKEN", "s3cret") | |
| 50 | + | return c, clk, c.registerRunner("alpha", "ubuntu-latest") | |
| 51 | + | } | |
| 52 | + | ||
| 53 | + | // fireAt moves the clock to the minute, ticks the scheduler and returns the job | |
| 54 | + | // of the scheduled run it started. | |
| 55 | + | func (c *ciHarness) fireAt(clk *testClock, runner *fakeRunner, at string) (*pb.Run, *pb.Job) { | |
| 56 | + | c.t.Helper() | |
| 57 | + | before := len(scheduleRuns(c.runs())) | |
| 58 | + | clk.set(at) | |
| 59 | + | c.forge.CI.ScheduleNow(c.ctx) | |
| 60 | + | runs := scheduleRuns(c.runs()) | |
| 61 | + | require.Len(c.t, runs, before+1, "one scheduled run") | |
| 62 | + | job := runner.await() | |
| 63 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 64 | + | return runs[0], job // newest first | |
| 65 | + | } | |
| 66 | + | ||
| 67 | + | func (c *ciHarness) repoToken(label string) string { | |
| 68 | + | return c.token(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), "proj") | |
| 69 | + | } | |
| 70 | + | ||
| 71 | + | func TestScheduleOnABranchAnAgentMadeTheDefaultIsNotTrusted(t *testing.T) { | |
| 72 | + | c, clk, runner := newScheduleSecretsHarness(t) | |
| 73 | + | // The repository is empty: the first branch pushed becomes the default | |
| 74 | + | // branch, whoever pushes it. | |
| 75 | + | secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), []string{"agent/"}, "proj") | |
| 76 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "agent/x") | |
| 77 | + | c.writeFile(c.wt, ".github/workflows/nightly.yml", []byte(scheduleSecretWorkflow)) | |
| 78 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(secret, "/proj.git"), "agent/x") | |
| 79 | + | def, _, err := c.forge.git.DefaultBranch(c.ctx, c.name("proj")) | |
| 80 | + | require.NoError(t, err) | |
| 81 | + | require.Equal(t, "agent/x", def, "the agent's branch is the default branch") | |
| 82 | + | ||
| 83 | + | run, job := c.fireAt(clk, runner, "2026-03-04 10:01:00") | |
| 84 | + | assert.False(t, run.GetTrusted(), "the owner's secrets do not run for an agent's workflow") | |
| 85 | + | assert.Equal(t, "claude", run.GetActor(), "the actor is who pushed the tip") | |
| 86 | + | assert.False(t, job.GetTrusted()) | |
| 87 | + | assert.Empty(t, job.GetSecrets()) | |
| 88 | + | } | |
| 89 | + | ||
| 90 | + | func TestScheduleActorFollowsTheLastPusherOfTheDefaultBranch(t *testing.T) { | |
| 91 | + | c, clk, runner := newScheduleSecretsHarness(t) | |
| 92 | + | ||
| 93 | + | // The owner pushes main: trusted, with the secrets. | |
| 94 | + | c.push("main", map[string]string{".github/workflows/nightly.yml": scheduleSecretWorkflow}, "") | |
| 95 | + | run, job := c.fireAt(clk, runner, "2026-03-04 10:01:00") | |
| 96 | + | assert.True(t, run.GetTrusted()) | |
| 97 | + | assert.Equal(t, "root", run.GetActor()) | |
| 98 | + | assert.Equal(t, map[string]string{"NIGHTLY_TOKEN": "s3cret"}, job.GetSecrets()) | |
| 99 | + | ||
| 100 | + | // A token scoped to the repository (not the owner) pushes main: untrusted. | |
| 101 | + | tok := c.repoToken("scoped") | |
| 102 | + | c.writeFile(c.wt, "x.txt", []byte("x")) | |
| 103 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(tok, "/proj.git"), "main") | |
| 104 | + | run, job = c.fireAt(clk, runner, "2026-03-04 10:02:00") | |
| 105 | + | assert.False(t, run.GetTrusted(), "a repository-scoped token is not the owner") | |
| 106 | + | assert.Empty(t, job.GetSecrets()) | |
| 107 | + | ||
| 108 | + | // The owner pushes again: trusted again. | |
| 109 | + | c.push("main", map[string]string{"y.txt": "y"}, "") | |
| 110 | + | run, _ = c.fireAt(clk, runner, "2026-03-04 10:03:00") | |
| 111 | + | assert.True(t, run.GetTrusted()) | |
| 112 | + | ||
| 113 | + | // The record is for one commit: a tip it does not name is not trusted. | |
| 114 | + | require.NoError(t, c.forge.store.CI().PutRefTip(c.ctx, domain.CIRefTip{ | |
| 115 | + | Repo: c.name("proj"), Ref: "refs/heads/main", Sha: "1111111111111111111111111111111111111111", Actor: "root", Trusted: true, | |
| 116 | + | })) | |
| 117 | + | run, job = c.fireAt(clk, runner, "2026-03-04 10:04:00") | |
| 118 | + | assert.False(t, run.GetTrusted(), "the record names another commit") | |
| 119 | + | assert.Empty(t, job.GetSecrets()) | |
| 120 | + | } | |
| 121 | + | ||
| 122 | + | func TestScheduleAfterAMergeIsTheMergersToo(t *testing.T) { | |
| 123 | + | c, clk, runner := newScheduleSecretsHarness(t) | |
| 124 | + | c.push("main", map[string]string{".github/workflows/nightly.yml": scheduleSecretWorkflow}, "") | |
| 125 | + | tok := c.repoToken("scoped") | |
| 126 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "feat") | |
| 127 | + | c.writeFile(c.wt, "f.txt", []byte("f")) | |
| 128 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(tok, "/proj.git"), "feat") | |
| 129 | + | n := c.openMR(c.root, "feat") | |
| 130 | + | c.waitFor("the head ref", func() bool { return c.ref(domain.PullHeadRef(n)) != "" }) | |
| 131 | + | head := c.ref(domain.PullHeadRef(n)) | |
| 132 | + | ||
| 133 | + | // The owner merges: main is theirs from now on. | |
| 134 | + | _, err := c.root.mr.MergeMergeRequest(c.ctx, connect.NewRequest(&pb.MergeMergeRequestRequest{Repo: "proj", Number: n, HeadId: head})) | |
| 135 | + | require.NoError(t, err) | |
| 136 | + | run, job := c.fireAt(clk, runner, "2026-03-04 10:01:00") | |
| 137 | + | assert.True(t, run.GetTrusted(), "merged by the owner") | |
| 138 | + | assert.Equal(t, map[string]string{"NIGHTLY_TOKEN": "s3cret"}, job.GetSecrets()) | |
| 139 | + | } | |
| 140 | + | ||
| 141 | + | func TestRefTipRecordsGoWithTheRepository(t *testing.T) { | |
| 142 | + | c, _, _ := newScheduleSecretsHarness(t) | |
| 143 | + | c.push("main", map[string]string{"a.txt": "a"}, "") | |
| 144 | + | tips := c.forge.store.CI() | |
| 145 | + | got, err := tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |
| 146 | + | require.NoError(t, err) | |
| 147 | + | assert.True(t, got.Trusted) | |
| 148 | + | assert.Equal(t, c.ref("refs/heads/main"), got.Sha) | |
| 149 | + | ||
| 150 | + | _, err = c.root.repo.DeleteRepo(c.ctx, connect.NewRequest(&pb.DeleteRepoRequest{Repo: "proj"})) | |
| 151 | + | require.NoError(t, err) | |
| 152 | + | _, err = tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |
| 153 | + | assert.ErrorIs(t, err, domain.ErrNotFound, "deleted with the repository") | |
| 154 | + | } | |
| 155 | + | ||
| 156 | + | // GO-004: the record is written before the ref moves, so the sequence owner A, | |
| 157 | + | // someone else B, someone else A (the same commit as the owner's) never leaves | |
| 158 | + | // the owner's record on the tip, however late the post-receive of the last push | |
| 159 | + | // is. A push that is refused after its record was written leaves a record that | |
| 160 | + | // matches no tip. | |
| 161 | + | func TestScheduleABAOfThePusherIsNeverTrusted(t *testing.T) { | |
| 162 | + | c, clk, runner := newScheduleSecretsHarness(t) | |
| 163 | + | tips := c.forge.store.CI() | |
| 164 | + | c.push("main", map[string]string{".github/workflows/nightly.yml": scheduleSecretWorkflow}, "") | |
| 165 | + | commitA := c.ref("refs/heads/main") | |
| 166 | + | run, _ := c.fireAt(clk, runner, "2026-03-04 10:01:00") | |
| 167 | + | require.True(t, run.GetTrusted(), "the owner's A is trusted") | |
| 168 | + | ||
| 169 | + | // B and A again, by a repository-scoped token. | |
| 170 | + | tok := c.repoToken("scoped") | |
| 171 | + | c.writeFile(c.wt, "b.txt", []byte("b")) | |
| 172 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(tok, "/proj.git"), "main") | |
| 173 | + | rec, err := tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |
| 174 | + | require.NoError(t, err) | |
| 175 | + | assert.False(t, rec.Trusted, "B was recorded as the token's before the ref moved") | |
| 176 | + | c.mustGit(c.wt, "push", "-q", "-f", c.urlAuth(tok, "/proj.git"), commitA+":refs/heads/main") | |
| 177 | + | require.Equal(t, commitA, c.ref("refs/heads/main"), "main is back at the owner's commit") | |
| 178 | + | ||
| 179 | + | // The record of that push is the token's, not the owner's: A is untrusted. | |
| 180 | + | rec, err = tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |
| 181 | + | require.NoError(t, err) | |
| 182 | + | assert.Equal(t, commitA, rec.Sha) | |
| 183 | + | assert.False(t, rec.Trusted) | |
| 184 | + | run, job := c.fireAt(clk, runner, "2026-03-04 10:02:00") | |
| 185 | + | assert.False(t, run.GetTrusted(), "owner A, token B, token A: not the owner's") | |
| 186 | + | assert.Empty(t, job.GetSecrets()) | |
| 187 | + | ||
| 188 | + | // A push that moves nothing (the owner pushes A where A already is) is not a | |
| 189 | + | // push for git: it endorses nothing. | |
| 190 | + | c.mustGit(c.wt, "push", "-q", "-f", c.urlAuth(adminToken, "/proj.git"), commitA+":refs/heads/main") | |
| 191 | + | run, _ = c.fireAt(clk, runner, "2026-03-04 10:03:00") | |
| 192 | + | assert.False(t, run.GetTrusted(), "still the token's") | |
| 193 | + | ||
| 194 | + | // The owner moves the tip: trusted again. | |
| 195 | + | c.push("main", map[string]string{"owner.txt": "o"}, "") | |
| 196 | + | run, _ = c.fireAt(clk, runner, "2026-03-04 10:04:00") | |
| 197 | + | assert.True(t, run.GetTrusted()) | |
| 198 | + | tip := c.ref("refs/heads/main") | |
| 199 | + | ||
| 200 | + | // A push refused after the decision recorded it (here the check alone runs, | |
| 201 | + | // then git never moves the ref): the record names a commit the tip does not | |
| 202 | + | // have, and the schedule fails closed. | |
| 203 | + | scoped, err := c.forge.Auth.Authenticate(c.ctx, tok) | |
| 204 | + | require.NoError(t, err) | |
| 205 | + | never := strings.Repeat("2", 40) | |
| 206 | + | require.NoError(t, c.forge.Pushes.Check(c.ctx, scoped, c.name("proj"), | |
| 207 | + | []domain.RefUpdate{{Ref: "refs/heads/main", Old: tip, New: never}}, ports.PushObjects{})) | |
| 208 | + | left, err := c.forge.store.CI().GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |
| 209 | + | require.NoError(t, err) | |
| 210 | + | assert.Equal(t, never, left.Sha, "the refused push left its record behind") | |
| 211 | + | run, _ = c.fireAt(clk, runner, "2026-03-04 10:05:00") | |
| 212 | + | assert.False(t, run.GetTrusted(), "a record for a commit that never became the tip: fail closed") | |
| 213 | + | } | |
| 214 | + | ||
| 215 | + | // GO-006: a merge that deletes its source branch drops the record of that | |
| 216 | + | // branch, and a merge records the target before it moves. | |
| 217 | + | func TestMergeRecordsTheTargetAndDropsTheSourceRecord(t *testing.T) { | |
| 218 | + | c, _, _ := newScheduleSecretsHarness(t) | |
| 219 | + | tips := c.forge.store.CI() | |
| 220 | + | c.push("main", map[string]string{"a.txt": "a"}, "") | |
| 221 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "feat") | |
| 222 | + | c.push("feat", map[string]string{"f.txt": "f"}, "") | |
| 223 | + | _, err := tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/feat") | |
| 224 | + | require.NoError(t, err, "the pushed branch has a record") | |
| 225 | + | n := c.openMR(c.root, "feat") | |
| 226 | + | c.waitFor("the head ref", func() bool { return c.ref(domain.PullHeadRef(n)) != "" }) | |
| 227 | + | ||
| 228 | + | _, err = c.root.mr.MergeMergeRequest(c.ctx, connect.NewRequest(&pb.MergeMergeRequestRequest{ | |
| 229 | + | Repo: "proj", Number: n, HeadId: c.ref(domain.PullHeadRef(n)), DeleteSourceBranch: true, | |
| 230 | + | })) | |
| 231 | + | require.NoError(t, err) | |
| 232 | + | ||
| 233 | + | _, err = tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/feat") | |
| 234 | + | assert.ErrorIs(t, err, domain.ErrNotFound, "the deleted source branch has no record") | |
| 235 | + | main, err := tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |
| 236 | + | require.NoError(t, err) | |
| 237 | + | assert.Equal(t, c.ref("refs/heads/main"), main.Sha, "the merge recorded the new tip") | |
| 238 | + | assert.True(t, main.Trusted) | |
| 239 | + | ||
| 240 | + | // A pushed branch that is deleted by a push loses its record too. | |
| 241 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "gone") | |
| 242 | + | c.push("gone", map[string]string{"g.txt": "g"}, "") | |
| 243 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), ":refs/heads/gone") | |
| 244 | + | _, err = tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/gone") | |
| 245 | + | assert.ErrorIs(t, err, domain.ErrNotFound) | |
| 246 | + | } | |
| 247 | + | ||
| 248 | + | // GO-005: a dispatched run is trusted only when the caller is and the ref was | |
| 249 | + | // moved to its commit by a trusted principal, for every caller. | |
| 250 | + | func TestDispatchTrustFollowsTheRefsPusher(t *testing.T) { | |
| 251 | + | c := newCIHarness(t, nil) | |
| 252 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 253 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow, ".github/workflows/d.yml": jobTokenDispatcher}, "") | |
| 254 | + | jobs := runner.jobsByKey(4) | |
| 255 | + | in := map[string]string{"version": "1"} | |
| 256 | + | ||
| 257 | + | // An agent's branch. | |
| 258 | + | secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), []string{"agent/"}, "proj") | |
| 259 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "agent/x") | |
| 260 | + | c.writeFile(c.wt, "x.txt", []byte("x")) | |
| 261 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(secret, "/proj.git"), "agent/x") | |
| 262 | + | c.waitFor("the agent's push run", func() bool { return len(c.runsOf("dispatcher")) >= 2 }) | |
| 263 | + | ||
| 264 | + | run, err := c.dispatch(c.root, "deploy.yml", "main", in) | |
| 265 | + | require.NoError(t, err) | |
| 266 | + | assert.True(t, run.GetTrusted(), "the owner dispatching what the owner pushed") | |
| 267 | + | ||
| 268 | + | run, err = c.dispatch(c.root, "deploy.yml", "agent/x", in) | |
| 269 | + | require.NoError(t, err) | |
| 270 | + | assert.False(t, run.GetTrusted(), "the owner dispatching an agent's branch") | |
| 271 | + | ||
| 272 | + | run, err = c.dispatch(c.api(jobs["with"].GetGithubToken()), "deploy.yml", "agent/x", in) | |
| 273 | + | require.NoError(t, err) | |
| 274 | + | assert.False(t, run.GetTrusted(), "a trusted job token dispatching an agent's branch") | |
| 275 | + | run, err = c.dispatch(c.api(jobs["with"].GetGithubToken()), "deploy.yml", "main", in) | |
| 276 | + | require.NoError(t, err) | |
| 277 | + | assert.True(t, run.GetTrusted(), "a trusted job token dispatching the owner's branch") | |
| 278 | + | ||
| 279 | + | // What the jobs of those runs received: the untrusted ones got no secrets. | |
| 280 | + | got := map[int64]*pb.Job{} | |
| 281 | + | for range 4 { | |
| 282 | + | j := runner.await() | |
| 283 | + | got[j.GetRunId()] = j | |
| 284 | + | } | |
| 285 | + | for _, r := range c.runs() { | |
| 286 | + | if r.GetEvent() != "workflow_dispatch" { | |
| 287 | + | continue | |
| 288 | + | } | |
| 289 | + | if j := got[r.GetId()]; j != nil { | |
| 290 | + | assert.Equal(t, r.GetTrusted(), j.GetTrusted(), "the job follows its run") | |
| 291 | + | if !r.GetTrusted() { | |
| 292 | + | assert.Empty(t, j.GetSecrets()) | |
| 293 | + | } | |
| 294 | + | } | |
| 295 | + | } | |
| 296 | + | } | |
| 297 | + | ||
| 298 | + | // GO-007: the trust of a dispatch is judged on the object and the commit it | |
| 299 | + | // resolved, not on the tag as it is read afterwards. The tag is a lightweight | |
| 300 | + | // one at commit A when the dispatch resolves it; before the trust is judged it | |
| 301 | + | // is retargeted by the owner to an annotated tag on commit B, whose record is | |
| 302 | + | // trusted: A runs, and must not borrow that record. | |
| 303 | + | func TestDispatchOnATagRetargetedMeanwhileIsNotTrusted(t *testing.T) { | |
| 304 | + | var retarget func() | |
| 305 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |
| 306 | + | cfg.DispatchResolved = func() { | |
| 307 | + | if retarget != nil { | |
| 308 | + | retarget() | |
| 309 | + | } | |
| 310 | + | } | |
| 311 | + | }) | |
| 312 | + | c.registerRunner("alpha", "ubuntu-latest") | |
| 313 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow}, "") | |
| 314 | + | c.mustGit(c.wt, "tag", "v1") | |
| 315 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "v1") | |
| 316 | + | commitA := c.ref("refs/heads/main") | |
| 317 | + | in := map[string]string{"version": "1"} | |
| 318 | + | ||
| 319 | + | run, err := c.dispatch(c.root, "deploy.yml", "v1", in) | |
| 320 | + | require.NoError(t, err) | |
| 321 | + | require.True(t, run.GetTrusted(), "without a retarget the owner's own tag is trusted") | |
| 322 | + | ||
| 323 | + | retarget = func() { | |
| 324 | + | retarget = nil // once | |
| 325 | + | c.writeFile(c.wt, "b.txt", []byte("b")) | |
| 326 | + | c.mustGit(c.wt, "tag", "-a", "-f", "-m", "annotated", "v1") | |
| 327 | + | c.mustGit(c.wt, "push", "-q", "-f", c.urlAuth(adminToken, "/proj.git"), "v1") | |
| 328 | + | } | |
| 329 | + | run, err = c.dispatch(c.root, "deploy.yml", "v1", in) | |
| 330 | + | require.NoError(t, err) | |
| 331 | + | assert.Equal(t, commitA, run.GetSha(), "the dispatch runs the commit it resolved") | |
| 332 | + | assert.False(t, run.GetTrusted(), "the tag moved to another object meanwhile: the record of the new one is not A's") | |
| 333 | + | ||
| 334 | + | // The tag really moved: dispatching it now is trusted again, on the new commit. | |
| 335 | + | run, err = c.dispatch(c.root, "deploy.yml", "v1", in) | |
| 336 | + | require.NoError(t, err) | |
| 337 | + | assert.NotEqual(t, commitA, run.GetSha()) | |
| 338 | + | assert.True(t, run.GetTrusted()) | |
| 339 | + | } | |
| 340 | + | ||
| 341 | + | // A tag is judged by its own record, like a branch. | |
| 342 | + | func TestDispatchOnATagFollowsItsPusher(t *testing.T) { | |
| 343 | + | c := newCIHarness(t, nil) | |
| 344 | + | c.registerRunner("alpha", "ubuntu-latest") | |
| 345 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow}, "") | |
| 346 | + | c.mustGit(c.wt, "tag", "v1") | |
| 347 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "v1") | |
| 348 | + | c.mustGit(c.wt, "tag", "-a", "-m", "annotated", "v2") | |
| 349 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "v2") | |
| 350 | + | in := map[string]string{"version": "1"} | |
| 351 | + | ||
| 352 | + | run, err := c.dispatch(c.root, "deploy.yml", "v1", in) | |
| 353 | + | require.NoError(t, err) | |
| 354 | + | assert.True(t, run.GetTrusted(), "a lightweight tag the owner pushed") | |
| 355 | + | run, err = c.dispatch(c.root, "deploy.yml", "v2", in) | |
| 356 | + | require.NoError(t, err) | |
| 357 | + | assert.True(t, run.GetTrusted(), "an annotated tag the owner pushed (the record names the tag object)") | |
| 358 | + | ||
| 359 | + | tok := c.repoToken("scoped") | |
| 360 | + | c.writeFile(c.wt, "t.txt", []byte("t")) | |
| 361 | + | c.mustGit(c.wt, "tag", "v3") | |
| 362 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(tok, "/proj.git"), "v3") | |
| 363 | + | run, err = c.dispatch(c.root, "deploy.yml", "v3", in) | |
| 364 | + | require.NoError(t, err) | |
| 365 | + | assert.False(t, run.GetTrusted(), "a tag a scoped token pushed") | |
| 366 | + | } | |
| @@ -0,0 +1,366 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package app | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "strings" | |||
| 7 | + | "testing" | |||
| 8 | + | "time" | |||
| 9 | + | ||||
| 10 | + | "connectrpc.com/connect" | |||
| 11 | + | "github.com/stretchr/testify/assert" | |||
| 12 | + | "github.com/stretchr/testify/require" | |||
| 13 | + | ||||
| 14 | + | pb "bnema.dev/ermite-api/gen/ermite" | |||
| 15 | + | "bnema.dev/ermite/internal/config" | |||
| 16 | + | "bnema.dev/ermite/internal/domain" | |||
| 17 | + | "bnema.dev/ermite/internal/ports" | |||
| 18 | + | "bnema.dev/ermite/internal/usecase" | |||
| 19 | + | ) | |||
| 20 | + | ||||
| 21 | + | // CI-3 D1, S1: a scheduled run is run as whoever moved the default branch to | |||
| 22 | + | // its tip, with their trust. | |||
| 23 | + | ||||
| 24 | + | const scheduleSecretWorkflow = `name: nightly | |||
| 25 | + | on: | |||
| 26 | + | schedule: | |||
| 27 | + | - cron: '* * * * *' | |||
| 28 | + | jobs: | |||
| 29 | + | j: | |||
| 30 | + | runs-on: ubuntu-latest | |||
| 31 | + | steps: | |||
| 32 | + | - run: echo ${{ secrets.NIGHTLY_TOKEN }} | |||
| 33 | + | ` | |||
| 34 | + | ||||
| 35 | + | func newScheduleSecretsHarness(t *testing.T) (*ciHarness, *testClock, *fakeRunner) { | |||
| 36 | + | t.Helper() | |||
| 37 | + | clk := &testClock{} | |||
| 38 | + | clk.set("2026-03-04 10:00:30") | |||
| 39 | + | h := newHarnessWith(t, harnessOpts{env: map[string]string{config.EnvSecretsKey: secretsKeyA}}, func(o *options) { | |||
| 40 | + | o.ci = func(cfg *usecase.CIConfig) { | |||
| 41 | + | cfg.Clock = clk | |||
| 42 | + | cfg.ScheduleTick = time.Hour | |||
| 43 | + | } | |||
| 44 | + | }) | |||
| 45 | + | c := &ciHarness{harness: h, root: h.api(adminToken), repo: "proj"} | |||
| 46 | + | _, err := h.forge.Repos.Create(h.ctx, domain.Root(), ports.CreateRepoParams{Name: h.name("proj"), Visibility: domain.VisibilityPrivate}) | |||
| 47 | + | require.NoError(t, err) | |||
| 48 | + | c.wt = h.workTree("main") | |||
| 49 | + | c.setSecret(repoScope("proj"), "NIGHTLY_TOKEN", "s3cret") | |||
| 50 | + | return c, clk, c.registerRunner("alpha", "ubuntu-latest") | |||
| 51 | + | } | |||
| 52 | + | ||||
| 53 | + | // fireAt moves the clock to the minute, ticks the scheduler and returns the job | |||
| 54 | + | // of the scheduled run it started. | |||
| 55 | + | func (c *ciHarness) fireAt(clk *testClock, runner *fakeRunner, at string) (*pb.Run, *pb.Job) { | |||
| 56 | + | c.t.Helper() | |||
| 57 | + | before := len(scheduleRuns(c.runs())) | |||
| 58 | + | clk.set(at) | |||
| 59 | + | c.forge.CI.ScheduleNow(c.ctx) | |||
| 60 | + | runs := scheduleRuns(c.runs()) | |||
| 61 | + | require.Len(c.t, runs, before+1, "one scheduled run") | |||
| 62 | + | job := runner.await() | |||
| 63 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 64 | + | return runs[0], job // newest first | |||
| 65 | + | } | |||
| 66 | + | ||||
| 67 | + | func (c *ciHarness) repoToken(label string) string { | |||
| 68 | + | return c.token(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), "proj") | |||
| 69 | + | } | |||
| 70 | + | ||||
| 71 | + | func TestScheduleOnABranchAnAgentMadeTheDefaultIsNotTrusted(t *testing.T) { | |||
| 72 | + | c, clk, runner := newScheduleSecretsHarness(t) | |||
| 73 | + | // The repository is empty: the first branch pushed becomes the default | |||
| 74 | + | // branch, whoever pushes it. | |||
| 75 | + | secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), []string{"agent/"}, "proj") | |||
| 76 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "agent/x") | |||
| 77 | + | c.writeFile(c.wt, ".github/workflows/nightly.yml", []byte(scheduleSecretWorkflow)) | |||
| 78 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(secret, "/proj.git"), "agent/x") | |||
| 79 | + | def, _, err := c.forge.git.DefaultBranch(c.ctx, c.name("proj")) | |||
| 80 | + | require.NoError(t, err) | |||
| 81 | + | require.Equal(t, "agent/x", def, "the agent's branch is the default branch") | |||
| 82 | + | ||||
| 83 | + | run, job := c.fireAt(clk, runner, "2026-03-04 10:01:00") | |||
| 84 | + | assert.False(t, run.GetTrusted(), "the owner's secrets do not run for an agent's workflow") | |||
| 85 | + | assert.Equal(t, "claude", run.GetActor(), "the actor is who pushed the tip") | |||
| 86 | + | assert.False(t, job.GetTrusted()) | |||
| 87 | + | assert.Empty(t, job.GetSecrets()) | |||
| 88 | + | } | |||
| 89 | + | ||||
| 90 | + | func TestScheduleActorFollowsTheLastPusherOfTheDefaultBranch(t *testing.T) { | |||
| 91 | + | c, clk, runner := newScheduleSecretsHarness(t) | |||
| 92 | + | ||||
| 93 | + | // The owner pushes main: trusted, with the secrets. | |||
| 94 | + | c.push("main", map[string]string{".github/workflows/nightly.yml": scheduleSecretWorkflow}, "") | |||
| 95 | + | run, job := c.fireAt(clk, runner, "2026-03-04 10:01:00") | |||
| 96 | + | assert.True(t, run.GetTrusted()) | |||
| 97 | + | assert.Equal(t, "root", run.GetActor()) | |||
| 98 | + | assert.Equal(t, map[string]string{"NIGHTLY_TOKEN": "s3cret"}, job.GetSecrets()) | |||
| 99 | + | ||||
| 100 | + | // A token scoped to the repository (not the owner) pushes main: untrusted. | |||
| 101 | + | tok := c.repoToken("scoped") | |||
| 102 | + | c.writeFile(c.wt, "x.txt", []byte("x")) | |||
| 103 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(tok, "/proj.git"), "main") | |||
| 104 | + | run, job = c.fireAt(clk, runner, "2026-03-04 10:02:00") | |||
| 105 | + | assert.False(t, run.GetTrusted(), "a repository-scoped token is not the owner") | |||
| 106 | + | assert.Empty(t, job.GetSecrets()) | |||
| 107 | + | ||||
| 108 | + | // The owner pushes again: trusted again. | |||
| 109 | + | c.push("main", map[string]string{"y.txt": "y"}, "") | |||
| 110 | + | run, _ = c.fireAt(clk, runner, "2026-03-04 10:03:00") | |||
| 111 | + | assert.True(t, run.GetTrusted()) | |||
| 112 | + | ||||
| 113 | + | // The record is for one commit: a tip it does not name is not trusted. | |||
| 114 | + | require.NoError(t, c.forge.store.CI().PutRefTip(c.ctx, domain.CIRefTip{ | |||
| 115 | + | Repo: c.name("proj"), Ref: "refs/heads/main", Sha: "1111111111111111111111111111111111111111", Actor: "root", Trusted: true, | |||
| 116 | + | })) | |||
| 117 | + | run, job = c.fireAt(clk, runner, "2026-03-04 10:04:00") | |||
| 118 | + | assert.False(t, run.GetTrusted(), "the record names another commit") | |||
| 119 | + | assert.Empty(t, job.GetSecrets()) | |||
| 120 | + | } | |||
| 121 | + | ||||
| 122 | + | func TestScheduleAfterAMergeIsTheMergersToo(t *testing.T) { | |||
| 123 | + | c, clk, runner := newScheduleSecretsHarness(t) | |||
| 124 | + | c.push("main", map[string]string{".github/workflows/nightly.yml": scheduleSecretWorkflow}, "") | |||
| 125 | + | tok := c.repoToken("scoped") | |||
| 126 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "feat") | |||
| 127 | + | c.writeFile(c.wt, "f.txt", []byte("f")) | |||
| 128 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(tok, "/proj.git"), "feat") | |||
| 129 | + | n := c.openMR(c.root, "feat") | |||
| 130 | + | c.waitFor("the head ref", func() bool { return c.ref(domain.PullHeadRef(n)) != "" }) | |||
| 131 | + | head := c.ref(domain.PullHeadRef(n)) | |||
| 132 | + | ||||
| 133 | + | // The owner merges: main is theirs from now on. | |||
| 134 | + | _, err := c.root.mr.MergeMergeRequest(c.ctx, connect.NewRequest(&pb.MergeMergeRequestRequest{Repo: "proj", Number: n, HeadId: head})) | |||
| 135 | + | require.NoError(t, err) | |||
| 136 | + | run, job := c.fireAt(clk, runner, "2026-03-04 10:01:00") | |||
| 137 | + | assert.True(t, run.GetTrusted(), "merged by the owner") | |||
| 138 | + | assert.Equal(t, map[string]string{"NIGHTLY_TOKEN": "s3cret"}, job.GetSecrets()) | |||
| 139 | + | } | |||
| 140 | + | ||||
| 141 | + | func TestRefTipRecordsGoWithTheRepository(t *testing.T) { | |||
| 142 | + | c, _, _ := newScheduleSecretsHarness(t) | |||
| 143 | + | c.push("main", map[string]string{"a.txt": "a"}, "") | |||
| 144 | + | tips := c.forge.store.CI() | |||
| 145 | + | got, err := tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |||
| 146 | + | require.NoError(t, err) | |||
| 147 | + | assert.True(t, got.Trusted) | |||
| 148 | + | assert.Equal(t, c.ref("refs/heads/main"), got.Sha) | |||
| 149 | + | ||||
| 150 | + | _, err = c.root.repo.DeleteRepo(c.ctx, connect.NewRequest(&pb.DeleteRepoRequest{Repo: "proj"})) | |||
| 151 | + | require.NoError(t, err) | |||
| 152 | + | _, err = tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |||
| 153 | + | assert.ErrorIs(t, err, domain.ErrNotFound, "deleted with the repository") | |||
| 154 | + | } | |||
| 155 | + | ||||
| 156 | + | // GO-004: the record is written before the ref moves, so the sequence owner A, | |||
| 157 | + | // someone else B, someone else A (the same commit as the owner's) never leaves | |||
| 158 | + | // the owner's record on the tip, however late the post-receive of the last push | |||
| 159 | + | // is. A push that is refused after its record was written leaves a record that | |||
| 160 | + | // matches no tip. | |||
| 161 | + | func TestScheduleABAOfThePusherIsNeverTrusted(t *testing.T) { | |||
| 162 | + | c, clk, runner := newScheduleSecretsHarness(t) | |||
| 163 | + | tips := c.forge.store.CI() | |||
| 164 | + | c.push("main", map[string]string{".github/workflows/nightly.yml": scheduleSecretWorkflow}, "") | |||
| 165 | + | commitA := c.ref("refs/heads/main") | |||
| 166 | + | run, _ := c.fireAt(clk, runner, "2026-03-04 10:01:00") | |||
| 167 | + | require.True(t, run.GetTrusted(), "the owner's A is trusted") | |||
| 168 | + | ||||
| 169 | + | // B and A again, by a repository-scoped token. | |||
| 170 | + | tok := c.repoToken("scoped") | |||
| 171 | + | c.writeFile(c.wt, "b.txt", []byte("b")) | |||
| 172 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(tok, "/proj.git"), "main") | |||
| 173 | + | rec, err := tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |||
| 174 | + | require.NoError(t, err) | |||
| 175 | + | assert.False(t, rec.Trusted, "B was recorded as the token's before the ref moved") | |||
| 176 | + | c.mustGit(c.wt, "push", "-q", "-f", c.urlAuth(tok, "/proj.git"), commitA+":refs/heads/main") | |||
| 177 | + | require.Equal(t, commitA, c.ref("refs/heads/main"), "main is back at the owner's commit") | |||
| 178 | + | ||||
| 179 | + | // The record of that push is the token's, not the owner's: A is untrusted. | |||
| 180 | + | rec, err = tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |||
| 181 | + | require.NoError(t, err) | |||
| 182 | + | assert.Equal(t, commitA, rec.Sha) | |||
| 183 | + | assert.False(t, rec.Trusted) | |||
| 184 | + | run, job := c.fireAt(clk, runner, "2026-03-04 10:02:00") | |||
| 185 | + | assert.False(t, run.GetTrusted(), "owner A, token B, token A: not the owner's") | |||
| 186 | + | assert.Empty(t, job.GetSecrets()) | |||
| 187 | + | ||||
| 188 | + | // A push that moves nothing (the owner pushes A where A already is) is not a | |||
| 189 | + | // push for git: it endorses nothing. | |||
| 190 | + | c.mustGit(c.wt, "push", "-q", "-f", c.urlAuth(adminToken, "/proj.git"), commitA+":refs/heads/main") | |||
| 191 | + | run, _ = c.fireAt(clk, runner, "2026-03-04 10:03:00") | |||
| 192 | + | assert.False(t, run.GetTrusted(), "still the token's") | |||
| 193 | + | ||||
| 194 | + | // The owner moves the tip: trusted again. | |||
| 195 | + | c.push("main", map[string]string{"owner.txt": "o"}, "") | |||
| 196 | + | run, _ = c.fireAt(clk, runner, "2026-03-04 10:04:00") | |||
| 197 | + | assert.True(t, run.GetTrusted()) | |||
| 198 | + | tip := c.ref("refs/heads/main") | |||
| 199 | + | ||||
| 200 | + | // A push refused after the decision recorded it (here the check alone runs, | |||
| 201 | + | // then git never moves the ref): the record names a commit the tip does not | |||
| 202 | + | // have, and the schedule fails closed. | |||
| 203 | + | scoped, err := c.forge.Auth.Authenticate(c.ctx, tok) | |||
| 204 | + | require.NoError(t, err) | |||
| 205 | + | never := strings.Repeat("2", 40) | |||
| 206 | + | require.NoError(t, c.forge.Pushes.Check(c.ctx, scoped, c.name("proj"), | |||
| 207 | + | []domain.RefUpdate{{Ref: "refs/heads/main", Old: tip, New: never}}, ports.PushObjects{})) | |||
| 208 | + | left, err := c.forge.store.CI().GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |||
| 209 | + | require.NoError(t, err) | |||
| 210 | + | assert.Equal(t, never, left.Sha, "the refused push left its record behind") | |||
| 211 | + | run, _ = c.fireAt(clk, runner, "2026-03-04 10:05:00") | |||
| 212 | + | assert.False(t, run.GetTrusted(), "a record for a commit that never became the tip: fail closed") | |||
| 213 | + | } | |||
| 214 | + | ||||
| 215 | + | // GO-006: a merge that deletes its source branch drops the record of that | |||
| 216 | + | // branch, and a merge records the target before it moves. | |||
| 217 | + | func TestMergeRecordsTheTargetAndDropsTheSourceRecord(t *testing.T) { | |||
| 218 | + | c, _, _ := newScheduleSecretsHarness(t) | |||
| 219 | + | tips := c.forge.store.CI() | |||
| 220 | + | c.push("main", map[string]string{"a.txt": "a"}, "") | |||
| 221 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "feat") | |||
| 222 | + | c.push("feat", map[string]string{"f.txt": "f"}, "") | |||
| 223 | + | _, err := tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/feat") | |||
| 224 | + | require.NoError(t, err, "the pushed branch has a record") | |||
| 225 | + | n := c.openMR(c.root, "feat") | |||
| 226 | + | c.waitFor("the head ref", func() bool { return c.ref(domain.PullHeadRef(n)) != "" }) | |||
| 227 | + | ||||
| 228 | + | _, err = c.root.mr.MergeMergeRequest(c.ctx, connect.NewRequest(&pb.MergeMergeRequestRequest{ | |||
| 229 | + | Repo: "proj", Number: n, HeadId: c.ref(domain.PullHeadRef(n)), DeleteSourceBranch: true, | |||
| 230 | + | })) | |||
| 231 | + | require.NoError(t, err) | |||
| 232 | + | ||||
| 233 | + | _, err = tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/feat") | |||
| 234 | + | assert.ErrorIs(t, err, domain.ErrNotFound, "the deleted source branch has no record") | |||
| 235 | + | main, err := tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/main") | |||
| 236 | + | require.NoError(t, err) | |||
| 237 | + | assert.Equal(t, c.ref("refs/heads/main"), main.Sha, "the merge recorded the new tip") | |||
| 238 | + | assert.True(t, main.Trusted) | |||
| 239 | + | ||||
| 240 | + | // A pushed branch that is deleted by a push loses its record too. | |||
| 241 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "gone") | |||
| 242 | + | c.push("gone", map[string]string{"g.txt": "g"}, "") | |||
| 243 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), ":refs/heads/gone") | |||
| 244 | + | _, err = tips.GetRefTip(c.ctx, c.name("proj"), "refs/heads/gone") | |||
| 245 | + | assert.ErrorIs(t, err, domain.ErrNotFound) | |||
| 246 | + | } | |||
| 247 | + | ||||
| 248 | + | // GO-005: a dispatched run is trusted only when the caller is and the ref was | |||
| 249 | + | // moved to its commit by a trusted principal, for every caller. | |||
| 250 | + | func TestDispatchTrustFollowsTheRefsPusher(t *testing.T) { | |||
| 251 | + | c := newCIHarness(t, nil) | |||
| 252 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 253 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow, ".github/workflows/d.yml": jobTokenDispatcher}, "") | |||
| 254 | + | jobs := runner.jobsByKey(4) | |||
| 255 | + | in := map[string]string{"version": "1"} | |||
| 256 | + | ||||
| 257 | + | // An agent's branch. | |||
| 258 | + | secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), []string{"agent/"}, "proj") | |||
| 259 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "agent/x") | |||
| 260 | + | c.writeFile(c.wt, "x.txt", []byte("x")) | |||
| 261 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(secret, "/proj.git"), "agent/x") | |||
| 262 | + | c.waitFor("the agent's push run", func() bool { return len(c.runsOf("dispatcher")) >= 2 }) | |||
| 263 | + | ||||
| 264 | + | run, err := c.dispatch(c.root, "deploy.yml", "main", in) | |||
| 265 | + | require.NoError(t, err) | |||
| 266 | + | assert.True(t, run.GetTrusted(), "the owner dispatching what the owner pushed") | |||
| 267 | + | ||||
| 268 | + | run, err = c.dispatch(c.root, "deploy.yml", "agent/x", in) | |||
| 269 | + | require.NoError(t, err) | |||
| 270 | + | assert.False(t, run.GetTrusted(), "the owner dispatching an agent's branch") | |||
| 271 | + | ||||
| 272 | + | run, err = c.dispatch(c.api(jobs["with"].GetGithubToken()), "deploy.yml", "agent/x", in) | |||
| 273 | + | require.NoError(t, err) | |||
| 274 | + | assert.False(t, run.GetTrusted(), "a trusted job token dispatching an agent's branch") | |||
| 275 | + | run, err = c.dispatch(c.api(jobs["with"].GetGithubToken()), "deploy.yml", "main", in) | |||
| 276 | + | require.NoError(t, err) | |||
| 277 | + | assert.True(t, run.GetTrusted(), "a trusted job token dispatching the owner's branch") | |||
| 278 | + | ||||
| 279 | + | // What the jobs of those runs received: the untrusted ones got no secrets. | |||
| 280 | + | got := map[int64]*pb.Job{} | |||
| 281 | + | for range 4 { | |||
| 282 | + | j := runner.await() | |||
| 283 | + | got[j.GetRunId()] = j | |||
| 284 | + | } | |||
| 285 | + | for _, r := range c.runs() { | |||
| 286 | + | if r.GetEvent() != "workflow_dispatch" { | |||
| 287 | + | continue | |||
| 288 | + | } | |||
| 289 | + | if j := got[r.GetId()]; j != nil { | |||
| 290 | + | assert.Equal(t, r.GetTrusted(), j.GetTrusted(), "the job follows its run") | |||
| 291 | + | if !r.GetTrusted() { | |||
| 292 | + | assert.Empty(t, j.GetSecrets()) | |||
| 293 | + | } | |||
| 294 | + | } | |||
| 295 | + | } | |||
| 296 | + | } | |||
| 297 | + | ||||
| 298 | + | // GO-007: the trust of a dispatch is judged on the object and the commit it | |||
| 299 | + | // resolved, not on the tag as it is read afterwards. The tag is a lightweight | |||
| 300 | + | // one at commit A when the dispatch resolves it; before the trust is judged it | |||
| 301 | + | // is retargeted by the owner to an annotated tag on commit B, whose record is | |||
| 302 | + | // trusted: A runs, and must not borrow that record. | |||
| 303 | + | func TestDispatchOnATagRetargetedMeanwhileIsNotTrusted(t *testing.T) { | |||
| 304 | + | var retarget func() | |||
| 305 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |||
| 306 | + | cfg.DispatchResolved = func() { | |||
| 307 | + | if retarget != nil { | |||
| 308 | + | retarget() | |||
| 309 | + | } | |||
| 310 | + | } | |||
| 311 | + | }) | |||
| 312 | + | c.registerRunner("alpha", "ubuntu-latest") | |||
| 313 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow}, "") | |||
| 314 | + | c.mustGit(c.wt, "tag", "v1") | |||
| 315 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "v1") | |||
| 316 | + | commitA := c.ref("refs/heads/main") | |||
| 317 | + | in := map[string]string{"version": "1"} | |||
| 318 | + | ||||
| 319 | + | run, err := c.dispatch(c.root, "deploy.yml", "v1", in) | |||
| 320 | + | require.NoError(t, err) | |||
| 321 | + | require.True(t, run.GetTrusted(), "without a retarget the owner's own tag is trusted") | |||
| 322 | + | ||||
| 323 | + | retarget = func() { | |||
| 324 | + | retarget = nil // once | |||
| 325 | + | c.writeFile(c.wt, "b.txt", []byte("b")) | |||
| 326 | + | c.mustGit(c.wt, "tag", "-a", "-f", "-m", "annotated", "v1") | |||
| 327 | + | c.mustGit(c.wt, "push", "-q", "-f", c.urlAuth(adminToken, "/proj.git"), "v1") | |||
| 328 | + | } | |||
| 329 | + | run, err = c.dispatch(c.root, "deploy.yml", "v1", in) | |||
| 330 | + | require.NoError(t, err) | |||
| 331 | + | assert.Equal(t, commitA, run.GetSha(), "the dispatch runs the commit it resolved") | |||
| 332 | + | assert.False(t, run.GetTrusted(), "the tag moved to another object meanwhile: the record of the new one is not A's") | |||
| 333 | + | ||||
| 334 | + | // The tag really moved: dispatching it now is trusted again, on the new commit. | |||
| 335 | + | run, err = c.dispatch(c.root, "deploy.yml", "v1", in) | |||
| 336 | + | require.NoError(t, err) | |||
| 337 | + | assert.NotEqual(t, commitA, run.GetSha()) | |||
| 338 | + | assert.True(t, run.GetTrusted()) | |||
| 339 | + | } | |||
| 340 | + | ||||
| 341 | + | // A tag is judged by its own record, like a branch. | |||
| 342 | + | func TestDispatchOnATagFollowsItsPusher(t *testing.T) { | |||
| 343 | + | c := newCIHarness(t, nil) | |||
| 344 | + | c.registerRunner("alpha", "ubuntu-latest") | |||
| 345 | + | c.push("main", map[string]string{".github/workflows/deploy.yml": dispatchWorkflow}, "") | |||
| 346 | + | c.mustGit(c.wt, "tag", "v1") | |||
| 347 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "v1") | |||
| 348 | + | c.mustGit(c.wt, "tag", "-a", "-m", "annotated", "v2") | |||
| 349 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "v2") | |||
| 350 | + | in := map[string]string{"version": "1"} | |||
| 351 | + | ||||
| 352 | + | run, err := c.dispatch(c.root, "deploy.yml", "v1", in) | |||
| 353 | + | require.NoError(t, err) | |||
| 354 | + | assert.True(t, run.GetTrusted(), "a lightweight tag the owner pushed") | |||
| 355 | + | run, err = c.dispatch(c.root, "deploy.yml", "v2", in) | |||
| 356 | + | require.NoError(t, err) | |||
| 357 | + | assert.True(t, run.GetTrusted(), "an annotated tag the owner pushed (the record names the tag object)") | |||
| 358 | + | ||||
| 359 | + | tok := c.repoToken("scoped") | |||
| 360 | + | c.writeFile(c.wt, "t.txt", []byte("t")) | |||
| 361 | + | c.mustGit(c.wt, "tag", "v3") | |||
| 362 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(tok, "/proj.git"), "v3") | |||
| 363 | + | run, err = c.dispatch(c.root, "deploy.yml", "v3", in) | |||
| 364 | + | require.NoError(t, err) | |||
| 365 | + | assert.False(t, run.GetTrusted(), "a tag a scoped token pushed") | |||
| 366 | + | } | |||
A internal/app/ci_workflowrun_test.go +168 -0
| @@ -0,0 +1,168 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package app | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "fmt" | |
| 7 | + | "testing" | |
| 8 | + | "time" | |
| 9 | + | ||
| 10 | + | "github.com/stretchr/testify/assert" | |
| 11 | + | ||
| 12 | + | pb "bnema.dev/ermite-api/gen/ermite" | |
| 13 | + | "bnema.dev/ermite/internal/domain" | |
| 14 | + | "bnema.dev/ermite/internal/usecase" | |
| 15 | + | ) | |
| 16 | + | ||
| 17 | + | // CI-3 D5, AC5: workflow_run. | |
| 18 | + | ||
| 19 | + | const triggeringWorkflow = "name: ci\non: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n" | |
| 20 | + | ||
| 21 | + | func followWorkflow(name, on string) map[string]string { | |
| 22 | + | return map[string]string{".github/workflows/" + name + ".yml": fmt.Sprintf( | |
| 23 | + | "name: %s\non:\n workflow_run:\n%s\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n", name, on)} | |
| 24 | + | } | |
| 25 | + | ||
| 26 | + | func merge(maps ...map[string]string) map[string]string { | |
| 27 | + | out := map[string]string{} | |
| 28 | + | for _, m := range maps { | |
| 29 | + | for k, v := range m { | |
| 30 | + | out[k] = v | |
| 31 | + | } | |
| 32 | + | } | |
| 33 | + | return out | |
| 34 | + | } | |
| 35 | + | ||
| 36 | + | func (c *ciHarness) runsOf(workflowName string) []*pb.Run { | |
| 37 | + | var out []*pb.Run | |
| 38 | + | for _, r := range c.runs() { | |
| 39 | + | if r.GetWorkflowName() == workflowName { | |
| 40 | + | out = append(out, r) | |
| 41 | + | } | |
| 42 | + | } | |
| 43 | + | return out | |
| 44 | + | } | |
| 45 | + | ||
| 46 | + | func TestWorkflowRunRequestedCompletedFiltersAndPayload(t *testing.T) { | |
| 47 | + | c := newCIHarness(t, nil) | |
| 48 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 49 | + | c.push("main", merge( | |
| 50 | + | map[string]string{".github/workflows/ci.yml": triggeringWorkflow}, | |
| 51 | + | followWorkflow("after", " workflows: [ci]\n types: [completed]\n branches: [main]"), | |
| 52 | + | followWorkflow("started", " workflows: [ci]\n types: [requested]"), | |
| 53 | + | followWorkflow("release-only", " workflows: [ci]\n branches: [release]"), | |
| 54 | + | followWorkflow("other-name", " workflows: [nobody]"), | |
| 55 | + | ), "") | |
| 56 | + | c.waitFor("ci and the run it requested", func() bool { return len(c.runs()) == 2 }) | |
| 57 | + | assert.Len(t, c.runsOf("started"), 1, "requested fires when the run is created") | |
| 58 | + | assert.Empty(t, c.runsOf("after"), "completed has not happened yet") | |
| 59 | + | ||
| 60 | + | // Both jobs are given to the runner: finish them. | |
| 61 | + | for range 2 { | |
| 62 | + | j := runner.await() | |
| 63 | + | runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_FAILURE, nil, "ok") | |
| 64 | + | } | |
| 65 | + | c.waitFor("the completed run", func() bool { return len(c.runsOf("after")) == 1 }) | |
| 66 | + | assert.Empty(t, c.runsOf("release-only"), "the branches filter applies to the head branch of the triggering run") | |
| 67 | + | assert.Empty(t, c.runsOf("other-name")) | |
| 68 | + | assert.Len(t, c.runsOf("started"), 1, "requested does not fire on completion") | |
| 69 | + | ||
| 70 | + | after := c.runsOf("after")[0] | |
| 71 | + | assert.Equal(t, "workflow_run", after.GetEvent()) | |
| 72 | + | assert.Equal(t, "refs/heads/main", after.GetRef(), "it runs on the default branch") | |
| 73 | + | assert.True(t, after.GetTrusted()) | |
| 74 | + | main := c.ref("refs/heads/main") | |
| 75 | + | assert.Equal(t, main, after.GetSha()) | |
| 76 | + | ||
| 77 | + | job := runner.await() | |
| 78 | + | gh := c.ghContext(job) | |
| 79 | + | assert.Equal(t, "workflow_run", gh["event_name"]) | |
| 80 | + | ev := gh["event"].(map[string]any) | |
| 81 | + | assert.Equal(t, "completed", ev["action"]) | |
| 82 | + | wr := ev["workflow_run"].(map[string]any) | |
| 83 | + | trig := c.runsOf("ci")[0] | |
| 84 | + | assert.EqualValues(t, trig.GetId(), wr["id"]) | |
| 85 | + | assert.Equal(t, "ci", wr["name"]) | |
| 86 | + | assert.Equal(t, "main", wr["head_branch"]) | |
| 87 | + | assert.Equal(t, main, wr["head_sha"]) | |
| 88 | + | assert.Equal(t, "push", wr["event"]) | |
| 89 | + | assert.Equal(t, "completed", wr["status"]) | |
| 90 | + | assert.Equal(t, "failure", wr["conclusion"], "the conclusion of the triggering run") | |
| 91 | + | assert.EqualValues(t, trig.GetNumber(), wr["run_number"]) | |
| 92 | + | assert.EqualValues(t, 1, wr["run_attempt"]) | |
| 93 | + | assert.Equal(t, ".github/workflows/ci.yml", wr["path"]) | |
| 94 | + | assert.NotEmpty(t, wr["actor"].(map[string]any)["login"]) | |
| 95 | + | wf := ev["workflow"].(map[string]any) | |
| 96 | + | assert.Equal(t, "ci", wf["name"]) | |
| 97 | + | assert.Equal(t, ".github/workflows/ci.yml", wf["path"]) | |
| 98 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 99 | + | } | |
| 100 | + | ||
| 101 | + | func TestWorkflowRunInheritsTrustAndNeverRaisesIt(t *testing.T) { | |
| 102 | + | c := newCIHarness(t, nil) | |
| 103 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 104 | + | c.push("main", merge( | |
| 105 | + | map[string]string{".github/workflows/ci.yml": triggeringWorkflow}, | |
| 106 | + | followWorkflow("after", " workflows: [ci]\n types: [completed]"), | |
| 107 | + | ), "") | |
| 108 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 109 | + | c.waitFor("the owner's workflow_run", func() bool { return len(c.runsOf("after")) == 1 }) | |
| 110 | + | assert.True(t, c.runsOf("after")[0].GetTrusted()) | |
| 111 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 112 | + | ||
| 113 | + | // An agent pushes a branch: its push run is untrusted, and so is the | |
| 114 | + | // workflow_run it triggers, though that workflow is the owner's, on main. | |
| 115 | + | secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), []string{"agent/claude/"}, "proj") | |
| 116 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "agent/claude/x") | |
| 117 | + | c.writeFile(c.wt, "a.txt", []byte("a")) | |
| 118 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(secret, "/proj.git"), "agent/claude/x") | |
| 119 | + | c.waitFor("the agent's push run", func() bool { return len(c.runsOf("ci")) == 2 }) | |
| 120 | + | agentRun := c.runsOf("ci")[0] | |
| 121 | + | assert.False(t, agentRun.GetTrusted()) | |
| 122 | + | job := runner.await() | |
| 123 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 124 | + | c.waitFor("the agent's workflow_run", func() bool { return len(c.runsOf("after")) == 2 }) | |
| 125 | + | follow := c.runsOf("after")[0] | |
| 126 | + | assert.False(t, follow.GetTrusted(), "never raised") | |
| 127 | + | assert.Equal(t, agentRun.GetActor(), follow.GetActor(), "the actor of the triggering run") | |
| 128 | + | next := runner.await() | |
| 129 | + | assert.False(t, next.GetTrusted()) | |
| 130 | + | assert.Empty(t, next.GetSecrets()) | |
| 131 | + | assert.Equal(t, "agent/claude/x", c.ghContext(next)["event"].(map[string]any)["workflow_run"].(map[string]any)["head_branch"]) | |
| 132 | + | runner.runJob(next, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 133 | + | } | |
| 134 | + | ||
| 135 | + | func TestWorkflowRunChainStopsAtDepthThree(t *testing.T) { | |
| 136 | + | // handled tells when the workflows waiting for an event have been looked at: | |
| 137 | + | // a barrier instead of a wait for something that must not happen. | |
| 138 | + | handled := make(chan [2]any, 64) | |
| 139 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |
| 140 | + | cfg.RunEventHandled = func(id int64, action string) { handled <- [2]any{id, action} } | |
| 141 | + | }) | |
| 142 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |
| 143 | + | files := map[string]string{".github/workflows/a.yml": "name: a\non: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"} | |
| 144 | + | prev := "a" | |
| 145 | + | for _, next := range []string{"b", "c", "d", "e"} { | |
| 146 | + | files = merge(files, followWorkflow(next, " workflows: ["+prev+"]\n types: [completed]")) | |
| 147 | + | prev = next | |
| 148 | + | } | |
| 149 | + | c.push("main", files, "") | |
| 150 | + | for _, name := range []string{"a", "b", "c", "d"} { | |
| 151 | + | c.waitFor("the run of "+name, func() bool { return len(c.runsOf(name)) == 1 }) | |
| 152 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |
| 153 | + | } | |
| 154 | + | c.waitFor("d completes", func() bool { return c.runsOf("d")[0].GetStatus() == pb.CiStatus_CI_STATUS_COMPLETED }) | |
| 155 | + | // The next level would be e: three workflow_run levels (b, c, d) are the most. | |
| 156 | + | // Wait until the "completed" event of d was handled, then look: e did not start. | |
| 157 | + | dID := c.runsOf("d")[0].GetId() | |
| 158 | + | for done := false; !done; { | |
| 159 | + | select { | |
| 160 | + | case ev := <-handled: | |
| 161 | + | done = ev == [2]any{dID, "completed"} | |
| 162 | + | case <-time.After(15 * time.Second): | |
| 163 | + | t.Fatal("the completed event of d was never handled") | |
| 164 | + | } | |
| 165 | + | } | |
| 166 | + | assert.Empty(t, c.runsOf("e"), "a chain of more than three levels stops") | |
| 167 | + | assert.Len(t, c.runs(), 4) | |
| 168 | + | } | |
| @@ -0,0 +1,168 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package app | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "fmt" | |||
| 7 | + | "testing" | |||
| 8 | + | "time" | |||
| 9 | + | ||||
| 10 | + | "github.com/stretchr/testify/assert" | |||
| 11 | + | ||||
| 12 | + | pb "bnema.dev/ermite-api/gen/ermite" | |||
| 13 | + | "bnema.dev/ermite/internal/domain" | |||
| 14 | + | "bnema.dev/ermite/internal/usecase" | |||
| 15 | + | ) | |||
| 16 | + | ||||
| 17 | + | // CI-3 D5, AC5: workflow_run. | |||
| 18 | + | ||||
| 19 | + | const triggeringWorkflow = "name: ci\non: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n" | |||
| 20 | + | ||||
| 21 | + | func followWorkflow(name, on string) map[string]string { | |||
| 22 | + | return map[string]string{".github/workflows/" + name + ".yml": fmt.Sprintf( | |||
| 23 | + | "name: %s\non:\n workflow_run:\n%s\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n", name, on)} | |||
| 24 | + | } | |||
| 25 | + | ||||
| 26 | + | func merge(maps ...map[string]string) map[string]string { | |||
| 27 | + | out := map[string]string{} | |||
| 28 | + | for _, m := range maps { | |||
| 29 | + | for k, v := range m { | |||
| 30 | + | out[k] = v | |||
| 31 | + | } | |||
| 32 | + | } | |||
| 33 | + | return out | |||
| 34 | + | } | |||
| 35 | + | ||||
| 36 | + | func (c *ciHarness) runsOf(workflowName string) []*pb.Run { | |||
| 37 | + | var out []*pb.Run | |||
| 38 | + | for _, r := range c.runs() { | |||
| 39 | + | if r.GetWorkflowName() == workflowName { | |||
| 40 | + | out = append(out, r) | |||
| 41 | + | } | |||
| 42 | + | } | |||
| 43 | + | return out | |||
| 44 | + | } | |||
| 45 | + | ||||
| 46 | + | func TestWorkflowRunRequestedCompletedFiltersAndPayload(t *testing.T) { | |||
| 47 | + | c := newCIHarness(t, nil) | |||
| 48 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 49 | + | c.push("main", merge( | |||
| 50 | + | map[string]string{".github/workflows/ci.yml": triggeringWorkflow}, | |||
| 51 | + | followWorkflow("after", " workflows: [ci]\n types: [completed]\n branches: [main]"), | |||
| 52 | + | followWorkflow("started", " workflows: [ci]\n types: [requested]"), | |||
| 53 | + | followWorkflow("release-only", " workflows: [ci]\n branches: [release]"), | |||
| 54 | + | followWorkflow("other-name", " workflows: [nobody]"), | |||
| 55 | + | ), "") | |||
| 56 | + | c.waitFor("ci and the run it requested", func() bool { return len(c.runs()) == 2 }) | |||
| 57 | + | assert.Len(t, c.runsOf("started"), 1, "requested fires when the run is created") | |||
| 58 | + | assert.Empty(t, c.runsOf("after"), "completed has not happened yet") | |||
| 59 | + | ||||
| 60 | + | // Both jobs are given to the runner: finish them. | |||
| 61 | + | for range 2 { | |||
| 62 | + | j := runner.await() | |||
| 63 | + | runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_FAILURE, nil, "ok") | |||
| 64 | + | } | |||
| 65 | + | c.waitFor("the completed run", func() bool { return len(c.runsOf("after")) == 1 }) | |||
| 66 | + | assert.Empty(t, c.runsOf("release-only"), "the branches filter applies to the head branch of the triggering run") | |||
| 67 | + | assert.Empty(t, c.runsOf("other-name")) | |||
| 68 | + | assert.Len(t, c.runsOf("started"), 1, "requested does not fire on completion") | |||
| 69 | + | ||||
| 70 | + | after := c.runsOf("after")[0] | |||
| 71 | + | assert.Equal(t, "workflow_run", after.GetEvent()) | |||
| 72 | + | assert.Equal(t, "refs/heads/main", after.GetRef(), "it runs on the default branch") | |||
| 73 | + | assert.True(t, after.GetTrusted()) | |||
| 74 | + | main := c.ref("refs/heads/main") | |||
| 75 | + | assert.Equal(t, main, after.GetSha()) | |||
| 76 | + | ||||
| 77 | + | job := runner.await() | |||
| 78 | + | gh := c.ghContext(job) | |||
| 79 | + | assert.Equal(t, "workflow_run", gh["event_name"]) | |||
| 80 | + | ev := gh["event"].(map[string]any) | |||
| 81 | + | assert.Equal(t, "completed", ev["action"]) | |||
| 82 | + | wr := ev["workflow_run"].(map[string]any) | |||
| 83 | + | trig := c.runsOf("ci")[0] | |||
| 84 | + | assert.EqualValues(t, trig.GetId(), wr["id"]) | |||
| 85 | + | assert.Equal(t, "ci", wr["name"]) | |||
| 86 | + | assert.Equal(t, "main", wr["head_branch"]) | |||
| 87 | + | assert.Equal(t, main, wr["head_sha"]) | |||
| 88 | + | assert.Equal(t, "push", wr["event"]) | |||
| 89 | + | assert.Equal(t, "completed", wr["status"]) | |||
| 90 | + | assert.Equal(t, "failure", wr["conclusion"], "the conclusion of the triggering run") | |||
| 91 | + | assert.EqualValues(t, trig.GetNumber(), wr["run_number"]) | |||
| 92 | + | assert.EqualValues(t, 1, wr["run_attempt"]) | |||
| 93 | + | assert.Equal(t, ".github/workflows/ci.yml", wr["path"]) | |||
| 94 | + | assert.NotEmpty(t, wr["actor"].(map[string]any)["login"]) | |||
| 95 | + | wf := ev["workflow"].(map[string]any) | |||
| 96 | + | assert.Equal(t, "ci", wf["name"]) | |||
| 97 | + | assert.Equal(t, ".github/workflows/ci.yml", wf["path"]) | |||
| 98 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 99 | + | } | |||
| 100 | + | ||||
| 101 | + | func TestWorkflowRunInheritsTrustAndNeverRaisesIt(t *testing.T) { | |||
| 102 | + | c := newCIHarness(t, nil) | |||
| 103 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 104 | + | c.push("main", merge( | |||
| 105 | + | map[string]string{".github/workflows/ci.yml": triggeringWorkflow}, | |||
| 106 | + | followWorkflow("after", " workflows: [ci]\n types: [completed]"), | |||
| 107 | + | ), "") | |||
| 108 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 109 | + | c.waitFor("the owner's workflow_run", func() bool { return len(c.runsOf("after")) == 1 }) | |||
| 110 | + | assert.True(t, c.runsOf("after")[0].GetTrusted()) | |||
| 111 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 112 | + | ||||
| 113 | + | // An agent pushes a branch: its push run is untrusted, and so is the | |||
| 114 | + | // workflow_run it triggers, though that workflow is the owner's, on main. | |||
| 115 | + | secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), []string{"agent/claude/"}, "proj") | |||
| 116 | + | c.mustGit(c.wt, "checkout", "-q", "-b", "agent/claude/x") | |||
| 117 | + | c.writeFile(c.wt, "a.txt", []byte("a")) | |||
| 118 | + | c.mustGit(c.wt, "push", "-q", c.urlAuth(secret, "/proj.git"), "agent/claude/x") | |||
| 119 | + | c.waitFor("the agent's push run", func() bool { return len(c.runsOf("ci")) == 2 }) | |||
| 120 | + | agentRun := c.runsOf("ci")[0] | |||
| 121 | + | assert.False(t, agentRun.GetTrusted()) | |||
| 122 | + | job := runner.await() | |||
| 123 | + | runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 124 | + | c.waitFor("the agent's workflow_run", func() bool { return len(c.runsOf("after")) == 2 }) | |||
| 125 | + | follow := c.runsOf("after")[0] | |||
| 126 | + | assert.False(t, follow.GetTrusted(), "never raised") | |||
| 127 | + | assert.Equal(t, agentRun.GetActor(), follow.GetActor(), "the actor of the triggering run") | |||
| 128 | + | next := runner.await() | |||
| 129 | + | assert.False(t, next.GetTrusted()) | |||
| 130 | + | assert.Empty(t, next.GetSecrets()) | |||
| 131 | + | assert.Equal(t, "agent/claude/x", c.ghContext(next)["event"].(map[string]any)["workflow_run"].(map[string]any)["head_branch"]) | |||
| 132 | + | runner.runJob(next, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 133 | + | } | |||
| 134 | + | ||||
| 135 | + | func TestWorkflowRunChainStopsAtDepthThree(t *testing.T) { | |||
| 136 | + | // handled tells when the workflows waiting for an event have been looked at: | |||
| 137 | + | // a barrier instead of a wait for something that must not happen. | |||
| 138 | + | handled := make(chan [2]any, 64) | |||
| 139 | + | c := newCIHarness(t, func(cfg *usecase.CIConfig) { | |||
| 140 | + | cfg.RunEventHandled = func(id int64, action string) { handled <- [2]any{id, action} } | |||
| 141 | + | }) | |||
| 142 | + | runner := c.registerRunner("alpha", "ubuntu-latest") | |||
| 143 | + | files := map[string]string{".github/workflows/a.yml": "name: a\non: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"} | |||
| 144 | + | prev := "a" | |||
| 145 | + | for _, next := range []string{"b", "c", "d", "e"} { | |||
| 146 | + | files = merge(files, followWorkflow(next, " workflows: ["+prev+"]\n types: [completed]")) | |||
| 147 | + | prev = next | |||
| 148 | + | } | |||
| 149 | + | c.push("main", files, "") | |||
| 150 | + | for _, name := range []string{"a", "b", "c", "d"} { | |||
| 151 | + | c.waitFor("the run of "+name, func() bool { return len(c.runsOf(name)) == 1 }) | |||
| 152 | + | runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok") | |||
| 153 | + | } | |||
| 154 | + | c.waitFor("d completes", func() bool { return c.runsOf("d")[0].GetStatus() == pb.CiStatus_CI_STATUS_COMPLETED }) | |||
| 155 | + | // The next level would be e: three workflow_run levels (b, c, d) are the most. | |||
| 156 | + | // Wait until the "completed" event of d was handled, then look: e did not start. | |||
| 157 | + | dID := c.runsOf("d")[0].GetId() | |||
| 158 | + | for done := false; !done; { | |||
| 159 | + | select { | |||
| 160 | + | case ev := <-handled: | |||
| 161 | + | done = ev == [2]any{dID, "completed"} | |||
| 162 | + | case <-time.After(15 * time.Second): | |||
| 163 | + | t.Fatal("the completed event of d was never handled") | |||
| 164 | + | } | |||
| 165 | + | } | |||
| 166 | + | assert.Empty(t, c.runsOf("e"), "a chain of more than three levels stops") | |||
| 167 | + | assert.Len(t, c.runs(), 4) | |||
| 168 | + | } | |||
M internal/app/cli_e2e_test.go +1 -1
| @@ -326,7 +326,7 @@ func TestAC6CLIEndToEnd(t *testing.T) { | |||
| 326 | 326 | ver := decode[versionOut](t, res.stdout) | |
| 327 | 327 | assert.Equal(t, c.url, ver.Server.URL) | |
| 328 | 328 | assert.NotEmpty(t, ver.Server.Version) | |
| 329 | - | assert.ElementsMatch(t, []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.secrets"}, ver.Server.Features) | |
| 329 | + | assert.ElementsMatch(t, []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.dispatch", "ci.secrets"}, ver.Server.Features) | |
| 330 | 330 | assert.Equal(t, []string{"passkeys"}, ver.Server.MissingFeatures) | |
| 331 | 331 | stdout, stderr, code = c.bin(t, secret, "version") | |
| 332 | 332 | require.Equal(t, 0, code, stderr) | |
| @@ -326,7 +326,7 @@ func TestAC6CLIEndToEnd(t *testing.T) { | |||||
| 326 | ver := decode[versionOut](t, res.stdout) | 326 | ver := decode[versionOut](t, res.stdout) | ||
| 327 | assert.Equal(t, c.url, ver.Server.URL) | 327 | assert.Equal(t, c.url, ver.Server.URL) | ||
| 328 | assert.NotEmpty(t, ver.Server.Version) | 328 | assert.NotEmpty(t, ver.Server.Version) | ||
| 329 | - | assert.ElementsMatch(t, []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.secrets"}, ver.Server.Features) | 329 | + | assert.ElementsMatch(t, []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "git.list_paths", "token.branch_prefixes", "ci.runs", "ci.runs.filters", "ci.runners", "ci.actions", "ci.annotations", "ci.summaries", "ci.concurrency", "ci.dispatch", "ci.secrets"}, ver.Server.Features) |
| 330 | assert.Equal(t, []string{"passkeys"}, ver.Server.MissingFeatures) | 330 | assert.Equal(t, []string{"passkeys"}, ver.Server.MissingFeatures) | ||
| 331 | stdout, stderr, code = c.bin(t, secret, "version") | 331 | stdout, stderr, code = c.bin(t, secret, "version") | ||
| 332 | require.Equal(t, 0, code, stderr) | 332 | require.Equal(t, 0, code, stderr) | ||
M internal/app/cli_run.go +3 -2
| @@ -253,12 +253,12 @@ func newRunListCmd(env *cliEnv) *cobra.Command { | |||
| 253 | 253 | rows := make([][]string, 0, len(runs)) | |
| 254 | 254 | for _, r := range runs { | |
| 255 | 255 | rows = append(rows, []string{ | |
| 256 | - | strconv.FormatInt(r.GetId(), 10), runResult(r.GetStatus(), r.GetConclusion()), r.GetWorkflowName(), | |
| 256 | + | strconv.FormatInt(r.GetId(), 10), runResult(r.GetStatus(), r.GetConclusion()), r.GetWorkflowName(), r.GetEvent(), | |
| 257 | 257 | refText(r.GetRef()), shortSha(r.GetSha()), | |
| 258 | 258 | r.GetActor(), shortTime(r.GetCreatedAt(), "-"), r.GetTitle(), | |
| 259 | 259 | }) | |
| 260 | 260 | } | |
| 261 | - | return table(cmd.OutOrStdout(), []string{"ID", "RESULT", "WORKFLOW", "REF", "COMMIT", "ACTOR", "CREATED", "TITLE"}, rows) | |
| 261 | + | return table(cmd.OutOrStdout(), []string{"ID", "RESULT", "WORKFLOW", "EVENT", "REF", "COMMIT", "ACTOR", "CREATED", "TITLE"}, rows) | |
| 262 | 262 | } | |
| 263 | 263 | return cmd | |
| 264 | 264 | } | |
| @@ -357,6 +357,7 @@ func printRun(w io.Writer, r *ermite.GetRunResponse) error { | |||
| 357 | 357 | pairs := [][2]string{ | |
| 358 | 358 | {"run", fmt.Sprintf("%d (#%d of %s)", run.GetId(), run.GetNumber(), run.GetWorkflowName())}, | |
| 359 | 359 | {"result", runResult(run.GetStatus(), run.GetConclusion())}, | |
| 360 | + | {"event", run.GetEvent()}, | |
| 360 | 361 | {"ref", run.GetRef()}, | |
| 361 | 362 | {"commit", run.GetSha()}, | |
| 362 | 363 | {"title", run.GetTitle()}, | |
| @@ -253,12 +253,12 @@ func newRunListCmd(env *cliEnv) *cobra.Command { | |||||
| 253 | rows := make([][]string, 0, len(runs)) | 253 | rows := make([][]string, 0, len(runs)) | ||
| 254 | for _, r := range runs { | 254 | for _, r := range runs { | ||
| 255 | rows = append(rows, []string{ | 255 | rows = append(rows, []string{ | ||
| 256 | - | strconv.FormatInt(r.GetId(), 10), runResult(r.GetStatus(), r.GetConclusion()), r.GetWorkflowName(), | 256 | + | strconv.FormatInt(r.GetId(), 10), runResult(r.GetStatus(), r.GetConclusion()), r.GetWorkflowName(), r.GetEvent(), |
| 257 | refText(r.GetRef()), shortSha(r.GetSha()), | 257 | refText(r.GetRef()), shortSha(r.GetSha()), | ||
| 258 | r.GetActor(), shortTime(r.GetCreatedAt(), "-"), r.GetTitle(), | 258 | r.GetActor(), shortTime(r.GetCreatedAt(), "-"), r.GetTitle(), | ||
| 259 | }) | 259 | }) | ||
| 260 | } | 260 | } | ||
| 261 | - | return table(cmd.OutOrStdout(), []string{"ID", "RESULT", "WORKFLOW", "REF", "COMMIT", "ACTOR", "CREATED", "TITLE"}, rows) | 261 | + | return table(cmd.OutOrStdout(), []string{"ID", "RESULT", "WORKFLOW", "EVENT", "REF", "COMMIT", "ACTOR", "CREATED", "TITLE"}, rows) |
| 262 | } | 262 | } | ||
| 263 | return cmd | 263 | return cmd | ||
| 264 | } | 264 | } | ||
| @@ -357,6 +357,7 @@ func printRun(w io.Writer, r *ermite.GetRunResponse) error { | |||||
| 357 | pairs := [][2]string{ | 357 | pairs := [][2]string{ | ||
| 358 | {"run", fmt.Sprintf("%d (#%d of %s)", run.GetId(), run.GetNumber(), run.GetWorkflowName())}, | 358 | {"run", fmt.Sprintf("%d (#%d of %s)", run.GetId(), run.GetNumber(), run.GetWorkflowName())}, | ||
| 359 | {"result", runResult(run.GetStatus(), run.GetConclusion())}, | 359 | {"result", runResult(run.GetStatus(), run.GetConclusion())}, | ||
| 360 | + | {"event", run.GetEvent()}, | |||
| 360 | {"ref", run.GetRef()}, | 361 | {"ref", run.GetRef()}, | ||
| 361 | {"commit", run.GetSha()}, | 362 | {"commit", run.GetSha()}, | ||
| 362 | {"title", run.GetTitle()}, | 363 | {"title", run.GetTitle()}, | ||
M internal/app/cli_version.go +1 -0
| @@ -119,5 +119,6 @@ var featureUse = map[string]string{ | |||
| 119 | 119 | client.FeatureCIRuns: "`ermite run` needs: the commands will fail on this server", | |
| 120 | 120 | client.FeatureCIRunners: "`ermite runner` needs: the commands will fail on this server", | |
| 121 | 121 | client.FeatureCISecrets: "`ermite secret`, `variable` and `environment` need: the commands will fail on this server", | |
| 122 | + | client.FeatureCIDispatch: "`ermite workflow run` needs: the command will fail on this server", | |
| 122 | 123 | client.FeaturePasskeys: "`ermite web passkey` needs: the commands will fail on this server (passkeys need an https public URL with a DNS host name)", | |
| 123 | 124 | } | |
| @@ -119,5 +119,6 @@ var featureUse = map[string]string{ | |||||
| 119 | client.FeatureCIRuns: "`ermite run` needs: the commands will fail on this server", | 119 | client.FeatureCIRuns: "`ermite run` needs: the commands will fail on this server", | ||
| 120 | client.FeatureCIRunners: "`ermite runner` needs: the commands will fail on this server", | 120 | client.FeatureCIRunners: "`ermite runner` needs: the commands will fail on this server", | ||
| 121 | client.FeatureCISecrets: "`ermite secret`, `variable` and `environment` need: the commands will fail on this server", | 121 | client.FeatureCISecrets: "`ermite secret`, `variable` and `environment` need: the commands will fail on this server", | ||
| 122 | + | client.FeatureCIDispatch: "`ermite workflow run` needs: the command will fail on this server", | |||
| 122 | client.FeaturePasskeys: "`ermite web passkey` needs: the commands will fail on this server (passkeys need an https public URL with a DNS host name)", | 123 | client.FeaturePasskeys: "`ermite web passkey` needs: the commands will fail on this server (passkeys need an https public URL with a DNS host name)", | ||
| 123 | } | 124 | } | ||
M internal/app/cli_version_test.go +5 -5
| @@ -17,7 +17,7 @@ import ( | |||
| 17 | 17 | ) | |
| 18 | 18 | ||
| 19 | 19 | // allFeatures are the features this client uses. | |
| 20 | - | var allFeatures = []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "token.branch_prefixes", "ci.runs", "ci.runners", "ci.secrets", "passkeys"} | |
| 20 | + | var allFeatures = []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "token.branch_prefixes", "ci.runs", "ci.runners", "ci.secrets", "ci.dispatch", "passkeys"} | |
| 21 | 21 | ||
| 22 | 22 | // Every feature the client uses needs a sentence for the version warning. | |
| 23 | 23 | func TestEveryUsedFeatureExplainsWhatBreaks(t *testing.T) { | |
| @@ -45,15 +45,15 @@ func TestVersionWithServer(t *testing.T) { | |||
| 45 | 45 | f := versionFake(allFeatures...) | |
| 46 | 46 | url := f.serve(t) | |
| 47 | 47 | res := runCLI(t, envFor(url), "", "version").ok(t) | |
| 48 | - | assert.Equal(t, "client ermite v0.0.1\nserver v0.4.0 at "+url+"\nfeatures repo.update, events.resume, merge_requests, merge_requests.delete_branch, signing_key, session, token.branch_prefixes, ci.runs, ci.runners, ci.secrets, passkeys\n", res.stdout) | |
| 48 | + | assert.Equal(t, "client ermite v0.0.1\nserver v0.4.0 at "+url+"\nfeatures repo.update, events.resume, merge_requests, merge_requests.delete_branch, signing_key, session, token.branch_prefixes, ci.runs, ci.runners, ci.secrets, ci.dispatch, passkeys\n", res.stdout) | |
| 49 | 49 | assert.Empty(t, res.stderr, "no warning when every feature is there") | |
| 50 | 50 | ||
| 51 | 51 | res = runCLI(t, envFor(url), "", "version", "--json").ok(t) | |
| 52 | - | assert.JSONEq(t, `{"client":{"version":"v0.0.1"},"server":{"url":"`+url+`","version":"v0.4.0","features":["repo.update","events.resume","merge_requests","merge_requests.delete_branch","signing_key","session","token.branch_prefixes","ci.runs","ci.runners","ci.secrets","passkeys"],"missing_features":[]}}`, res.stdout) | |
| 52 | + | assert.JSONEq(t, `{"client":{"version":"v0.0.1"},"server":{"url":"`+url+`","version":"v0.4.0","features":["repo.update","events.resume","merge_requests","merge_requests.delete_branch","signing_key","session","token.branch_prefixes","ci.runs","ci.runners","ci.secrets","ci.dispatch","passkeys"],"missing_features":[]}}`, res.stdout) | |
| 53 | 53 | } | |
| 54 | 54 | ||
| 55 | 55 | func TestVersionWarnsAboutMissingFeatures(t *testing.T) { | |
| 56 | - | f := versionFake("repo.update", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "token.branch_prefixes", "ci.runs", "ci.runners", "ci.secrets", "passkeys") | |
| 56 | + | f := versionFake("repo.update", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "token.branch_prefixes", "ci.runs", "ci.runners", "ci.secrets", "ci.dispatch", "passkeys") | |
| 57 | 57 | url := f.serve(t) | |
| 58 | 58 | res := runCLI(t, envFor(url), "", "version").ok(t) | |
| 59 | 59 | assert.Contains(t, res.stdout, "v0.4.0") | |
| @@ -61,7 +61,7 @@ func TestVersionWarnsAboutMissingFeatures(t *testing.T) { | |||
| 61 | 61 | assert.NotContains(t, res.stderr, "repo.update") | |
| 62 | 62 | ||
| 63 | 63 | res = runCLI(t, envFor(url), "", "version", "--json").ok(t) | |
| 64 | - | assert.JSONEq(t, `{"client":{"version":"v0.0.1"},"server":{"url":"`+url+`","version":"v0.4.0","features":["repo.update","merge_requests","merge_requests.delete_branch","signing_key","session","token.branch_prefixes","ci.runs","ci.runners","ci.secrets","passkeys"],"missing_features":["events.resume"]}}`, res.stdout) | |
| 64 | + | assert.JSONEq(t, `{"client":{"version":"v0.0.1"},"server":{"url":"`+url+`","version":"v0.4.0","features":["repo.update","merge_requests","merge_requests.delete_branch","signing_key","session","token.branch_prefixes","ci.runs","ci.runners","ci.secrets","ci.dispatch","passkeys"],"missing_features":["events.resume"]}}`, res.stdout) | |
| 65 | 65 | assert.Contains(t, res.stderr, "events.resume", "the warning also goes to stderr with --json") | |
| 66 | 66 | ||
| 67 | 67 | f = versionFake() | |
| @@ -17,7 +17,7 @@ import ( | |||||
| 17 | ) | 17 | ) | ||
| 18 | 18 | ||||
| 19 | // allFeatures are the features this client uses. | 19 | // allFeatures are the features this client uses. | ||
| 20 | - | var allFeatures = []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "token.branch_prefixes", "ci.runs", "ci.runners", "ci.secrets", "passkeys"} | 20 | + | var allFeatures = []string{"repo.update", "events.resume", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "token.branch_prefixes", "ci.runs", "ci.runners", "ci.secrets", "ci.dispatch", "passkeys"} |
| 21 | 21 | ||||
| 22 | // Every feature the client uses needs a sentence for the version warning. | 22 | // Every feature the client uses needs a sentence for the version warning. | ||
| 23 | func TestEveryUsedFeatureExplainsWhatBreaks(t *testing.T) { | 23 | func TestEveryUsedFeatureExplainsWhatBreaks(t *testing.T) { | ||
| @@ -45,15 +45,15 @@ func TestVersionWithServer(t *testing.T) { | |||||
| 45 | f := versionFake(allFeatures...) | 45 | f := versionFake(allFeatures...) | ||
| 46 | url := f.serve(t) | 46 | url := f.serve(t) | ||
| 47 | res := runCLI(t, envFor(url), "", "version").ok(t) | 47 | res := runCLI(t, envFor(url), "", "version").ok(t) | ||
| 48 | - | assert.Equal(t, "client ermite v0.0.1\nserver v0.4.0 at "+url+"\nfeatures repo.update, events.resume, merge_requests, merge_requests.delete_branch, signing_key, session, token.branch_prefixes, ci.runs, ci.runners, ci.secrets, passkeys\n", res.stdout) | 48 | + | assert.Equal(t, "client ermite v0.0.1\nserver v0.4.0 at "+url+"\nfeatures repo.update, events.resume, merge_requests, merge_requests.delete_branch, signing_key, session, token.branch_prefixes, ci.runs, ci.runners, ci.secrets, ci.dispatch, passkeys\n", res.stdout) |
| 49 | assert.Empty(t, res.stderr, "no warning when every feature is there") | 49 | assert.Empty(t, res.stderr, "no warning when every feature is there") | ||
| 50 | 50 | ||||
| 51 | res = runCLI(t, envFor(url), "", "version", "--json").ok(t) | 51 | res = runCLI(t, envFor(url), "", "version", "--json").ok(t) | ||
| 52 | - | assert.JSONEq(t, `{"client":{"version":"v0.0.1"},"server":{"url":"`+url+`","version":"v0.4.0","features":["repo.update","events.resume","merge_requests","merge_requests.delete_branch","signing_key","session","token.branch_prefixes","ci.runs","ci.runners","ci.secrets","passkeys"],"missing_features":[]}}`, res.stdout) | 52 | + | assert.JSONEq(t, `{"client":{"version":"v0.0.1"},"server":{"url":"`+url+`","version":"v0.4.0","features":["repo.update","events.resume","merge_requests","merge_requests.delete_branch","signing_key","session","token.branch_prefixes","ci.runs","ci.runners","ci.secrets","ci.dispatch","passkeys"],"missing_features":[]}}`, res.stdout) |
| 53 | } | 53 | } | ||
| 54 | 54 | ||||
| 55 | func TestVersionWarnsAboutMissingFeatures(t *testing.T) { | 55 | func TestVersionWarnsAboutMissingFeatures(t *testing.T) { | ||
| 56 | - | f := versionFake("repo.update", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "token.branch_prefixes", "ci.runs", "ci.runners", "ci.secrets", "passkeys") | 56 | + | f := versionFake("repo.update", "merge_requests", "merge_requests.delete_branch", "signing_key", "session", "token.branch_prefixes", "ci.runs", "ci.runners", "ci.secrets", "ci.dispatch", "passkeys") |
| 57 | url := f.serve(t) | 57 | url := f.serve(t) | ||
| 58 | res := runCLI(t, envFor(url), "", "version").ok(t) | 58 | res := runCLI(t, envFor(url), "", "version").ok(t) | ||
| 59 | assert.Contains(t, res.stdout, "v0.4.0") | 59 | assert.Contains(t, res.stdout, "v0.4.0") | ||
| @@ -61,7 +61,7 @@ func TestVersionWarnsAboutMissingFeatures(t *testing.T) { | |||||
| 61 | assert.NotContains(t, res.stderr, "repo.update") | 61 | assert.NotContains(t, res.stderr, "repo.update") | ||
| 62 | 62 | ||||
| 63 | res = runCLI(t, envFor(url), "", "version", "--json").ok(t) | 63 | res = runCLI(t, envFor(url), "", "version", "--json").ok(t) | ||
| 64 | - | assert.JSONEq(t, `{"client":{"version":"v0.0.1"},"server":{"url":"`+url+`","version":"v0.4.0","features":["repo.update","merge_requests","merge_requests.delete_branch","signing_key","session","token.branch_prefixes","ci.runs","ci.runners","ci.secrets","passkeys"],"missing_features":["events.resume"]}}`, res.stdout) | 64 | + | assert.JSONEq(t, `{"client":{"version":"v0.0.1"},"server":{"url":"`+url+`","version":"v0.4.0","features":["repo.update","merge_requests","merge_requests.delete_branch","signing_key","session","token.branch_prefixes","ci.runs","ci.runners","ci.secrets","ci.dispatch","passkeys"],"missing_features":["events.resume"]}}`, res.stdout) |
| 65 | assert.Contains(t, res.stderr, "events.resume", "the warning also goes to stderr with --json") | 65 | assert.Contains(t, res.stderr, "events.resume", "the warning also goes to stderr with --json") | ||
| 66 | 66 | ||||
| 67 | f = versionFake() | 67 | f = versionFake() | ||
A internal/app/cli_workflow.go +85 -0
| @@ -0,0 +1,85 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package app | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "fmt" | |
| 7 | + | "strings" | |
| 8 | + | ||
| 9 | + | "connectrpc.com/connect" | |
| 10 | + | "github.com/spf13/cobra" | |
| 11 | + | ||
| 12 | + | "bnema.dev/ermite-api/gen/ermite" | |
| 13 | + | ) | |
| 14 | + | ||
| 15 | + | // `ermite workflow run`: start a workflow that has a workflow_dispatch trigger, | |
| 16 | + | // like `gh workflow run`. | |
| 17 | + | ||
| 18 | + | func newWorkflowCmd(env *cliEnv) *cobra.Command { | |
| 19 | + | wf := &cobra.Command{ | |
| 20 | + | Use: "workflow", | |
| 21 | + | Short: "Start workflows by hand", | |
| 22 | + | } | |
| 23 | + | wf.AddCommand(newWorkflowRunCmd(env)) | |
| 24 | + | return wf | |
| 25 | + | } | |
| 26 | + | ||
| 27 | + | // parseInputFlags reads the -f key=value flags. The first = separates the name | |
| 28 | + | // from the value, so a value may hold =; a repeated name takes the last value. | |
| 29 | + | func parseInputFlags(flags []string) (map[string]string, error) { | |
| 30 | + | if len(flags) == 0 { | |
| 31 | + | return nil, nil | |
| 32 | + | } | |
| 33 | + | out := make(map[string]string, len(flags)) | |
| 34 | + | for _, f := range flags { | |
| 35 | + | k, v, ok := strings.Cut(f, "=") | |
| 36 | + | if !ok || k == "" { | |
| 37 | + | return nil, fmt.Errorf("invalid input %q: use -f name=value", f) | |
| 38 | + | } | |
| 39 | + | out[k] = v | |
| 40 | + | } | |
| 41 | + | return out, nil | |
| 42 | + | } | |
| 43 | + | ||
| 44 | + | func newWorkflowRunCmd(env *cliEnv) *cobra.Command { | |
| 45 | + | var repo, ref string | |
| 46 | + | var fields []string | |
| 47 | + | cmd := &cobra.Command{ | |
| 48 | + | Use: "run <workflow file> --repo <repo>", | |
| 49 | + | Short: "Run a workflow that has a workflow_dispatch trigger", | |
| 50 | + | Long: "Start a run of a workflow file on a branch or a tag (the default branch without --ref). The workflow is read at\n" + | |
| 51 | + | "that ref and must have a workflow_dispatch trigger; -f name=value gives the inputs it declares (checked by type,\n" + | |
| 52 | + | "required inputs and choices). The workflow file is its path or its file name. Needs the ci:write scope." + jsonRunHelp, | |
| 53 | + | Example: " ermite workflow run release.yml --repo app -f version=1.2.3\n" + | |
| 54 | + | " ermite workflow run .github/workflows/ci.yml --repo app --ref feature/x", | |
| 55 | + | Args: cobra.ExactArgs(1), | |
| 56 | + | ValidArgsFunction: cobra.NoFileCompletions, | |
| 57 | + | } | |
| 58 | + | addRepoFlag(cmd, env, &repo) | |
| 59 | + | cmd.Flags().StringVar(&ref, "ref", "", "the branch or tag to run on (default: the default branch)") | |
| 60 | + | cmd.Flags().StringArrayVarP(&fields, "field", "f", nil, "an input of the workflow as name=value (repeatable)") | |
| 61 | + | asJSON := addJSONFlag(cmd) | |
| 62 | + | cmd.RunE = func(cmd *cobra.Command, args []string) error { | |
| 63 | + | inputs, err := parseInputFlags(fields) | |
| 64 | + | if err != nil { | |
| 65 | + | return err | |
| 66 | + | } | |
| 67 | + | c, err := env.client(cmd) | |
| 68 | + | if err != nil { | |
| 69 | + | return err | |
| 70 | + | } | |
| 71 | + | res, err := c.Runs().DispatchWorkflow(env.ctx(cmd), connect.NewRequest(&ermite.DispatchWorkflowRequest{ | |
| 72 | + | Repo: repo, WorkflowPath: args[0], Ref: ref, Inputs: inputs, | |
| 73 | + | })) | |
| 74 | + | if err != nil { | |
| 75 | + | return err | |
| 76 | + | } | |
| 77 | + | run := res.Msg.GetRun() | |
| 78 | + | if *asJSON { | |
| 79 | + | return writeJSON(cmd.OutOrStdout(), toJSONRun(run)) | |
| 80 | + | } | |
| 81 | + | noticef(cmd, "run %d of %s started on %s", run.GetId(), run.GetWorkflowName(), refText(run.GetRef())) | |
| 82 | + | return nil | |
| 83 | + | } | |
| 84 | + | return cmd | |
| 85 | + | } | |
| @@ -0,0 +1,85 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package app | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "fmt" | |||
| 7 | + | "strings" | |||
| 8 | + | ||||
| 9 | + | "connectrpc.com/connect" | |||
| 10 | + | "github.com/spf13/cobra" | |||
| 11 | + | ||||
| 12 | + | "bnema.dev/ermite-api/gen/ermite" | |||
| 13 | + | ) | |||
| 14 | + | ||||
| 15 | + | // `ermite workflow run`: start a workflow that has a workflow_dispatch trigger, | |||
| 16 | + | // like `gh workflow run`. | |||
| 17 | + | ||||
| 18 | + | func newWorkflowCmd(env *cliEnv) *cobra.Command { | |||
| 19 | + | wf := &cobra.Command{ | |||
| 20 | + | Use: "workflow", | |||
| 21 | + | Short: "Start workflows by hand", | |||
| 22 | + | } | |||
| 23 | + | wf.AddCommand(newWorkflowRunCmd(env)) | |||
| 24 | + | return wf | |||
| 25 | + | } | |||
| 26 | + | ||||
| 27 | + | // parseInputFlags reads the -f key=value flags. The first = separates the name | |||
| 28 | + | // from the value, so a value may hold =; a repeated name takes the last value. | |||
| 29 | + | func parseInputFlags(flags []string) (map[string]string, error) { | |||
| 30 | + | if len(flags) == 0 { | |||
| 31 | + | return nil, nil | |||
| 32 | + | } | |||
| 33 | + | out := make(map[string]string, len(flags)) | |||
| 34 | + | for _, f := range flags { | |||
| 35 | + | k, v, ok := strings.Cut(f, "=") | |||
| 36 | + | if !ok || k == "" { | |||
| 37 | + | return nil, fmt.Errorf("invalid input %q: use -f name=value", f) | |||
| 38 | + | } | |||
| 39 | + | out[k] = v | |||
| 40 | + | } | |||
| 41 | + | return out, nil | |||
| 42 | + | } | |||
| 43 | + | ||||
| 44 | + | func newWorkflowRunCmd(env *cliEnv) *cobra.Command { | |||
| 45 | + | var repo, ref string | |||
| 46 | + | var fields []string | |||
| 47 | + | cmd := &cobra.Command{ | |||
| 48 | + | Use: "run <workflow file> --repo <repo>", | |||
| 49 | + | Short: "Run a workflow that has a workflow_dispatch trigger", | |||
| 50 | + | Long: "Start a run of a workflow file on a branch or a tag (the default branch without --ref). The workflow is read at\n" + | |||
| 51 | + | "that ref and must have a workflow_dispatch trigger; -f name=value gives the inputs it declares (checked by type,\n" + | |||
| 52 | + | "required inputs and choices). The workflow file is its path or its file name. Needs the ci:write scope." + jsonRunHelp, | |||
| 53 | + | Example: " ermite workflow run release.yml --repo app -f version=1.2.3\n" + | |||
| 54 | + | " ermite workflow run .github/workflows/ci.yml --repo app --ref feature/x", | |||
| 55 | + | Args: cobra.ExactArgs(1), | |||
| 56 | + | ValidArgsFunction: cobra.NoFileCompletions, | |||
| 57 | + | } | |||
| 58 | + | addRepoFlag(cmd, env, &repo) | |||
| 59 | + | cmd.Flags().StringVar(&ref, "ref", "", "the branch or tag to run on (default: the default branch)") | |||
| 60 | + | cmd.Flags().StringArrayVarP(&fields, "field", "f", nil, "an input of the workflow as name=value (repeatable)") | |||
| 61 | + | asJSON := addJSONFlag(cmd) | |||
| 62 | + | cmd.RunE = func(cmd *cobra.Command, args []string) error { | |||
| 63 | + | inputs, err := parseInputFlags(fields) | |||
| 64 | + | if err != nil { | |||
| 65 | + | return err | |||
| 66 | + | } | |||
| 67 | + | c, err := env.client(cmd) | |||
| 68 | + | if err != nil { | |||
| 69 | + | return err | |||
| 70 | + | } | |||
| 71 | + | res, err := c.Runs().DispatchWorkflow(env.ctx(cmd), connect.NewRequest(&ermite.DispatchWorkflowRequest{ | |||
| 72 | + | Repo: repo, WorkflowPath: args[0], Ref: ref, Inputs: inputs, | |||
| 73 | + | })) | |||
| 74 | + | if err != nil { | |||
| 75 | + | return err | |||
| 76 | + | } | |||
| 77 | + | run := res.Msg.GetRun() | |||
| 78 | + | if *asJSON { | |||
| 79 | + | return writeJSON(cmd.OutOrStdout(), toJSONRun(run)) | |||
| 80 | + | } | |||
| 81 | + | noticef(cmd, "run %d of %s started on %s", run.GetId(), run.GetWorkflowName(), refText(run.GetRef())) | |||
| 82 | + | return nil | |||
| 83 | + | } | |||
| 84 | + | return cmd | |||
| 85 | + | } | |||
M internal/app/forge.go +3 -2
| @@ -256,6 +256,7 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e | |||
| 256 | 256 | Proxy: gitremote.ProxyEnv(os.Getenv), | |
| 257 | 257 | })), | |
| 258 | 258 | Tokens: store.Tokens(), | |
| 259 | + | RefTips: store.CI(), | |
| 259 | 260 | Settings: f.Settings, | |
| 260 | 261 | Workflows: opts.workflows(githubraw.New(githubraw.Config{})), | |
| 261 | 262 | } | |
| @@ -264,7 +265,7 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e | |||
| 264 | 265 | opts.ci(&ciCfg) | |
| 265 | 266 | } | |
| 266 | 267 | f.CI = usecase.NewCI(ctx, ciCfg) | |
| 267 | - | f.Pushes.WithWorkflowTrigger(f.CI).WithWorkflowGuard(git) | |
| 268 | + | f.Pushes.WithWorkflowTrigger(f.CI).WithWorkflowGuard(git).WithRefTips(store.CI()) | |
| 268 | 269 | f.Repos.WithCIRunCleaner(f.CI) | |
| 269 | 270 | f.CI.Start() | |
| 270 | 271 | f.EventLog = usecase.NewEvents(ctx, f.Repos, f.Bus, clock, usecase.EventsConfig{ | |
| @@ -284,7 +285,7 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e | |||
| 284 | 285 | f.MergeRequests = usecase.NewMergeRequests(usecase.MergeRequestsConfig{ | |
| 285 | 286 | Meta: store, Store: store.MergeRequests(), Repos: git, Git: git, Signer: signer, | |
| 286 | 287 | Committer: committer, Bus: f.Bus, Clock: clock, Maint: f.maint, | |
| 287 | - | CI: f.CI, Checks: f.CI, PullRequests: f.CI, | |
| 288 | + | CI: f.CI, Checks: f.CI, PullRequests: f.CI, RefTips: store.CI(), | |
| 288 | 289 | }) | |
| 289 | 290 | f.CI.SetPullGate(f.MergeRequests) | |
| 290 | 291 | log.Info().Str("fingerprint", signer.PublicKey().Fingerprint).Msg("commit signing key ready") | |
| @@ -256,6 +256,7 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e | |||||
| 256 | Proxy: gitremote.ProxyEnv(os.Getenv), | 256 | Proxy: gitremote.ProxyEnv(os.Getenv), | ||
| 257 | })), | 257 | })), | ||
| 258 | Tokens: store.Tokens(), | 258 | Tokens: store.Tokens(), | ||
| 259 | + | RefTips: store.CI(), | |||
| 259 | Settings: f.Settings, | 260 | Settings: f.Settings, | ||
| 260 | Workflows: opts.workflows(githubraw.New(githubraw.Config{})), | 261 | Workflows: opts.workflows(githubraw.New(githubraw.Config{})), | ||
| 261 | } | 262 | } | ||
| @@ -264,7 +265,7 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e | |||||
| 264 | opts.ci(&ciCfg) | 265 | opts.ci(&ciCfg) | ||
| 265 | } | 266 | } | ||
| 266 | f.CI = usecase.NewCI(ctx, ciCfg) | 267 | f.CI = usecase.NewCI(ctx, ciCfg) | ||
| 267 | - | f.Pushes.WithWorkflowTrigger(f.CI).WithWorkflowGuard(git) | 268 | + | f.Pushes.WithWorkflowTrigger(f.CI).WithWorkflowGuard(git).WithRefTips(store.CI()) |
| 268 | f.Repos.WithCIRunCleaner(f.CI) | 269 | f.Repos.WithCIRunCleaner(f.CI) | ||
| 269 | f.CI.Start() | 270 | f.CI.Start() | ||
| 270 | f.EventLog = usecase.NewEvents(ctx, f.Repos, f.Bus, clock, usecase.EventsConfig{ | 271 | f.EventLog = usecase.NewEvents(ctx, f.Repos, f.Bus, clock, usecase.EventsConfig{ | ||
| @@ -284,7 +285,7 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e | |||||
| 284 | f.MergeRequests = usecase.NewMergeRequests(usecase.MergeRequestsConfig{ | 285 | f.MergeRequests = usecase.NewMergeRequests(usecase.MergeRequestsConfig{ | ||
| 285 | Meta: store, Store: store.MergeRequests(), Repos: git, Git: git, Signer: signer, | 286 | Meta: store, Store: store.MergeRequests(), Repos: git, Git: git, Signer: signer, | ||
| 286 | Committer: committer, Bus: f.Bus, Clock: clock, Maint: f.maint, | 287 | Committer: committer, Bus: f.Bus, Clock: clock, Maint: f.maint, | ||
| 287 | - | CI: f.CI, Checks: f.CI, PullRequests: f.CI, | 288 | + | CI: f.CI, Checks: f.CI, PullRequests: f.CI, RefTips: store.CI(), |
| 288 | }) | 289 | }) | ||
| 289 | f.CI.SetPullGate(f.MergeRequests) | 290 | f.CI.SetPullGate(f.MergeRequests) | ||
| 290 | log.Info().Str("fingerprint", signer.PublicKey().Fingerprint).Msg("commit signing key ready") | 291 | log.Info().Str("fingerprint", signer.PublicKey().Fingerprint).Msg("commit signing key ready") | ||
M internal/app/root.go +1 -1
| @@ -32,7 +32,7 @@ func newRootCmdWith(env *cliEnv) *cobra.Command { | |||
| 32 | 32 | newServeCmd(), newHookCmd(), | |
| 33 | 33 | newVersionCmd(env), newRepoCmd(env), newTokenCmd(env), newWatchCmd(env), | |
| 34 | 34 | newMRCmd(env), newSigningKeyCmd(env), newWebCmd(env), newConfigCmd(env), | |
| 35 | - | newRunCmd(env), newRunnerCmd(env), newSecretCmd(env), newVariableCmd(env), newEnvironmentCmd(env), | |
| 35 | + | newRunCmd(env), newWorkflowCmd(env), newRunnerCmd(env), newSecretCmd(env), newVariableCmd(env), newEnvironmentCmd(env), | |
| 36 | 36 | ) | |
| 37 | 37 | return root | |
| 38 | 38 | } | |
| @@ -32,7 +32,7 @@ func newRootCmdWith(env *cliEnv) *cobra.Command { | |||||
| 32 | newServeCmd(), newHookCmd(), | 32 | newServeCmd(), newHookCmd(), | ||
| 33 | newVersionCmd(env), newRepoCmd(env), newTokenCmd(env), newWatchCmd(env), | 33 | newVersionCmd(env), newRepoCmd(env), newTokenCmd(env), newWatchCmd(env), | ||
| 34 | newMRCmd(env), newSigningKeyCmd(env), newWebCmd(env), newConfigCmd(env), | 34 | newMRCmd(env), newSigningKeyCmd(env), newWebCmd(env), newConfigCmd(env), | ||
| 35 | - | newRunCmd(env), newRunnerCmd(env), newSecretCmd(env), newVariableCmd(env), newEnvironmentCmd(env), | 35 | + | newRunCmd(env), newWorkflowCmd(env), newRunnerCmd(env), newSecretCmd(env), newVariableCmd(env), newEnvironmentCmd(env), |
| 36 | ) | 36 | ) | ||
| 37 | return root | 37 | return root | ||
| 38 | } | 38 | } | ||
A internal/domain/ci_cron.go +134 -0
| @@ -0,0 +1,134 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package domain | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "fmt" | |
| 7 | + | "strconv" | |
| 8 | + | "strings" | |
| 9 | + | "time" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | // Cron is a parsed "cron:" of a schedule trigger: five fields (minute, hour, day | |
| 13 | + | // of month, month, day of week), evaluated in UTC (CI-3 D5; the "timezone" key | |
| 14 | + | // is ignored). The syntax is that of GitHub's schedule: "*", numbers, lists | |
| 15 | + | // "a,b", ranges "a-b", steps "*/n", "a/n" and "a-b/n", the names jan-dec and | |
| 16 | + | // sun-sat in any case; the day of week is 0-6 (Sunday is 0). | |
| 17 | + | // | |
| 18 | + | // As in Vixie cron, when both the day of month and the day of week are | |
| 19 | + | // restricted (neither starts with "*") a day matches if either does; otherwise | |
| 20 | + | // both must. | |
| 21 | + | type Cron struct { | |
| 22 | + | minute, hour, dom, month, dow uint64 // bit sets | |
| 23 | + | domStar, dowStar bool | |
| 24 | + | } | |
| 25 | + | ||
| 26 | + | type cronField struct { | |
| 27 | + | min, max int | |
| 28 | + | names map[string]int | |
| 29 | + | } | |
| 30 | + | ||
| 31 | + | var ( | |
| 32 | + | cronMinuteField = cronField{0, 59, nil} | |
| 33 | + | cronHourField = cronField{0, 23, nil} | |
| 34 | + | cronDomField = cronField{1, 31, nil} | |
| 35 | + | cronMonthField = cronField{1, 12, map[string]int{"jan": 1, "feb": 2, "mar": 3, "apr": 4, "may": 5, "jun": 6, "jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12}} | |
| 36 | + | cronDowField = cronField{0, 6, map[string]int{"sun": 0, "mon": 1, "tue": 2, "wed": 3, "thu": 4, "fri": 5, "sat": 6}} | |
| 37 | + | ) | |
| 38 | + | ||
| 39 | + | // ParseCron parses the five fields of a cron expression separated by spaces. | |
| 40 | + | func ParseCron(s string) (Cron, error) { | |
| 41 | + | parts := strings.Fields(s) | |
| 42 | + | if len(parts) != 5 { | |
| 43 | + | return Cron{}, fmt.Errorf("%w: cron %q must have 5 fields: minute hour day-of-month month day-of-week", ErrInvalid, s) | |
| 44 | + | } | |
| 45 | + | var c Cron | |
| 46 | + | var err error | |
| 47 | + | for i, f := range []struct { | |
| 48 | + | spec cronField | |
| 49 | + | dst *uint64 | |
| 50 | + | }{ | |
| 51 | + | {cronMinuteField, &c.minute}, {cronHourField, &c.hour}, {cronDomField, &c.dom}, {cronMonthField, &c.month}, {cronDowField, &c.dow}, | |
| 52 | + | } { | |
| 53 | + | if *f.dst, err = parseCronField(parts[i], f.spec); err != nil { | |
| 54 | + | return Cron{}, fmt.Errorf("%w: cron %q: %v", ErrInvalid, s, err) | |
| 55 | + | } | |
| 56 | + | } | |
| 57 | + | c.domStar, c.dowStar = strings.HasPrefix(parts[2], "*"), strings.HasPrefix(parts[4], "*") | |
| 58 | + | return c, nil | |
| 59 | + | } | |
| 60 | + | ||
| 61 | + | func parseCronField(s string, f cronField) (uint64, error) { | |
| 62 | + | var set uint64 | |
| 63 | + | for _, item := range strings.Split(s, ",") { | |
| 64 | + | if item == "" { | |
| 65 | + | return 0, fmt.Errorf("empty item in %q", s) | |
| 66 | + | } | |
| 67 | + | rangePart, stepPart, hasStep := strings.Cut(item, "/") | |
| 68 | + | step := 1 | |
| 69 | + | if hasStep { | |
| 70 | + | n, err := strconv.Atoi(stepPart) | |
| 71 | + | if err != nil || n < 1 || strings.Trim(stepPart, "0123456789") != "" { | |
| 72 | + | return 0, fmt.Errorf("invalid step %q", stepPart) | |
| 73 | + | } | |
| 74 | + | // A step larger than the field is accepted, as cron does, and keeps | |
| 75 | + | // only the first value: it is clamped to the size of the field, which | |
| 76 | + | // has the same meaning and cannot overflow the loop below. | |
| 77 | + | step = min(n, f.max-f.min+1) | |
| 78 | + | } | |
| 79 | + | lo, hi := f.min, f.max | |
| 80 | + | if rangePart != "*" { | |
| 81 | + | a, b, isRange := strings.Cut(rangePart, "-") | |
| 82 | + | var err error | |
| 83 | + | if lo, err = cronValue(a, f); err != nil { | |
| 84 | + | return 0, err | |
| 85 | + | } | |
| 86 | + | switch { | |
| 87 | + | case isRange: | |
| 88 | + | if hi, err = cronValue(b, f); err != nil { | |
| 89 | + | return 0, err | |
| 90 | + | } | |
| 91 | + | if hi < lo { | |
| 92 | + | return 0, fmt.Errorf("range %q goes backwards", rangePart) | |
| 93 | + | } | |
| 94 | + | case hasStep: | |
| 95 | + | // "a/n" is "a-max/n". | |
| 96 | + | default: | |
| 97 | + | hi = lo | |
| 98 | + | } | |
| 99 | + | } | |
| 100 | + | for v := lo; v <= hi; v += step { | |
| 101 | + | set |= 1 << uint(v) | |
| 102 | + | if step > hi-v { | |
| 103 | + | break // the next value is past the range (and v+step could overflow) | |
| 104 | + | } | |
| 105 | + | } | |
| 106 | + | } | |
| 107 | + | return set, nil | |
| 108 | + | } | |
| 109 | + | ||
| 110 | + | func cronValue(s string, f cronField) (int, error) { | |
| 111 | + | if v, ok := f.names[strings.ToLower(s)]; ok { | |
| 112 | + | return v, nil | |
| 113 | + | } | |
| 114 | + | n, err := strconv.Atoi(s) | |
| 115 | + | if err != nil || strings.Trim(s, "0123456789") != "" || n < f.min || n > f.max { | |
| 116 | + | return 0, fmt.Errorf("invalid value %q (%d-%d)", s, f.min, f.max) | |
| 117 | + | } | |
| 118 | + | return n, nil | |
| 119 | + | } | |
| 120 | + | ||
| 121 | + | // Matches reports whether the minute of t (in UTC) is one the expression fires | |
| 122 | + | // at. | |
| 123 | + | func (c Cron) Matches(t time.Time) bool { | |
| 124 | + | t = t.UTC() | |
| 125 | + | if c.minute&(1<<uint(t.Minute())) == 0 || c.hour&(1<<uint(t.Hour())) == 0 || c.month&(1<<uint(t.Month())) == 0 { | |
| 126 | + | return false | |
| 127 | + | } | |
| 128 | + | domOK := c.dom&(1<<uint(t.Day())) != 0 | |
| 129 | + | dowOK := c.dow&(1<<uint(t.Weekday())) != 0 | |
| 130 | + | if !c.domStar && !c.dowStar { | |
| 131 | + | return domOK || dowOK | |
| 132 | + | } | |
| 133 | + | return domOK && dowOK | |
| 134 | + | } | |
| @@ -0,0 +1,134 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package domain | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "fmt" | |||
| 7 | + | "strconv" | |||
| 8 | + | "strings" | |||
| 9 | + | "time" | |||
| 10 | + | ) | |||
| 11 | + | ||||
| 12 | + | // Cron is a parsed "cron:" of a schedule trigger: five fields (minute, hour, day | |||
| 13 | + | // of month, month, day of week), evaluated in UTC (CI-3 D5; the "timezone" key | |||
| 14 | + | // is ignored). The syntax is that of GitHub's schedule: "*", numbers, lists | |||
| 15 | + | // "a,b", ranges "a-b", steps "*/n", "a/n" and "a-b/n", the names jan-dec and | |||
| 16 | + | // sun-sat in any case; the day of week is 0-6 (Sunday is 0). | |||
| 17 | + | // | |||
| 18 | + | // As in Vixie cron, when both the day of month and the day of week are | |||
| 19 | + | // restricted (neither starts with "*") a day matches if either does; otherwise | |||
| 20 | + | // both must. | |||
| 21 | + | type Cron struct { | |||
| 22 | + | minute, hour, dom, month, dow uint64 // bit sets | |||
| 23 | + | domStar, dowStar bool | |||
| 24 | + | } | |||
| 25 | + | ||||
| 26 | + | type cronField struct { | |||
| 27 | + | min, max int | |||
| 28 | + | names map[string]int | |||
| 29 | + | } | |||
| 30 | + | ||||
| 31 | + | var ( | |||
| 32 | + | cronMinuteField = cronField{0, 59, nil} | |||
| 33 | + | cronHourField = cronField{0, 23, nil} | |||
| 34 | + | cronDomField = cronField{1, 31, nil} | |||
| 35 | + | cronMonthField = cronField{1, 12, map[string]int{"jan": 1, "feb": 2, "mar": 3, "apr": 4, "may": 5, "jun": 6, "jul": 7, "aug": 8, "sep": 9, "oct": 10, "nov": 11, "dec": 12}} | |||
| 36 | + | cronDowField = cronField{0, 6, map[string]int{"sun": 0, "mon": 1, "tue": 2, "wed": 3, "thu": 4, "fri": 5, "sat": 6}} | |||
| 37 | + | ) | |||
| 38 | + | ||||
| 39 | + | // ParseCron parses the five fields of a cron expression separated by spaces. | |||
| 40 | + | func ParseCron(s string) (Cron, error) { | |||
| 41 | + | parts := strings.Fields(s) | |||
| 42 | + | if len(parts) != 5 { | |||
| 43 | + | return Cron{}, fmt.Errorf("%w: cron %q must have 5 fields: minute hour day-of-month month day-of-week", ErrInvalid, s) | |||
| 44 | + | } | |||
| 45 | + | var c Cron | |||
| 46 | + | var err error | |||
| 47 | + | for i, f := range []struct { | |||
| 48 | + | spec cronField | |||
| 49 | + | dst *uint64 | |||
| 50 | + | }{ | |||
| 51 | + | {cronMinuteField, &c.minute}, {cronHourField, &c.hour}, {cronDomField, &c.dom}, {cronMonthField, &c.month}, {cronDowField, &c.dow}, | |||
| 52 | + | } { | |||
| 53 | + | if *f.dst, err = parseCronField(parts[i], f.spec); err != nil { | |||
| 54 | + | return Cron{}, fmt.Errorf("%w: cron %q: %v", ErrInvalid, s, err) | |||
| 55 | + | } | |||
| 56 | + | } | |||
| 57 | + | c.domStar, c.dowStar = strings.HasPrefix(parts[2], "*"), strings.HasPrefix(parts[4], "*") | |||
| 58 | + | return c, nil | |||
| 59 | + | } | |||
| 60 | + | ||||
| 61 | + | func parseCronField(s string, f cronField) (uint64, error) { | |||
| 62 | + | var set uint64 | |||
| 63 | + | for _, item := range strings.Split(s, ",") { | |||
| 64 | + | if item == "" { | |||
| 65 | + | return 0, fmt.Errorf("empty item in %q", s) | |||
| 66 | + | } | |||
| 67 | + | rangePart, stepPart, hasStep := strings.Cut(item, "/") | |||
| 68 | + | step := 1 | |||
| 69 | + | if hasStep { | |||
| 70 | + | n, err := strconv.Atoi(stepPart) | |||
| 71 | + | if err != nil || n < 1 || strings.Trim(stepPart, "0123456789") != "" { | |||
| 72 | + | return 0, fmt.Errorf("invalid step %q", stepPart) | |||
| 73 | + | } | |||
| 74 | + | // A step larger than the field is accepted, as cron does, and keeps | |||
| 75 | + | // only the first value: it is clamped to the size of the field, which | |||
| 76 | + | // has the same meaning and cannot overflow the loop below. | |||
| 77 | + | step = min(n, f.max-f.min+1) | |||
| 78 | + | } | |||
| 79 | + | lo, hi := f.min, f.max | |||
| 80 | + | if rangePart != "*" { | |||
| 81 | + | a, b, isRange := strings.Cut(rangePart, "-") | |||
| 82 | + | var err error | |||
| 83 | + | if lo, err = cronValue(a, f); err != nil { | |||
| 84 | + | return 0, err | |||
| 85 | + | } | |||
| 86 | + | switch { | |||
| 87 | + | case isRange: | |||
| 88 | + | if hi, err = cronValue(b, f); err != nil { | |||
| 89 | + | return 0, err | |||
| 90 | + | } | |||
| 91 | + | if hi < lo { | |||
| 92 | + | return 0, fmt.Errorf("range %q goes backwards", rangePart) | |||
| 93 | + | } | |||
| 94 | + | case hasStep: | |||
| 95 | + | // "a/n" is "a-max/n". | |||
| 96 | + | default: | |||
| 97 | + | hi = lo | |||
| 98 | + | } | |||
| 99 | + | } | |||
| 100 | + | for v := lo; v <= hi; v += step { | |||
| 101 | + | set |= 1 << uint(v) | |||
| 102 | + | if step > hi-v { | |||
| 103 | + | break // the next value is past the range (and v+step could overflow) | |||
| 104 | + | } | |||
| 105 | + | } | |||
| 106 | + | } | |||
| 107 | + | return set, nil | |||
| 108 | + | } | |||
| 109 | + | ||||
| 110 | + | func cronValue(s string, f cronField) (int, error) { | |||
| 111 | + | if v, ok := f.names[strings.ToLower(s)]; ok { | |||
| 112 | + | return v, nil | |||
| 113 | + | } | |||
| 114 | + | n, err := strconv.Atoi(s) | |||
| 115 | + | if err != nil || strings.Trim(s, "0123456789") != "" || n < f.min || n > f.max { | |||
| 116 | + | return 0, fmt.Errorf("invalid value %q (%d-%d)", s, f.min, f.max) | |||
| 117 | + | } | |||
| 118 | + | return n, nil | |||
| 119 | + | } | |||
| 120 | + | ||||
| 121 | + | // Matches reports whether the minute of t (in UTC) is one the expression fires | |||
| 122 | + | // at. | |||
| 123 | + | func (c Cron) Matches(t time.Time) bool { | |||
| 124 | + | t = t.UTC() | |||
| 125 | + | if c.minute&(1<<uint(t.Minute())) == 0 || c.hour&(1<<uint(t.Hour())) == 0 || c.month&(1<<uint(t.Month())) == 0 { | |||
| 126 | + | return false | |||
| 127 | + | } | |||
| 128 | + | domOK := c.dom&(1<<uint(t.Day())) != 0 | |||
| 129 | + | dowOK := c.dow&(1<<uint(t.Weekday())) != 0 | |||
| 130 | + | if !c.domStar && !c.dowStar { | |||
| 131 | + | return domOK || dowOK | |||
| 132 | + | } | |||
| 133 | + | return domOK && dowOK | |||
| 134 | + | } | |||
A internal/domain/ci_cron_test.go +107 -0
| @@ -0,0 +1,107 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package domain_test | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "testing" | |
| 7 | + | "time" | |
| 8 | + | ||
| 9 | + | "github.com/stretchr/testify/assert" | |
| 10 | + | "github.com/stretchr/testify/require" | |
| 11 | + | ||
| 12 | + | "bnema.dev/ermite/internal/domain" | |
| 13 | + | ) | |
| 14 | + | ||
| 15 | + | func at(t *testing.T, s string) time.Time { | |
| 16 | + | t.Helper() | |
| 17 | + | v, err := time.Parse("2006-01-02 15:04", s) | |
| 18 | + | require.NoError(t, err) | |
| 19 | + | return v | |
| 20 | + | } | |
| 21 | + | ||
| 22 | + | func TestCronMatches(t *testing.T) { | |
| 23 | + | tests := []struct { | |
| 24 | + | spec string | |
| 25 | + | yes []string | |
| 26 | + | no []string | |
| 27 | + | extra string | |
| 28 | + | }{ | |
| 29 | + | {"* * * * *", []string{"2026-03-04 05:06", "2026-12-31 23:59"}, nil, ""}, | |
| 30 | + | {"*/15 * * * *", []string{"2026-03-04 05:00", "2026-03-04 05:45"}, []string{"2026-03-04 05:46", "2026-03-04 05:14"}, ""}, | |
| 31 | + | {"30 4 * * *", []string{"2026-03-04 04:30"}, []string{"2026-03-04 04:31", "2026-03-04 05:30"}, ""}, | |
| 32 | + | {"0 0 1 1 *", []string{"2027-01-01 00:00"}, []string{"2027-01-02 00:00", "2027-02-01 00:00"}, ""}, | |
| 33 | + | {"5,10-12 */6 * * *", []string{"2026-03-04 06:05", "2026-03-04 12:11"}, []string{"2026-03-04 06:06", "2026-03-04 07:05"}, ""}, | |
| 34 | + | {"10/20 * * * *", []string{"2026-03-04 00:10", "2026-03-04 00:30", "2026-03-04 00:50"}, []string{"2026-03-04 00:00", "2026-03-04 00:20"}, "a/n runs from a to the maximum"}, | |
| 35 | + | {"0 12 * JAN-mar mon-fri", []string{"2026-02-17 12:00"}, []string{"2026-02-21 12:00", "2026-04-01 12:00"}, "names in any case"}, | |
| 36 | + | {"0 0 * * 0", []string{"2026-03-01 00:00"}, []string{"2026-03-02 00:00"}, "0 is Sunday"}, | |
| 37 | + | // Both day fields restricted: either matches (2026-03-13 is a Friday). | |
| 38 | + | {"0 0 13 * fri", []string{"2026-03-13 00:00", "2026-03-20 00:00", "2026-04-13 00:00"}, []string{"2026-03-14 00:00"}, "day of month OR day of week"}, | |
| 39 | + | // One of them a star: both must. | |
| 40 | + | {"0 0 13 * *", []string{"2026-03-13 00:00"}, []string{"2026-03-20 00:00"}, ""}, | |
| 41 | + | {"0 0 * * fri", []string{"2026-03-13 00:00"}, []string{"2026-03-12 00:00"}, ""}, | |
| 42 | + | } | |
| 43 | + | for _, tt := range tests { | |
| 44 | + | c, err := domain.ParseCron(tt.spec) | |
| 45 | + | require.NoError(t, err, tt.spec) | |
| 46 | + | for _, y := range tt.yes { | |
| 47 | + | assert.True(t, c.Matches(at(t, y)), "%s at %s %s", tt.spec, y, tt.extra) | |
| 48 | + | } | |
| 49 | + | for _, n := range tt.no { | |
| 50 | + | assert.False(t, c.Matches(at(t, n)), "%s at %s %s", tt.spec, n, tt.extra) | |
| 51 | + | } | |
| 52 | + | } | |
| 53 | + | // Only the UTC minute counts, whatever the zone of the time given. | |
| 54 | + | c, err := domain.ParseCron("0 10 * * *") | |
| 55 | + | require.NoError(t, err) | |
| 56 | + | paris := time.FixedZone("CEST", 2*3600) | |
| 57 | + | assert.True(t, c.Matches(time.Date(2026, 6, 1, 12, 0, 0, 0, paris))) | |
| 58 | + | assert.False(t, c.Matches(time.Date(2026, 6, 1, 10, 0, 0, 0, paris))) | |
| 59 | + | } | |
| 60 | + | ||
| 61 | + | func TestCronLeapDay(t *testing.T) { | |
| 62 | + | c, err := domain.ParseCron("0 0 29 2 *") | |
| 63 | + | require.NoError(t, err) | |
| 64 | + | assert.True(t, c.Matches(at(t, "2028-02-29 00:00")), "2028 is a leap year") | |
| 65 | + | assert.False(t, c.Matches(at(t, "2028-02-28 00:00"))) | |
| 66 | + | assert.False(t, c.Matches(at(t, "2028-03-01 00:00"))) | |
| 67 | + | for d := at(t, "2027-02-01 00:00"); d.Month() == time.February; d = d.AddDate(0, 0, 1) { | |
| 68 | + | assert.False(t, c.Matches(d), "no 29 February in 2027: %s", d) | |
| 69 | + | } | |
| 70 | + | any31, err := domain.ParseCron("0 0 31 * *") | |
| 71 | + | require.NoError(t, err) | |
| 72 | + | assert.True(t, any31.Matches(at(t, "2026-03-31 00:00"))) | |
| 73 | + | assert.False(t, any31.Matches(at(t, "2026-04-30 00:00")), "no 31st in April") | |
| 74 | + | } | |
| 75 | + | ||
| 76 | + | // A step larger than the field keeps its first value only, and cannot overflow. | |
| 77 | + | func TestCronHugeSteps(t *testing.T) { | |
| 78 | + | for _, spec := range []string{"59/9223372036854775807 * * * *", "*/9223372036854775807 * * * *", "0-30/100 * * * *"} { | |
| 79 | + | c, err := domain.ParseCron(spec) | |
| 80 | + | require.NoError(t, err, spec) | |
| 81 | + | var got []int | |
| 82 | + | for m := range 60 { | |
| 83 | + | if c.Matches(at(t, "2026-03-04 05:00").Add(time.Duration(m) * time.Minute)) { | |
| 84 | + | got = append(got, m) | |
| 85 | + | } | |
| 86 | + | } | |
| 87 | + | switch spec[:2] { | |
| 88 | + | case "59": | |
| 89 | + | assert.Equal(t, []int{59}, got, spec) | |
| 90 | + | default: | |
| 91 | + | assert.Equal(t, []int{0}, got, spec) | |
| 92 | + | } | |
| 93 | + | } | |
| 94 | + | _, err := domain.ParseCron("*/99999999999999999999 * * * *") | |
| 95 | + | require.Error(t, err, "beyond an int64") | |
| 96 | + | } | |
| 97 | + | ||
| 98 | + | func TestParseCronRefuses(t *testing.T) { | |
| 99 | + | for _, bad := range []string{ | |
| 100 | + | "", "* * * *", "* * * * * *", "60 * * * *", "* 24 * * *", "* * 0 * *", "* * 32 * *", "* * * 13 *", "* * * * 7", | |
| 101 | + | "*/0 * * * *", "*/x * * * *", "5-3 * * * *", "1,,2 * * * *", "-1 * * * *", "1- * * * *", "foo * * * *", "* * * jun-foo *", "+5 * * * *", "0x5 * * * *", | |
| 102 | + | } { | |
| 103 | + | _, err := domain.ParseCron(bad) | |
| 104 | + | require.Error(t, err, bad) | |
| 105 | + | assert.ErrorIs(t, err, domain.ErrInvalid, bad) | |
| 106 | + | } | |
| 107 | + | } | |
| @@ -0,0 +1,107 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package domain_test | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "testing" | |||
| 7 | + | "time" | |||
| 8 | + | ||||
| 9 | + | "github.com/stretchr/testify/assert" | |||
| 10 | + | "github.com/stretchr/testify/require" | |||
| 11 | + | ||||
| 12 | + | "bnema.dev/ermite/internal/domain" | |||
| 13 | + | ) | |||
| 14 | + | ||||
| 15 | + | func at(t *testing.T, s string) time.Time { | |||
| 16 | + | t.Helper() | |||
| 17 | + | v, err := time.Parse("2006-01-02 15:04", s) | |||
| 18 | + | require.NoError(t, err) | |||
| 19 | + | return v | |||
| 20 | + | } | |||
| 21 | + | ||||
| 22 | + | func TestCronMatches(t *testing.T) { | |||
| 23 | + | tests := []struct { | |||
| 24 | + | spec string | |||
| 25 | + | yes []string | |||
| 26 | + | no []string | |||
| 27 | + | extra string | |||
| 28 | + | }{ | |||
| 29 | + | {"* * * * *", []string{"2026-03-04 05:06", "2026-12-31 23:59"}, nil, ""}, | |||
| 30 | + | {"*/15 * * * *", []string{"2026-03-04 05:00", "2026-03-04 05:45"}, []string{"2026-03-04 05:46", "2026-03-04 05:14"}, ""}, | |||
| 31 | + | {"30 4 * * *", []string{"2026-03-04 04:30"}, []string{"2026-03-04 04:31", "2026-03-04 05:30"}, ""}, | |||
| 32 | + | {"0 0 1 1 *", []string{"2027-01-01 00:00"}, []string{"2027-01-02 00:00", "2027-02-01 00:00"}, ""}, | |||
| 33 | + | {"5,10-12 */6 * * *", []string{"2026-03-04 06:05", "2026-03-04 12:11"}, []string{"2026-03-04 06:06", "2026-03-04 07:05"}, ""}, | |||
| 34 | + | {"10/20 * * * *", []string{"2026-03-04 00:10", "2026-03-04 00:30", "2026-03-04 00:50"}, []string{"2026-03-04 00:00", "2026-03-04 00:20"}, "a/n runs from a to the maximum"}, | |||
| 35 | + | {"0 12 * JAN-mar mon-fri", []string{"2026-02-17 12:00"}, []string{"2026-02-21 12:00", "2026-04-01 12:00"}, "names in any case"}, | |||
| 36 | + | {"0 0 * * 0", []string{"2026-03-01 00:00"}, []string{"2026-03-02 00:00"}, "0 is Sunday"}, | |||
| 37 | + | // Both day fields restricted: either matches (2026-03-13 is a Friday). | |||
| 38 | + | {"0 0 13 * fri", []string{"2026-03-13 00:00", "2026-03-20 00:00", "2026-04-13 00:00"}, []string{"2026-03-14 00:00"}, "day of month OR day of week"}, | |||
| 39 | + | // One of them a star: both must. | |||
| 40 | + | {"0 0 13 * *", []string{"2026-03-13 00:00"}, []string{"2026-03-20 00:00"}, ""}, | |||
| 41 | + | {"0 0 * * fri", []string{"2026-03-13 00:00"}, []string{"2026-03-12 00:00"}, ""}, | |||
| 42 | + | } | |||
| 43 | + | for _, tt := range tests { | |||
| 44 | + | c, err := domain.ParseCron(tt.spec) | |||
| 45 | + | require.NoError(t, err, tt.spec) | |||
| 46 | + | for _, y := range tt.yes { | |||
| 47 | + | assert.True(t, c.Matches(at(t, y)), "%s at %s %s", tt.spec, y, tt.extra) | |||
| 48 | + | } | |||
| 49 | + | for _, n := range tt.no { | |||
| 50 | + | assert.False(t, c.Matches(at(t, n)), "%s at %s %s", tt.spec, n, tt.extra) | |||
| 51 | + | } | |||
| 52 | + | } | |||
| 53 | + | // Only the UTC minute counts, whatever the zone of the time given. | |||
| 54 | + | c, err := domain.ParseCron("0 10 * * *") | |||
| 55 | + | require.NoError(t, err) | |||
| 56 | + | paris := time.FixedZone("CEST", 2*3600) | |||
| 57 | + | assert.True(t, c.Matches(time.Date(2026, 6, 1, 12, 0, 0, 0, paris))) | |||
| 58 | + | assert.False(t, c.Matches(time.Date(2026, 6, 1, 10, 0, 0, 0, paris))) | |||
| 59 | + | } | |||
| 60 | + | ||||
| 61 | + | func TestCronLeapDay(t *testing.T) { | |||
| 62 | + | c, err := domain.ParseCron("0 0 29 2 *") | |||
| 63 | + | require.NoError(t, err) | |||
| 64 | + | assert.True(t, c.Matches(at(t, "2028-02-29 00:00")), "2028 is a leap year") | |||
| 65 | + | assert.False(t, c.Matches(at(t, "2028-02-28 00:00"))) | |||
| 66 | + | assert.False(t, c.Matches(at(t, "2028-03-01 00:00"))) | |||
| 67 | + | for d := at(t, "2027-02-01 00:00"); d.Month() == time.February; d = d.AddDate(0, 0, 1) { | |||
| 68 | + | assert.False(t, c.Matches(d), "no 29 February in 2027: %s", d) | |||
| 69 | + | } | |||
| 70 | + | any31, err := domain.ParseCron("0 0 31 * *") | |||
| 71 | + | require.NoError(t, err) | |||
| 72 | + | assert.True(t, any31.Matches(at(t, "2026-03-31 00:00"))) | |||
| 73 | + | assert.False(t, any31.Matches(at(t, "2026-04-30 00:00")), "no 31st in April") | |||
| 74 | + | } | |||
| 75 | + | ||||
| 76 | + | // A step larger than the field keeps its first value only, and cannot overflow. | |||
| 77 | + | func TestCronHugeSteps(t *testing.T) { | |||
| 78 | + | for _, spec := range []string{"59/9223372036854775807 * * * *", "*/9223372036854775807 * * * *", "0-30/100 * * * *"} { | |||
| 79 | + | c, err := domain.ParseCron(spec) | |||
| 80 | + | require.NoError(t, err, spec) | |||
| 81 | + | var got []int | |||
| 82 | + | for m := range 60 { | |||
| 83 | + | if c.Matches(at(t, "2026-03-04 05:00").Add(time.Duration(m) * time.Minute)) { | |||
| 84 | + | got = append(got, m) | |||
| 85 | + | } | |||
| 86 | + | } | |||
| 87 | + | switch spec[:2] { | |||
| 88 | + | case "59": | |||
| 89 | + | assert.Equal(t, []int{59}, got, spec) | |||
| 90 | + | default: | |||
| 91 | + | assert.Equal(t, []int{0}, got, spec) | |||
| 92 | + | } | |||
| 93 | + | } | |||
| 94 | + | _, err := domain.ParseCron("*/99999999999999999999 * * * *") | |||
| 95 | + | require.Error(t, err, "beyond an int64") | |||
| 96 | + | } | |||
| 97 | + | ||||
| 98 | + | func TestParseCronRefuses(t *testing.T) { | |||
| 99 | + | for _, bad := range []string{ | |||
| 100 | + | "", "* * * *", "* * * * * *", "60 * * * *", "* 24 * * *", "* * 0 * *", "* * 32 * *", "* * * 13 *", "* * * * 7", | |||
| 101 | + | "*/0 * * * *", "*/x * * * *", "5-3 * * * *", "1,,2 * * * *", "-1 * * * *", "1- * * * *", "foo * * * *", "* * * jun-foo *", "+5 * * * *", "0x5 * * * *", | |||
| 102 | + | } { | |||
| 103 | + | _, err := domain.ParseCron(bad) | |||
| 104 | + | require.Error(t, err, bad) | |||
| 105 | + | assert.ErrorIs(t, err, domain.ErrInvalid, bad) | |||
| 106 | + | } | |||
| 107 | + | } | |||
A internal/domain/ci_tip.go +42 -0
| @@ -0,0 +1,42 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package domain | |
| 4 | + | ||
| 5 | + | // CIRefTip records who moves a branch or a tag to an object (CI-3, D1): the | |
| 6 | + | // actor and pusher of the push or merge that sets Ref to Sha, and whether that | |
| 7 | + | // principal was trusted (TrustedActor). | |
| 8 | + | // | |
| 9 | + | // It is what a scheduled run takes its actor and its trust from (GitHub runs a | |
| 10 | + | // schedule as the author of the last commit of the default branch), and what a | |
| 11 | + | // dispatched run's trust is capped by: the default branch is not necessarily | |
| 12 | + | // the owner's work (a token limited to some branches can push the first branch | |
| 13 | + | // of an empty repository, which becomes the default branch), and an agent's | |
| 14 | + | // branch is not the owner's work either. | |
| 15 | + | // | |
| 16 | + | // The record is written BEFORE the ref moves (in the pre-receive decision of a | |
| 17 | + | // push, and inside the merge before it updates the target), so that no moment | |
| 18 | + | // exists when the tip is the new commit and its record the previous pusher's; | |
| 19 | + | // writing it after would let a slow post-receive of an agent's push leave the | |
| 20 | + | // owner's record on a tip the agent has since moved back to the same commit | |
| 21 | + | // (owner A, agent B, agent A). It names the object it was written for, and is | |
| 22 | + | // valid only while the ref still points to it. A push that is refused after the | |
| 23 | + | // record was written leaves a record that matches no tip: not trusted (fails | |
| 24 | + | // closed). A tip with no record (a push from before records existed, a record | |
| 25 | + | // that could not be written) is not trusted either. | |
| 26 | + | // | |
| 27 | + | // Two concurrent pushes of the same object by the owner and by an agent can | |
| 28 | + | // leave the owner's record: the object is the same and the owner pushed it, so | |
| 29 | + | // the content is endorsed by the owner. That case is accepted. | |
| 30 | + | type CIRefTip struct { | |
| 31 | + | Repo RepoName | |
| 32 | + | // Ref is the full name: "refs/heads/main" or "refs/tags/v1". | |
| 33 | + | Ref string | |
| 34 | + | // Sha is the object the ref is set to (the tag object for an annotated tag). | |
| 35 | + | Sha string | |
| 36 | + | // Actor is who the run shows, as for a push (authorOf). | |
| 37 | + | Actor string | |
| 38 | + | // Pusher is the principal whose rights the actions of other repositories are | |
| 39 | + | // resolved with. | |
| 40 | + | Pusher CIPusher | |
| 41 | + | Trusted bool | |
| 42 | + | } | |
| @@ -0,0 +1,42 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package domain | |||
| 4 | + | ||||
| 5 | + | // CIRefTip records who moves a branch or a tag to an object (CI-3, D1): the | |||
| 6 | + | // actor and pusher of the push or merge that sets Ref to Sha, and whether that | |||
| 7 | + | // principal was trusted (TrustedActor). | |||
| 8 | + | // | |||
| 9 | + | // It is what a scheduled run takes its actor and its trust from (GitHub runs a | |||
| 10 | + | // schedule as the author of the last commit of the default branch), and what a | |||
| 11 | + | // dispatched run's trust is capped by: the default branch is not necessarily | |||
| 12 | + | // the owner's work (a token limited to some branches can push the first branch | |||
| 13 | + | // of an empty repository, which becomes the default branch), and an agent's | |||
| 14 | + | // branch is not the owner's work either. | |||
| 15 | + | // | |||
| 16 | + | // The record is written BEFORE the ref moves (in the pre-receive decision of a | |||
| 17 | + | // push, and inside the merge before it updates the target), so that no moment | |||
| 18 | + | // exists when the tip is the new commit and its record the previous pusher's; | |||
| 19 | + | // writing it after would let a slow post-receive of an agent's push leave the | |||
| 20 | + | // owner's record on a tip the agent has since moved back to the same commit | |||
| 21 | + | // (owner A, agent B, agent A). It names the object it was written for, and is | |||
| 22 | + | // valid only while the ref still points to it. A push that is refused after the | |||
| 23 | + | // record was written leaves a record that matches no tip: not trusted (fails | |||
| 24 | + | // closed). A tip with no record (a push from before records existed, a record | |||
| 25 | + | // that could not be written) is not trusted either. | |||
| 26 | + | // | |||
| 27 | + | // Two concurrent pushes of the same object by the owner and by an agent can | |||
| 28 | + | // leave the owner's record: the object is the same and the owner pushed it, so | |||
| 29 | + | // the content is endorsed by the owner. That case is accepted. | |||
| 30 | + | type CIRefTip struct { | |||
| 31 | + | Repo RepoName | |||
| 32 | + | // Ref is the full name: "refs/heads/main" or "refs/tags/v1". | |||
| 33 | + | Ref string | |||
| 34 | + | // Sha is the object the ref is set to (the tag object for an annotated tag). | |||
| 35 | + | Sha string | |||
| 36 | + | // Actor is who the run shows, as for a push (authorOf). | |||
| 37 | + | Actor string | |||
| 38 | + | // Pusher is the principal whose rights the actions of other repositories are | |||
| 39 | + | // resolved with. | |||
| 40 | + | Pusher CIPusher | |||
| 41 | + | Trusted bool | |||
| 42 | + | } | |||
M internal/ports/ci.go +30 -0
| @@ -82,6 +82,19 @@ type CIRepoTokenStore interface { | |||
| 82 | 82 | GetRepoToken(ctx context.Context, id string) (domain.CIRepoToken, error) | |
| 83 | 83 | } | |
| 84 | 84 | ||
| 85 | + | // CIRefTipStore keeps, per branch or tag (full ref name), who moves it to its | |
| 86 | + | // object (domain.CIRefTip). The record is written before the ref moves. | |
| 87 | + | // *kvstore.CI implements it; the records go with the repository. | |
| 88 | + | type CIRefTipStore interface { | |
| 89 | + | // PutRefTip stores the record of a ref, replacing the earlier one. A | |
| 90 | + | // repository that does not exist is domain.ErrNotFound. | |
| 91 | + | PutRefTip(ctx context.Context, t domain.CIRefTip) error | |
| 92 | + | // GetRefTip returns the record of a ref, or domain.ErrNotFound. | |
| 93 | + | GetRefTip(ctx context.Context, repo domain.RepoName, ref string) (domain.CIRefTip, error) | |
| 94 | + | // DeleteRefTip removes the record of a ref; a missing one is not an error. | |
| 95 | + | DeleteRefTip(ctx context.Context, repo domain.RepoName, ref string) error | |
| 96 | + | } | |
| 97 | + | ||
| 85 | 98 | // CIStore keeps CI runs, jobs and checks. The store derives its own indexes | |
| 86 | 99 | // (the queue of ready jobs, the active jobs, the open runs, the job tokens) | |
| 87 | 100 | // from the records it writes. | |
| @@ -170,6 +183,9 @@ type CIStore interface { | |||
| 170 | 183 | type CIRunCleaner interface { | |
| 171 | 184 | RunIDs(ctx context.Context, repo domain.RepoName) ([]int64, error) | |
| 172 | 185 | RemoveLogs(ctx context.Context, runIDs []int64) error | |
| 186 | + | // ForgetRepo drops what CI keeps in memory for the deleted repository (its | |
| 187 | + | // cached schedules). | |
| 188 | + | ForgetRepo(repo domain.RepoName) | |
| 173 | 189 | } | |
| 174 | 190 | ||
| 175 | 191 | // CIChecksReader is the read side of the checks, for the merge rule and the | |
| @@ -255,6 +271,16 @@ type CISummaryStore interface { | |||
| 255 | 271 | RunIDs(ctx context.Context) ([]int64, error) | |
| 256 | 272 | } | |
| 257 | 273 | ||
| 274 | + | // DispatchParams are the arguments of RunService.DispatchWorkflow. | |
| 275 | + | type DispatchParams struct { | |
| 276 | + | // WorkflowPath is the path of the workflow file, or its file name. | |
| 277 | + | WorkflowPath string | |
| 278 | + | // Ref is a branch or a tag, short or full name; empty is the default branch. | |
| 279 | + | Ref string | |
| 280 | + | // Inputs are the values of the inputs by name, as text. | |
| 281 | + | Inputs map[string]string | |
| 282 | + | } | |
| 283 | + | ||
| 258 | 284 | // WorkflowTrigger starts the workflows that match pushed refs. OnPush only | |
| 259 | 285 | // queues the work and returns: planning happens in the background and its | |
| 260 | 286 | // errors never reach the push. | |
| @@ -663,6 +689,10 @@ type RunService interface { | |||
| 663 | 689 | GetJobLog(ctx context.Context, p domain.Principal, repo domain.RepoName, params GetJobLogParams, send func(LogBatch) error) error | |
| 664 | 690 | CancelRun(ctx context.Context, p domain.Principal, repo domain.RepoName, runID int64) (domain.CIRun, error) | |
| 665 | 691 | RerunRun(ctx context.Context, p domain.Principal, repo domain.RepoName, runID int64, failedOnly bool) (domain.CIRun, error) | |
| 692 | + | // DispatchWorkflow starts a run of a workflow file by hand (workflow_dispatch). | |
| 693 | + | // The caller needs ci:write on the repository, or is a job token with the | |
| 694 | + | // permission actions: write. The run is returned queued. | |
| 695 | + | DispatchWorkflow(ctx context.Context, p domain.Principal, repo domain.RepoName, params DispatchParams) (domain.CIRun, error) | |
| 666 | 696 | ListChecks(ctx context.Context, p domain.Principal, repo domain.RepoName, sha string) (ChecksResult, error) | |
| 667 | 697 | // ListAnnotations returns the annotations of a run attempt (0 for the | |
| 668 | 698 | // latest), of one job when jobID is not empty. Each job gives those of the | |
| @@ -82,6 +82,19 @@ type CIRepoTokenStore interface { | |||||
| 82 | GetRepoToken(ctx context.Context, id string) (domain.CIRepoToken, error) | 82 | GetRepoToken(ctx context.Context, id string) (domain.CIRepoToken, error) | ||
| 83 | } | 83 | } | ||
| 84 | 84 | ||||
| 85 | + | // CIRefTipStore keeps, per branch or tag (full ref name), who moves it to its | |||
| 86 | + | // object (domain.CIRefTip). The record is written before the ref moves. | |||
| 87 | + | // *kvstore.CI implements it; the records go with the repository. | |||
| 88 | + | type CIRefTipStore interface { | |||
| 89 | + | // PutRefTip stores the record of a ref, replacing the earlier one. A | |||
| 90 | + | // repository that does not exist is domain.ErrNotFound. | |||
| 91 | + | PutRefTip(ctx context.Context, t domain.CIRefTip) error | |||
| 92 | + | // GetRefTip returns the record of a ref, or domain.ErrNotFound. | |||
| 93 | + | GetRefTip(ctx context.Context, repo domain.RepoName, ref string) (domain.CIRefTip, error) | |||
| 94 | + | // DeleteRefTip removes the record of a ref; a missing one is not an error. | |||
| 95 | + | DeleteRefTip(ctx context.Context, repo domain.RepoName, ref string) error | |||
| 96 | + | } | |||
| 97 | + | ||||
| 85 | // CIStore keeps CI runs, jobs and checks. The store derives its own indexes | 98 | // CIStore keeps CI runs, jobs and checks. The store derives its own indexes | ||
| 86 | // (the queue of ready jobs, the active jobs, the open runs, the job tokens) | 99 | // (the queue of ready jobs, the active jobs, the open runs, the job tokens) | ||
| 87 | // from the records it writes. | 100 | // from the records it writes. | ||
| @@ -170,6 +183,9 @@ type CIStore interface { | |||||
| 170 | type CIRunCleaner interface { | 183 | type CIRunCleaner interface { | ||
| 171 | RunIDs(ctx context.Context, repo domain.RepoName) ([]int64, error) | 184 | RunIDs(ctx context.Context, repo domain.RepoName) ([]int64, error) | ||
| 172 | RemoveLogs(ctx context.Context, runIDs []int64) error | 185 | RemoveLogs(ctx context.Context, runIDs []int64) error | ||
| 186 | + | // ForgetRepo drops what CI keeps in memory for the deleted repository (its | |||
| 187 | + | // cached schedules). | |||
| 188 | + | ForgetRepo(repo domain.RepoName) | |||
| 173 | } | 189 | } | ||
| 174 | 190 | ||||
| 175 | // CIChecksReader is the read side of the checks, for the merge rule and the | 191 | // CIChecksReader is the read side of the checks, for the merge rule and the | ||
| @@ -255,6 +271,16 @@ type CISummaryStore interface { | |||||
| 255 | RunIDs(ctx context.Context) ([]int64, error) | 271 | RunIDs(ctx context.Context) ([]int64, error) | ||
| 256 | } | 272 | } | ||
| 257 | 273 | ||||
| 274 | + | // DispatchParams are the arguments of RunService.DispatchWorkflow. | |||
| 275 | + | type DispatchParams struct { | |||
| 276 | + | // WorkflowPath is the path of the workflow file, or its file name. | |||
| 277 | + | WorkflowPath string | |||
| 278 | + | // Ref is a branch or a tag, short or full name; empty is the default branch. | |||
| 279 | + | Ref string | |||
| 280 | + | // Inputs are the values of the inputs by name, as text. | |||
| 281 | + | Inputs map[string]string | |||
| 282 | + | } | |||
| 283 | + | ||||
| 258 | // WorkflowTrigger starts the workflows that match pushed refs. OnPush only | 284 | // WorkflowTrigger starts the workflows that match pushed refs. OnPush only | ||
| 259 | // queues the work and returns: planning happens in the background and its | 285 | // queues the work and returns: planning happens in the background and its | ||
| 260 | // errors never reach the push. | 286 | // errors never reach the push. | ||
| @@ -663,6 +689,10 @@ type RunService interface { | |||||
| 663 | GetJobLog(ctx context.Context, p domain.Principal, repo domain.RepoName, params GetJobLogParams, send func(LogBatch) error) error | 689 | GetJobLog(ctx context.Context, p domain.Principal, repo domain.RepoName, params GetJobLogParams, send func(LogBatch) error) error | ||
| 664 | CancelRun(ctx context.Context, p domain.Principal, repo domain.RepoName, runID int64) (domain.CIRun, error) | 690 | CancelRun(ctx context.Context, p domain.Principal, repo domain.RepoName, runID int64) (domain.CIRun, error) | ||
| 665 | RerunRun(ctx context.Context, p domain.Principal, repo domain.RepoName, runID int64, failedOnly bool) (domain.CIRun, error) | 691 | RerunRun(ctx context.Context, p domain.Principal, repo domain.RepoName, runID int64, failedOnly bool) (domain.CIRun, error) | ||
| 692 | + | // DispatchWorkflow starts a run of a workflow file by hand (workflow_dispatch). | |||
| 693 | + | // The caller needs ci:write on the repository, or is a job token with the | |||
| 694 | + | // permission actions: write. The run is returned queued. | |||
| 695 | + | DispatchWorkflow(ctx context.Context, p domain.Principal, repo domain.RepoName, params DispatchParams) (domain.CIRun, error) | |||
| 666 | ListChecks(ctx context.Context, p domain.Principal, repo domain.RepoName, sha string) (ChecksResult, error) | 696 | ListChecks(ctx context.Context, p domain.Principal, repo domain.RepoName, sha string) (ChecksResult, error) | ||
| 667 | // ListAnnotations returns the annotations of a run attempt (0 for the | 697 | // ListAnnotations returns the annotations of a run attempt (0 for the | ||
| 668 | // latest), of one job when jobID is not empty. Each job gives those of the | 698 | // latest), of one job when jobID is not empty. Each job gives those of the | ||
A internal/ports/mocks/CIRefTipStore.go +240 -0
| @@ -0,0 +1,240 @@ | |||
| 1 | + | // Code generated by mockery; DO NOT EDIT. | |
| 2 | + | // github.com/vektra/mockery | |
| 3 | + | // template: testify | |
| 4 | + | ||
| 5 | + | package mocks | |
| 6 | + | ||
| 7 | + | import ( | |
| 8 | + | "context" | |
| 9 | + | ||
| 10 | + | "bnema.dev/ermite/internal/domain" | |
| 11 | + | mock "github.com/stretchr/testify/mock" | |
| 12 | + | ) | |
| 13 | + | ||
| 14 | + | // NewCIRefTipStore creates a new instance of CIRefTipStore. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. | |
| 15 | + | // The first argument is typically a *testing.T value. | |
| 16 | + | func NewCIRefTipStore(t interface { | |
| 17 | + | mock.TestingT | |
| 18 | + | Cleanup(func()) | |
| 19 | + | }) *CIRefTipStore { | |
| 20 | + | if helper, ok := t.(interface{ Helper() }); ok { | |
| 21 | + | helper.Helper() | |
| 22 | + | } | |
| 23 | + | ||
| 24 | + | mock := &CIRefTipStore{} | |
| 25 | + | mock.Mock.Test(t) | |
| 26 | + | ||
| 27 | + | t.Cleanup(func() { | |
| 28 | + | if helper, ok := t.(interface{ Helper() }); ok { | |
| 29 | + | helper.Helper() | |
| 30 | + | } | |
| 31 | + | mock.AssertExpectations(t) | |
| 32 | + | }) | |
| 33 | + | ||
| 34 | + | return mock | |
| 35 | + | } | |
| 36 | + | ||
| 37 | + | // CIRefTipStore is an autogenerated mock type for the CIRefTipStore type | |
| 38 | + | type CIRefTipStore struct { | |
| 39 | + | mock.Mock | |
| 40 | + | } | |
| 41 | + | ||
| 42 | + | type CIRefTipStore_Expecter struct { | |
| 43 | + | mock *mock.Mock | |
| 44 | + | } | |
| 45 | + | ||
| 46 | + | func (_m *CIRefTipStore) EXPECT() *CIRefTipStore_Expecter { | |
| 47 | + | return &CIRefTipStore_Expecter{mock: &_m.Mock} | |
| 48 | + | } | |
| 49 | + | ||
| 50 | + | // DeleteRefTip provides a mock function for the type CIRefTipStore | |
| 51 | + | func (_mock *CIRefTipStore) DeleteRefTip(ctx context.Context, repo domain.RepoName, ref string) error { | |
| 52 | + | ret := _mock.Called(ctx, repo, ref) | |
| 53 | + | ||
| 54 | + | if len(ret) == 0 { | |
| 55 | + | panic("no return value specified for DeleteRefTip") | |
| 56 | + | } | |
| 57 | + | ||
| 58 | + | var r0 error | |
| 59 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string) error); ok { | |
| 60 | + | r0 = returnFunc(ctx, repo, ref) | |
| 61 | + | } else { | |
| 62 | + | r0 = ret.Error(0) | |
| 63 | + | } | |
| 64 | + | return r0 | |
| 65 | + | } | |
| 66 | + | ||
| 67 | + | // CIRefTipStore_DeleteRefTip_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'DeleteRefTip' | |
| 68 | + | type CIRefTipStore_DeleteRefTip_Call struct { | |
| 69 | + | *mock.Call | |
| 70 | + | } | |
| 71 | + | ||
| 72 | + | // DeleteRefTip is a helper method to define mock.On call | |
| 73 | + | // - ctx context.Context | |
| 74 | + | // - repo domain.RepoName | |
| 75 | + | // - ref string | |
| 76 | + | func (_e *CIRefTipStore_Expecter) DeleteRefTip(ctx any, repo any, ref any) *CIRefTipStore_DeleteRefTip_Call { | |
| 77 | + | return &CIRefTipStore_DeleteRefTip_Call{Call: _e.mock.On("DeleteRefTip", ctx, repo, ref)} | |
| 78 | + | } | |
| 79 | + | ||
| 80 | + | func (_c *CIRefTipStore_DeleteRefTip_Call) Run(run func(ctx context.Context, repo domain.RepoName, ref string)) *CIRefTipStore_DeleteRefTip_Call { | |
| 81 | + | _c.Call.Run(func(args mock.Arguments) { | |
| 82 | + | var arg0 context.Context | |
| 83 | + | if args[0] != nil { | |
| 84 | + | arg0 = args[0].(context.Context) | |
| 85 | + | } | |
| 86 | + | var arg1 domain.RepoName | |
| 87 | + | if args[1] != nil { | |
| 88 | + | arg1 = args[1].(domain.RepoName) | |
| 89 | + | } | |
| 90 | + | var arg2 string | |
| 91 | + | if args[2] != nil { | |
| 92 | + | arg2 = args[2].(string) | |
| 93 | + | } | |
| 94 | + | run( | |
| 95 | + | arg0, | |
| 96 | + | arg1, | |
| 97 | + | arg2, | |
| 98 | + | ) | |
| 99 | + | }) | |
| 100 | + | return _c | |
| 101 | + | } | |
| 102 | + | ||
| 103 | + | func (_c *CIRefTipStore_DeleteRefTip_Call) Return(err error) *CIRefTipStore_DeleteRefTip_Call { | |
| 104 | + | _c.Call.Return(err) | |
| 105 | + | return _c | |
| 106 | + | } | |
| 107 | + | ||
| 108 | + | func (_c *CIRefTipStore_DeleteRefTip_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, ref string) error) *CIRefTipStore_DeleteRefTip_Call { | |
| 109 | + | _c.Call.Return(run) | |
| 110 | + | return _c | |
| 111 | + | } | |
| 112 | + | ||
| 113 | + | // GetRefTip provides a mock function for the type CIRefTipStore | |
| 114 | + | func (_mock *CIRefTipStore) GetRefTip(ctx context.Context, repo domain.RepoName, ref string) (domain.CIRefTip, error) { | |
| 115 | + | ret := _mock.Called(ctx, repo, ref) | |
| 116 | + | ||
| 117 | + | if len(ret) == 0 { | |
| 118 | + | panic("no return value specified for GetRefTip") | |
| 119 | + | } | |
| 120 | + | ||
| 121 | + | var r0 domain.CIRefTip | |
| 122 | + | var r1 error | |
| 123 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string) (domain.CIRefTip, error)); ok { | |
| 124 | + | return returnFunc(ctx, repo, ref) | |
| 125 | + | } | |
| 126 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string) domain.CIRefTip); ok { | |
| 127 | + | r0 = returnFunc(ctx, repo, ref) | |
| 128 | + | } else { | |
| 129 | + | r0 = ret.Get(0).(domain.CIRefTip) | |
| 130 | + | } | |
| 131 | + | if returnFunc, ok := ret.Get(1).(func(context.Context, domain.RepoName, string) error); ok { | |
| 132 | + | r1 = returnFunc(ctx, repo, ref) | |
| 133 | + | } else { | |
| 134 | + | r1 = ret.Error(1) | |
| 135 | + | } | |
| 136 | + | return r0, r1 | |
| 137 | + | } | |
| 138 | + | ||
| 139 | + | // CIRefTipStore_GetRefTip_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetRefTip' | |
| 140 | + | type CIRefTipStore_GetRefTip_Call struct { | |
| 141 | + | *mock.Call | |
| 142 | + | } | |
| 143 | + | ||
| 144 | + | // GetRefTip is a helper method to define mock.On call | |
| 145 | + | // - ctx context.Context | |
| 146 | + | // - repo domain.RepoName | |
| 147 | + | // - ref string | |
| 148 | + | func (_e *CIRefTipStore_Expecter) GetRefTip(ctx any, repo any, ref any) *CIRefTipStore_GetRefTip_Call { | |
| 149 | + | return &CIRefTipStore_GetRefTip_Call{Call: _e.mock.On("GetRefTip", ctx, repo, ref)} | |
| 150 | + | } | |
| 151 | + | ||
| 152 | + | func (_c *CIRefTipStore_GetRefTip_Call) Run(run func(ctx context.Context, repo domain.RepoName, ref string)) *CIRefTipStore_GetRefTip_Call { | |
| 153 | + | _c.Call.Run(func(args mock.Arguments) { | |
| 154 | + | var arg0 context.Context | |
| 155 | + | if args[0] != nil { | |
| 156 | + | arg0 = args[0].(context.Context) | |
| 157 | + | } | |
| 158 | + | var arg1 domain.RepoName | |
| 159 | + | if args[1] != nil { | |
| 160 | + | arg1 = args[1].(domain.RepoName) | |
| 161 | + | } | |
| 162 | + | var arg2 string | |
| 163 | + | if args[2] != nil { | |
| 164 | + | arg2 = args[2].(string) | |
| 165 | + | } | |
| 166 | + | run( | |
| 167 | + | arg0, | |
| 168 | + | arg1, | |
| 169 | + | arg2, | |
| 170 | + | ) | |
| 171 | + | }) | |
| 172 | + | return _c | |
| 173 | + | } | |
| 174 | + | ||
| 175 | + | func (_c *CIRefTipStore_GetRefTip_Call) Return(cIRefTip domain.CIRefTip, err error) *CIRefTipStore_GetRefTip_Call { | |
| 176 | + | _c.Call.Return(cIRefTip, err) | |
| 177 | + | return _c | |
| 178 | + | } | |
| 179 | + | ||
| 180 | + | func (_c *CIRefTipStore_GetRefTip_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, ref string) (domain.CIRefTip, error)) *CIRefTipStore_GetRefTip_Call { | |
| 181 | + | _c.Call.Return(run) | |
| 182 | + | return _c | |
| 183 | + | } | |
| 184 | + | ||
| 185 | + | // PutRefTip provides a mock function for the type CIRefTipStore | |
| 186 | + | func (_mock *CIRefTipStore) PutRefTip(ctx context.Context, t domain.CIRefTip) error { | |
| 187 | + | ret := _mock.Called(ctx, t) | |
| 188 | + | ||
| 189 | + | if len(ret) == 0 { | |
| 190 | + | panic("no return value specified for PutRefTip") | |
| 191 | + | } | |
| 192 | + | ||
| 193 | + | var r0 error | |
| 194 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.CIRefTip) error); ok { | |
| 195 | + | r0 = returnFunc(ctx, t) | |
| 196 | + | } else { | |
| 197 | + | r0 = ret.Error(0) | |
| 198 | + | } | |
| 199 | + | return r0 | |
| 200 | + | } | |
| 201 | + | ||
| 202 | + | // CIRefTipStore_PutRefTip_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PutRefTip' | |
| 203 | + | type CIRefTipStore_PutRefTip_Call struct { | |
| 204 | + | *mock.Call | |
| 205 | + | } | |
| 206 | + | ||
| 207 | + | // PutRefTip is a helper method to define mock.On call | |
| 208 | + | // - ctx context.Context | |
| 209 | + | // - t domain.CIRefTip | |
| 210 | + | func (_e *CIRefTipStore_Expecter) PutRefTip(ctx any, t any) *CIRefTipStore_PutRefTip_Call { | |
| 211 | + | return &CIRefTipStore_PutRefTip_Call{Call: _e.mock.On("PutRefTip", ctx, t)} | |
| 212 | + | } | |
| 213 | + | ||
| 214 | + | func (_c *CIRefTipStore_PutRefTip_Call) Run(run func(ctx context.Context, t domain.CIRefTip)) *CIRefTipStore_PutRefTip_Call { | |
| 215 | + | _c.Call.Run(func(args mock.Arguments) { | |
| 216 | + | var arg0 context.Context | |
| 217 | + | if args[0] != nil { | |
| 218 | + | arg0 = args[0].(context.Context) | |
| 219 | + | } | |
| 220 | + | var arg1 domain.CIRefTip | |
| 221 | + | if args[1] != nil { | |
| 222 | + | arg1 = args[1].(domain.CIRefTip) | |
| 223 | + | } | |
| 224 | + | run( | |
| 225 | + | arg0, | |
| 226 | + | arg1, | |
| 227 | + | ) | |
| 228 | + | }) | |
| 229 | + | return _c | |
| 230 | + | } | |
| 231 | + | ||
| 232 | + | func (_c *CIRefTipStore_PutRefTip_Call) Return(err error) *CIRefTipStore_PutRefTip_Call { | |
| 233 | + | _c.Call.Return(err) | |
| 234 | + | return _c | |
| 235 | + | } | |
| 236 | + | ||
| 237 | + | func (_c *CIRefTipStore_PutRefTip_Call) RunAndReturn(run func(ctx context.Context, t domain.CIRefTip) error) *CIRefTipStore_PutRefTip_Call { | |
| 238 | + | _c.Call.Return(run) | |
| 239 | + | return _c | |
| 240 | + | } | |
| @@ -0,0 +1,240 @@ | |||||
| 1 | + | // Code generated by mockery; DO NOT EDIT. | |||
| 2 | + | // github.com/vektra/mockery | |||
| 3 | + | // template: testify | |||
| 4 | + | ||||
| 5 | + | package mocks | |||
| 6 | + | ||||
| 7 | + | import ( | |||
| 8 | + | "context" | |||
| 9 | + | ||||
| 10 | + | "bnema.dev/ermite/internal/domain" | |||
| 11 | + | mock "github.com/stretchr/testify/mock" | |||
| 12 | + | ) | |||
| 13 | + | ||||
| 14 | + | // NewCIRefTipStore creates a new instance of CIRefTipStore. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations. | |||
| 15 | + | // The first argument is typically a *testing.T value. | |||
| 16 | + | func NewCIRefTipStore(t interface { | |||
| 17 | + | mock.TestingT | |||
| 18 | + | Cleanup(func()) | |||
| 19 | + | }) *CIRefTipStore { | |||
| 20 | + | if helper, ok := t.(interface{ Helper() }); ok { | |||
| 21 | + | helper.Helper() | |||
| 22 | + | } | |||
| 23 | + | ||||
| 24 | + | mock := &CIRefTipStore{} | |||
| 25 | + | mock.Mock.Test(t) | |||
| 26 | + | ||||
| 27 | + | t.Cleanup(func() { | |||
| 28 | + | if helper, ok := t.(interface{ Helper() }); ok { | |||
| 29 | + | helper.Helper() | |||
| 30 | + | } | |||
| 31 | + | mock.AssertExpectations(t) | |||
| 32 | + | }) | |||
| 33 | + | ||||
| 34 | + | return mock | |||
| 35 | + | } | |||
| 36 | + | ||||
| 37 | + | // CIRefTipStore is an autogenerated mock type for the CIRefTipStore type | |||
| 38 | + | type CIRefTipStore struct { | |||
| 39 | + | mock.Mock | |||
| 40 | + | } | |||
| 41 | + | ||||
| 42 | + | type CIRefTipStore_Expecter struct { | |||
| 43 | + | mock *mock.Mock | |||
| 44 | + | } | |||
| 45 | + | ||||
| 46 | + | func (_m *CIRefTipStore) EXPECT() *CIRefTipStore_Expecter { | |||
| 47 | + | return &CIRefTipStore_Expecter{mock: &_m.Mock} | |||
| 48 | + | } | |||
| 49 | + | ||||
| 50 | + | // DeleteRefTip provides a mock function for the type CIRefTipStore | |||
| 51 | + | func (_mock *CIRefTipStore) DeleteRefTip(ctx context.Context, repo domain.RepoName, ref string) error { | |||
| 52 | + | ret := _mock.Called(ctx, repo, ref) | |||
| 53 | + | ||||
| 54 | + | if len(ret) == 0 { | |||
| 55 | + | panic("no return value specified for DeleteRefTip") | |||
| 56 | + | } | |||
| 57 | + | ||||
| 58 | + | var r0 error | |||
| 59 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string) error); ok { | |||
| 60 | + | r0 = returnFunc(ctx, repo, ref) | |||
| 61 | + | } else { | |||
| 62 | + | r0 = ret.Error(0) | |||
| 63 | + | } | |||
| 64 | + | return r0 | |||
| 65 | + | } | |||
| 66 | + | ||||
| 67 | + | // CIRefTipStore_DeleteRefTip_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'DeleteRefTip' | |||
| 68 | + | type CIRefTipStore_DeleteRefTip_Call struct { | |||
| 69 | + | *mock.Call | |||
| 70 | + | } | |||
| 71 | + | ||||
| 72 | + | // DeleteRefTip is a helper method to define mock.On call | |||
| 73 | + | // - ctx context.Context | |||
| 74 | + | // - repo domain.RepoName | |||
| 75 | + | // - ref string | |||
| 76 | + | func (_e *CIRefTipStore_Expecter) DeleteRefTip(ctx any, repo any, ref any) *CIRefTipStore_DeleteRefTip_Call { | |||
| 77 | + | return &CIRefTipStore_DeleteRefTip_Call{Call: _e.mock.On("DeleteRefTip", ctx, repo, ref)} | |||
| 78 | + | } | |||
| 79 | + | ||||
| 80 | + | func (_c *CIRefTipStore_DeleteRefTip_Call) Run(run func(ctx context.Context, repo domain.RepoName, ref string)) *CIRefTipStore_DeleteRefTip_Call { | |||
| 81 | + | _c.Call.Run(func(args mock.Arguments) { | |||
| 82 | + | var arg0 context.Context | |||
| 83 | + | if args[0] != nil { | |||
| 84 | + | arg0 = args[0].(context.Context) | |||
| 85 | + | } | |||
| 86 | + | var arg1 domain.RepoName | |||
| 87 | + | if args[1] != nil { | |||
| 88 | + | arg1 = args[1].(domain.RepoName) | |||
| 89 | + | } | |||
| 90 | + | var arg2 string | |||
| 91 | + | if args[2] != nil { | |||
| 92 | + | arg2 = args[2].(string) | |||
| 93 | + | } | |||
| 94 | + | run( | |||
| 95 | + | arg0, | |||
| 96 | + | arg1, | |||
| 97 | + | arg2, | |||
| 98 | + | ) | |||
| 99 | + | }) | |||
| 100 | + | return _c | |||
| 101 | + | } | |||
| 102 | + | ||||
| 103 | + | func (_c *CIRefTipStore_DeleteRefTip_Call) Return(err error) *CIRefTipStore_DeleteRefTip_Call { | |||
| 104 | + | _c.Call.Return(err) | |||
| 105 | + | return _c | |||
| 106 | + | } | |||
| 107 | + | ||||
| 108 | + | func (_c *CIRefTipStore_DeleteRefTip_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, ref string) error) *CIRefTipStore_DeleteRefTip_Call { | |||
| 109 | + | _c.Call.Return(run) | |||
| 110 | + | return _c | |||
| 111 | + | } | |||
| 112 | + | ||||
| 113 | + | // GetRefTip provides a mock function for the type CIRefTipStore | |||
| 114 | + | func (_mock *CIRefTipStore) GetRefTip(ctx context.Context, repo domain.RepoName, ref string) (domain.CIRefTip, error) { | |||
| 115 | + | ret := _mock.Called(ctx, repo, ref) | |||
| 116 | + | ||||
| 117 | + | if len(ret) == 0 { | |||
| 118 | + | panic("no return value specified for GetRefTip") | |||
| 119 | + | } | |||
| 120 | + | ||||
| 121 | + | var r0 domain.CIRefTip | |||
| 122 | + | var r1 error | |||
| 123 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string) (domain.CIRefTip, error)); ok { | |||
| 124 | + | return returnFunc(ctx, repo, ref) | |||
| 125 | + | } | |||
| 126 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string) domain.CIRefTip); ok { | |||
| 127 | + | r0 = returnFunc(ctx, repo, ref) | |||
| 128 | + | } else { | |||
| 129 | + | r0 = ret.Get(0).(domain.CIRefTip) | |||
| 130 | + | } | |||
| 131 | + | if returnFunc, ok := ret.Get(1).(func(context.Context, domain.RepoName, string) error); ok { | |||
| 132 | + | r1 = returnFunc(ctx, repo, ref) | |||
| 133 | + | } else { | |||
| 134 | + | r1 = ret.Error(1) | |||
| 135 | + | } | |||
| 136 | + | return r0, r1 | |||
| 137 | + | } | |||
| 138 | + | ||||
| 139 | + | // CIRefTipStore_GetRefTip_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'GetRefTip' | |||
| 140 | + | type CIRefTipStore_GetRefTip_Call struct { | |||
| 141 | + | *mock.Call | |||
| 142 | + | } | |||
| 143 | + | ||||
| 144 | + | // GetRefTip is a helper method to define mock.On call | |||
| 145 | + | // - ctx context.Context | |||
| 146 | + | // - repo domain.RepoName | |||
| 147 | + | // - ref string | |||
| 148 | + | func (_e *CIRefTipStore_Expecter) GetRefTip(ctx any, repo any, ref any) *CIRefTipStore_GetRefTip_Call { | |||
| 149 | + | return &CIRefTipStore_GetRefTip_Call{Call: _e.mock.On("GetRefTip", ctx, repo, ref)} | |||
| 150 | + | } | |||
| 151 | + | ||||
| 152 | + | func (_c *CIRefTipStore_GetRefTip_Call) Run(run func(ctx context.Context, repo domain.RepoName, ref string)) *CIRefTipStore_GetRefTip_Call { | |||
| 153 | + | _c.Call.Run(func(args mock.Arguments) { | |||
| 154 | + | var arg0 context.Context | |||
| 155 | + | if args[0] != nil { | |||
| 156 | + | arg0 = args[0].(context.Context) | |||
| 157 | + | } | |||
| 158 | + | var arg1 domain.RepoName | |||
| 159 | + | if args[1] != nil { | |||
| 160 | + | arg1 = args[1].(domain.RepoName) | |||
| 161 | + | } | |||
| 162 | + | var arg2 string | |||
| 163 | + | if args[2] != nil { | |||
| 164 | + | arg2 = args[2].(string) | |||
| 165 | + | } | |||
| 166 | + | run( | |||
| 167 | + | arg0, | |||
| 168 | + | arg1, | |||
| 169 | + | arg2, | |||
| 170 | + | ) | |||
| 171 | + | }) | |||
| 172 | + | return _c | |||
| 173 | + | } | |||
| 174 | + | ||||
| 175 | + | func (_c *CIRefTipStore_GetRefTip_Call) Return(cIRefTip domain.CIRefTip, err error) *CIRefTipStore_GetRefTip_Call { | |||
| 176 | + | _c.Call.Return(cIRefTip, err) | |||
| 177 | + | return _c | |||
| 178 | + | } | |||
| 179 | + | ||||
| 180 | + | func (_c *CIRefTipStore_GetRefTip_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, ref string) (domain.CIRefTip, error)) *CIRefTipStore_GetRefTip_Call { | |||
| 181 | + | _c.Call.Return(run) | |||
| 182 | + | return _c | |||
| 183 | + | } | |||
| 184 | + | ||||
| 185 | + | // PutRefTip provides a mock function for the type CIRefTipStore | |||
| 186 | + | func (_mock *CIRefTipStore) PutRefTip(ctx context.Context, t domain.CIRefTip) error { | |||
| 187 | + | ret := _mock.Called(ctx, t) | |||
| 188 | + | ||||
| 189 | + | if len(ret) == 0 { | |||
| 190 | + | panic("no return value specified for PutRefTip") | |||
| 191 | + | } | |||
| 192 | + | ||||
| 193 | + | var r0 error | |||
| 194 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.CIRefTip) error); ok { | |||
| 195 | + | r0 = returnFunc(ctx, t) | |||
| 196 | + | } else { | |||
| 197 | + | r0 = ret.Error(0) | |||
| 198 | + | } | |||
| 199 | + | return r0 | |||
| 200 | + | } | |||
| 201 | + | ||||
| 202 | + | // CIRefTipStore_PutRefTip_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PutRefTip' | |||
| 203 | + | type CIRefTipStore_PutRefTip_Call struct { | |||
| 204 | + | *mock.Call | |||
| 205 | + | } | |||
| 206 | + | ||||
| 207 | + | // PutRefTip is a helper method to define mock.On call | |||
| 208 | + | // - ctx context.Context | |||
| 209 | + | // - t domain.CIRefTip | |||
| 210 | + | func (_e *CIRefTipStore_Expecter) PutRefTip(ctx any, t any) *CIRefTipStore_PutRefTip_Call { | |||
| 211 | + | return &CIRefTipStore_PutRefTip_Call{Call: _e.mock.On("PutRefTip", ctx, t)} | |||
| 212 | + | } | |||
| 213 | + | ||||
| 214 | + | func (_c *CIRefTipStore_PutRefTip_Call) Run(run func(ctx context.Context, t domain.CIRefTip)) *CIRefTipStore_PutRefTip_Call { | |||
| 215 | + | _c.Call.Run(func(args mock.Arguments) { | |||
| 216 | + | var arg0 context.Context | |||
| 217 | + | if args[0] != nil { | |||
| 218 | + | arg0 = args[0].(context.Context) | |||
| 219 | + | } | |||
| 220 | + | var arg1 domain.CIRefTip | |||
| 221 | + | if args[1] != nil { | |||
| 222 | + | arg1 = args[1].(domain.CIRefTip) | |||
| 223 | + | } | |||
| 224 | + | run( | |||
| 225 | + | arg0, | |||
| 226 | + | arg1, | |||
| 227 | + | ) | |||
| 228 | + | }) | |||
| 229 | + | return _c | |||
| 230 | + | } | |||
| 231 | + | ||||
| 232 | + | func (_c *CIRefTipStore_PutRefTip_Call) Return(err error) *CIRefTipStore_PutRefTip_Call { | |||
| 233 | + | _c.Call.Return(err) | |||
| 234 | + | return _c | |||
| 235 | + | } | |||
| 236 | + | ||||
| 237 | + | func (_c *CIRefTipStore_PutRefTip_Call) RunAndReturn(run func(ctx context.Context, t domain.CIRefTip) error) *CIRefTipStore_PutRefTip_Call { | |||
| 238 | + | _c.Call.Return(run) | |||
| 239 | + | return _c | |||
| 240 | + | } | |||
M internal/ports/mocks/CIRunCleaner.go +40 -0
| @@ -47,6 +47,46 @@ func (_m *CIRunCleaner) EXPECT() *CIRunCleaner_Expecter { | |||
| 47 | 47 | return &CIRunCleaner_Expecter{mock: &_m.Mock} | |
| 48 | 48 | } | |
| 49 | 49 | ||
| 50 | + | // ForgetRepo provides a mock function for the type CIRunCleaner | |
| 51 | + | func (_mock *CIRunCleaner) ForgetRepo(repo domain.RepoName) { | |
| 52 | + | _mock.Called(repo) | |
| 53 | + | return | |
| 54 | + | } | |
| 55 | + | ||
| 56 | + | // CIRunCleaner_ForgetRepo_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ForgetRepo' | |
| 57 | + | type CIRunCleaner_ForgetRepo_Call struct { | |
| 58 | + | *mock.Call | |
| 59 | + | } | |
| 60 | + | ||
| 61 | + | // ForgetRepo is a helper method to define mock.On call | |
| 62 | + | // - repo domain.RepoName | |
| 63 | + | func (_e *CIRunCleaner_Expecter) ForgetRepo(repo any) *CIRunCleaner_ForgetRepo_Call { | |
| 64 | + | return &CIRunCleaner_ForgetRepo_Call{Call: _e.mock.On("ForgetRepo", repo)} | |
| 65 | + | } | |
| 66 | + | ||
| 67 | + | func (_c *CIRunCleaner_ForgetRepo_Call) Run(run func(repo domain.RepoName)) *CIRunCleaner_ForgetRepo_Call { | |
| 68 | + | _c.Call.Run(func(args mock.Arguments) { | |
| 69 | + | var arg0 domain.RepoName | |
| 70 | + | if args[0] != nil { | |
| 71 | + | arg0 = args[0].(domain.RepoName) | |
| 72 | + | } | |
| 73 | + | run( | |
| 74 | + | arg0, | |
| 75 | + | ) | |
| 76 | + | }) | |
| 77 | + | return _c | |
| 78 | + | } | |
| 79 | + | ||
| 80 | + | func (_c *CIRunCleaner_ForgetRepo_Call) Return() *CIRunCleaner_ForgetRepo_Call { | |
| 81 | + | _c.Call.Return() | |
| 82 | + | return _c | |
| 83 | + | } | |
| 84 | + | ||
| 85 | + | func (_c *CIRunCleaner_ForgetRepo_Call) RunAndReturn(run func(repo domain.RepoName)) *CIRunCleaner_ForgetRepo_Call { | |
| 86 | + | _c.Run(run) | |
| 87 | + | return _c | |
| 88 | + | } | |
| 89 | + | ||
| 50 | 90 | // RemoveLogs provides a mock function for the type CIRunCleaner | |
| 51 | 91 | func (_mock *CIRunCleaner) RemoveLogs(ctx context.Context, runIDs []int64) error { | |
| 52 | 92 | ret := _mock.Called(ctx, runIDs) | |
| @@ -47,6 +47,46 @@ func (_m *CIRunCleaner) EXPECT() *CIRunCleaner_Expecter { | |||||
| 47 | return &CIRunCleaner_Expecter{mock: &_m.Mock} | 47 | return &CIRunCleaner_Expecter{mock: &_m.Mock} | ||
| 48 | } | 48 | } | ||
| 49 | 49 | ||||
| 50 | + | // ForgetRepo provides a mock function for the type CIRunCleaner | |||
| 51 | + | func (_mock *CIRunCleaner) ForgetRepo(repo domain.RepoName) { | |||
| 52 | + | _mock.Called(repo) | |||
| 53 | + | return | |||
| 54 | + | } | |||
| 55 | + | ||||
| 56 | + | // CIRunCleaner_ForgetRepo_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ForgetRepo' | |||
| 57 | + | type CIRunCleaner_ForgetRepo_Call struct { | |||
| 58 | + | *mock.Call | |||
| 59 | + | } | |||
| 60 | + | ||||
| 61 | + | // ForgetRepo is a helper method to define mock.On call | |||
| 62 | + | // - repo domain.RepoName | |||
| 63 | + | func (_e *CIRunCleaner_Expecter) ForgetRepo(repo any) *CIRunCleaner_ForgetRepo_Call { | |||
| 64 | + | return &CIRunCleaner_ForgetRepo_Call{Call: _e.mock.On("ForgetRepo", repo)} | |||
| 65 | + | } | |||
| 66 | + | ||||
| 67 | + | func (_c *CIRunCleaner_ForgetRepo_Call) Run(run func(repo domain.RepoName)) *CIRunCleaner_ForgetRepo_Call { | |||
| 68 | + | _c.Call.Run(func(args mock.Arguments) { | |||
| 69 | + | var arg0 domain.RepoName | |||
| 70 | + | if args[0] != nil { | |||
| 71 | + | arg0 = args[0].(domain.RepoName) | |||
| 72 | + | } | |||
| 73 | + | run( | |||
| 74 | + | arg0, | |||
| 75 | + | ) | |||
| 76 | + | }) | |||
| 77 | + | return _c | |||
| 78 | + | } | |||
| 79 | + | ||||
| 80 | + | func (_c *CIRunCleaner_ForgetRepo_Call) Return() *CIRunCleaner_ForgetRepo_Call { | |||
| 81 | + | _c.Call.Return() | |||
| 82 | + | return _c | |||
| 83 | + | } | |||
| 84 | + | ||||
| 85 | + | func (_c *CIRunCleaner_ForgetRepo_Call) RunAndReturn(run func(repo domain.RepoName)) *CIRunCleaner_ForgetRepo_Call { | |||
| 86 | + | _c.Run(run) | |||
| 87 | + | return _c | |||
| 88 | + | } | |||
| 89 | + | ||||
| 50 | // RemoveLogs provides a mock function for the type CIRunCleaner | 90 | // RemoveLogs provides a mock function for the type CIRunCleaner | ||
| 51 | func (_mock *CIRunCleaner) RemoveLogs(ctx context.Context, runIDs []int64) error { | 91 | func (_mock *CIRunCleaner) RemoveLogs(ctx context.Context, runIDs []int64) error { | ||
| 52 | ret := _mock.Called(ctx, runIDs) | 92 | ret := _mock.Called(ctx, runIDs) | ||
M internal/ports/mocks/RunService.go +78 -0
| @@ -126,6 +126,84 @@ func (_c *RunService_CancelRun_Call) RunAndReturn(run func(ctx context.Context, | |||
| 126 | 126 | return _c | |
| 127 | 127 | } | |
| 128 | 128 | ||
| 129 | + | // DispatchWorkflow provides a mock function for the type RunService | |
| 130 | + | func (_mock *RunService) DispatchWorkflow(ctx context.Context, p domain.Principal, repo domain.RepoName, params ports.DispatchParams) (domain.CIRun, error) { | |
| 131 | + | ret := _mock.Called(ctx, p, repo, params) | |
| 132 | + | ||
| 133 | + | if len(ret) == 0 { | |
| 134 | + | panic("no return value specified for DispatchWorkflow") | |
| 135 | + | } | |
| 136 | + | ||
| 137 | + | var r0 domain.CIRun | |
| 138 | + | var r1 error | |
| 139 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.Principal, domain.RepoName, ports.DispatchParams) (domain.CIRun, error)); ok { | |
| 140 | + | return returnFunc(ctx, p, repo, params) | |
| 141 | + | } | |
| 142 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.Principal, domain.RepoName, ports.DispatchParams) domain.CIRun); ok { | |
| 143 | + | r0 = returnFunc(ctx, p, repo, params) | |
| 144 | + | } else { | |
| 145 | + | r0 = ret.Get(0).(domain.CIRun) | |
| 146 | + | } | |
| 147 | + | if returnFunc, ok := ret.Get(1).(func(context.Context, domain.Principal, domain.RepoName, ports.DispatchParams) error); ok { | |
| 148 | + | r1 = returnFunc(ctx, p, repo, params) | |
| 149 | + | } else { | |
| 150 | + | r1 = ret.Error(1) | |
| 151 | + | } | |
| 152 | + | return r0, r1 | |
| 153 | + | } | |
| 154 | + | ||
| 155 | + | // RunService_DispatchWorkflow_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'DispatchWorkflow' | |
| 156 | + | type RunService_DispatchWorkflow_Call struct { | |
| 157 | + | *mock.Call | |
| 158 | + | } | |
| 159 | + | ||
| 160 | + | // DispatchWorkflow is a helper method to define mock.On call | |
| 161 | + | // - ctx context.Context | |
| 162 | + | // - p domain.Principal | |
| 163 | + | // - repo domain.RepoName | |
| 164 | + | // - params ports.DispatchParams | |
| 165 | + | func (_e *RunService_Expecter) DispatchWorkflow(ctx any, p any, repo any, params any) *RunService_DispatchWorkflow_Call { | |
| 166 | + | return &RunService_DispatchWorkflow_Call{Call: _e.mock.On("DispatchWorkflow", ctx, p, repo, params)} | |
| 167 | + | } | |
| 168 | + | ||
| 169 | + | func (_c *RunService_DispatchWorkflow_Call) Run(run func(ctx context.Context, p domain.Principal, repo domain.RepoName, params ports.DispatchParams)) *RunService_DispatchWorkflow_Call { | |
| 170 | + | _c.Call.Run(func(args mock.Arguments) { | |
| 171 | + | var arg0 context.Context | |
| 172 | + | if args[0] != nil { | |
| 173 | + | arg0 = args[0].(context.Context) | |
| 174 | + | } | |
| 175 | + | var arg1 domain.Principal | |
| 176 | + | if args[1] != nil { | |
| 177 | + | arg1 = args[1].(domain.Principal) | |
| 178 | + | } | |
| 179 | + | var arg2 domain.RepoName | |
| 180 | + | if args[2] != nil { | |
| 181 | + | arg2 = args[2].(domain.RepoName) | |
| 182 | + | } | |
| 183 | + | var arg3 ports.DispatchParams | |
| 184 | + | if args[3] != nil { | |
| 185 | + | arg3 = args[3].(ports.DispatchParams) | |
| 186 | + | } | |
| 187 | + | run( | |
| 188 | + | arg0, | |
| 189 | + | arg1, | |
| 190 | + | arg2, | |
| 191 | + | arg3, | |
| 192 | + | ) | |
| 193 | + | }) | |
| 194 | + | return _c | |
| 195 | + | } | |
| 196 | + | ||
| 197 | + | func (_c *RunService_DispatchWorkflow_Call) Return(cIRun domain.CIRun, err error) *RunService_DispatchWorkflow_Call { | |
| 198 | + | _c.Call.Return(cIRun, err) | |
| 199 | + | return _c | |
| 200 | + | } | |
| 201 | + | ||
| 202 | + | func (_c *RunService_DispatchWorkflow_Call) RunAndReturn(run func(ctx context.Context, p domain.Principal, repo domain.RepoName, params ports.DispatchParams) (domain.CIRun, error)) *RunService_DispatchWorkflow_Call { | |
| 203 | + | _c.Call.Return(run) | |
| 204 | + | return _c | |
| 205 | + | } | |
| 206 | + | ||
| 129 | 207 | // GetJob provides a mock function for the type RunService | |
| 130 | 208 | func (_mock *RunService) GetJob(ctx context.Context, p domain.Principal, repo domain.RepoName, jobID string, attempt int32) (ports.JobView, error) { | |
| 131 | 209 | ret := _mock.Called(ctx, p, repo, jobID, attempt) | |
| @@ -126,6 +126,84 @@ func (_c *RunService_CancelRun_Call) RunAndReturn(run func(ctx context.Context, | |||||
| 126 | return _c | 126 | return _c | ||
| 127 | } | 127 | } | ||
| 128 | 128 | ||||
| 129 | + | // DispatchWorkflow provides a mock function for the type RunService | |||
| 130 | + | func (_mock *RunService) DispatchWorkflow(ctx context.Context, p domain.Principal, repo domain.RepoName, params ports.DispatchParams) (domain.CIRun, error) { | |||
| 131 | + | ret := _mock.Called(ctx, p, repo, params) | |||
| 132 | + | ||||
| 133 | + | if len(ret) == 0 { | |||
| 134 | + | panic("no return value specified for DispatchWorkflow") | |||
| 135 | + | } | |||
| 136 | + | ||||
| 137 | + | var r0 domain.CIRun | |||
| 138 | + | var r1 error | |||
| 139 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.Principal, domain.RepoName, ports.DispatchParams) (domain.CIRun, error)); ok { | |||
| 140 | + | return returnFunc(ctx, p, repo, params) | |||
| 141 | + | } | |||
| 142 | + | if returnFunc, ok := ret.Get(0).(func(context.Context, domain.Principal, domain.RepoName, ports.DispatchParams) domain.CIRun); ok { | |||
| 143 | + | r0 = returnFunc(ctx, p, repo, params) | |||
| 144 | + | } else { | |||
| 145 | + | r0 = ret.Get(0).(domain.CIRun) | |||
| 146 | + | } | |||
| 147 | + | if returnFunc, ok := ret.Get(1).(func(context.Context, domain.Principal, domain.RepoName, ports.DispatchParams) error); ok { | |||
| 148 | + | r1 = returnFunc(ctx, p, repo, params) | |||
| 149 | + | } else { | |||
| 150 | + | r1 = ret.Error(1) | |||
| 151 | + | } | |||
| 152 | + | return r0, r1 | |||
| 153 | + | } | |||
| 154 | + | ||||
| 155 | + | // RunService_DispatchWorkflow_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'DispatchWorkflow' | |||
| 156 | + | type RunService_DispatchWorkflow_Call struct { | |||
| 157 | + | *mock.Call | |||
| 158 | + | } | |||
| 159 | + | ||||
| 160 | + | // DispatchWorkflow is a helper method to define mock.On call | |||
| 161 | + | // - ctx context.Context | |||
| 162 | + | // - p domain.Principal | |||
| 163 | + | // - repo domain.RepoName | |||
| 164 | + | // - params ports.DispatchParams | |||
| 165 | + | func (_e *RunService_Expecter) DispatchWorkflow(ctx any, p any, repo any, params any) *RunService_DispatchWorkflow_Call { | |||
| 166 | + | return &RunService_DispatchWorkflow_Call{Call: _e.mock.On("DispatchWorkflow", ctx, p, repo, params)} | |||
| 167 | + | } | |||
| 168 | + | ||||
| 169 | + | func (_c *RunService_DispatchWorkflow_Call) Run(run func(ctx context.Context, p domain.Principal, repo domain.RepoName, params ports.DispatchParams)) *RunService_DispatchWorkflow_Call { | |||
| 170 | + | _c.Call.Run(func(args mock.Arguments) { | |||
| 171 | + | var arg0 context.Context | |||
| 172 | + | if args[0] != nil { | |||
| 173 | + | arg0 = args[0].(context.Context) | |||
| 174 | + | } | |||
| 175 | + | var arg1 domain.Principal | |||
| 176 | + | if args[1] != nil { | |||
| 177 | + | arg1 = args[1].(domain.Principal) | |||
| 178 | + | } | |||
| 179 | + | var arg2 domain.RepoName | |||
| 180 | + | if args[2] != nil { | |||
| 181 | + | arg2 = args[2].(domain.RepoName) | |||
| 182 | + | } | |||
| 183 | + | var arg3 ports.DispatchParams | |||
| 184 | + | if args[3] != nil { | |||
| 185 | + | arg3 = args[3].(ports.DispatchParams) | |||
| 186 | + | } | |||
| 187 | + | run( | |||
| 188 | + | arg0, | |||
| 189 | + | arg1, | |||
| 190 | + | arg2, | |||
| 191 | + | arg3, | |||
| 192 | + | ) | |||
| 193 | + | }) | |||
| 194 | + | return _c | |||
| 195 | + | } | |||
| 196 | + | ||||
| 197 | + | func (_c *RunService_DispatchWorkflow_Call) Return(cIRun domain.CIRun, err error) *RunService_DispatchWorkflow_Call { | |||
| 198 | + | _c.Call.Return(cIRun, err) | |||
| 199 | + | return _c | |||
| 200 | + | } | |||
| 201 | + | ||||
| 202 | + | func (_c *RunService_DispatchWorkflow_Call) RunAndReturn(run func(ctx context.Context, p domain.Principal, repo domain.RepoName, params ports.DispatchParams) (domain.CIRun, error)) *RunService_DispatchWorkflow_Call { | |||
| 203 | + | _c.Call.Return(run) | |||
| 204 | + | return _c | |||
| 205 | + | } | |||
| 206 | + | ||||
| 129 | // GetJob provides a mock function for the type RunService | 207 | // GetJob provides a mock function for the type RunService | ||
| 130 | func (_mock *RunService) GetJob(ctx context.Context, p domain.Principal, repo domain.RepoName, jobID string, attempt int32) (ports.JobView, error) { | 208 | func (_mock *RunService) GetJob(ctx context.Context, p domain.Principal, repo domain.RepoName, jobID string, attempt int32) (ports.JobView, error) { | ||
| 131 | ret := _mock.Called(ctx, p, repo, jobID, attempt) | 209 | ret := _mock.Called(ctx, p, repo, jobID, attempt) | ||
M internal/usecase/ci.go +39 -3
| @@ -141,6 +141,27 @@ type CIConfig struct { | |||
| 141 | 141 | // transaction removes (default 256, far below the size limit of a kv batch); | |
| 142 | 142 | // the others go with the next decisions and the reconciliation. | |
| 143 | 143 | ConcurrencyPruneBatch int | |
| 144 | + | // ScheduleTick is how often the scheduler looks at the clock (default one | |
| 145 | + | // second) and ScheduleBudget the repositories it handles per tick (default | |
| 146 | + | // 64); see ci_schedule.go. | |
| 147 | + | ScheduleTick time.Duration | |
| 148 | + | ScheduleBudget int | |
| 149 | + | // RefTips holds who moves each branch and tag to its object, written before | |
| 150 | + | // the ref moves (Pushes, MergeRequests), for the actor and trust of scheduled | |
| 151 | + | // runs and the trust of dispatched ones (ci_tips.go). Nil: they are never | |
| 152 | + | // trusted. Leave it unset rather than assigning a typed nil pointer. | |
| 153 | + | RefTips ports.CIRefTipStore | |
| 154 | + | // DispatchResolved, when set, is called by DispatchWorkflow after it resolved | |
| 155 | + | // its ref and before it judges the trust of the run. Tests use it to move the | |
| 156 | + | // ref in between. | |
| 157 | + | DispatchResolved func() | |
| 158 | + | // RunEventHandled, when set, is called after the workflows waiting for a | |
| 159 | + | // "requested" or "completed" event of a run were started (or none matched). | |
| 160 | + | // Tests use it as a barrier. | |
| 161 | + | RunEventHandled func(runID int64, action string) | |
| 162 | + | // ScheduleVisit, when set, is called with each repository the scheduler | |
| 163 | + | // looks at. Tests use it. | |
| 164 | + | ScheduleVisit func(repo domain.RepoName) | |
| 144 | 165 | } | |
| 145 | 166 | ||
| 146 | 167 | func (c CIConfig) withDefaults() CIConfig { | |
| @@ -174,6 +195,10 @@ func (c CIConfig) withDefaults() CIConfig { | |||
| 174 | 195 | if c.ConcurrencyPruneBatch <= 0 { | |
| 175 | 196 | c.ConcurrencyPruneBatch = DefaultCIConcurrencyPruneBatch | |
| 176 | 197 | } | |
| 198 | + | def(&c.ScheduleTick, DefaultCIScheduleTick) | |
| 199 | + | if c.ScheduleBudget <= 0 { | |
| 200 | + | c.ScheduleBudget = DefaultCIScheduleBudget | |
| 201 | + | } | |
| 177 | 202 | return c | |
| 178 | 203 | } | |
| 179 | 204 | ||
| @@ -188,7 +213,11 @@ type CI struct { | |||
| 188 | 213 | cfg CIConfig | |
| 189 | 214 | boot time.Time | |
| 190 | 215 | queue chan ciTrigger | |
| 191 | - | pull *pullQueue // the work of the merge requests (ci_pull.go) | |
| 216 | + | // runEvents are the requested and completed events of runs, for the | |
| 217 | + | // workflow_run workflows (ci_workflowrun.go). | |
| 218 | + | runEvents chan runEvent | |
| 219 | + | sched *scheduler // the schedule trigger (ci_schedule.go) | |
| 220 | + | pull *pullQueue // the work of the merge requests (ci_pull.go) | |
| 192 | 221 | ||
| 193 | 222 | pullFileLists pullFilesCache // the files of the merge requests (ci_pullfiles.go) | |
| 194 | 223 | ||
| @@ -243,7 +272,7 @@ func NewCI(ctx context.Context, cfg CIConfig) *CI { | |||
| 243 | 272 | cfg = cfg.withDefaults() | |
| 244 | 273 | run, stop := context.WithCancel(context.WithoutCancel(ctx)) | |
| 245 | 274 | return &CI{ | |
| 246 | - | cfg: cfg, boot: cfg.Clock.Now(), queue: make(chan ciTrigger, cfg.TriggerQueue), pull: newPullQueue(cfg.TriggerQueue, cfg.PullRefreshes, cfg.PullAttempts), | |
| 275 | + | cfg: cfg, boot: cfg.Clock.Now(), queue: make(chan ciTrigger, cfg.TriggerQueue), runEvents: make(chan runEvent, cfg.TriggerQueue), sched: newScheduler(cfg.Clock.Now()), pull: newPullQueue(cfg.TriggerQueue, cfg.PullRefreshes, cfg.PullAttempts), | |
| 247 | 276 | ctx: run, stop: stop, flows: newWfCache(64), remoteSlots: make(chan struct{}, cfg.RemoteConcurrency), | |
| 248 | 277 | heartbeats: map[string]time.Time{}, seen: map[string]time.Time{}, polls: map[string]pollRecord{}, | |
| 249 | 278 | archiveTokens: map[string]map[string]string{}, | |
| @@ -254,7 +283,9 @@ func NewCI(ctx context.Context, cfg CIConfig) *CI { | |||
| 254 | 283 | // Start runs the trigger worker and the watchdog, and plans again the runs | |
| 255 | 284 | // that were open when the server stopped. | |
| 256 | 285 | func (c *CI) Start() { | |
| 257 | - | c.wg.Add(4) | |
| 286 | + | c.wg.Add(6) | |
| 287 | + | go c.scheduleLoop() | |
| 288 | + | go c.runEventLoop() | |
| 258 | 289 | go c.pullLoop() | |
| 259 | 290 | go c.refreshLoop() | |
| 260 | 291 | go c.triggerLoop() | |
| @@ -426,6 +457,7 @@ func (rs *runState) flush(ctx context.Context) error { | |||
| 426 | 457 | now := c.now() | |
| 427 | 458 | var w ports.CIWrite | |
| 428 | 459 | var changedJobs []int | |
| 460 | + | completing := false | |
| 429 | 461 | for i := range rs.jobs { | |
| 430 | 462 | if rs.dirty[rs.jobs[i].ID] { | |
| 431 | 463 | w.Jobs = append(w.Jobs, rs.jobs[i]) | |
| @@ -439,6 +471,7 @@ func (rs *runState) flush(ctx context.Context) error { | |||
| 439 | 471 | } | |
| 440 | 472 | } | |
| 441 | 473 | if next := rs.recompute(now); next != rs.run { | |
| 474 | + | completing = rs.run.Status != domain.CICompleted && next.Status == domain.CICompleted | |
| 442 | 475 | rs.run, rs.runDirty = next, true | |
| 443 | 476 | } | |
| 444 | 477 | } | |
| @@ -486,6 +519,9 @@ func (rs *runState) flush(ctx context.Context) error { | |||
| 486 | 519 | if changedRun { | |
| 487 | 520 | c.cfg.Bus.Publish(ctx, domain.RunUpdated{Run: rs.run, At: now, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt}) | |
| 488 | 521 | } | |
| 522 | + | if completing { | |
| 523 | + | c.onRunEvent(ctx, rs.run, runCompleted) | |
| 524 | + | } | |
| 489 | 525 | return nil | |
| 490 | 526 | } | |
| 491 | 527 | ||
| @@ -141,6 +141,27 @@ type CIConfig struct { | |||||
| 141 | // transaction removes (default 256, far below the size limit of a kv batch); | 141 | // transaction removes (default 256, far below the size limit of a kv batch); | ||
| 142 | // the others go with the next decisions and the reconciliation. | 142 | // the others go with the next decisions and the reconciliation. | ||
| 143 | ConcurrencyPruneBatch int | 143 | ConcurrencyPruneBatch int | ||
| 144 | + | // ScheduleTick is how often the scheduler looks at the clock (default one | |||
| 145 | + | // second) and ScheduleBudget the repositories it handles per tick (default | |||
| 146 | + | // 64); see ci_schedule.go. | |||
| 147 | + | ScheduleTick time.Duration | |||
| 148 | + | ScheduleBudget int | |||
| 149 | + | // RefTips holds who moves each branch and tag to its object, written before | |||
| 150 | + | // the ref moves (Pushes, MergeRequests), for the actor and trust of scheduled | |||
| 151 | + | // runs and the trust of dispatched ones (ci_tips.go). Nil: they are never | |||
| 152 | + | // trusted. Leave it unset rather than assigning a typed nil pointer. | |||
| 153 | + | RefTips ports.CIRefTipStore | |||
| 154 | + | // DispatchResolved, when set, is called by DispatchWorkflow after it resolved | |||
| 155 | + | // its ref and before it judges the trust of the run. Tests use it to move the | |||
| 156 | + | // ref in between. | |||
| 157 | + | DispatchResolved func() | |||
| 158 | + | // RunEventHandled, when set, is called after the workflows waiting for a | |||
| 159 | + | // "requested" or "completed" event of a run were started (or none matched). | |||
| 160 | + | // Tests use it as a barrier. | |||
| 161 | + | RunEventHandled func(runID int64, action string) | |||
| 162 | + | // ScheduleVisit, when set, is called with each repository the scheduler | |||
| 163 | + | // looks at. Tests use it. | |||
| 164 | + | ScheduleVisit func(repo domain.RepoName) | |||
| 144 | } | 165 | } | ||
| 145 | 166 | ||||
| 146 | func (c CIConfig) withDefaults() CIConfig { | 167 | func (c CIConfig) withDefaults() CIConfig { | ||
| @@ -174,6 +195,10 @@ func (c CIConfig) withDefaults() CIConfig { | |||||
| 174 | if c.ConcurrencyPruneBatch <= 0 { | 195 | if c.ConcurrencyPruneBatch <= 0 { | ||
| 175 | c.ConcurrencyPruneBatch = DefaultCIConcurrencyPruneBatch | 196 | c.ConcurrencyPruneBatch = DefaultCIConcurrencyPruneBatch | ||
| 176 | } | 197 | } | ||
| 198 | + | def(&c.ScheduleTick, DefaultCIScheduleTick) | |||
| 199 | + | if c.ScheduleBudget <= 0 { | |||
| 200 | + | c.ScheduleBudget = DefaultCIScheduleBudget | |||
| 201 | + | } | |||
| 177 | return c | 202 | return c | ||
| 178 | } | 203 | } | ||
| 179 | 204 | ||||
| @@ -188,7 +213,11 @@ type CI struct { | |||||
| 188 | cfg CIConfig | 213 | cfg CIConfig | ||
| 189 | boot time.Time | 214 | boot time.Time | ||
| 190 | queue chan ciTrigger | 215 | queue chan ciTrigger | ||
| 191 | - | pull *pullQueue // the work of the merge requests (ci_pull.go) | 216 | + | // runEvents are the requested and completed events of runs, for the |
| 217 | + | // workflow_run workflows (ci_workflowrun.go). | |||
| 218 | + | runEvents chan runEvent | |||
| 219 | + | sched *scheduler // the schedule trigger (ci_schedule.go) | |||
| 220 | + | pull *pullQueue // the work of the merge requests (ci_pull.go) | |||
| 192 | 221 | ||||
| 193 | pullFileLists pullFilesCache // the files of the merge requests (ci_pullfiles.go) | 222 | pullFileLists pullFilesCache // the files of the merge requests (ci_pullfiles.go) | ||
| 194 | 223 | ||||
| @@ -243,7 +272,7 @@ func NewCI(ctx context.Context, cfg CIConfig) *CI { | |||||
| 243 | cfg = cfg.withDefaults() | 272 | cfg = cfg.withDefaults() | ||
| 244 | run, stop := context.WithCancel(context.WithoutCancel(ctx)) | 273 | run, stop := context.WithCancel(context.WithoutCancel(ctx)) | ||
| 245 | return &CI{ | 274 | return &CI{ | ||
| 246 | - | cfg: cfg, boot: cfg.Clock.Now(), queue: make(chan ciTrigger, cfg.TriggerQueue), pull: newPullQueue(cfg.TriggerQueue, cfg.PullRefreshes, cfg.PullAttempts), | 275 | + | cfg: cfg, boot: cfg.Clock.Now(), queue: make(chan ciTrigger, cfg.TriggerQueue), runEvents: make(chan runEvent, cfg.TriggerQueue), sched: newScheduler(cfg.Clock.Now()), pull: newPullQueue(cfg.TriggerQueue, cfg.PullRefreshes, cfg.PullAttempts), |
| 247 | ctx: run, stop: stop, flows: newWfCache(64), remoteSlots: make(chan struct{}, cfg.RemoteConcurrency), | 276 | ctx: run, stop: stop, flows: newWfCache(64), remoteSlots: make(chan struct{}, cfg.RemoteConcurrency), | ||
| 248 | heartbeats: map[string]time.Time{}, seen: map[string]time.Time{}, polls: map[string]pollRecord{}, | 277 | heartbeats: map[string]time.Time{}, seen: map[string]time.Time{}, polls: map[string]pollRecord{}, | ||
| 249 | archiveTokens: map[string]map[string]string{}, | 278 | archiveTokens: map[string]map[string]string{}, | ||
| @@ -254,7 +283,9 @@ func NewCI(ctx context.Context, cfg CIConfig) *CI { | |||||
| 254 | // Start runs the trigger worker and the watchdog, and plans again the runs | 283 | // Start runs the trigger worker and the watchdog, and plans again the runs | ||
| 255 | // that were open when the server stopped. | 284 | // that were open when the server stopped. | ||
| 256 | func (c *CI) Start() { | 285 | func (c *CI) Start() { | ||
| 257 | - | c.wg.Add(4) | 286 | + | c.wg.Add(6) |
| 287 | + | go c.scheduleLoop() | |||
| 288 | + | go c.runEventLoop() | |||
| 258 | go c.pullLoop() | 289 | go c.pullLoop() | ||
| 259 | go c.refreshLoop() | 290 | go c.refreshLoop() | ||
| 260 | go c.triggerLoop() | 291 | go c.triggerLoop() | ||
| @@ -426,6 +457,7 @@ func (rs *runState) flush(ctx context.Context) error { | |||||
| 426 | now := c.now() | 457 | now := c.now() | ||
| 427 | var w ports.CIWrite | 458 | var w ports.CIWrite | ||
| 428 | var changedJobs []int | 459 | var changedJobs []int | ||
| 460 | + | completing := false | |||
| 429 | for i := range rs.jobs { | 461 | for i := range rs.jobs { | ||
| 430 | if rs.dirty[rs.jobs[i].ID] { | 462 | if rs.dirty[rs.jobs[i].ID] { | ||
| 431 | w.Jobs = append(w.Jobs, rs.jobs[i]) | 463 | w.Jobs = append(w.Jobs, rs.jobs[i]) | ||
| @@ -439,6 +471,7 @@ func (rs *runState) flush(ctx context.Context) error { | |||||
| 439 | } | 471 | } | ||
| 440 | } | 472 | } | ||
| 441 | if next := rs.recompute(now); next != rs.run { | 473 | if next := rs.recompute(now); next != rs.run { | ||
| 474 | + | completing = rs.run.Status != domain.CICompleted && next.Status == domain.CICompleted | |||
| 442 | rs.run, rs.runDirty = next, true | 475 | rs.run, rs.runDirty = next, true | ||
| 443 | } | 476 | } | ||
| 444 | } | 477 | } | ||
| @@ -486,6 +519,9 @@ func (rs *runState) flush(ctx context.Context) error { | |||||
| 486 | if changedRun { | 519 | if changedRun { | ||
| 487 | c.cfg.Bus.Publish(ctx, domain.RunUpdated{Run: rs.run, At: now, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt}) | 520 | c.cfg.Bus.Publish(ctx, domain.RunUpdated{Run: rs.run, At: now, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt}) | ||
| 488 | } | 521 | } | ||
| 522 | + | if completing { | |||
| 523 | + | c.onRunEvent(ctx, rs.run, runCompleted) | |||
| 524 | + | } | |||
| 489 | return nil | 525 | return nil | ||
| 490 | } | 526 | } | ||
| 491 | 527 | ||||
M internal/usecase/ci_concurrency.go +1 -1
| @@ -616,7 +616,7 @@ func (rs *runState) evalRunConcurrency(ctx context.Context, wf *workflow.Workflo | |||
| 616 | 616 | if err != nil { | |
| 617 | 617 | return domain.CIConcurrency{}, err | |
| 618 | 618 | } | |
| 619 | - | env := template.Env{Contexts: map[string]expr.Value{"github": gh, "inputs": expr.NewDictionary(), "vars": vars}} | |
| 619 | + | env := template.Env{Contexts: map[string]expr.Value{"github": gh, "inputs": rs.inputsOf(domain.CIJob{}), "vars": vars}} | |
| 620 | 620 | return evalConcurrency(ctx, wf.Concurrency, env) | |
| 621 | 621 | } | |
| 622 | 622 | ||
| @@ -616,7 +616,7 @@ func (rs *runState) evalRunConcurrency(ctx context.Context, wf *workflow.Workflo | |||||
| 616 | if err != nil { | 616 | if err != nil { | ||
| 617 | return domain.CIConcurrency{}, err | 617 | return domain.CIConcurrency{}, err | ||
| 618 | } | 618 | } | ||
| 619 | - | env := template.Env{Contexts: map[string]expr.Value{"github": gh, "inputs": expr.NewDictionary(), "vars": vars}} | 619 | + | env := template.Env{Contexts: map[string]expr.Value{"github": gh, "inputs": rs.inputsOf(domain.CIJob{}), "vars": vars}} |
| 620 | return evalConcurrency(ctx, wf.Concurrency, env) | 620 | return evalConcurrency(ctx, wf.Concurrency, env) | ||
| 621 | } | 621 | } | ||
| 622 | 622 | ||||
A internal/usecase/ci_dispatch.go +232 -0
| @@ -0,0 +1,232 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package usecase | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "errors" | |
| 8 | + | "fmt" | |
| 9 | + | "io" | |
| 10 | + | "path" | |
| 11 | + | "strings" | |
| 12 | + | ||
| 13 | + | "bnema.dev/ermite-actions/workflow" | |
| 14 | + | "bnema.dev/ermite/internal/domain" | |
| 15 | + | "bnema.dev/ermite/internal/ports" | |
| 16 | + | ) | |
| 17 | + | ||
| 18 | + | // jobTokenActor is the actor of a run dispatched with a job token, GitHub's name | |
| 19 | + | // for the account behind a GITHUB_TOKEN. | |
| 20 | + | const jobTokenActor = "github-actions[bot]" | |
| 21 | + | ||
| 22 | + | // DispatchWorkflow starts a run of a workflow file by hand (workflow_dispatch, | |
| 23 | + | // CI-3 D5), like `gh workflow run`. | |
| 24 | + | // | |
| 25 | + | // Who may: a principal with ci:write on the repository, or the holder of a job | |
| 26 | + | // token whose permissions include "actions: write" (GitHub's one exception to | |
| 27 | + | // "events made with GITHUB_TOKEN start no run"; a token of a run that is not | |
| 28 | + | // trusted never has it, TokenPermissions lowers it, so a job token that gets | |
| 29 | + | // here belongs to a trusted run). A token limited to branch prefixes may only | |
| 30 | + | // dispatch on a branch under them, and its run is not trusted (D1). Whoever | |
| 31 | + | // calls, the run is trusted only when the ref it runs on was moved to its commit | |
| 32 | + | // by a trusted principal (ci_tips.go). | |
| 33 | + | // | |
| 34 | + | // The workflow is read at ref (the default branch when empty) and must declare | |
| 35 | + | // workflow_dispatch; the inputs are checked against its declaration. | |
| 36 | + | func (c *CI) DispatchWorkflow(ctx context.Context, p domain.Principal, name domain.RepoName, params ports.DispatchParams) (domain.CIRun, error) { | |
| 37 | + | repo, actor, pusher, trusted, err := c.dispatchCaller(ctx, p, name) | |
| 38 | + | if err != nil { | |
| 39 | + | return domain.CIRun{}, err | |
| 40 | + | } | |
| 41 | + | wfPath, err := dispatchPath(params.WorkflowPath) | |
| 42 | + | if err != nil { | |
| 43 | + | return domain.CIRun{}, err | |
| 44 | + | } | |
| 45 | + | pin, err := c.dispatchRef(ctx, name, params.Ref) | |
| 46 | + | if err != nil { | |
| 47 | + | return domain.CIRun{}, err | |
| 48 | + | } | |
| 49 | + | ref, commit := pin.ref, pin.commit | |
| 50 | + | if c.cfg.DispatchResolved != nil { | |
| 51 | + | c.cfg.DispatchResolved() | |
| 52 | + | } | |
| 53 | + | if !p.AllowsRef(ref) { | |
| 54 | + | return domain.CIRun{}, fmt.Errorf("%w: this token may not write %s", domain.ErrForbidden, ref) | |
| 55 | + | } | |
| 56 | + | content, err := c.readWorkflowAt(ctx, name, commit, wfPath) | |
| 57 | + | if err != nil { | |
| 58 | + | return domain.CIRun{}, err | |
| 59 | + | } | |
| 60 | + | wf, perrs := workflow.Parse(wfPath, content) | |
| 61 | + | if len(perrs) > 0 { | |
| 62 | + | lines := make([]string, 0, len(perrs)) | |
| 63 | + | for _, e := range perrs { | |
| 64 | + | lines = append(lines, e.Error()) | |
| 65 | + | } | |
| 66 | + | return domain.CIRun{}, fmt.Errorf("%w: %s is not a valid workflow:\n%s", domain.ErrInvalid, wfPath, strings.Join(lines, "\n")) | |
| 67 | + | } | |
| 68 | + | ev := wf.On.Event(eventDispatch) | |
| 69 | + | if ev == nil { | |
| 70 | + | return domain.CIRun{}, fmt.Errorf("%w: %s does not have a %s trigger", domain.ErrPreconditionFailed, wfPath, eventDispatch) | |
| 71 | + | } | |
| 72 | + | values, err := resolveDispatchInputs(ev.Inputs, params.Inputs) | |
| 73 | + | if err != nil { | |
| 74 | + | return domain.CIRun{}, err | |
| 75 | + | } | |
| 76 | + | ||
| 77 | + | f := pushFacts{ | |
| 78 | + | update: domain.RefUpdate{Ref: ref, Old: domain.ZeroOID, New: commit.String()}, commit: commit, | |
| 79 | + | custom: &customEvent{name: eventDispatch, title: workflowName(wf, wfPath), inputs: values}, | |
| 80 | + | } | |
| 81 | + | if def, _, err := c.cfg.Repos.DefaultBranch(ctx, name); err == nil { | |
| 82 | + | f.defaultBranch = def | |
| 83 | + | } | |
| 84 | + | // The run executes the code of the ref, which the caller did not necessarily | |
| 85 | + | // write: it is trusted only if the caller is AND whoever moved the ref to this | |
| 86 | + | // commit was (domain.CIRefTip), as a re-run of an agent's run by the owner | |
| 87 | + | // does not unlock secrets. A ref with no record is not trusted. This holds for | |
| 88 | + | // every caller, the owner and a job token alike. | |
| 89 | + | trusted = trusted && c.refTrusted(ctx, name, pin) | |
| 90 | + | t := ciTrigger{repo: name, actor: actor, pusher: pusher, trusted: trusted, at: c.now()} | |
| 91 | + | if problem := c.matrixProblem(ctx, repo, t, f, wfPath, wf); problem.text != "" { | |
| 92 | + | return domain.CIRun{}, fmt.Errorf("%w: %s", domain.ErrInvalid, problem.text) | |
| 93 | + | } | |
| 94 | + | run, err := c.launchRun(ctx, repo, t, f, wfPath, string(content), wf) | |
| 95 | + | if err != nil { | |
| 96 | + | return domain.CIRun{}, err | |
| 97 | + | } | |
| 98 | + | if run.ID == 0 { | |
| 99 | + | return domain.CIRun{}, domain.ErrNotFound | |
| 100 | + | } | |
| 101 | + | ctxLog(ctx).Info().Str("repo", name.String()).Int64("run_id", run.ID).Str("workflow", wfPath).Str("ref", ref). | |
| 102 | + | Str("principal", p.Label()).Bool("trusted", trusted).Msg("ci workflow dispatched") | |
| 103 | + | return run, nil | |
| 104 | + | } | |
| 105 | + | ||
| 106 | + | // dispatchCaller applies the access rule of a dispatch and returns who the run | |
| 107 | + | // is for. | |
| 108 | + | func (c *CI) dispatchCaller(ctx context.Context, p domain.Principal, name domain.RepoName) (repo domain.Repo, actor string, pusher domain.CIPusher, trusted bool, err error) { | |
| 109 | + | if p.ViaJobToken() { | |
| 110 | + | repo, err = lookupRepo(ctx, c.cfg.Meta, name) | |
| 111 | + | if err != nil { | |
| 112 | + | return repo, "", pusher, false, err | |
| 113 | + | } | |
| 114 | + | // The permission is judged on its own: a job of this repository with | |
| 115 | + | // actions: write may dispatch even when its contents permission is none | |
| 116 | + | // (a private repository is then unreadable to it). JobPermits is true only | |
| 117 | + | // for the job's own token on its own repository, so any other caller of | |
| 118 | + | // a repository it cannot read still gets not found. | |
| 119 | + | switch { | |
| 120 | + | case p.JobPermits("actions", domain.PermissionWrite, name): | |
| 121 | + | return repo, jobTokenActor, domain.PusherOf(p), true, nil | |
| 122 | + | case !canRead(p, repo): | |
| 123 | + | return repo, "", pusher, false, domain.ErrNotFound | |
| 124 | + | } | |
| 125 | + | return repo, "", pusher, false, fmt.Errorf("%w: a job token needs the permission actions: write to dispatch a workflow", domain.ErrForbidden) | |
| 126 | + | } | |
| 127 | + | repo, err = c.writableRepo(ctx, p, name) | |
| 128 | + | if err != nil { | |
| 129 | + | return repo, "", pusher, false, err | |
| 130 | + | } | |
| 131 | + | return repo, authorOf(p), domain.PusherOf(p), domain.TrustedActor(p), nil | |
| 132 | + | } | |
| 133 | + | ||
| 134 | + | // dispatchPath checks the workflow file of a dispatch: a file of | |
| 135 | + | // .github/workflows, named by its path or by its file name. | |
| 136 | + | func dispatchPath(s string) (string, error) { | |
| 137 | + | s = strings.TrimSpace(s) | |
| 138 | + | if s == "" { | |
| 139 | + | return "", fmt.Errorf("%w: the workflow file is required", domain.ErrInvalid) | |
| 140 | + | } | |
| 141 | + | if !strings.Contains(s, "/") { | |
| 142 | + | s = workflowDir + "/" + s | |
| 143 | + | } | |
| 144 | + | ext := strings.ToLower(path.Ext(s)) | |
| 145 | + | if path.Dir(s) != workflowDir || (ext != ".yml" && ext != ".yaml") { | |
| 146 | + | return "", fmt.Errorf("%w: %q is not a workflow file of %s", domain.ErrInvalid, s, workflowDir) | |
| 147 | + | } | |
| 148 | + | return s, nil | |
| 149 | + | } | |
| 150 | + | ||
| 151 | + | // pinnedRef is what a dispatch resolved its ref to, once: the full name, the | |
| 152 | + | // commit the run executes and the object the ref points to (the commit for a | |
| 153 | + | // branch and a lightweight tag, the tag object for an annotated tag). The trust | |
| 154 | + | // of the run is judged on exactly these, never on the ref as it is read later | |
| 155 | + | // (ci_tips.go). | |
| 156 | + | type pinnedRef struct { | |
| 157 | + | ref string | |
| 158 | + | commit, object domain.ObjectID | |
| 159 | + | } | |
| 160 | + | ||
| 161 | + | // dispatchRef resolves the ref of a dispatch: a branch or a tag, by short or | |
| 162 | + | // full name; the default branch when empty. A tag is resolved to its commit and | |
| 163 | + | // its own object together; a tag that moves between the two reads is refused, | |
| 164 | + | // so the commit and the object always belong to one state of the ref. | |
| 165 | + | func (c *CI) dispatchRef(ctx context.Context, name domain.RepoName, given string) (pinnedRef, error) { | |
| 166 | + | given = strings.TrimSpace(given) | |
| 167 | + | if given == "" { | |
| 168 | + | def, exists, err := c.cfg.Repos.DefaultBranch(ctx, name) | |
| 169 | + | if err != nil { | |
| 170 | + | return pinnedRef{}, err | |
| 171 | + | } | |
| 172 | + | if !exists { | |
| 173 | + | return pinnedRef{}, fmt.Errorf("%w: the default branch %q does not exist", domain.ErrNotFound, def) | |
| 174 | + | } | |
| 175 | + | given = "refs/heads/" + def | |
| 176 | + | } | |
| 177 | + | rev, err := domain.ParseRevision(given) | |
| 178 | + | if err != nil { | |
| 179 | + | return pinnedRef{}, err | |
| 180 | + | } | |
| 181 | + | res, err := c.cfg.Reader.ResolveRevision(ctx, name, rev) | |
| 182 | + | if err != nil { | |
| 183 | + | return pinnedRef{}, err | |
| 184 | + | } | |
| 185 | + | pin := pinnedRef{ref: res.Ref, commit: res.CommitID, object: res.CommitID} | |
| 186 | + | switch { | |
| 187 | + | case strings.HasPrefix(res.Ref, "refs/heads/"): | |
| 188 | + | return pin, nil | |
| 189 | + | case !strings.HasPrefix(res.Ref, "refs/tags/"): | |
| 190 | + | return pinnedRef{}, fmt.Errorf("%w: %q is not a branch or a tag", domain.ErrInvalid, given) | |
| 191 | + | } | |
| 192 | + | refs, err := c.cfg.Reader.ListRefs(ctx, name) | |
| 193 | + | if err != nil { | |
| 194 | + | return pinnedRef{}, err | |
| 195 | + | } | |
| 196 | + | for _, r := range refs { | |
| 197 | + | if r.Name != res.Ref { | |
| 198 | + | continue | |
| 199 | + | } | |
| 200 | + | if r.CommitID != res.CommitID || r.ID.IsZero() { | |
| 201 | + | break // moved between the two reads | |
| 202 | + | } | |
| 203 | + | pin.object = r.ID | |
| 204 | + | return pin, nil | |
| 205 | + | } | |
| 206 | + | return pinnedRef{}, fmt.Errorf("%w: tag %q moved while it was resolved, try again", domain.ErrConflict, res.Ref) | |
| 207 | + | } | |
| 208 | + | ||
| 209 | + | // readWorkflowAt reads a workflow file at a commit, bounded; a missing file is | |
| 210 | + | // domain.ErrNotFound. | |
| 211 | + | func (c *CI) readWorkflowAt(ctx context.Context, repo domain.RepoName, commit domain.ObjectID, wfPath string) ([]byte, error) { | |
| 212 | + | tp, err := domain.ParseTreePath(wfPath) | |
| 213 | + | if err != nil { | |
| 214 | + | return nil, err | |
| 215 | + | } | |
| 216 | + | _, rc, err := c.cfg.Reader.OpenBlob(ctx, repo, domain.RevisionOf(commit), tp) | |
| 217 | + | if err != nil { | |
| 218 | + | if errors.Is(err, domain.ErrNotFound) || errors.Is(err, domain.ErrNotFile) { | |
| 219 | + | return nil, fmt.Errorf("%w: workflow %s", domain.ErrNotFound, wfPath) | |
| 220 | + | } | |
| 221 | + | return nil, fmt.Errorf("read %s: %w", wfPath, err) | |
| 222 | + | } | |
| 223 | + | content, err := io.ReadAll(io.LimitReader(rc, maxCIWorkflowBytes+1)) | |
| 224 | + | _ = rc.Close() | |
| 225 | + | if err != nil { | |
| 226 | + | return nil, fmt.Errorf("read %s: %w", wfPath, err) | |
| 227 | + | } | |
| 228 | + | if len(content) > maxCIWorkflowBytes { | |
| 229 | + | return nil, fmt.Errorf("%w: %s is larger than 1 MiB", domain.ErrInvalid, wfPath) | |
| 230 | + | } | |
| 231 | + | return content, nil | |
| 232 | + | } | |
| @@ -0,0 +1,232 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package usecase | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "context" | |||
| 7 | + | "errors" | |||
| 8 | + | "fmt" | |||
| 9 | + | "io" | |||
| 10 | + | "path" | |||
| 11 | + | "strings" | |||
| 12 | + | ||||
| 13 | + | "bnema.dev/ermite-actions/workflow" | |||
| 14 | + | "bnema.dev/ermite/internal/domain" | |||
| 15 | + | "bnema.dev/ermite/internal/ports" | |||
| 16 | + | ) | |||
| 17 | + | ||||
| 18 | + | // jobTokenActor is the actor of a run dispatched with a job token, GitHub's name | |||
| 19 | + | // for the account behind a GITHUB_TOKEN. | |||
| 20 | + | const jobTokenActor = "github-actions[bot]" | |||
| 21 | + | ||||
| 22 | + | // DispatchWorkflow starts a run of a workflow file by hand (workflow_dispatch, | |||
| 23 | + | // CI-3 D5), like `gh workflow run`. | |||
| 24 | + | // | |||
| 25 | + | // Who may: a principal with ci:write on the repository, or the holder of a job | |||
| 26 | + | // token whose permissions include "actions: write" (GitHub's one exception to | |||
| 27 | + | // "events made with GITHUB_TOKEN start no run"; a token of a run that is not | |||
| 28 | + | // trusted never has it, TokenPermissions lowers it, so a job token that gets | |||
| 29 | + | // here belongs to a trusted run). A token limited to branch prefixes may only | |||
| 30 | + | // dispatch on a branch under them, and its run is not trusted (D1). Whoever | |||
| 31 | + | // calls, the run is trusted only when the ref it runs on was moved to its commit | |||
| 32 | + | // by a trusted principal (ci_tips.go). | |||
| 33 | + | // | |||
| 34 | + | // The workflow is read at ref (the default branch when empty) and must declare | |||
| 35 | + | // workflow_dispatch; the inputs are checked against its declaration. | |||
| 36 | + | func (c *CI) DispatchWorkflow(ctx context.Context, p domain.Principal, name domain.RepoName, params ports.DispatchParams) (domain.CIRun, error) { | |||
| 37 | + | repo, actor, pusher, trusted, err := c.dispatchCaller(ctx, p, name) | |||
| 38 | + | if err != nil { | |||
| 39 | + | return domain.CIRun{}, err | |||
| 40 | + | } | |||
| 41 | + | wfPath, err := dispatchPath(params.WorkflowPath) | |||
| 42 | + | if err != nil { | |||
| 43 | + | return domain.CIRun{}, err | |||
| 44 | + | } | |||
| 45 | + | pin, err := c.dispatchRef(ctx, name, params.Ref) | |||
| 46 | + | if err != nil { | |||
| 47 | + | return domain.CIRun{}, err | |||
| 48 | + | } | |||
| 49 | + | ref, commit := pin.ref, pin.commit | |||
| 50 | + | if c.cfg.DispatchResolved != nil { | |||
| 51 | + | c.cfg.DispatchResolved() | |||
| 52 | + | } | |||
| 53 | + | if !p.AllowsRef(ref) { | |||
| 54 | + | return domain.CIRun{}, fmt.Errorf("%w: this token may not write %s", domain.ErrForbidden, ref) | |||
| 55 | + | } | |||
| 56 | + | content, err := c.readWorkflowAt(ctx, name, commit, wfPath) | |||
| 57 | + | if err != nil { | |||
| 58 | + | return domain.CIRun{}, err | |||
| 59 | + | } | |||
| 60 | + | wf, perrs := workflow.Parse(wfPath, content) | |||
| 61 | + | if len(perrs) > 0 { | |||
| 62 | + | lines := make([]string, 0, len(perrs)) | |||
| 63 | + | for _, e := range perrs { | |||
| 64 | + | lines = append(lines, e.Error()) | |||
| 65 | + | } | |||
| 66 | + | return domain.CIRun{}, fmt.Errorf("%w: %s is not a valid workflow:\n%s", domain.ErrInvalid, wfPath, strings.Join(lines, "\n")) | |||
| 67 | + | } | |||
| 68 | + | ev := wf.On.Event(eventDispatch) | |||
| 69 | + | if ev == nil { | |||
| 70 | + | return domain.CIRun{}, fmt.Errorf("%w: %s does not have a %s trigger", domain.ErrPreconditionFailed, wfPath, eventDispatch) | |||
| 71 | + | } | |||
| 72 | + | values, err := resolveDispatchInputs(ev.Inputs, params.Inputs) | |||
| 73 | + | if err != nil { | |||
| 74 | + | return domain.CIRun{}, err | |||
| 75 | + | } | |||
| 76 | + | ||||
| 77 | + | f := pushFacts{ | |||
| 78 | + | update: domain.RefUpdate{Ref: ref, Old: domain.ZeroOID, New: commit.String()}, commit: commit, | |||
| 79 | + | custom: &customEvent{name: eventDispatch, title: workflowName(wf, wfPath), inputs: values}, | |||
| 80 | + | } | |||
| 81 | + | if def, _, err := c.cfg.Repos.DefaultBranch(ctx, name); err == nil { | |||
| 82 | + | f.defaultBranch = def | |||
| 83 | + | } | |||
| 84 | + | // The run executes the code of the ref, which the caller did not necessarily | |||
| 85 | + | // write: it is trusted only if the caller is AND whoever moved the ref to this | |||
| 86 | + | // commit was (domain.CIRefTip), as a re-run of an agent's run by the owner | |||
| 87 | + | // does not unlock secrets. A ref with no record is not trusted. This holds for | |||
| 88 | + | // every caller, the owner and a job token alike. | |||
| 89 | + | trusted = trusted && c.refTrusted(ctx, name, pin) | |||
| 90 | + | t := ciTrigger{repo: name, actor: actor, pusher: pusher, trusted: trusted, at: c.now()} | |||
| 91 | + | if problem := c.matrixProblem(ctx, repo, t, f, wfPath, wf); problem.text != "" { | |||
| 92 | + | return domain.CIRun{}, fmt.Errorf("%w: %s", domain.ErrInvalid, problem.text) | |||
| 93 | + | } | |||
| 94 | + | run, err := c.launchRun(ctx, repo, t, f, wfPath, string(content), wf) | |||
| 95 | + | if err != nil { | |||
| 96 | + | return domain.CIRun{}, err | |||
| 97 | + | } | |||
| 98 | + | if run.ID == 0 { | |||
| 99 | + | return domain.CIRun{}, domain.ErrNotFound | |||
| 100 | + | } | |||
| 101 | + | ctxLog(ctx).Info().Str("repo", name.String()).Int64("run_id", run.ID).Str("workflow", wfPath).Str("ref", ref). | |||
| 102 | + | Str("principal", p.Label()).Bool("trusted", trusted).Msg("ci workflow dispatched") | |||
| 103 | + | return run, nil | |||
| 104 | + | } | |||
| 105 | + | ||||
| 106 | + | // dispatchCaller applies the access rule of a dispatch and returns who the run | |||
| 107 | + | // is for. | |||
| 108 | + | func (c *CI) dispatchCaller(ctx context.Context, p domain.Principal, name domain.RepoName) (repo domain.Repo, actor string, pusher domain.CIPusher, trusted bool, err error) { | |||
| 109 | + | if p.ViaJobToken() { | |||
| 110 | + | repo, err = lookupRepo(ctx, c.cfg.Meta, name) | |||
| 111 | + | if err != nil { | |||
| 112 | + | return repo, "", pusher, false, err | |||
| 113 | + | } | |||
| 114 | + | // The permission is judged on its own: a job of this repository with | |||
| 115 | + | // actions: write may dispatch even when its contents permission is none | |||
| 116 | + | // (a private repository is then unreadable to it). JobPermits is true only | |||
| 117 | + | // for the job's own token on its own repository, so any other caller of | |||
| 118 | + | // a repository it cannot read still gets not found. | |||
| 119 | + | switch { | |||
| 120 | + | case p.JobPermits("actions", domain.PermissionWrite, name): | |||
| 121 | + | return repo, jobTokenActor, domain.PusherOf(p), true, nil | |||
| 122 | + | case !canRead(p, repo): | |||
| 123 | + | return repo, "", pusher, false, domain.ErrNotFound | |||
| 124 | + | } | |||
| 125 | + | return repo, "", pusher, false, fmt.Errorf("%w: a job token needs the permission actions: write to dispatch a workflow", domain.ErrForbidden) | |||
| 126 | + | } | |||
| 127 | + | repo, err = c.writableRepo(ctx, p, name) | |||
| 128 | + | if err != nil { | |||
| 129 | + | return repo, "", pusher, false, err | |||
| 130 | + | } | |||
| 131 | + | return repo, authorOf(p), domain.PusherOf(p), domain.TrustedActor(p), nil | |||
| 132 | + | } | |||
| 133 | + | ||||
| 134 | + | // dispatchPath checks the workflow file of a dispatch: a file of | |||
| 135 | + | // .github/workflows, named by its path or by its file name. | |||
| 136 | + | func dispatchPath(s string) (string, error) { | |||
| 137 | + | s = strings.TrimSpace(s) | |||
| 138 | + | if s == "" { | |||
| 139 | + | return "", fmt.Errorf("%w: the workflow file is required", domain.ErrInvalid) | |||
| 140 | + | } | |||
| 141 | + | if !strings.Contains(s, "/") { | |||
| 142 | + | s = workflowDir + "/" + s | |||
| 143 | + | } | |||
| 144 | + | ext := strings.ToLower(path.Ext(s)) | |||
| 145 | + | if path.Dir(s) != workflowDir || (ext != ".yml" && ext != ".yaml") { | |||
| 146 | + | return "", fmt.Errorf("%w: %q is not a workflow file of %s", domain.ErrInvalid, s, workflowDir) | |||
| 147 | + | } | |||
| 148 | + | return s, nil | |||
| 149 | + | } | |||
| 150 | + | ||||
| 151 | + | // pinnedRef is what a dispatch resolved its ref to, once: the full name, the | |||
| 152 | + | // commit the run executes and the object the ref points to (the commit for a | |||
| 153 | + | // branch and a lightweight tag, the tag object for an annotated tag). The trust | |||
| 154 | + | // of the run is judged on exactly these, never on the ref as it is read later | |||
| 155 | + | // (ci_tips.go). | |||
| 156 | + | type pinnedRef struct { | |||
| 157 | + | ref string | |||
| 158 | + | commit, object domain.ObjectID | |||
| 159 | + | } | |||
| 160 | + | ||||
| 161 | + | // dispatchRef resolves the ref of a dispatch: a branch or a tag, by short or | |||
| 162 | + | // full name; the default branch when empty. A tag is resolved to its commit and | |||
| 163 | + | // its own object together; a tag that moves between the two reads is refused, | |||
| 164 | + | // so the commit and the object always belong to one state of the ref. | |||
| 165 | + | func (c *CI) dispatchRef(ctx context.Context, name domain.RepoName, given string) (pinnedRef, error) { | |||
| 166 | + | given = strings.TrimSpace(given) | |||
| 167 | + | if given == "" { | |||
| 168 | + | def, exists, err := c.cfg.Repos.DefaultBranch(ctx, name) | |||
| 169 | + | if err != nil { | |||
| 170 | + | return pinnedRef{}, err | |||
| 171 | + | } | |||
| 172 | + | if !exists { | |||
| 173 | + | return pinnedRef{}, fmt.Errorf("%w: the default branch %q does not exist", domain.ErrNotFound, def) | |||
| 174 | + | } | |||
| 175 | + | given = "refs/heads/" + def | |||
| 176 | + | } | |||
| 177 | + | rev, err := domain.ParseRevision(given) | |||
| 178 | + | if err != nil { | |||
| 179 | + | return pinnedRef{}, err | |||
| 180 | + | } | |||
| 181 | + | res, err := c.cfg.Reader.ResolveRevision(ctx, name, rev) | |||
| 182 | + | if err != nil { | |||
| 183 | + | return pinnedRef{}, err | |||
| 184 | + | } | |||
| 185 | + | pin := pinnedRef{ref: res.Ref, commit: res.CommitID, object: res.CommitID} | |||
| 186 | + | switch { | |||
| 187 | + | case strings.HasPrefix(res.Ref, "refs/heads/"): | |||
| 188 | + | return pin, nil | |||
| 189 | + | case !strings.HasPrefix(res.Ref, "refs/tags/"): | |||
| 190 | + | return pinnedRef{}, fmt.Errorf("%w: %q is not a branch or a tag", domain.ErrInvalid, given) | |||
| 191 | + | } | |||
| 192 | + | refs, err := c.cfg.Reader.ListRefs(ctx, name) | |||
| 193 | + | if err != nil { | |||
| 194 | + | return pinnedRef{}, err | |||
| 195 | + | } | |||
| 196 | + | for _, r := range refs { | |||
| 197 | + | if r.Name != res.Ref { | |||
| 198 | + | continue | |||
| 199 | + | } | |||
| 200 | + | if r.CommitID != res.CommitID || r.ID.IsZero() { | |||
| 201 | + | break // moved between the two reads | |||
| 202 | + | } | |||
| 203 | + | pin.object = r.ID | |||
| 204 | + | return pin, nil | |||
| 205 | + | } | |||
| 206 | + | return pinnedRef{}, fmt.Errorf("%w: tag %q moved while it was resolved, try again", domain.ErrConflict, res.Ref) | |||
| 207 | + | } | |||
| 208 | + | ||||
| 209 | + | // readWorkflowAt reads a workflow file at a commit, bounded; a missing file is | |||
| 210 | + | // domain.ErrNotFound. | |||
| 211 | + | func (c *CI) readWorkflowAt(ctx context.Context, repo domain.RepoName, commit domain.ObjectID, wfPath string) ([]byte, error) { | |||
| 212 | + | tp, err := domain.ParseTreePath(wfPath) | |||
| 213 | + | if err != nil { | |||
| 214 | + | return nil, err | |||
| 215 | + | } | |||
| 216 | + | _, rc, err := c.cfg.Reader.OpenBlob(ctx, repo, domain.RevisionOf(commit), tp) | |||
| 217 | + | if err != nil { | |||
| 218 | + | if errors.Is(err, domain.ErrNotFound) || errors.Is(err, domain.ErrNotFile) { | |||
| 219 | + | return nil, fmt.Errorf("%w: workflow %s", domain.ErrNotFound, wfPath) | |||
| 220 | + | } | |||
| 221 | + | return nil, fmt.Errorf("read %s: %w", wfPath, err) | |||
| 222 | + | } | |||
| 223 | + | content, err := io.ReadAll(io.LimitReader(rc, maxCIWorkflowBytes+1)) | |||
| 224 | + | _ = rc.Close() | |||
| 225 | + | if err != nil { | |||
| 226 | + | return nil, fmt.Errorf("read %s: %w", wfPath, err) | |||
| 227 | + | } | |||
| 228 | + | if len(content) > maxCIWorkflowBytes { | |||
| 229 | + | return nil, fmt.Errorf("%w: %s is larger than 1 MiB", domain.ErrInvalid, wfPath) | |||
| 230 | + | } | |||
| 231 | + | return content, nil | |||
| 232 | + | } | |||
A internal/usecase/ci_events.go +274 -0
| @@ -0,0 +1,274 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package usecase | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "fmt" | |
| 7 | + | "math" | |
| 8 | + | "slices" | |
| 9 | + | "strconv" | |
| 10 | + | "strings" | |
| 11 | + | ||
| 12 | + | "bnema.dev/ermite-actions/expr" | |
| 13 | + | "bnema.dev/ermite-actions/workflow" | |
| 14 | + | "bnema.dev/ermite/internal/domain" | |
| 15 | + | ) | |
| 16 | + | ||
| 17 | + | // The events workflow_dispatch, schedule and workflow_run (CI-3, D5). They share | |
| 18 | + | // the run machinery of the push (startRun); what differs is the github context | |
| 19 | + | // and, for workflow_dispatch, the typed inputs. | |
| 20 | + | ||
| 21 | + | const ( | |
| 22 | + | eventDispatch = "workflow_dispatch" | |
| 23 | + | eventSchedule = "schedule" | |
| 24 | + | eventWorkflowRun = "workflow_run" | |
| 25 | + | ||
| 26 | + | // maxDispatchInputs is the number of top-level inputs of a workflow_dispatch | |
| 27 | + | // (docs.github.com, "Events that trigger workflows", workflow_dispatch: up to | |
| 28 | + | // 25 top-level inputs; recalled from memory, not fetched), and | |
| 29 | + | // maxDispatchPayload the size of their values together in characters (the | |
| 30 | + | // REST reference of "Create a workflow dispatch event": at most 65,535). | |
| 31 | + | maxDispatchInputs = 25 | |
| 32 | + | maxDispatchPayload = 65535 | |
| 33 | + | ) | |
| 34 | + | ||
| 35 | + | // customEvent is what a run of workflow_dispatch, schedule or workflow_run is | |
| 36 | + | // for, beside the commit (pushFacts.custom). | |
| 37 | + | type customEvent struct { | |
| 38 | + | name string | |
| 39 | + | title string | |
| 40 | + | // inputs are the values of github.event.inputs (strings), nil when the | |
| 41 | + | // workflow declares none (workflow_dispatch). | |
| 42 | + | inputs map[string]string | |
| 43 | + | // cron is the schedule entry that fired (schedule). | |
| 44 | + | cron string | |
| 45 | + | // triggered is the run a workflow_run is for. | |
| 46 | + | triggered *workflowRunFacts | |
| 47 | + | } | |
| 48 | + | ||
| 49 | + | // workflowRunFacts is the triggering run of a workflow_run. | |
| 50 | + | type workflowRunFacts struct { | |
| 51 | + | run domain.CIRun | |
| 52 | + | // action is "requested" or "completed". | |
| 53 | + | action string | |
| 54 | + | // headBranch is the branch of the triggering run: the source branch of a | |
| 55 | + | // merge request, else the branch or tag the run is on. | |
| 56 | + | headBranch string | |
| 57 | + | } | |
| 58 | + | ||
| 59 | + | // resolveDispatchInputs checks the values given to a workflow_dispatch against | |
| 60 | + | // the inputs the workflow declares and returns github.event.inputs: the value | |
| 61 | + | // given, else the default, else (for a choice) its first option; an optional | |
| 62 | + | // input with none of them is absent. Errors are domain.ErrInvalid. | |
| 63 | + | func resolveDispatchInputs(defs []workflow.Input, given map[string]string) (map[string]string, error) { | |
| 64 | + | invalid := func(format string, args ...any) error { | |
| 65 | + | return fmt.Errorf("%w: %s", domain.ErrInvalid, fmt.Sprintf(format, args...)) | |
| 66 | + | } | |
| 67 | + | if len(defs) > maxDispatchInputs { | |
| 68 | + | return nil, invalid("the workflow declares %d inputs: at most %d are allowed", len(defs), maxDispatchInputs) | |
| 69 | + | } | |
| 70 | + | var unknown []string | |
| 71 | + | for k := range given { | |
| 72 | + | if !slices.ContainsFunc(defs, func(in workflow.Input) bool { return in.Name == k }) { | |
| 73 | + | unknown = append(unknown, k) | |
| 74 | + | } | |
| 75 | + | } | |
| 76 | + | if len(unknown) > 0 { | |
| 77 | + | slices.Sort(unknown) | |
| 78 | + | return nil, invalid("Unexpected inputs provided: [%s]", strings.Join(unknown, ", ")) | |
| 79 | + | } | |
| 80 | + | out := map[string]string{} | |
| 81 | + | size := 0 | |
| 82 | + | for _, in := range defs { | |
| 83 | + | v, ok := given[in.Name] | |
| 84 | + | if ok && v == "" && in.Required && !in.HasDefault { | |
| 85 | + | ok = false | |
| 86 | + | } | |
| 87 | + | if !ok { | |
| 88 | + | switch { | |
| 89 | + | case in.HasDefault: | |
| 90 | + | v, ok = in.Default, true | |
| 91 | + | case in.Required: | |
| 92 | + | return nil, invalid("Required input '%s' not provided", in.Name) | |
| 93 | + | case in.Type == "choice" && len(in.Options) > 0: | |
| 94 | + | v, ok = in.Options[0], true | |
| 95 | + | } | |
| 96 | + | } | |
| 97 | + | if !ok { | |
| 98 | + | continue | |
| 99 | + | } | |
| 100 | + | if err := checkInputValue(in, v); err != nil { | |
| 101 | + | return nil, err | |
| 102 | + | } | |
| 103 | + | size += len(in.Name) + len(v) | |
| 104 | + | out[in.Name] = v | |
| 105 | + | } | |
| 106 | + | if size > maxDispatchPayload { | |
| 107 | + | return nil, invalid("the inputs are larger than %d characters", maxDispatchPayload) | |
| 108 | + | } | |
| 109 | + | return out, nil | |
| 110 | + | } | |
| 111 | + | ||
| 112 | + | // checkInputValue checks the text of one input against its type. | |
| 113 | + | func checkInputValue(in workflow.Input, v string) error { | |
| 114 | + | bad := func(what string) error { | |
| 115 | + | return fmt.Errorf("%w: input '%s' %s", domain.ErrInvalid, in.Name, what) | |
| 116 | + | } | |
| 117 | + | switch in.Type { | |
| 118 | + | case "boolean": | |
| 119 | + | if v != "true" && v != "false" { | |
| 120 | + | return bad("must be 'true' or 'false'") | |
| 121 | + | } | |
| 122 | + | case "number": | |
| 123 | + | if _, ok := parseInputNumber(v); !ok { | |
| 124 | + | return bad("must be a number") | |
| 125 | + | } | |
| 126 | + | case "choice": | |
| 127 | + | if !slices.Contains(in.Options, v) { | |
| 128 | + | return bad("must be one of: " + strings.Join(in.Options, ", ")) | |
| 129 | + | } | |
| 130 | + | case "environment": | |
| 131 | + | if _, err := domain.ParseEnvironmentName(v); err != nil { | |
| 132 | + | return bad("must be the name of an environment") | |
| 133 | + | } | |
| 134 | + | } | |
| 135 | + | return nil | |
| 136 | + | } | |
| 137 | + | ||
| 138 | + | // parseInputNumber reads a decimal number: no hexadecimal, no underscores, no | |
| 139 | + | // NaN or infinity. | |
| 140 | + | func parseInputNumber(v string) (float64, bool) { | |
| 141 | + | v = strings.TrimSpace(v) | |
| 142 | + | if v == "" || strings.ContainsAny(v, "xX_pP") { | |
| 143 | + | return 0, false | |
| 144 | + | } | |
| 145 | + | f, err := strconv.ParseFloat(v, 64) | |
| 146 | + | if err != nil || math.IsNaN(f) || math.IsInf(f, 0) { | |
| 147 | + | return 0, false | |
| 148 | + | } | |
| 149 | + | return f, true | |
| 150 | + | } | |
| 151 | + | ||
| 152 | + | // typedInputs is the "inputs" context of a workflow_dispatch run: each declared | |
| 153 | + | // input with the type of its declaration (boolean, number or string), the zero | |
| 154 | + | // of the type when it has no value. | |
| 155 | + | func typedInputs(defs []workflow.Input, values map[string]string) *expr.Dictionary { | |
| 156 | + | d := expr.NewDictionary() | |
| 157 | + | for _, in := range defs { | |
| 158 | + | s, ok := values[in.Name] | |
| 159 | + | switch in.Type { | |
| 160 | + | case "boolean": | |
| 161 | + | d.Add(in.Name, expr.Boolean(ok && s == "true")) | |
| 162 | + | case "number": | |
| 163 | + | f, _ := parseInputNumber(s) | |
| 164 | + | d.Add(in.Name, expr.Number(f)) | |
| 165 | + | default: | |
| 166 | + | d.Add(in.Name, expr.String(s)) | |
| 167 | + | } | |
| 168 | + | } | |
| 169 | + | return d | |
| 170 | + | } | |
| 171 | + | ||
| 172 | + | // eventInputs is the "inputs" context of the top level of a run: the typed | |
| 173 | + | // inputs of a workflow_dispatch run (read back from github.event.inputs of its | |
| 174 | + | // github context), empty for every other event. | |
| 175 | + | func eventInputs(wf *workflow.Workflow, githubJSON string) *expr.Dictionary { | |
| 176 | + | if wf == nil || wf.On == nil || !strings.Contains(githubJSON, `"event_name":"`+eventDispatch+`"`) { | |
| 177 | + | return expr.NewDictionary() | |
| 178 | + | } | |
| 179 | + | ev := wf.On.Event(eventDispatch) | |
| 180 | + | if ev == nil { | |
| 181 | + | return expr.NewDictionary() | |
| 182 | + | } | |
| 183 | + | values := map[string]string{} | |
| 184 | + | if v, err := expr.FromJSON([]byte(githubJSON)); err == nil { | |
| 185 | + | if gh, ok := v.(*expr.Dictionary); ok { | |
| 186 | + | if e, ok := gh.Get("event"); ok { | |
| 187 | + | if ed, ok := e.(*expr.Dictionary); ok { | |
| 188 | + | if in, ok := ed.Get("inputs"); ok { | |
| 189 | + | if id, ok := in.(*expr.Dictionary); ok { | |
| 190 | + | for _, p := range id.Pairs() { | |
| 191 | + | values[p.Key] = expr.CoerceString(p.Value) | |
| 192 | + | } | |
| 193 | + | } | |
| 194 | + | } | |
| 195 | + | } | |
| 196 | + | } | |
| 197 | + | } | |
| 198 | + | } | |
| 199 | + | return typedInputs(ev.Inputs, values) | |
| 200 | + | } | |
| 201 | + | ||
| 202 | + | // customGithubJSON is the github context of a workflow_dispatch, schedule or | |
| 203 | + | // workflow_run run. | |
| 204 | + | func (c *CI) customGithubJSON(repo domain.Repo, run domain.CIRun, f pushFacts) string { | |
| 205 | + | u := c.urls(repo) | |
| 206 | + | id := c.identity(run.Actor) | |
| 207 | + | cu := f.custom | |
| 208 | + | event := map[string]any{ | |
| 209 | + | "repository": c.repoObj(repo, u, f.defaultBranch, run.CreatedAt), | |
| 210 | + | "sender": u.userObj(id, false), | |
| 211 | + | } | |
| 212 | + | switch cu.name { | |
| 213 | + | case eventDispatch: | |
| 214 | + | event["ref"] = run.Ref | |
| 215 | + | event["workflow"] = run.WorkflowPath | |
| 216 | + | if cu.inputs != nil { | |
| 217 | + | event["inputs"] = cu.inputs | |
| 218 | + | } else { | |
| 219 | + | event["inputs"] = nil | |
| 220 | + | } | |
| 221 | + | case eventSchedule: | |
| 222 | + | event["schedule"] = cu.cron | |
| 223 | + | case eventWorkflowRun: | |
| 224 | + | t := cu.triggered | |
| 225 | + | event["action"] = t.action | |
| 226 | + | event["workflow_run"] = c.workflowRunObj(repo, u, t) | |
| 227 | + | event["workflow"] = c.workflowObj(u, t.run.WorkflowPath, t.run.WorkflowName, t.run.CreatedAt) | |
| 228 | + | } | |
| 229 | + | refName, refType := strings.TrimPrefix(run.Ref, "refs/heads/"), "branch" | |
| 230 | + | if tag, ok := strings.CutPrefix(run.Ref, "refs/tags/"); ok { | |
| 231 | + | refName, refType = tag, "tag" | |
| 232 | + | } | |
| 233 | + | return c.githubContext(repo, run, ghEvent{name: cu.name, refName: refName, refType: refType, payload: event}) | |
| 234 | + | } | |
| 235 | + | ||
| 236 | + | // workflowObj is the "workflow" object of a payload. | |
| 237 | + | func (c *CI) workflowObj(u ghURLs, path, name string, at interface{ Unix() int64 }) map[string]any { | |
| 238 | + | id := stableID("workflow", u.full+"\x00"+path) | |
| 239 | + | return map[string]any{ | |
| 240 | + | "id": id, "node_id": nodeID("02", "Workflow", id), "name": name, "path": path, "state": "active", | |
| 241 | + | "url": u.api + "/repos/" + u.full + "/actions/workflows/" + strconv.FormatInt(id, 10), | |
| 242 | + | "html_url": u.repo + "/blob/HEAD/" + path, | |
| 243 | + | "badge_url": u.repo + "/workflows/" + path + "/badge.svg", | |
| 244 | + | "created_at": at.Unix(), "updated_at": at.Unix(), | |
| 245 | + | } | |
| 246 | + | } | |
| 247 | + | ||
| 248 | + | // workflowRunObj is the "workflow_run" object of the payload: the fields ermite | |
| 249 | + | // can fill about the triggering run. | |
| 250 | + | func (c *CI) workflowRunObj(repo domain.Repo, u ghURLs, t *workflowRunFacts) map[string]any { | |
| 251 | + | r := t.run | |
| 252 | + | var conclusion any | |
| 253 | + | if r.Conclusion != domain.CINoConclusion { | |
| 254 | + | conclusion = string(r.Conclusion) | |
| 255 | + | } | |
| 256 | + | id := strconv.FormatInt(r.ID, 10) | |
| 257 | + | actor := u.userObj(c.identity(r.Actor), false) | |
| 258 | + | obj := map[string]any{ | |
| 259 | + | "id": r.ID, "name": r.WorkflowName, "display_title": r.Title, "head_branch": t.headBranch, "head_sha": r.CheckSha(), | |
| 260 | + | "path": r.WorkflowPath, "run_number": r.Number, "run_attempt": r.Attempt, "event": r.Event, | |
| 261 | + | "status": string(r.Status), "conclusion": conclusion, | |
| 262 | + | "workflow_id": stableID("workflow", u.full+"\x00"+r.WorkflowPath), | |
| 263 | + | "url": u.api + "/repos/" + u.full + "/actions/runs/" + id, | |
| 264 | + | "html_url": u.repo + "/actions/runs/" + id, | |
| 265 | + | "actor": actor, "triggering_actor": actor, | |
| 266 | + | "created_at": r.CreatedAt.UTC().Format("2006-01-02T15:04:05Z07:00"), | |
| 267 | + | "updated_at": c.now().Format("2006-01-02T15:04:05Z07:00"), | |
| 268 | + | "repository": c.repoObj(repo, u, "", r.CreatedAt), | |
| 269 | + | } | |
| 270 | + | if !r.StartedAt.IsZero() { | |
| 271 | + | obj["run_started_at"] = r.StartedAt.UTC().Format("2006-01-02T15:04:05Z07:00") | |
| 272 | + | } | |
| 273 | + | return obj | |
| 274 | + | } | |
| @@ -0,0 +1,274 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package usecase | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "fmt" | |||
| 7 | + | "math" | |||
| 8 | + | "slices" | |||
| 9 | + | "strconv" | |||
| 10 | + | "strings" | |||
| 11 | + | ||||
| 12 | + | "bnema.dev/ermite-actions/expr" | |||
| 13 | + | "bnema.dev/ermite-actions/workflow" | |||
| 14 | + | "bnema.dev/ermite/internal/domain" | |||
| 15 | + | ) | |||
| 16 | + | ||||
| 17 | + | // The events workflow_dispatch, schedule and workflow_run (CI-3, D5). They share | |||
| 18 | + | // the run machinery of the push (startRun); what differs is the github context | |||
| 19 | + | // and, for workflow_dispatch, the typed inputs. | |||
| 20 | + | ||||
| 21 | + | const ( | |||
| 22 | + | eventDispatch = "workflow_dispatch" | |||
| 23 | + | eventSchedule = "schedule" | |||
| 24 | + | eventWorkflowRun = "workflow_run" | |||
| 25 | + | ||||
| 26 | + | // maxDispatchInputs is the number of top-level inputs of a workflow_dispatch | |||
| 27 | + | // (docs.github.com, "Events that trigger workflows", workflow_dispatch: up to | |||
| 28 | + | // 25 top-level inputs; recalled from memory, not fetched), and | |||
| 29 | + | // maxDispatchPayload the size of their values together in characters (the | |||
| 30 | + | // REST reference of "Create a workflow dispatch event": at most 65,535). | |||
| 31 | + | maxDispatchInputs = 25 | |||
| 32 | + | maxDispatchPayload = 65535 | |||
| 33 | + | ) | |||
| 34 | + | ||||
| 35 | + | // customEvent is what a run of workflow_dispatch, schedule or workflow_run is | |||
| 36 | + | // for, beside the commit (pushFacts.custom). | |||
| 37 | + | type customEvent struct { | |||
| 38 | + | name string | |||
| 39 | + | title string | |||
| 40 | + | // inputs are the values of github.event.inputs (strings), nil when the | |||
| 41 | + | // workflow declares none (workflow_dispatch). | |||
| 42 | + | inputs map[string]string | |||
| 43 | + | // cron is the schedule entry that fired (schedule). | |||
| 44 | + | cron string | |||
| 45 | + | // triggered is the run a workflow_run is for. | |||
| 46 | + | triggered *workflowRunFacts | |||
| 47 | + | } | |||
| 48 | + | ||||
| 49 | + | // workflowRunFacts is the triggering run of a workflow_run. | |||
| 50 | + | type workflowRunFacts struct { | |||
| 51 | + | run domain.CIRun | |||
| 52 | + | // action is "requested" or "completed". | |||
| 53 | + | action string | |||
| 54 | + | // headBranch is the branch of the triggering run: the source branch of a | |||
| 55 | + | // merge request, else the branch or tag the run is on. | |||
| 56 | + | headBranch string | |||
| 57 | + | } | |||
| 58 | + | ||||
| 59 | + | // resolveDispatchInputs checks the values given to a workflow_dispatch against | |||
| 60 | + | // the inputs the workflow declares and returns github.event.inputs: the value | |||
| 61 | + | // given, else the default, else (for a choice) its first option; an optional | |||
| 62 | + | // input with none of them is absent. Errors are domain.ErrInvalid. | |||
| 63 | + | func resolveDispatchInputs(defs []workflow.Input, given map[string]string) (map[string]string, error) { | |||
| 64 | + | invalid := func(format string, args ...any) error { | |||
| 65 | + | return fmt.Errorf("%w: %s", domain.ErrInvalid, fmt.Sprintf(format, args...)) | |||
| 66 | + | } | |||
| 67 | + | if len(defs) > maxDispatchInputs { | |||
| 68 | + | return nil, invalid("the workflow declares %d inputs: at most %d are allowed", len(defs), maxDispatchInputs) | |||
| 69 | + | } | |||
| 70 | + | var unknown []string | |||
| 71 | + | for k := range given { | |||
| 72 | + | if !slices.ContainsFunc(defs, func(in workflow.Input) bool { return in.Name == k }) { | |||
| 73 | + | unknown = append(unknown, k) | |||
| 74 | + | } | |||
| 75 | + | } | |||
| 76 | + | if len(unknown) > 0 { | |||
| 77 | + | slices.Sort(unknown) | |||
| 78 | + | return nil, invalid("Unexpected inputs provided: [%s]", strings.Join(unknown, ", ")) | |||
| 79 | + | } | |||
| 80 | + | out := map[string]string{} | |||
| 81 | + | size := 0 | |||
| 82 | + | for _, in := range defs { | |||
| 83 | + | v, ok := given[in.Name] | |||
| 84 | + | if ok && v == "" && in.Required && !in.HasDefault { | |||
| 85 | + | ok = false | |||
| 86 | + | } | |||
| 87 | + | if !ok { | |||
| 88 | + | switch { | |||
| 89 | + | case in.HasDefault: | |||
| 90 | + | v, ok = in.Default, true | |||
| 91 | + | case in.Required: | |||
| 92 | + | return nil, invalid("Required input '%s' not provided", in.Name) | |||
| 93 | + | case in.Type == "choice" && len(in.Options) > 0: | |||
| 94 | + | v, ok = in.Options[0], true | |||
| 95 | + | } | |||
| 96 | + | } | |||
| 97 | + | if !ok { | |||
| 98 | + | continue | |||
| 99 | + | } | |||
| 100 | + | if err := checkInputValue(in, v); err != nil { | |||
| 101 | + | return nil, err | |||
| 102 | + | } | |||
| 103 | + | size += len(in.Name) + len(v) | |||
| 104 | + | out[in.Name] = v | |||
| 105 | + | } | |||
| 106 | + | if size > maxDispatchPayload { | |||
| 107 | + | return nil, invalid("the inputs are larger than %d characters", maxDispatchPayload) | |||
| 108 | + | } | |||
| 109 | + | return out, nil | |||
| 110 | + | } | |||
| 111 | + | ||||
| 112 | + | // checkInputValue checks the text of one input against its type. | |||
| 113 | + | func checkInputValue(in workflow.Input, v string) error { | |||
| 114 | + | bad := func(what string) error { | |||
| 115 | + | return fmt.Errorf("%w: input '%s' %s", domain.ErrInvalid, in.Name, what) | |||
| 116 | + | } | |||
| 117 | + | switch in.Type { | |||
| 118 | + | case "boolean": | |||
| 119 | + | if v != "true" && v != "false" { | |||
| 120 | + | return bad("must be 'true' or 'false'") | |||
| 121 | + | } | |||
| 122 | + | case "number": | |||
| 123 | + | if _, ok := parseInputNumber(v); !ok { | |||
| 124 | + | return bad("must be a number") | |||
| 125 | + | } | |||
| 126 | + | case "choice": | |||
| 127 | + | if !slices.Contains(in.Options, v) { | |||
| 128 | + | return bad("must be one of: " + strings.Join(in.Options, ", ")) | |||
| 129 | + | } | |||
| 130 | + | case "environment": | |||
| 131 | + | if _, err := domain.ParseEnvironmentName(v); err != nil { | |||
| 132 | + | return bad("must be the name of an environment") | |||
| 133 | + | } | |||
| 134 | + | } | |||
| 135 | + | return nil | |||
| 136 | + | } | |||
| 137 | + | ||||
| 138 | + | // parseInputNumber reads a decimal number: no hexadecimal, no underscores, no | |||
| 139 | + | // NaN or infinity. | |||
| 140 | + | func parseInputNumber(v string) (float64, bool) { | |||
| 141 | + | v = strings.TrimSpace(v) | |||
| 142 | + | if v == "" || strings.ContainsAny(v, "xX_pP") { | |||
| 143 | + | return 0, false | |||
| 144 | + | } | |||
| 145 | + | f, err := strconv.ParseFloat(v, 64) | |||
| 146 | + | if err != nil || math.IsNaN(f) || math.IsInf(f, 0) { | |||
| 147 | + | return 0, false | |||
| 148 | + | } | |||
| 149 | + | return f, true | |||
| 150 | + | } | |||
| 151 | + | ||||
| 152 | + | // typedInputs is the "inputs" context of a workflow_dispatch run: each declared | |||
| 153 | + | // input with the type of its declaration (boolean, number or string), the zero | |||
| 154 | + | // of the type when it has no value. | |||
| 155 | + | func typedInputs(defs []workflow.Input, values map[string]string) *expr.Dictionary { | |||
| 156 | + | d := expr.NewDictionary() | |||
| 157 | + | for _, in := range defs { | |||
| 158 | + | s, ok := values[in.Name] | |||
| 159 | + | switch in.Type { | |||
| 160 | + | case "boolean": | |||
| 161 | + | d.Add(in.Name, expr.Boolean(ok && s == "true")) | |||
| 162 | + | case "number": | |||
| 163 | + | f, _ := parseInputNumber(s) | |||
| 164 | + | d.Add(in.Name, expr.Number(f)) | |||
| 165 | + | default: | |||
| 166 | + | d.Add(in.Name, expr.String(s)) | |||
| 167 | + | } | |||
| 168 | + | } | |||
| 169 | + | return d | |||
| 170 | + | } | |||
| 171 | + | ||||
| 172 | + | // eventInputs is the "inputs" context of the top level of a run: the typed | |||
| 173 | + | // inputs of a workflow_dispatch run (read back from github.event.inputs of its | |||
| 174 | + | // github context), empty for every other event. | |||
| 175 | + | func eventInputs(wf *workflow.Workflow, githubJSON string) *expr.Dictionary { | |||
| 176 | + | if wf == nil || wf.On == nil || !strings.Contains(githubJSON, `"event_name":"`+eventDispatch+`"`) { | |||
| 177 | + | return expr.NewDictionary() | |||
| 178 | + | } | |||
| 179 | + | ev := wf.On.Event(eventDispatch) | |||
| 180 | + | if ev == nil { | |||
| 181 | + | return expr.NewDictionary() | |||
| 182 | + | } | |||
| 183 | + | values := map[string]string{} | |||
| 184 | + | if v, err := expr.FromJSON([]byte(githubJSON)); err == nil { | |||
| 185 | + | if gh, ok := v.(*expr.Dictionary); ok { | |||
| 186 | + | if e, ok := gh.Get("event"); ok { | |||
| 187 | + | if ed, ok := e.(*expr.Dictionary); ok { | |||
| 188 | + | if in, ok := ed.Get("inputs"); ok { | |||
| 189 | + | if id, ok := in.(*expr.Dictionary); ok { | |||
| 190 | + | for _, p := range id.Pairs() { | |||
| 191 | + | values[p.Key] = expr.CoerceString(p.Value) | |||
| 192 | + | } | |||
| 193 | + | } | |||
| 194 | + | } | |||
| 195 | + | } | |||
| 196 | + | } | |||
| 197 | + | } | |||
| 198 | + | } | |||
| 199 | + | return typedInputs(ev.Inputs, values) | |||
| 200 | + | } | |||
| 201 | + | ||||
| 202 | + | // customGithubJSON is the github context of a workflow_dispatch, schedule or | |||
| 203 | + | // workflow_run run. | |||
| 204 | + | func (c *CI) customGithubJSON(repo domain.Repo, run domain.CIRun, f pushFacts) string { | |||
| 205 | + | u := c.urls(repo) | |||
| 206 | + | id := c.identity(run.Actor) | |||
| 207 | + | cu := f.custom | |||
| 208 | + | event := map[string]any{ | |||
| 209 | + | "repository": c.repoObj(repo, u, f.defaultBranch, run.CreatedAt), | |||
| 210 | + | "sender": u.userObj(id, false), | |||
| 211 | + | } | |||
| 212 | + | switch cu.name { | |||
| 213 | + | case eventDispatch: | |||
| 214 | + | event["ref"] = run.Ref | |||
| 215 | + | event["workflow"] = run.WorkflowPath | |||
| 216 | + | if cu.inputs != nil { | |||
| 217 | + | event["inputs"] = cu.inputs | |||
| 218 | + | } else { | |||
| 219 | + | event["inputs"] = nil | |||
| 220 | + | } | |||
| 221 | + | case eventSchedule: | |||
| 222 | + | event["schedule"] = cu.cron | |||
| 223 | + | case eventWorkflowRun: | |||
| 224 | + | t := cu.triggered | |||
| 225 | + | event["action"] = t.action | |||
| 226 | + | event["workflow_run"] = c.workflowRunObj(repo, u, t) | |||
| 227 | + | event["workflow"] = c.workflowObj(u, t.run.WorkflowPath, t.run.WorkflowName, t.run.CreatedAt) | |||
| 228 | + | } | |||
| 229 | + | refName, refType := strings.TrimPrefix(run.Ref, "refs/heads/"), "branch" | |||
| 230 | + | if tag, ok := strings.CutPrefix(run.Ref, "refs/tags/"); ok { | |||
| 231 | + | refName, refType = tag, "tag" | |||
| 232 | + | } | |||
| 233 | + | return c.githubContext(repo, run, ghEvent{name: cu.name, refName: refName, refType: refType, payload: event}) | |||
| 234 | + | } | |||
| 235 | + | ||||
| 236 | + | // workflowObj is the "workflow" object of a payload. | |||
| 237 | + | func (c *CI) workflowObj(u ghURLs, path, name string, at interface{ Unix() int64 }) map[string]any { | |||
| 238 | + | id := stableID("workflow", u.full+"\x00"+path) | |||
| 239 | + | return map[string]any{ | |||
| 240 | + | "id": id, "node_id": nodeID("02", "Workflow", id), "name": name, "path": path, "state": "active", | |||
| 241 | + | "url": u.api + "/repos/" + u.full + "/actions/workflows/" + strconv.FormatInt(id, 10), | |||
| 242 | + | "html_url": u.repo + "/blob/HEAD/" + path, | |||
| 243 | + | "badge_url": u.repo + "/workflows/" + path + "/badge.svg", | |||
| 244 | + | "created_at": at.Unix(), "updated_at": at.Unix(), | |||
| 245 | + | } | |||
| 246 | + | } | |||
| 247 | + | ||||
| 248 | + | // workflowRunObj is the "workflow_run" object of the payload: the fields ermite | |||
| 249 | + | // can fill about the triggering run. | |||
| 250 | + | func (c *CI) workflowRunObj(repo domain.Repo, u ghURLs, t *workflowRunFacts) map[string]any { | |||
| 251 | + | r := t.run | |||
| 252 | + | var conclusion any | |||
| 253 | + | if r.Conclusion != domain.CINoConclusion { | |||
| 254 | + | conclusion = string(r.Conclusion) | |||
| 255 | + | } | |||
| 256 | + | id := strconv.FormatInt(r.ID, 10) | |||
| 257 | + | actor := u.userObj(c.identity(r.Actor), false) | |||
| 258 | + | obj := map[string]any{ | |||
| 259 | + | "id": r.ID, "name": r.WorkflowName, "display_title": r.Title, "head_branch": t.headBranch, "head_sha": r.CheckSha(), | |||
| 260 | + | "path": r.WorkflowPath, "run_number": r.Number, "run_attempt": r.Attempt, "event": r.Event, | |||
| 261 | + | "status": string(r.Status), "conclusion": conclusion, | |||
| 262 | + | "workflow_id": stableID("workflow", u.full+"\x00"+r.WorkflowPath), | |||
| 263 | + | "url": u.api + "/repos/" + u.full + "/actions/runs/" + id, | |||
| 264 | + | "html_url": u.repo + "/actions/runs/" + id, | |||
| 265 | + | "actor": actor, "triggering_actor": actor, | |||
| 266 | + | "created_at": r.CreatedAt.UTC().Format("2006-01-02T15:04:05Z07:00"), | |||
| 267 | + | "updated_at": c.now().Format("2006-01-02T15:04:05Z07:00"), | |||
| 268 | + | "repository": c.repoObj(repo, u, "", r.CreatedAt), | |||
| 269 | + | } | |||
| 270 | + | if !r.StartedAt.IsZero() { | |||
| 271 | + | obj["run_started_at"] = r.StartedAt.UTC().Format("2006-01-02T15:04:05Z07:00") | |||
| 272 | + | } | |||
| 273 | + | return obj | |||
| 274 | + | } | |||
A internal/usecase/ci_events_test.go +73 -0
| @@ -0,0 +1,73 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package usecase_test | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "fmt" | |
| 7 | + | "testing" | |
| 8 | + | ||
| 9 | + | "github.com/stretchr/testify/assert" | |
| 10 | + | "github.com/stretchr/testify/require" | |
| 11 | + | ||
| 12 | + | "bnema.dev/ermite-actions/workflow" | |
| 13 | + | "bnema.dev/ermite/internal/domain" | |
| 14 | + | "bnema.dev/ermite/internal/usecase" | |
| 15 | + | ) | |
| 16 | + | ||
| 17 | + | func dispatchInputs(t *testing.T, inputs string) []workflow.Input { | |
| 18 | + | t.Helper() | |
| 19 | + | wf, errs := workflow.Parse("d.yml", []byte("on:\n workflow_dispatch:\n inputs:\n"+inputs+"\njobs:\n j:\n runs-on: x\n steps:\n - run: 'true'\n")) | |
| 20 | + | require.Empty(t, errs) | |
| 21 | + | return wf.On.Event("workflow_dispatch").Inputs | |
| 22 | + | } | |
| 23 | + | ||
| 24 | + | func TestResolveDispatchInputs(t *testing.T) { | |
| 25 | + | defs := dispatchInputs(t, ` s: {type: string, required: true} | |
| 26 | + | b: {type: boolean, default: true} | |
| 27 | + | n: {type: number, default: 5} | |
| 28 | + | c: {type: choice, options: [x, y]} | |
| 29 | + | e: {type: environment} | |
| 30 | + | o: {description: optional} | |
| 31 | + | `) | |
| 32 | + | got, err := usecase.ResolveDispatchInputs(defs, map[string]string{"s": "v"}) | |
| 33 | + | require.NoError(t, err) | |
| 34 | + | assert.Equal(t, map[string]string{"s": "v", "b": "true", "n": "5", "c": "x"}, got, "defaults, first option of a choice, optional inputs absent") | |
| 35 | + | ||
| 36 | + | got, err = usecase.ResolveDispatchInputs(defs, map[string]string{"s": "v", "b": "false", "n": "-1.5", "c": "y", "e": "prod", "o": ""}) | |
| 37 | + | require.NoError(t, err) | |
| 38 | + | assert.Equal(t, map[string]string{"s": "v", "b": "false", "n": "-1.5", "c": "y", "e": "prod", "o": ""}, got) | |
| 39 | + | ||
| 40 | + | for name, in := range map[string]map[string]string{ | |
| 41 | + | "required missing": {}, | |
| 42 | + | "required empty": {"s": ""}, | |
| 43 | + | "unknown": {"s": "v", "zz": "1"}, | |
| 44 | + | "boolean": {"s": "v", "b": "yes"}, | |
| 45 | + | "number": {"s": "v", "n": "five"}, | |
| 46 | + | "number hex": {"s": "v", "n": "0x1"}, | |
| 47 | + | "number nan": {"s": "v", "n": "NaN"}, | |
| 48 | + | "choice": {"s": "v", "c": "z"}, | |
| 49 | + | "environment": {"s": "v", "e": " padded"}, | |
| 50 | + | } { | |
| 51 | + | _, err := usecase.ResolveDispatchInputs(defs, in) | |
| 52 | + | require.Error(t, err, name) | |
| 53 | + | assert.ErrorIs(t, err, domain.ErrInvalid, name) | |
| 54 | + | } | |
| 55 | + | } | |
| 56 | + | ||
| 57 | + | func TestResolveDispatchInputsLimits(t *testing.T) { | |
| 58 | + | var spec string | |
| 59 | + | for i := range 26 { | |
| 60 | + | spec += fmt.Sprintf(" i%d: {}\n", i) | |
| 61 | + | } | |
| 62 | + | _, err := usecase.ResolveDispatchInputs(dispatchInputs(t, spec), nil) | |
| 63 | + | require.Error(t, err, "more than 25 inputs") | |
| 64 | + | assert.Contains(t, err.Error(), "at most 25") | |
| 65 | + | ||
| 66 | + | defs := dispatchInputs(t, " a: {}\n") | |
| 67 | + | big := make([]byte, 65535) | |
| 68 | + | for i := range big { | |
| 69 | + | big[i] = 'x' | |
| 70 | + | } | |
| 71 | + | _, err = usecase.ResolveDispatchInputs(defs, map[string]string{"a": string(big)}) | |
| 72 | + | require.Error(t, err, "payload over 65,535 characters") | |
| 73 | + | } | |
| @@ -0,0 +1,73 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package usecase_test | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "fmt" | |||
| 7 | + | "testing" | |||
| 8 | + | ||||
| 9 | + | "github.com/stretchr/testify/assert" | |||
| 10 | + | "github.com/stretchr/testify/require" | |||
| 11 | + | ||||
| 12 | + | "bnema.dev/ermite-actions/workflow" | |||
| 13 | + | "bnema.dev/ermite/internal/domain" | |||
| 14 | + | "bnema.dev/ermite/internal/usecase" | |||
| 15 | + | ) | |||
| 16 | + | ||||
| 17 | + | func dispatchInputs(t *testing.T, inputs string) []workflow.Input { | |||
| 18 | + | t.Helper() | |||
| 19 | + | wf, errs := workflow.Parse("d.yml", []byte("on:\n workflow_dispatch:\n inputs:\n"+inputs+"\njobs:\n j:\n runs-on: x\n steps:\n - run: 'true'\n")) | |||
| 20 | + | require.Empty(t, errs) | |||
| 21 | + | return wf.On.Event("workflow_dispatch").Inputs | |||
| 22 | + | } | |||
| 23 | + | ||||
| 24 | + | func TestResolveDispatchInputs(t *testing.T) { | |||
| 25 | + | defs := dispatchInputs(t, ` s: {type: string, required: true} | |||
| 26 | + | b: {type: boolean, default: true} | |||
| 27 | + | n: {type: number, default: 5} | |||
| 28 | + | c: {type: choice, options: [x, y]} | |||
| 29 | + | e: {type: environment} | |||
| 30 | + | o: {description: optional} | |||
| 31 | + | `) | |||
| 32 | + | got, err := usecase.ResolveDispatchInputs(defs, map[string]string{"s": "v"}) | |||
| 33 | + | require.NoError(t, err) | |||
| 34 | + | assert.Equal(t, map[string]string{"s": "v", "b": "true", "n": "5", "c": "x"}, got, "defaults, first option of a choice, optional inputs absent") | |||
| 35 | + | ||||
| 36 | + | got, err = usecase.ResolveDispatchInputs(defs, map[string]string{"s": "v", "b": "false", "n": "-1.5", "c": "y", "e": "prod", "o": ""}) | |||
| 37 | + | require.NoError(t, err) | |||
| 38 | + | assert.Equal(t, map[string]string{"s": "v", "b": "false", "n": "-1.5", "c": "y", "e": "prod", "o": ""}, got) | |||
| 39 | + | ||||
| 40 | + | for name, in := range map[string]map[string]string{ | |||
| 41 | + | "required missing": {}, | |||
| 42 | + | "required empty": {"s": ""}, | |||
| 43 | + | "unknown": {"s": "v", "zz": "1"}, | |||
| 44 | + | "boolean": {"s": "v", "b": "yes"}, | |||
| 45 | + | "number": {"s": "v", "n": "five"}, | |||
| 46 | + | "number hex": {"s": "v", "n": "0x1"}, | |||
| 47 | + | "number nan": {"s": "v", "n": "NaN"}, | |||
| 48 | + | "choice": {"s": "v", "c": "z"}, | |||
| 49 | + | "environment": {"s": "v", "e": " padded"}, | |||
| 50 | + | } { | |||
| 51 | + | _, err := usecase.ResolveDispatchInputs(defs, in) | |||
| 52 | + | require.Error(t, err, name) | |||
| 53 | + | assert.ErrorIs(t, err, domain.ErrInvalid, name) | |||
| 54 | + | } | |||
| 55 | + | } | |||
| 56 | + | ||||
| 57 | + | func TestResolveDispatchInputsLimits(t *testing.T) { | |||
| 58 | + | var spec string | |||
| 59 | + | for i := range 26 { | |||
| 60 | + | spec += fmt.Sprintf(" i%d: {}\n", i) | |||
| 61 | + | } | |||
| 62 | + | _, err := usecase.ResolveDispatchInputs(dispatchInputs(t, spec), nil) | |||
| 63 | + | require.Error(t, err, "more than 25 inputs") | |||
| 64 | + | assert.Contains(t, err.Error(), "at most 25") | |||
| 65 | + | ||||
| 66 | + | defs := dispatchInputs(t, " a: {}\n") | |||
| 67 | + | big := make([]byte, 65535) | |||
| 68 | + | for i := range big { | |||
| 69 | + | big[i] = 'x' | |||
| 70 | + | } | |||
| 71 | + | _, err = usecase.ResolveDispatchInputs(defs, map[string]string{"a": string(big)}) | |||
| 72 | + | require.Error(t, err, "payload over 65,535 characters") | |||
| 73 | + | } | |||
M internal/usecase/ci_github.go +3 -0
| @@ -267,6 +267,9 @@ func (c *CI) githubJSON(ctx context.Context, repo domain.Repo, run domain.CIRun, | |||
| 267 | 267 | if f.pull != nil { | |
| 268 | 268 | return c.pullGithubJSON(repo, run, f) | |
| 269 | 269 | } | |
| 270 | + | if f.custom != nil { | |
| 271 | + | return c.customGithubJSON(repo, run, f) | |
| 272 | + | } | |
| 270 | 273 | u := c.urls(repo) | |
| 271 | 274 | id := c.identity(run.Actor) | |
| 272 | 275 | refName, refType := strings.TrimPrefix(run.Ref, "refs/heads/"), "branch" | |
| @@ -267,6 +267,9 @@ func (c *CI) githubJSON(ctx context.Context, repo domain.Repo, run domain.CIRun, | |||||
| 267 | if f.pull != nil { | 267 | if f.pull != nil { | ||
| 268 | return c.pullGithubJSON(repo, run, f) | 268 | return c.pullGithubJSON(repo, run, f) | ||
| 269 | } | 269 | } | ||
| 270 | + | if f.custom != nil { | |||
| 271 | + | return c.customGithubJSON(repo, run, f) | |||
| 272 | + | } | |||
| 270 | u := c.urls(repo) | 273 | u := c.urls(repo) | ||
| 271 | id := c.identity(run.Actor) | 274 | id := c.identity(run.Actor) | ||
| 272 | refName, refType := strings.TrimPrefix(run.Ref, "refs/heads/"), "branch" | 275 | refName, refType := strings.TrimPrefix(run.Ref, "refs/heads/"), "branch" | ||
M internal/usecase/ci_reusable.go +7 -0
| @@ -484,6 +484,13 @@ func (rs *runState) inputsOf(j domain.CIJob) *expr.Dictionary { | |||
| 484 | 484 | return d | |
| 485 | 485 | } | |
| 486 | 486 | } | |
| 487 | + | return expr.NewDictionary() | |
| 488 | + | } | |
| 489 | + | if j.Scope == "" && rs.run.Event == eventDispatch { | |
| 490 | + | // The inputs of a workflow_dispatch run, typed by their declaration. | |
| 491 | + | if wf, err := rs.c.flows.get(rs.run.ID, rs.run.WorkflowPath, rs.src.WorkflowYAML); err == nil { | |
| 492 | + | return eventInputs(wf, rs.src.GithubJSON) | |
| 493 | + | } | |
| 487 | 494 | } | |
| 488 | 495 | return expr.NewDictionary() | |
| 489 | 496 | } | |
| @@ -484,6 +484,13 @@ func (rs *runState) inputsOf(j domain.CIJob) *expr.Dictionary { | |||||
| 484 | return d | 484 | return d | ||
| 485 | } | 485 | } | ||
| 486 | } | 486 | } | ||
| 487 | + | return expr.NewDictionary() | |||
| 488 | + | } | |||
| 489 | + | if j.Scope == "" && rs.run.Event == eventDispatch { | |||
| 490 | + | // The inputs of a workflow_dispatch run, typed by their declaration. | |||
| 491 | + | if wf, err := rs.c.flows.get(rs.run.ID, rs.run.WorkflowPath, rs.src.WorkflowYAML); err == nil { | |||
| 492 | + | return eventInputs(wf, rs.src.GithubJSON) | |||
| 493 | + | } | |||
| 487 | } | 494 | } | ||
| 488 | return expr.NewDictionary() | 495 | return expr.NewDictionary() | ||
| 489 | } | 496 | } | ||
A internal/usecase/ci_schedule.go +443 -0
| @@ -0,0 +1,443 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package usecase | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "errors" | |
| 8 | + | "fmt" | |
| 9 | + | "runtime/debug" | |
| 10 | + | "slices" | |
| 11 | + | "strings" | |
| 12 | + | "sync" | |
| 13 | + | "time" | |
| 14 | + | ||
| 15 | + | "github.com/bnema/zerowrap" | |
| 16 | + | ||
| 17 | + | "bnema.dev/ermite-actions/workflow" | |
| 18 | + | "bnema.dev/ermite/internal/domain" | |
| 19 | + | ) | |
| 20 | + | ||
| 21 | + | // The schedule event (CI-3, D5). A scheduler goroutine looks, once per minute of | |
| 22 | + | // its clock, at the "cron" entries of the workflows of every repository's default | |
| 23 | + | // branch and starts the ones that match that minute (UTC; "timezone" is ignored). | |
| 24 | + | // | |
| 25 | + | // - The run is on the latest commit of the default branch. Its actor and its | |
| 26 | + | // trust are those of whoever pushed that commit (ci_tips.go), as on GitHub: | |
| 27 | + | // the default branch is not necessarily the owner's work. | |
| 28 | + | // - A minute is looked at once, when the clock first reaches it. Ticks missed | |
| 29 | + | // while ermite was down or busy are not made up: a pass that is still going | |
| 30 | + | // when the next minute starts is dropped, as GitHub drops late runs. | |
| 31 | + | // - The schedules of a repository are read once and cached with the commit | |
| 32 | + | // they were read at. A push (of any principal, a job token's included, which | |
| 33 | + | // starts no run but may change the file) marks the repository, and the | |
| 34 | + | // cache is dropped when its default branch moved. A repository that no | |
| 35 | + | // longer exists is forgotten at the next pass. | |
| 36 | + | // - The work is bounded: at most ScheduleBudget repositories are looked at per | |
| 37 | + | // tick, so a pass over many repositories spreads over several ticks (and | |
| 38 | + | // never starts a run late for a minute that is over). A pass dropped | |
| 39 | + | // before its end leaves a cursor: the next pass starts after the last | |
| 40 | + | // repository handled, so that no repository is starved. | |
| 41 | + | // | |
| 42 | + | // The cron entries of one workflow that match the same minute start one run of | |
| 43 | + | // it (the first in file order); the event payload is that entry's text. | |
| 44 | + | ||
| 45 | + | const ( | |
| 46 | + | // DefaultCIScheduleTick is how often the scheduler looks at its clock. | |
| 47 | + | DefaultCIScheduleTick = time.Second | |
| 48 | + | // DefaultCIScheduleBudget is the repositories the scheduler handles at most | |
| 49 | + | // per tick. | |
| 50 | + | DefaultCIScheduleBudget = 64 | |
| 51 | + | ) | |
| 52 | + | ||
| 53 | + | // scheduledCron is one cron entry of a workflow. | |
| 54 | + | type scheduledCron struct { | |
| 55 | + | spec string | |
| 56 | + | cron domain.Cron | |
| 57 | + | } | |
| 58 | + | ||
| 59 | + | // scheduledWorkflow is a workflow with at least one valid cron entry. | |
| 60 | + | type scheduledWorkflow struct { | |
| 61 | + | path string | |
| 62 | + | yaml []byte | |
| 63 | + | crons []scheduledCron | |
| 64 | + | } | |
| 65 | + | ||
| 66 | + | // scheduleEntry is what was read from one repository at tip. | |
| 67 | + | type scheduleEntry struct { | |
| 68 | + | tip domain.ObjectID | |
| 69 | + | files []scheduledWorkflow | |
| 70 | + | } | |
| 71 | + | ||
| 72 | + | // schedulePass is the look at one minute over the repositories. | |
| 73 | + | type schedulePass struct { | |
| 74 | + | minute time.Time | |
| 75 | + | repos []domain.RepoName | |
| 76 | + | next int | |
| 77 | + | } | |
| 78 | + | ||
| 79 | + | type scheduler struct { | |
| 80 | + | mu sync.Mutex | |
| 81 | + | dirty map[domain.RepoName]struct{} | |
| 82 | + | cache map[domain.RepoName]*scheduleEntry | |
| 83 | + | // passed is the minute of the last pass that went through every repository. | |
| 84 | + | passed time.Time | |
| 85 | + | ||
| 86 | + | // The fields below belong to the scheduler goroutine. | |
| 87 | + | last time.Time // the latest minute a pass was started for | |
| 88 | + | pass *schedulePass | |
| 89 | + | // cursor is the last repository a pass handled when it was dropped before | |
| 90 | + | // the end; the next pass starts after it, so that repositories beyond the | |
| 91 | + | // budget of a tick are not starved by the ones that come first. Zero when | |
| 92 | + | // the last pass went through every repository. | |
| 93 | + | cursor domain.RepoName | |
| 94 | + | } | |
| 95 | + | ||
| 96 | + | func newScheduler(boot time.Time) *scheduler { | |
| 97 | + | return &scheduler{ | |
| 98 | + | dirty: map[domain.RepoName]struct{}{}, cache: map[domain.RepoName]*scheduleEntry{}, | |
| 99 | + | // The minute ermite starts in belongs to the process before it. | |
| 100 | + | last: boot.UTC().Truncate(time.Minute), | |
| 101 | + | } | |
| 102 | + | } | |
| 103 | + | ||
| 104 | + | func (s *scheduler) get(repo domain.RepoName) *scheduleEntry { | |
| 105 | + | s.mu.Lock() | |
| 106 | + | defer s.mu.Unlock() | |
| 107 | + | return s.cache[repo] | |
| 108 | + | } | |
| 109 | + | ||
| 110 | + | func (s *scheduler) put(repo domain.RepoName, e *scheduleEntry) { | |
| 111 | + | s.mu.Lock() | |
| 112 | + | defer s.mu.Unlock() | |
| 113 | + | s.cache[repo] = e | |
| 114 | + | } | |
| 115 | + | ||
| 116 | + | func (s *scheduler) drop(repo domain.RepoName) { | |
| 117 | + | s.mu.Lock() | |
| 118 | + | defer s.mu.Unlock() | |
| 119 | + | delete(s.cache, repo) | |
| 120 | + | } | |
| 121 | + | ||
| 122 | + | // markDirty remembers that the default branch of a repository may have moved. | |
| 123 | + | func (s *scheduler) markDirty(repo domain.RepoName) { | |
| 124 | + | s.mu.Lock() | |
| 125 | + | defer s.mu.Unlock() | |
| 126 | + | if _, cached := s.cache[repo]; cached { | |
| 127 | + | s.dirty[repo] = struct{}{} | |
| 128 | + | } | |
| 129 | + | } | |
| 130 | + | ||
| 131 | + | func (s *scheduler) forgetDirty(repo domain.RepoName) { | |
| 132 | + | s.mu.Lock() | |
| 133 | + | defer s.mu.Unlock() | |
| 134 | + | delete(s.dirty, repo) | |
| 135 | + | } | |
| 136 | + | ||
| 137 | + | // takeDirty removes and returns at most n marked repositories. | |
| 138 | + | func (s *scheduler) takeDirty(n int) []domain.RepoName { | |
| 139 | + | s.mu.Lock() | |
| 140 | + | defer s.mu.Unlock() | |
| 141 | + | var out []domain.RepoName | |
| 142 | + | for r := range s.dirty { | |
| 143 | + | if len(out) == n { | |
| 144 | + | break | |
| 145 | + | } | |
| 146 | + | out = append(out, r) | |
| 147 | + | delete(s.dirty, r) | |
| 148 | + | } | |
| 149 | + | return out | |
| 150 | + | } | |
| 151 | + | ||
| 152 | + | // prune forgets the repositories that are not in keep. | |
| 153 | + | func (s *scheduler) prune(keep []domain.RepoName) { | |
| 154 | + | live := make(map[domain.RepoName]struct{}, len(keep)) | |
| 155 | + | for _, r := range keep { | |
| 156 | + | live[r] = struct{}{} | |
| 157 | + | } | |
| 158 | + | s.mu.Lock() | |
| 159 | + | defer s.mu.Unlock() | |
| 160 | + | for r := range s.cache { | |
| 161 | + | if _, ok := live[r]; !ok { | |
| 162 | + | delete(s.cache, r) | |
| 163 | + | } | |
| 164 | + | } | |
| 165 | + | for r := range s.dirty { | |
| 166 | + | if _, ok := live[r]; !ok { | |
| 167 | + | delete(s.dirty, r) | |
| 168 | + | } | |
| 169 | + | } | |
| 170 | + | } | |
| 171 | + | ||
| 172 | + | // CachedSchedules is the number of scheduled workflows the scheduler holds for a | |
| 173 | + | // repository, -1 when it holds nothing for it. For tests and diagnostics. | |
| 174 | + | func (c *CI) CachedSchedules(repo domain.RepoName) int { | |
| 175 | + | if e := c.sched.get(repo); e != nil { | |
| 176 | + | return len(e.files) | |
| 177 | + | } | |
| 178 | + | return -1 | |
| 179 | + | } | |
| 180 | + | ||
| 181 | + | // SchedulePassed is the minute of the last pass of the scheduler that went | |
| 182 | + | // through every repository (zero before the first). For tests and diagnostics. | |
| 183 | + | func (c *CI) SchedulePassed() time.Time { | |
| 184 | + | c.sched.mu.Lock() | |
| 185 | + | defer c.sched.mu.Unlock() | |
| 186 | + | return c.sched.passed | |
| 187 | + | } | |
| 188 | + | ||
| 189 | + | // markSchedulesDirty tells the scheduler that these updates may have moved the | |
| 190 | + | // default branch of the repository. | |
| 191 | + | func (c *CI) markSchedulesDirty(repo domain.RepoName, updates []domain.RefUpdate) { | |
| 192 | + | for _, u := range updates { | |
| 193 | + | if strings.HasPrefix(u.Ref, "refs/heads/") { | |
| 194 | + | c.sched.markDirty(repo) | |
| 195 | + | return | |
| 196 | + | } | |
| 197 | + | } | |
| 198 | + | } | |
| 199 | + | ||
| 200 | + | // ScheduleNow does one tick of the scheduler on the calling goroutine, with the | |
| 201 | + | // clock of the configuration. It is for tests (with a ScheduleTick too long to | |
| 202 | + | // fire) and operators' tooling; the server never needs to call it. It must not | |
| 203 | + | // run at the same time as the scheduler goroutine. | |
| 204 | + | func (c *CI) ScheduleNow(ctx context.Context) { | |
| 205 | + | c.guardSchedule(func(context.Context) { c.scheduleTick(ctx) }) | |
| 206 | + | } | |
| 207 | + | ||
| 208 | + | func (c *CI) scheduleLoop() { | |
| 209 | + | defer c.wg.Done() | |
| 210 | + | t := time.NewTicker(c.cfg.ScheduleTick) | |
| 211 | + | defer t.Stop() | |
| 212 | + | for { | |
| 213 | + | select { | |
| 214 | + | case <-c.ctx.Done(): | |
| 215 | + | return | |
| 216 | + | case <-t.C: | |
| 217 | + | } | |
| 218 | + | c.guardSchedule(c.scheduleTick) | |
| 219 | + | } | |
| 220 | + | } | |
| 221 | + | ||
| 222 | + | func (c *CI) guardSchedule(fn func(context.Context)) { | |
| 223 | + | log := zerowrap.FromCtx(c.ctx) | |
| 224 | + | defer func() { | |
| 225 | + | if r := recover(); r != nil { | |
| 226 | + | log.Error().Interface("panic", r).Bytes("stack", debug.Stack()).Msg("ci scheduler panicked") | |
| 227 | + | } | |
| 228 | + | }() | |
| 229 | + | fn(c.ctx) | |
| 230 | + | } | |
| 231 | + | ||
| 232 | + | // scheduleTick does one tick of work: it checks the repositories whose default | |
| 233 | + | // branch may have moved, starts a pass when the clock reached a new minute, and | |
| 234 | + | // carries on the pass within the budget. | |
| 235 | + | func (c *CI) scheduleTick(ctx context.Context) { | |
| 236 | + | s := c.sched | |
| 237 | + | budget := c.cfg.ScheduleBudget | |
| 238 | + | for _, repo := range s.takeDirty(budget) { | |
| 239 | + | budget-- | |
| 240 | + | if s.get(repo) != nil && !c.scheduleCurrent(ctx, repo) { | |
| 241 | + | s.drop(repo) | |
| 242 | + | } | |
| 243 | + | } | |
| 244 | + | if minute := c.now().Truncate(time.Minute); minute.After(s.last) { | |
| 245 | + | // A pass still going belongs to a minute that is over: it is dropped, and | |
| 246 | + | // so is every minute between. | |
| 247 | + | s.last = minute | |
| 248 | + | s.pass = c.startPass(ctx, minute) | |
| 249 | + | } | |
| 250 | + | p := s.pass | |
| 251 | + | for p != nil && p.next < len(p.repos) && budget > 0 && ctx.Err() == nil { | |
| 252 | + | repo := p.repos[p.next] | |
| 253 | + | p.next++ | |
| 254 | + | s.cursor = repo | |
| 255 | + | budget-- | |
| 256 | + | if c.cfg.ScheduleVisit != nil { | |
| 257 | + | c.cfg.ScheduleVisit(repo) | |
| 258 | + | } | |
| 259 | + | if err := c.scheduleRepo(ctx, repo, p.minute); err != nil && ctx.Err() == nil { | |
| 260 | + | ctxLog(ctx).Error().Err(err).Str("repo", repo.String()).Msg("ci schedule failed") | |
| 261 | + | } | |
| 262 | + | } | |
| 263 | + | if p != nil && p.next >= len(p.repos) { | |
| 264 | + | s.mu.Lock() | |
| 265 | + | s.passed = p.minute | |
| 266 | + | s.mu.Unlock() | |
| 267 | + | s.pass, s.cursor = nil, domain.RepoName{} | |
| 268 | + | } | |
| 269 | + | } | |
| 270 | + | ||
| 271 | + | // startPass lists the repositories to look at for a minute and forgets the | |
| 272 | + | // ones that are gone. Without a runner CI is off and nothing is looked at. | |
| 273 | + | func (c *CI) startPass(ctx context.Context, minute time.Time) *schedulePass { | |
| 274 | + | p := &schedulePass{minute: minute} | |
| 275 | + | repos, err := c.cfg.Meta.List(ctx) | |
| 276 | + | if err != nil { | |
| 277 | + | ctxLog(ctx).Error().Err(err).Msg("ci schedule: list the repositories") | |
| 278 | + | return p | |
| 279 | + | } | |
| 280 | + | names := make([]domain.RepoName, 0, len(repos)) | |
| 281 | + | for _, r := range repos { | |
| 282 | + | names = append(names, r.Name) | |
| 283 | + | } | |
| 284 | + | c.sched.prune(names) | |
| 285 | + | if runners, err := c.cfg.Store.ListRunners(ctx); err != nil || len(runners) == 0 { | |
| 286 | + | return p | |
| 287 | + | } | |
| 288 | + | // Start after the repository the dropped pass stopped at (the list is in | |
| 289 | + | // name order), and wrap around. | |
| 290 | + | start := 0 | |
| 291 | + | if !c.sched.cursor.IsZero() { | |
| 292 | + | for start < len(names) && names[start].String() <= c.sched.cursor.String() { | |
| 293 | + | start++ | |
| 294 | + | } | |
| 295 | + | } | |
| 296 | + | p.repos = append(slices.Clone(names[start:]), names[:start]...) | |
| 297 | + | return p | |
| 298 | + | } | |
| 299 | + | ||
| 300 | + | // scheduleCurrent reports whether the cached schedules of a repository were | |
| 301 | + | // read at the tip of its default branch as it is now. | |
| 302 | + | func (c *CI) scheduleCurrent(ctx context.Context, repo domain.RepoName) bool { | |
| 303 | + | e := c.sched.get(repo) | |
| 304 | + | if e == nil { | |
| 305 | + | return false | |
| 306 | + | } | |
| 307 | + | tip, ok := c.defaultTip(ctx, repo) | |
| 308 | + | return ok && tip == e.tip | |
| 309 | + | } | |
| 310 | + | ||
| 311 | + | // defaultTip is the latest commit of the default branch. | |
| 312 | + | func (c *CI) defaultTip(ctx context.Context, repo domain.RepoName) (domain.ObjectID, bool) { | |
| 313 | + | def, exists, err := c.cfg.Repos.DefaultBranch(ctx, repo) | |
| 314 | + | if err != nil || !exists { | |
| 315 | + | return domain.ObjectID{}, false | |
| 316 | + | } | |
| 317 | + | tip, err := c.cfg.Git.BranchHead(ctx, repo, def) | |
| 318 | + | if err != nil { | |
| 319 | + | return domain.ObjectID{}, false | |
| 320 | + | } | |
| 321 | + | return tip, true | |
| 322 | + | } | |
| 323 | + | ||
| 324 | + | // loadSchedules reads the cron entries of the workflows at the tip of the | |
| 325 | + | // default branch. A workflow that does not parse is skipped (the push of the | |
| 326 | + | // file reported it). | |
| 327 | + | func (c *CI) loadSchedules(ctx context.Context, repo domain.RepoName) (*scheduleEntry, error) { | |
| 328 | + | tip, ok := c.defaultTip(ctx, repo) | |
| 329 | + | if !ok { | |
| 330 | + | return &scheduleEntry{}, nil // no repository, no default branch, nothing to schedule | |
| 331 | + | } | |
| 332 | + | e := &scheduleEntry{tip: tip} | |
| 333 | + | files, err := c.workflowFiles(ctx, repo, tip) | |
| 334 | + | if err != nil { | |
| 335 | + | return nil, err | |
| 336 | + | } | |
| 337 | + | for _, f := range files { | |
| 338 | + | if f.problem != "" { | |
| 339 | + | continue | |
| 340 | + | } | |
| 341 | + | wf, perrs := workflow.Parse(f.path, f.content) | |
| 342 | + | if len(perrs) > 0 || wf.On == nil { | |
| 343 | + | continue | |
| 344 | + | } | |
| 345 | + | ev := wf.On.Event(eventSchedule) | |
| 346 | + | if ev == nil { | |
| 347 | + | continue | |
| 348 | + | } | |
| 349 | + | sw := scheduledWorkflow{path: f.path, yaml: f.content} | |
| 350 | + | for _, sc := range ev.Schedule { | |
| 351 | + | cron, err := domain.ParseCron(sc.Cron) | |
| 352 | + | if err != nil { | |
| 353 | + | continue | |
| 354 | + | } | |
| 355 | + | sw.crons = append(sw.crons, scheduledCron{spec: sc.Cron, cron: cron}) | |
| 356 | + | } | |
| 357 | + | if len(sw.crons) > 0 { | |
| 358 | + | e.files = append(e.files, sw) | |
| 359 | + | } | |
| 360 | + | } | |
| 361 | + | return e, nil | |
| 362 | + | } | |
| 363 | + | ||
| 364 | + | // dueFor returns, for each workflow of the entry that has an entry matching the | |
| 365 | + | // minute, the text of the first one that does. | |
| 366 | + | func dueFor(e *scheduleEntry, minute time.Time) (idx []int, specs []string) { | |
| 367 | + | for i, f := range e.files { | |
| 368 | + | for _, sc := range f.crons { | |
| 369 | + | if sc.cron.Matches(minute) { | |
| 370 | + | idx, specs = append(idx, i), append(specs, sc.spec) | |
| 371 | + | break | |
| 372 | + | } | |
| 373 | + | } | |
| 374 | + | } | |
| 375 | + | return idx, specs | |
| 376 | + | } | |
| 377 | + | ||
| 378 | + | // scheduleRepo starts the scheduled workflows of a repository that are due at | |
| 379 | + | // the minute. | |
| 380 | + | func (c *CI) scheduleRepo(ctx context.Context, repo domain.RepoName, minute time.Time) error { | |
| 381 | + | // The cache is of the commit the run will be on: it is read again when the | |
| 382 | + | // default branch moved (a push marked it, or the default branch was changed). | |
| 383 | + | tip, ok := c.defaultTip(ctx, repo) | |
| 384 | + | if !ok { | |
| 385 | + | c.sched.drop(repo) | |
| 386 | + | return nil | |
| 387 | + | } | |
| 388 | + | e := c.sched.get(repo) | |
| 389 | + | if e == nil || e.tip != tip { | |
| 390 | + | var err error | |
| 391 | + | if e, err = c.loadSchedules(ctx, repo); err != nil { | |
| 392 | + | return fmt.Errorf("read the schedules: %w", err) | |
| 393 | + | } | |
| 394 | + | c.sched.put(repo, e) | |
| 395 | + | } | |
| 396 | + | idx, specs := dueFor(e, minute) | |
| 397 | + | if len(idx) == 0 { | |
| 398 | + | return nil | |
| 399 | + | } | |
| 400 | + | meta, err := c.cfg.Meta.Get(ctx, repo) | |
| 401 | + | if err != nil { | |
| 402 | + | c.sched.drop(repo) | |
| 403 | + | return nil // deleted meanwhile | |
| 404 | + | } | |
| 405 | + | def, _, err := c.cfg.Repos.DefaultBranch(ctx, repo) | |
| 406 | + | if err != nil { | |
| 407 | + | return nil | |
| 408 | + | } | |
| 409 | + | commits, _, err := c.cfg.Git.ListRange(ctx, repo, domain.ObjectID{}, e.tip, 1) | |
| 410 | + | if err != nil || len(commits) == 0 { | |
| 411 | + | return fmt.Errorf("read the head commit of %s: %w", def, err) | |
| 412 | + | } | |
| 413 | + | var errs []error | |
| 414 | + | for n, i := range idx { | |
| 415 | + | errs = append(errs, c.fireSchedule(ctx, meta, def, e, commits[0], e.files[i], specs[n])) | |
| 416 | + | } | |
| 417 | + | return errors.Join(errs...) | |
| 418 | + | } | |
| 419 | + | ||
| 420 | + | // fireSchedule starts the run of one scheduled workflow. | |
| 421 | + | func (c *CI) fireSchedule(ctx context.Context, repo domain.Repo, def string, e *scheduleEntry, head domain.Commit, sw scheduledWorkflow, spec string) error { | |
| 422 | + | wf, perrs := workflow.Parse(sw.path, sw.yaml) | |
| 423 | + | if len(perrs) > 0 { | |
| 424 | + | return nil | |
| 425 | + | } | |
| 426 | + | facts := pushFacts{ | |
| 427 | + | update: domain.RefUpdate{Ref: "refs/heads/" + def, Old: domain.ZeroOID, New: e.tip.String()}, commit: e.tip, head: head, | |
| 428 | + | defaultBranch: def, custom: &customEvent{name: eventSchedule, cron: spec}, | |
| 429 | + | } | |
| 430 | + | // Whoever moved the default branch to this commit, with their trust | |
| 431 | + | // (ci_tips.go); the owner's name and no trust when that is not known. | |
| 432 | + | actor, pusher, trusted := c.scheduleActor(ctx, repo.Name, def, e.tip) | |
| 433 | + | t := ciTrigger{repo: repo.Name, actor: actor, pusher: pusher, trusted: trusted, at: c.now()} | |
| 434 | + | if problem := c.matrixProblem(ctx, repo, t, facts, sw.path, wf); problem.text != "" { | |
| 435 | + | name := sw.path | |
| 436 | + | if problem.named { | |
| 437 | + | name = workflowName(wf, sw.path) | |
| 438 | + | } | |
| 439 | + | return c.startInvalid(ctx, repo, t, facts, sw.path, name, problem.text) | |
| 440 | + | } | |
| 441 | + | _, err := c.launchRun(ctx, repo, t, facts, sw.path, string(sw.yaml), wf) | |
| 442 | + | return err | |
| 443 | + | } | |
| @@ -0,0 +1,443 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package usecase | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "context" | |||
| 7 | + | "errors" | |||
| 8 | + | "fmt" | |||
| 9 | + | "runtime/debug" | |||
| 10 | + | "slices" | |||
| 11 | + | "strings" | |||
| 12 | + | "sync" | |||
| 13 | + | "time" | |||
| 14 | + | ||||
| 15 | + | "github.com/bnema/zerowrap" | |||
| 16 | + | ||||
| 17 | + | "bnema.dev/ermite-actions/workflow" | |||
| 18 | + | "bnema.dev/ermite/internal/domain" | |||
| 19 | + | ) | |||
| 20 | + | ||||
| 21 | + | // The schedule event (CI-3, D5). A scheduler goroutine looks, once per minute of | |||
| 22 | + | // its clock, at the "cron" entries of the workflows of every repository's default | |||
| 23 | + | // branch and starts the ones that match that minute (UTC; "timezone" is ignored). | |||
| 24 | + | // | |||
| 25 | + | // - The run is on the latest commit of the default branch. Its actor and its | |||
| 26 | + | // trust are those of whoever pushed that commit (ci_tips.go), as on GitHub: | |||
| 27 | + | // the default branch is not necessarily the owner's work. | |||
| 28 | + | // - A minute is looked at once, when the clock first reaches it. Ticks missed | |||
| 29 | + | // while ermite was down or busy are not made up: a pass that is still going | |||
| 30 | + | // when the next minute starts is dropped, as GitHub drops late runs. | |||
| 31 | + | // - The schedules of a repository are read once and cached with the commit | |||
| 32 | + | // they were read at. A push (of any principal, a job token's included, which | |||
| 33 | + | // starts no run but may change the file) marks the repository, and the | |||
| 34 | + | // cache is dropped when its default branch moved. A repository that no | |||
| 35 | + | // longer exists is forgotten at the next pass. | |||
| 36 | + | // - The work is bounded: at most ScheduleBudget repositories are looked at per | |||
| 37 | + | // tick, so a pass over many repositories spreads over several ticks (and | |||
| 38 | + | // never starts a run late for a minute that is over). A pass dropped | |||
| 39 | + | // before its end leaves a cursor: the next pass starts after the last | |||
| 40 | + | // repository handled, so that no repository is starved. | |||
| 41 | + | // | |||
| 42 | + | // The cron entries of one workflow that match the same minute start one run of | |||
| 43 | + | // it (the first in file order); the event payload is that entry's text. | |||
| 44 | + | ||||
| 45 | + | const ( | |||
| 46 | + | // DefaultCIScheduleTick is how often the scheduler looks at its clock. | |||
| 47 | + | DefaultCIScheduleTick = time.Second | |||
| 48 | + | // DefaultCIScheduleBudget is the repositories the scheduler handles at most | |||
| 49 | + | // per tick. | |||
| 50 | + | DefaultCIScheduleBudget = 64 | |||
| 51 | + | ) | |||
| 52 | + | ||||
| 53 | + | // scheduledCron is one cron entry of a workflow. | |||
| 54 | + | type scheduledCron struct { | |||
| 55 | + | spec string | |||
| 56 | + | cron domain.Cron | |||
| 57 | + | } | |||
| 58 | + | ||||
| 59 | + | // scheduledWorkflow is a workflow with at least one valid cron entry. | |||
| 60 | + | type scheduledWorkflow struct { | |||
| 61 | + | path string | |||
| 62 | + | yaml []byte | |||
| 63 | + | crons []scheduledCron | |||
| 64 | + | } | |||
| 65 | + | ||||
| 66 | + | // scheduleEntry is what was read from one repository at tip. | |||
| 67 | + | type scheduleEntry struct { | |||
| 68 | + | tip domain.ObjectID | |||
| 69 | + | files []scheduledWorkflow | |||
| 70 | + | } | |||
| 71 | + | ||||
| 72 | + | // schedulePass is the look at one minute over the repositories. | |||
| 73 | + | type schedulePass struct { | |||
| 74 | + | minute time.Time | |||
| 75 | + | repos []domain.RepoName | |||
| 76 | + | next int | |||
| 77 | + | } | |||
| 78 | + | ||||
| 79 | + | type scheduler struct { | |||
| 80 | + | mu sync.Mutex | |||
| 81 | + | dirty map[domain.RepoName]struct{} | |||
| 82 | + | cache map[domain.RepoName]*scheduleEntry | |||
| 83 | + | // passed is the minute of the last pass that went through every repository. | |||
| 84 | + | passed time.Time | |||
| 85 | + | ||||
| 86 | + | // The fields below belong to the scheduler goroutine. | |||
| 87 | + | last time.Time // the latest minute a pass was started for | |||
| 88 | + | pass *schedulePass | |||
| 89 | + | // cursor is the last repository a pass handled when it was dropped before | |||
| 90 | + | // the end; the next pass starts after it, so that repositories beyond the | |||
| 91 | + | // budget of a tick are not starved by the ones that come first. Zero when | |||
| 92 | + | // the last pass went through every repository. | |||
| 93 | + | cursor domain.RepoName | |||
| 94 | + | } | |||
| 95 | + | ||||
| 96 | + | func newScheduler(boot time.Time) *scheduler { | |||
| 97 | + | return &scheduler{ | |||
| 98 | + | dirty: map[domain.RepoName]struct{}{}, cache: map[domain.RepoName]*scheduleEntry{}, | |||
| 99 | + | // The minute ermite starts in belongs to the process before it. | |||
| 100 | + | last: boot.UTC().Truncate(time.Minute), | |||
| 101 | + | } | |||
| 102 | + | } | |||
| 103 | + | ||||
| 104 | + | func (s *scheduler) get(repo domain.RepoName) *scheduleEntry { | |||
| 105 | + | s.mu.Lock() | |||
| 106 | + | defer s.mu.Unlock() | |||
| 107 | + | return s.cache[repo] | |||
| 108 | + | } | |||
| 109 | + | ||||
| 110 | + | func (s *scheduler) put(repo domain.RepoName, e *scheduleEntry) { | |||
| 111 | + | s.mu.Lock() | |||
| 112 | + | defer s.mu.Unlock() | |||
| 113 | + | s.cache[repo] = e | |||
| 114 | + | } | |||
| 115 | + | ||||
| 116 | + | func (s *scheduler) drop(repo domain.RepoName) { | |||
| 117 | + | s.mu.Lock() | |||
| 118 | + | defer s.mu.Unlock() | |||
| 119 | + | delete(s.cache, repo) | |||
| 120 | + | } | |||
| 121 | + | ||||
| 122 | + | // markDirty remembers that the default branch of a repository may have moved. | |||
| 123 | + | func (s *scheduler) markDirty(repo domain.RepoName) { | |||
| 124 | + | s.mu.Lock() | |||
| 125 | + | defer s.mu.Unlock() | |||
| 126 | + | if _, cached := s.cache[repo]; cached { | |||
| 127 | + | s.dirty[repo] = struct{}{} | |||
| 128 | + | } | |||
| 129 | + | } | |||
| 130 | + | ||||
| 131 | + | func (s *scheduler) forgetDirty(repo domain.RepoName) { | |||
| 132 | + | s.mu.Lock() | |||
| 133 | + | defer s.mu.Unlock() | |||
| 134 | + | delete(s.dirty, repo) | |||
| 135 | + | } | |||
| 136 | + | ||||
| 137 | + | // takeDirty removes and returns at most n marked repositories. | |||
| 138 | + | func (s *scheduler) takeDirty(n int) []domain.RepoName { | |||
| 139 | + | s.mu.Lock() | |||
| 140 | + | defer s.mu.Unlock() | |||
| 141 | + | var out []domain.RepoName | |||
| 142 | + | for r := range s.dirty { | |||
| 143 | + | if len(out) == n { | |||
| 144 | + | break | |||
| 145 | + | } | |||
| 146 | + | out = append(out, r) | |||
| 147 | + | delete(s.dirty, r) | |||
| 148 | + | } | |||
| 149 | + | return out | |||
| 150 | + | } | |||
| 151 | + | ||||
| 152 | + | // prune forgets the repositories that are not in keep. | |||
| 153 | + | func (s *scheduler) prune(keep []domain.RepoName) { | |||
| 154 | + | live := make(map[domain.RepoName]struct{}, len(keep)) | |||
| 155 | + | for _, r := range keep { | |||
| 156 | + | live[r] = struct{}{} | |||
| 157 | + | } | |||
| 158 | + | s.mu.Lock() | |||
| 159 | + | defer s.mu.Unlock() | |||
| 160 | + | for r := range s.cache { | |||
| 161 | + | if _, ok := live[r]; !ok { | |||
| 162 | + | delete(s.cache, r) | |||
| 163 | + | } | |||
| 164 | + | } | |||
| 165 | + | for r := range s.dirty { | |||
| 166 | + | if _, ok := live[r]; !ok { | |||
| 167 | + | delete(s.dirty, r) | |||
| 168 | + | } | |||
| 169 | + | } | |||
| 170 | + | } | |||
| 171 | + | ||||
| 172 | + | // CachedSchedules is the number of scheduled workflows the scheduler holds for a | |||
| 173 | + | // repository, -1 when it holds nothing for it. For tests and diagnostics. | |||
| 174 | + | func (c *CI) CachedSchedules(repo domain.RepoName) int { | |||
| 175 | + | if e := c.sched.get(repo); e != nil { | |||
| 176 | + | return len(e.files) | |||
| 177 | + | } | |||
| 178 | + | return -1 | |||
| 179 | + | } | |||
| 180 | + | ||||
| 181 | + | // SchedulePassed is the minute of the last pass of the scheduler that went | |||
| 182 | + | // through every repository (zero before the first). For tests and diagnostics. | |||
| 183 | + | func (c *CI) SchedulePassed() time.Time { | |||
| 184 | + | c.sched.mu.Lock() | |||
| 185 | + | defer c.sched.mu.Unlock() | |||
| 186 | + | return c.sched.passed | |||
| 187 | + | } | |||
| 188 | + | ||||
| 189 | + | // markSchedulesDirty tells the scheduler that these updates may have moved the | |||
| 190 | + | // default branch of the repository. | |||
| 191 | + | func (c *CI) markSchedulesDirty(repo domain.RepoName, updates []domain.RefUpdate) { | |||
| 192 | + | for _, u := range updates { | |||
| 193 | + | if strings.HasPrefix(u.Ref, "refs/heads/") { | |||
| 194 | + | c.sched.markDirty(repo) | |||
| 195 | + | return | |||
| 196 | + | } | |||
| 197 | + | } | |||
| 198 | + | } | |||
| 199 | + | ||||
| 200 | + | // ScheduleNow does one tick of the scheduler on the calling goroutine, with the | |||
| 201 | + | // clock of the configuration. It is for tests (with a ScheduleTick too long to | |||
| 202 | + | // fire) and operators' tooling; the server never needs to call it. It must not | |||
| 203 | + | // run at the same time as the scheduler goroutine. | |||
| 204 | + | func (c *CI) ScheduleNow(ctx context.Context) { | |||
| 205 | + | c.guardSchedule(func(context.Context) { c.scheduleTick(ctx) }) | |||
| 206 | + | } | |||
| 207 | + | ||||
| 208 | + | func (c *CI) scheduleLoop() { | |||
| 209 | + | defer c.wg.Done() | |||
| 210 | + | t := time.NewTicker(c.cfg.ScheduleTick) | |||
| 211 | + | defer t.Stop() | |||
| 212 | + | for { | |||
| 213 | + | select { | |||
| 214 | + | case <-c.ctx.Done(): | |||
| 215 | + | return | |||
| 216 | + | case <-t.C: | |||
| 217 | + | } | |||
| 218 | + | c.guardSchedule(c.scheduleTick) | |||
| 219 | + | } | |||
| 220 | + | } | |||
| 221 | + | ||||
| 222 | + | func (c *CI) guardSchedule(fn func(context.Context)) { | |||
| 223 | + | log := zerowrap.FromCtx(c.ctx) | |||
| 224 | + | defer func() { | |||
| 225 | + | if r := recover(); r != nil { | |||
| 226 | + | log.Error().Interface("panic", r).Bytes("stack", debug.Stack()).Msg("ci scheduler panicked") | |||
| 227 | + | } | |||
| 228 | + | }() | |||
| 229 | + | fn(c.ctx) | |||
| 230 | + | } | |||
| 231 | + | ||||
| 232 | + | // scheduleTick does one tick of work: it checks the repositories whose default | |||
| 233 | + | // branch may have moved, starts a pass when the clock reached a new minute, and | |||
| 234 | + | // carries on the pass within the budget. | |||
| 235 | + | func (c *CI) scheduleTick(ctx context.Context) { | |||
| 236 | + | s := c.sched | |||
| 237 | + | budget := c.cfg.ScheduleBudget | |||
| 238 | + | for _, repo := range s.takeDirty(budget) { | |||
| 239 | + | budget-- | |||
| 240 | + | if s.get(repo) != nil && !c.scheduleCurrent(ctx, repo) { | |||
| 241 | + | s.drop(repo) | |||
| 242 | + | } | |||
| 243 | + | } | |||
| 244 | + | if minute := c.now().Truncate(time.Minute); minute.After(s.last) { | |||
| 245 | + | // A pass still going belongs to a minute that is over: it is dropped, and | |||
| 246 | + | // so is every minute between. | |||
| 247 | + | s.last = minute | |||
| 248 | + | s.pass = c.startPass(ctx, minute) | |||
| 249 | + | } | |||
| 250 | + | p := s.pass | |||
| 251 | + | for p != nil && p.next < len(p.repos) && budget > 0 && ctx.Err() == nil { | |||
| 252 | + | repo := p.repos[p.next] | |||
| 253 | + | p.next++ | |||
| 254 | + | s.cursor = repo | |||
| 255 | + | budget-- | |||
| 256 | + | if c.cfg.ScheduleVisit != nil { | |||
| 257 | + | c.cfg.ScheduleVisit(repo) | |||
| 258 | + | } | |||
| 259 | + | if err := c.scheduleRepo(ctx, repo, p.minute); err != nil && ctx.Err() == nil { | |||
| 260 | + | ctxLog(ctx).Error().Err(err).Str("repo", repo.String()).Msg("ci schedule failed") | |||
| 261 | + | } | |||
| 262 | + | } | |||
| 263 | + | if p != nil && p.next >= len(p.repos) { | |||
| 264 | + | s.mu.Lock() | |||
| 265 | + | s.passed = p.minute | |||
| 266 | + | s.mu.Unlock() | |||
| 267 | + | s.pass, s.cursor = nil, domain.RepoName{} | |||
| 268 | + | } | |||
| 269 | + | } | |||
| 270 | + | ||||
| 271 | + | // startPass lists the repositories to look at for a minute and forgets the | |||
| 272 | + | // ones that are gone. Without a runner CI is off and nothing is looked at. | |||
| 273 | + | func (c *CI) startPass(ctx context.Context, minute time.Time) *schedulePass { | |||
| 274 | + | p := &schedulePass{minute: minute} | |||
| 275 | + | repos, err := c.cfg.Meta.List(ctx) | |||
| 276 | + | if err != nil { | |||
| 277 | + | ctxLog(ctx).Error().Err(err).Msg("ci schedule: list the repositories") | |||
| 278 | + | return p | |||
| 279 | + | } | |||
| 280 | + | names := make([]domain.RepoName, 0, len(repos)) | |||
| 281 | + | for _, r := range repos { | |||
| 282 | + | names = append(names, r.Name) | |||
| 283 | + | } | |||
| 284 | + | c.sched.prune(names) | |||
| 285 | + | if runners, err := c.cfg.Store.ListRunners(ctx); err != nil || len(runners) == 0 { | |||
| 286 | + | return p | |||
| 287 | + | } | |||
| 288 | + | // Start after the repository the dropped pass stopped at (the list is in | |||
| 289 | + | // name order), and wrap around. | |||
| 290 | + | start := 0 | |||
| 291 | + | if !c.sched.cursor.IsZero() { | |||
| 292 | + | for start < len(names) && names[start].String() <= c.sched.cursor.String() { | |||
| 293 | + | start++ | |||
| 294 | + | } | |||
| 295 | + | } | |||
| 296 | + | p.repos = append(slices.Clone(names[start:]), names[:start]...) | |||
| 297 | + | return p | |||
| 298 | + | } | |||
| 299 | + | ||||
| 300 | + | // scheduleCurrent reports whether the cached schedules of a repository were | |||
| 301 | + | // read at the tip of its default branch as it is now. | |||
| 302 | + | func (c *CI) scheduleCurrent(ctx context.Context, repo domain.RepoName) bool { | |||
| 303 | + | e := c.sched.get(repo) | |||
| 304 | + | if e == nil { | |||
| 305 | + | return false | |||
| 306 | + | } | |||
| 307 | + | tip, ok := c.defaultTip(ctx, repo) | |||
| 308 | + | return ok && tip == e.tip | |||
| 309 | + | } | |||
| 310 | + | ||||
| 311 | + | // defaultTip is the latest commit of the default branch. | |||
| 312 | + | func (c *CI) defaultTip(ctx context.Context, repo domain.RepoName) (domain.ObjectID, bool) { | |||
| 313 | + | def, exists, err := c.cfg.Repos.DefaultBranch(ctx, repo) | |||
| 314 | + | if err != nil || !exists { | |||
| 315 | + | return domain.ObjectID{}, false | |||
| 316 | + | } | |||
| 317 | + | tip, err := c.cfg.Git.BranchHead(ctx, repo, def) | |||
| 318 | + | if err != nil { | |||
| 319 | + | return domain.ObjectID{}, false | |||
| 320 | + | } | |||
| 321 | + | return tip, true | |||
| 322 | + | } | |||
| 323 | + | ||||
| 324 | + | // loadSchedules reads the cron entries of the workflows at the tip of the | |||
| 325 | + | // default branch. A workflow that does not parse is skipped (the push of the | |||
| 326 | + | // file reported it). | |||
| 327 | + | func (c *CI) loadSchedules(ctx context.Context, repo domain.RepoName) (*scheduleEntry, error) { | |||
| 328 | + | tip, ok := c.defaultTip(ctx, repo) | |||
| 329 | + | if !ok { | |||
| 330 | + | return &scheduleEntry{}, nil // no repository, no default branch, nothing to schedule | |||
| 331 | + | } | |||
| 332 | + | e := &scheduleEntry{tip: tip} | |||
| 333 | + | files, err := c.workflowFiles(ctx, repo, tip) | |||
| 334 | + | if err != nil { | |||
| 335 | + | return nil, err | |||
| 336 | + | } | |||
| 337 | + | for _, f := range files { | |||
| 338 | + | if f.problem != "" { | |||
| 339 | + | continue | |||
| 340 | + | } | |||
| 341 | + | wf, perrs := workflow.Parse(f.path, f.content) | |||
| 342 | + | if len(perrs) > 0 || wf.On == nil { | |||
| 343 | + | continue | |||
| 344 | + | } | |||
| 345 | + | ev := wf.On.Event(eventSchedule) | |||
| 346 | + | if ev == nil { | |||
| 347 | + | continue | |||
| 348 | + | } | |||
| 349 | + | sw := scheduledWorkflow{path: f.path, yaml: f.content} | |||
| 350 | + | for _, sc := range ev.Schedule { | |||
| 351 | + | cron, err := domain.ParseCron(sc.Cron) | |||
| 352 | + | if err != nil { | |||
| 353 | + | continue | |||
| 354 | + | } | |||
| 355 | + | sw.crons = append(sw.crons, scheduledCron{spec: sc.Cron, cron: cron}) | |||
| 356 | + | } | |||
| 357 | + | if len(sw.crons) > 0 { | |||
| 358 | + | e.files = append(e.files, sw) | |||
| 359 | + | } | |||
| 360 | + | } | |||
| 361 | + | return e, nil | |||
| 362 | + | } | |||
| 363 | + | ||||
| 364 | + | // dueFor returns, for each workflow of the entry that has an entry matching the | |||
| 365 | + | // minute, the text of the first one that does. | |||
| 366 | + | func dueFor(e *scheduleEntry, minute time.Time) (idx []int, specs []string) { | |||
| 367 | + | for i, f := range e.files { | |||
| 368 | + | for _, sc := range f.crons { | |||
| 369 | + | if sc.cron.Matches(minute) { | |||
| 370 | + | idx, specs = append(idx, i), append(specs, sc.spec) | |||
| 371 | + | break | |||
| 372 | + | } | |||
| 373 | + | } | |||
| 374 | + | } | |||
| 375 | + | return idx, specs | |||
| 376 | + | } | |||
| 377 | + | ||||
| 378 | + | // scheduleRepo starts the scheduled workflows of a repository that are due at | |||
| 379 | + | // the minute. | |||
| 380 | + | func (c *CI) scheduleRepo(ctx context.Context, repo domain.RepoName, minute time.Time) error { | |||
| 381 | + | // The cache is of the commit the run will be on: it is read again when the | |||
| 382 | + | // default branch moved (a push marked it, or the default branch was changed). | |||
| 383 | + | tip, ok := c.defaultTip(ctx, repo) | |||
| 384 | + | if !ok { | |||
| 385 | + | c.sched.drop(repo) | |||
| 386 | + | return nil | |||
| 387 | + | } | |||
| 388 | + | e := c.sched.get(repo) | |||
| 389 | + | if e == nil || e.tip != tip { | |||
| 390 | + | var err error | |||
| 391 | + | if e, err = c.loadSchedules(ctx, repo); err != nil { | |||
| 392 | + | return fmt.Errorf("read the schedules: %w", err) | |||
| 393 | + | } | |||
| 394 | + | c.sched.put(repo, e) | |||
| 395 | + | } | |||
| 396 | + | idx, specs := dueFor(e, minute) | |||
| 397 | + | if len(idx) == 0 { | |||
| 398 | + | return nil | |||
| 399 | + | } | |||
| 400 | + | meta, err := c.cfg.Meta.Get(ctx, repo) | |||
| 401 | + | if err != nil { | |||
| 402 | + | c.sched.drop(repo) | |||
| 403 | + | return nil // deleted meanwhile | |||
| 404 | + | } | |||
| 405 | + | def, _, err := c.cfg.Repos.DefaultBranch(ctx, repo) | |||
| 406 | + | if err != nil { | |||
| 407 | + | return nil | |||
| 408 | + | } | |||
| 409 | + | commits, _, err := c.cfg.Git.ListRange(ctx, repo, domain.ObjectID{}, e.tip, 1) | |||
| 410 | + | if err != nil || len(commits) == 0 { | |||
| 411 | + | return fmt.Errorf("read the head commit of %s: %w", def, err) | |||
| 412 | + | } | |||
| 413 | + | var errs []error | |||
| 414 | + | for n, i := range idx { | |||
| 415 | + | errs = append(errs, c.fireSchedule(ctx, meta, def, e, commits[0], e.files[i], specs[n])) | |||
| 416 | + | } | |||
| 417 | + | return errors.Join(errs...) | |||
| 418 | + | } | |||
| 419 | + | ||||
| 420 | + | // fireSchedule starts the run of one scheduled workflow. | |||
| 421 | + | func (c *CI) fireSchedule(ctx context.Context, repo domain.Repo, def string, e *scheduleEntry, head domain.Commit, sw scheduledWorkflow, spec string) error { | |||
| 422 | + | wf, perrs := workflow.Parse(sw.path, sw.yaml) | |||
| 423 | + | if len(perrs) > 0 { | |||
| 424 | + | return nil | |||
| 425 | + | } | |||
| 426 | + | facts := pushFacts{ | |||
| 427 | + | update: domain.RefUpdate{Ref: "refs/heads/" + def, Old: domain.ZeroOID, New: e.tip.String()}, commit: e.tip, head: head, | |||
| 428 | + | defaultBranch: def, custom: &customEvent{name: eventSchedule, cron: spec}, | |||
| 429 | + | } | |||
| 430 | + | // Whoever moved the default branch to this commit, with their trust | |||
| 431 | + | // (ci_tips.go); the owner's name and no trust when that is not known. | |||
| 432 | + | actor, pusher, trusted := c.scheduleActor(ctx, repo.Name, def, e.tip) | |||
| 433 | + | t := ciTrigger{repo: repo.Name, actor: actor, pusher: pusher, trusted: trusted, at: c.now()} | |||
| 434 | + | if problem := c.matrixProblem(ctx, repo, t, facts, sw.path, wf); problem.text != "" { | |||
| 435 | + | name := sw.path | |||
| 436 | + | if problem.named { | |||
| 437 | + | name = workflowName(wf, sw.path) | |||
| 438 | + | } | |||
| 439 | + | return c.startInvalid(ctx, repo, t, facts, sw.path, name, problem.text) | |||
| 440 | + | } | |||
| 441 | + | _, err := c.launchRun(ctx, repo, t, facts, sw.path, string(sw.yaml), wf) | |||
| 442 | + | return err | |||
| 443 | + | } | |||
A internal/usecase/ci_tips.go +99 -0
| @@ -0,0 +1,99 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package usecase | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "errors" | |
| 8 | + | "fmt" | |
| 9 | + | "strings" | |
| 10 | + | ||
| 11 | + | "bnema.dev/ermite/internal/domain" | |
| 12 | + | "bnema.dev/ermite/internal/ports" | |
| 13 | + | ) | |
| 14 | + | ||
| 15 | + | // Who moves a branch or a tag (CI-3, D1; domain.CIRefTip). A scheduled run has | |
| 16 | + | // no pusher of its own: like GitHub's, it is the run of the last commit of the | |
| 17 | + | // default branch, and it takes the actor and the trust of whoever moved that | |
| 18 | + | // branch to that commit. A dispatched run is capped by the trust of the ref it | |
| 19 | + | // runs on. Without this a token limited to some branches could push the first | |
| 20 | + | // branch of an empty repository (which becomes the default branch) with a cron | |
| 21 | + | // workflow, and have it run as the owner with the owner's secrets. | |
| 22 | + | // | |
| 23 | + | // The record is written BEFORE the ref moves: by the pre-receive decision of a | |
| 24 | + | // push (Pushes.Check) and by a merge before it updates the target, under the | |
| 25 | + | // merge lock. Written after, a delayed write could leave an earlier pusher's | |
| 26 | + | // record on a tip moved back to the same commit (owner A, agent B, agent A). | |
| 27 | + | // Writing it is part of the decision: when it fails the push or the merge is | |
| 28 | + | // refused, never let through unrecorded. A push that is refused later leaves a | |
| 29 | + | // record that names an object the ref does not have, which is not trusted. | |
| 30 | + | ||
| 31 | + | // recordRefTips stores who is about to move the branches and tags of a push or a | |
| 32 | + | // merge, and removes the records of the refs it deletes. actor is what the runs | |
| 33 | + | // show, pusher and trusted are as for a push run. refs/pull/* is ermite's own | |
| 34 | + | // and has none. | |
| 35 | + | func recordRefTips(ctx context.Context, tips ports.CIRefTipStore, repo domain.RepoName, updates []domain.RefUpdate, actor string, pusher domain.CIPusher, trusted bool) error { | |
| 36 | + | if tips == nil { | |
| 37 | + | return nil | |
| 38 | + | } | |
| 39 | + | for _, u := range updates { | |
| 40 | + | if !strings.HasPrefix(u.Ref, "refs/heads/") && !strings.HasPrefix(u.Ref, "refs/tags/") { | |
| 41 | + | continue | |
| 42 | + | } | |
| 43 | + | var err error | |
| 44 | + | if u.IsDelete() { | |
| 45 | + | err = tips.DeleteRefTip(ctx, repo, u.Ref) | |
| 46 | + | } else { | |
| 47 | + | err = tips.PutRefTip(ctx, domain.CIRefTip{Repo: repo, Ref: u.Ref, Sha: u.New, Actor: actor, Pusher: pusher, Trusted: trusted}) | |
| 48 | + | } | |
| 49 | + | if errors.Is(err, domain.ErrNotFound) { | |
| 50 | + | continue // the repository is gone: nothing to record, nothing to run | |
| 51 | + | } | |
| 52 | + | if err != nil { | |
| 53 | + | return fmt.Errorf("record who pushes %s: %w", u.Ref, err) | |
| 54 | + | } | |
| 55 | + | } | |
| 56 | + | return nil | |
| 57 | + | } | |
| 58 | + | ||
| 59 | + | // refTip returns the record of a ref when it names the object the ref has now; | |
| 60 | + | // ok is false when there is none, it is for another object, or it cannot be | |
| 61 | + | // read (logged): the caller then treats the ref as not trusted. | |
| 62 | + | func (c *CI) refTip(ctx context.Context, repo domain.RepoName, ref string, objects ...domain.ObjectID) (domain.CIRefTip, bool) { | |
| 63 | + | if c.cfg.RefTips == nil { | |
| 64 | + | return domain.CIRefTip{}, false | |
| 65 | + | } | |
| 66 | + | t, err := c.cfg.RefTips.GetRefTip(ctx, repo, ref) | |
| 67 | + | if err != nil { | |
| 68 | + | if !errors.Is(err, domain.ErrNotFound) { | |
| 69 | + | ctxLog(ctx).Error().Err(err).Str("repo", repo.String()).Str("ref", ref).Msg("ci: read who pushed a ref: it is not trusted") | |
| 70 | + | } | |
| 71 | + | return domain.CIRefTip{}, false | |
| 72 | + | } | |
| 73 | + | for _, o := range objects { | |
| 74 | + | if !o.IsZero() && t.Sha == o.String() { | |
| 75 | + | return t, true | |
| 76 | + | } | |
| 77 | + | } | |
| 78 | + | return domain.CIRefTip{}, false | |
| 79 | + | } | |
| 80 | + | ||
| 81 | + | // scheduleActor returns the actor, the pusher and the trust of a scheduled run | |
| 82 | + | // on the tip of a branch: those of the record of the branch when it names this | |
| 83 | + | // very commit; otherwise the owner's name, no pusher and no trust. | |
| 84 | + | func (c *CI) scheduleActor(ctx context.Context, repo domain.RepoName, branch string, tip domain.ObjectID) (actor string, pusher domain.CIPusher, trusted bool) { | |
| 85 | + | if t, ok := c.refTip(ctx, repo, "refs/heads/"+branch, tip); ok { | |
| 86 | + | return t.Actor, t.Pusher, t.Trusted | |
| 87 | + | } | |
| 88 | + | return authorOf(domain.Owner()), domain.CIPusher{}, false | |
| 89 | + | } | |
| 90 | + | ||
| 91 | + | // refTrusted reports whether the ref was set to exactly the pinned object by a | |
| 92 | + | // trusted principal. The record must name the object the dispatch resolved (the | |
| 93 | + | // commit, or the tag object of an annotated tag), which the dispatch captured | |
| 94 | + | // together with the commit it runs: the refs are not read again here, so a tag | |
| 95 | + | // retargeted since cannot lend its new record to the old commit. | |
| 96 | + | func (c *CI) refTrusted(ctx context.Context, repo domain.RepoName, pin pinnedRef) bool { | |
| 97 | + | t, ok := c.refTip(ctx, repo, pin.ref, pin.object) | |
| 98 | + | return ok && t.Trusted | |
| 99 | + | } | |
| @@ -0,0 +1,99 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package usecase | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "context" | |||
| 7 | + | "errors" | |||
| 8 | + | "fmt" | |||
| 9 | + | "strings" | |||
| 10 | + | ||||
| 11 | + | "bnema.dev/ermite/internal/domain" | |||
| 12 | + | "bnema.dev/ermite/internal/ports" | |||
| 13 | + | ) | |||
| 14 | + | ||||
| 15 | + | // Who moves a branch or a tag (CI-3, D1; domain.CIRefTip). A scheduled run has | |||
| 16 | + | // no pusher of its own: like GitHub's, it is the run of the last commit of the | |||
| 17 | + | // default branch, and it takes the actor and the trust of whoever moved that | |||
| 18 | + | // branch to that commit. A dispatched run is capped by the trust of the ref it | |||
| 19 | + | // runs on. Without this a token limited to some branches could push the first | |||
| 20 | + | // branch of an empty repository (which becomes the default branch) with a cron | |||
| 21 | + | // workflow, and have it run as the owner with the owner's secrets. | |||
| 22 | + | // | |||
| 23 | + | // The record is written BEFORE the ref moves: by the pre-receive decision of a | |||
| 24 | + | // push (Pushes.Check) and by a merge before it updates the target, under the | |||
| 25 | + | // merge lock. Written after, a delayed write could leave an earlier pusher's | |||
| 26 | + | // record on a tip moved back to the same commit (owner A, agent B, agent A). | |||
| 27 | + | // Writing it is part of the decision: when it fails the push or the merge is | |||
| 28 | + | // refused, never let through unrecorded. A push that is refused later leaves a | |||
| 29 | + | // record that names an object the ref does not have, which is not trusted. | |||
| 30 | + | ||||
| 31 | + | // recordRefTips stores who is about to move the branches and tags of a push or a | |||
| 32 | + | // merge, and removes the records of the refs it deletes. actor is what the runs | |||
| 33 | + | // show, pusher and trusted are as for a push run. refs/pull/* is ermite's own | |||
| 34 | + | // and has none. | |||
| 35 | + | func recordRefTips(ctx context.Context, tips ports.CIRefTipStore, repo domain.RepoName, updates []domain.RefUpdate, actor string, pusher domain.CIPusher, trusted bool) error { | |||
| 36 | + | if tips == nil { | |||
| 37 | + | return nil | |||
| 38 | + | } | |||
| 39 | + | for _, u := range updates { | |||
| 40 | + | if !strings.HasPrefix(u.Ref, "refs/heads/") && !strings.HasPrefix(u.Ref, "refs/tags/") { | |||
| 41 | + | continue | |||
| 42 | + | } | |||
| 43 | + | var err error | |||
| 44 | + | if u.IsDelete() { | |||
| 45 | + | err = tips.DeleteRefTip(ctx, repo, u.Ref) | |||
| 46 | + | } else { | |||
| 47 | + | err = tips.PutRefTip(ctx, domain.CIRefTip{Repo: repo, Ref: u.Ref, Sha: u.New, Actor: actor, Pusher: pusher, Trusted: trusted}) | |||
| 48 | + | } | |||
| 49 | + | if errors.Is(err, domain.ErrNotFound) { | |||
| 50 | + | continue // the repository is gone: nothing to record, nothing to run | |||
| 51 | + | } | |||
| 52 | + | if err != nil { | |||
| 53 | + | return fmt.Errorf("record who pushes %s: %w", u.Ref, err) | |||
| 54 | + | } | |||
| 55 | + | } | |||
| 56 | + | return nil | |||
| 57 | + | } | |||
| 58 | + | ||||
| 59 | + | // refTip returns the record of a ref when it names the object the ref has now; | |||
| 60 | + | // ok is false when there is none, it is for another object, or it cannot be | |||
| 61 | + | // read (logged): the caller then treats the ref as not trusted. | |||
| 62 | + | func (c *CI) refTip(ctx context.Context, repo domain.RepoName, ref string, objects ...domain.ObjectID) (domain.CIRefTip, bool) { | |||
| 63 | + | if c.cfg.RefTips == nil { | |||
| 64 | + | return domain.CIRefTip{}, false | |||
| 65 | + | } | |||
| 66 | + | t, err := c.cfg.RefTips.GetRefTip(ctx, repo, ref) | |||
| 67 | + | if err != nil { | |||
| 68 | + | if !errors.Is(err, domain.ErrNotFound) { | |||
| 69 | + | ctxLog(ctx).Error().Err(err).Str("repo", repo.String()).Str("ref", ref).Msg("ci: read who pushed a ref: it is not trusted") | |||
| 70 | + | } | |||
| 71 | + | return domain.CIRefTip{}, false | |||
| 72 | + | } | |||
| 73 | + | for _, o := range objects { | |||
| 74 | + | if !o.IsZero() && t.Sha == o.String() { | |||
| 75 | + | return t, true | |||
| 76 | + | } | |||
| 77 | + | } | |||
| 78 | + | return domain.CIRefTip{}, false | |||
| 79 | + | } | |||
| 80 | + | ||||
| 81 | + | // scheduleActor returns the actor, the pusher and the trust of a scheduled run | |||
| 82 | + | // on the tip of a branch: those of the record of the branch when it names this | |||
| 83 | + | // very commit; otherwise the owner's name, no pusher and no trust. | |||
| 84 | + | func (c *CI) scheduleActor(ctx context.Context, repo domain.RepoName, branch string, tip domain.ObjectID) (actor string, pusher domain.CIPusher, trusted bool) { | |||
| 85 | + | if t, ok := c.refTip(ctx, repo, "refs/heads/"+branch, tip); ok { | |||
| 86 | + | return t.Actor, t.Pusher, t.Trusted | |||
| 87 | + | } | |||
| 88 | + | return authorOf(domain.Owner()), domain.CIPusher{}, false | |||
| 89 | + | } | |||
| 90 | + | ||||
| 91 | + | // refTrusted reports whether the ref was set to exactly the pinned object by a | |||
| 92 | + | // trusted principal. The record must name the object the dispatch resolved (the | |||
| 93 | + | // commit, or the tag object of an annotated tag), which the dispatch captured | |||
| 94 | + | // together with the commit it runs: the refs are not read again here, so a tag | |||
| 95 | + | // retargeted since cannot lend its new record to the old commit. | |||
| 96 | + | func (c *CI) refTrusted(ctx context.Context, repo domain.RepoName, pin pinnedRef) bool { | |||
| 97 | + | t, ok := c.refTip(ctx, repo, pin.ref, pin.object) | |||
| 98 | + | return ok && t.Trusted | |||
| 99 | + | } | |||
M internal/usecase/ci_trigger.go +34 -10
| @@ -54,6 +54,9 @@ type ciTrigger struct { | |||
| 54 | 54 | // with GITHUB_TOKEN start no run): that is what stops a workflow that pushes | |
| 55 | 55 | // from running itself again. | |
| 56 | 56 | func (c *CI) OnPush(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, pusher domain.Principal) { | |
| 57 | + | // Whoever pushed, the workflow files of the default branch may have changed. | |
| 58 | + | // (Who pushed was recorded before the refs moved: Pushes.Check.) | |
| 59 | + | c.markSchedulesDirty(repo, updates) | |
| 57 | 60 | // The merge requests follow the push whoever made it (their refs); only a | |
| 58 | 61 | // push that is not a job's starts pull_request runs. | |
| 59 | 62 | defer c.onBranchesMoved(ctx, repo, updates, authorOf(pusher), domain.PusherOf(pusher), domain.TrustedActor(pusher), !pusher.ViaJobToken()) | |
| @@ -72,6 +75,7 @@ func (c *CI) OnPush(ctx context.Context, repo domain.RepoName, updates []domain. | |||
| 72 | 75 | // merging. For the same reason the run is trusted (gets secrets) only when the | |
| 73 | 76 | // merger is: the displayed actor is not the one that decides. | |
| 74 | 77 | func (c *CI) OnMerge(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, merger domain.Principal) { | |
| 78 | + | c.markSchedulesDirty(repo, updates) | |
| 75 | 79 | // The merge moved a target branch: the test merges of the other requests | |
| 76 | 80 | // into it are computed again (a merge starts no synchronize run), after the | |
| 77 | 81 | // push workflows are queued. | |
| @@ -181,13 +185,19 @@ type pushFacts struct { | |||
| 181 | 185 | // refs/pull/<n>/merge (or /head), commit the test merge commit (or the | |
| 182 | 186 | // head) and head the head commit. | |
| 183 | 187 | pull *pullFacts | |
| 188 | + | // custom is set for the events that are not a push or a merge request: | |
| 189 | + | // workflow_dispatch, schedule and workflow_run (ci_events.go). | |
| 190 | + | custom *customEvent | |
| 184 | 191 | } | |
| 185 | 192 | ||
| 186 | 193 | // eventName is the event of the run, as the run records and github.event_name | |
| 187 | 194 | // show it. | |
| 188 | 195 | func (f pushFacts) eventName() string { | |
| 189 | - | if f.pull != nil { | |
| 196 | + | switch { | |
| 197 | + | case f.pull != nil: | |
| 190 | 198 | return "pull_request" | |
| 199 | + | case f.custom != nil: | |
| 200 | + | return f.custom.name | |
| 191 | 201 | } | |
| 192 | 202 | return "push" | |
| 193 | 203 | } | |
| @@ -195,8 +205,11 @@ func (f pushFacts) eventName() string { | |||
| 195 | 205 | // runTitle is the title of a run without run-name: the commit message of a | |
| 196 | 206 | // push, the title of the merge request for a pull_request. | |
| 197 | 207 | func (f pushFacts) runTitle() string { | |
| 198 | - | if f.pull != nil { | |
| 208 | + | switch { | |
| 209 | + | case f.pull != nil: | |
| 199 | 210 | return firstLine(f.pull.mr.Title.String()) | |
| 211 | + | case f.custom != nil && f.custom.title != "": | |
| 212 | + | return firstLine(f.custom.title) | |
| 200 | 213 | } | |
| 201 | 214 | return firstLine(f.head.Message) | |
| 202 | 215 | } | |
| @@ -469,6 +482,8 @@ func (c *CI) startInvalid(ctx context.Context, repo domain.Repo, t ciTrigger, f | |||
| 469 | 482 | return err | |
| 470 | 483 | } | |
| 471 | 484 | c.publishRun(ctx, repo, draft.Run) | |
| 485 | + | c.onRunEvent(ctx, draft.Run, runRequested) | |
| 486 | + | c.onRunEvent(ctx, draft.Run, runCompleted) | |
| 472 | 487 | log := logRun(ctx, draft.Run) | |
| 473 | 488 | log.Info().Str("workflow", wfPath).Msg("ci run failed: invalid workflow file") | |
| 474 | 489 | return nil | |
| @@ -481,6 +496,13 @@ func (c *CI) publishRun(ctx context.Context, repo domain.Repo, run domain.CIRun) | |||
| 481 | 496 | // startRun creates the run of a matching workflow with one waiting record per | |
| 482 | 497 | // job of the file, then plans the jobs that can start. | |
| 483 | 498 | func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f pushFacts, wfPath, yaml string, wf *workflow.Workflow) error { | |
| 499 | + | _, err := c.launchRun(ctx, repo, t, f, wfPath, yaml, wf) | |
| 500 | + | return err | |
| 501 | + | } | |
| 502 | + | ||
| 503 | + | // launchRun is startRun that also returns the run it created, as it was when | |
| 504 | + | // created (the zero run when the repository or the merge request is gone). | |
| 505 | + | func (c *CI) launchRun(ctx context.Context, repo domain.Repo, t ciTrigger, f pushFacts, wfPath, yaml string, wf *workflow.Workflow) (domain.CIRun, error) { | |
| 484 | 506 | now := c.now() | |
| 485 | 507 | name := workflowName(wf, wfPath) | |
| 486 | 508 | // The title of a run without run-name is the commit message for a branch. | |
| @@ -498,7 +520,7 @@ func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f push | |||
| 498 | 520 | } | |
| 499 | 521 | vars, err := c.varsContext(ctx, repo.Name) | |
| 500 | 522 | if err != nil { | |
| 501 | - | return err | |
| 523 | + | return domain.CIRun{}, err | |
| 502 | 524 | } | |
| 503 | 525 | draft, err := c.createRun(ctx, f, repo.Name, wfPath, func(id, number int64) ports.CIRunDraft { | |
| 504 | 526 | run := domain.CIRun{ | |
| @@ -527,18 +549,19 @@ func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f push | |||
| 527 | 549 | return d | |
| 528 | 550 | }) | |
| 529 | 551 | if errors.Is(err, domain.ErrNotFound) || errors.Is(err, errPullClosed) { | |
| 530 | - | return nil | |
| 552 | + | return domain.CIRun{}, nil | |
| 531 | 553 | } | |
| 532 | 554 | if err != nil { | |
| 533 | - | return err | |
| 555 | + | return domain.CIRun{}, err | |
| 534 | 556 | } | |
| 535 | 557 | c.publishRun(ctx, repo, draft.Run) | |
| 536 | 558 | for _, j := range draft.Jobs { | |
| 537 | 559 | c.cfg.Bus.Publish(ctx, domain.JobUpdated{Job: j, At: now, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt}) | |
| 538 | 560 | } | |
| 561 | + | c.onRunEvent(ctx, draft.Run, runRequested) | |
| 539 | 562 | log := logRun(ctx, draft.Run) | |
| 540 | - | log.Info().Str("workflow", wfPath).Str("ref", f.update.Ref).Int("jobs", len(draft.Jobs)).Msg("ci run created") | |
| 541 | - | return c.advanceRun(ctx, draft.Run.ID) | |
| 563 | + | log.Info().Str("workflow", wfPath).Str("ref", f.update.Ref).Str("event", draft.Run.Event).Int("jobs", len(draft.Jobs)).Msg("ci run created") | |
| 564 | + | return draft.Run, c.advanceRun(ctx, draft.Run.ID) | |
| 542 | 565 | } | |
| 543 | 566 | ||
| 544 | 567 | // workflowName is the name of a workflow, the path of its file when it has no | |
| @@ -620,7 +643,7 @@ func (c *CI) evalRunName(ctx context.Context, wf *workflow.Workflow, githubJSON | |||
| 620 | 643 | return "", false | |
| 621 | 644 | } | |
| 622 | 645 | env := template.Env{Contexts: map[string]expr.Value{ | |
| 623 | - | "github": gh, "inputs": expr.NewDictionary(), "vars": vars, | |
| 646 | + | "github": gh, "inputs": eventInputs(wf, githubJSON), "vars": vars, | |
| 624 | 647 | }} | |
| 625 | 648 | name, err := template.EvaluateString(ctx, wf.RunName, env) | |
| 626 | 649 | if err != nil { | |
| @@ -652,7 +675,8 @@ func (c *CI) matrixProblem(ctx context.Context, repo domain.Repo, t ciTrigger, f | |||
| 652 | 675 | return matrixIssue{} // nothing to expand: no context to build | |
| 653 | 676 | } | |
| 654 | 677 | probe := domain.CIRun{Repo: repo.Name, Ref: f.update.Ref, Sha: f.commit.String(), Actor: t.actor, WorkflowPath: wfPath, WorkflowName: wf.Name, CreatedAt: c.now()} | |
| 655 | - | gh, err := expr.FromJSON([]byte(c.githubJSON(ctx, repo, probe, f))) | |
| 678 | + | ghJSON := c.githubJSON(ctx, repo, probe, f) | |
| 679 | + | gh, err := expr.FromJSON([]byte(ghJSON)) | |
| 656 | 680 | if err != nil { | |
| 657 | 681 | return matrixIssue{} | |
| 658 | 682 | } | |
| @@ -663,7 +687,7 @@ func (c *CI) matrixProblem(ctx context.Context, repo domain.Repo, t ciTrigger, f | |||
| 663 | 687 | vars = expr.NewDictionary() | |
| 664 | 688 | } | |
| 665 | 689 | env := template.Env{Contexts: map[string]expr.Value{ | |
| 666 | - | "github": gh, "needs": expr.NewDictionary(), "inputs": expr.NewDictionary(), "vars": vars, | |
| 690 | + | "github": gh, "needs": expr.NewDictionary(), "inputs": eventInputs(wf, ghJSON), "vars": vars, | |
| 667 | 691 | }} | |
| 668 | 692 | for _, j := range wf.Jobs { | |
| 669 | 693 | _, err := j.EvaluateStrategy(ctx, env) | |
| @@ -54,6 +54,9 @@ type ciTrigger struct { | |||||
| 54 | // with GITHUB_TOKEN start no run): that is what stops a workflow that pushes | 54 | // with GITHUB_TOKEN start no run): that is what stops a workflow that pushes | ||
| 55 | // from running itself again. | 55 | // from running itself again. | ||
| 56 | func (c *CI) OnPush(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, pusher domain.Principal) { | 56 | func (c *CI) OnPush(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, pusher domain.Principal) { | ||
| 57 | + | // Whoever pushed, the workflow files of the default branch may have changed. | |||
| 58 | + | // (Who pushed was recorded before the refs moved: Pushes.Check.) | |||
| 59 | + | c.markSchedulesDirty(repo, updates) | |||
| 57 | // The merge requests follow the push whoever made it (their refs); only a | 60 | // The merge requests follow the push whoever made it (their refs); only a | ||
| 58 | // push that is not a job's starts pull_request runs. | 61 | // push that is not a job's starts pull_request runs. | ||
| 59 | defer c.onBranchesMoved(ctx, repo, updates, authorOf(pusher), domain.PusherOf(pusher), domain.TrustedActor(pusher), !pusher.ViaJobToken()) | 62 | defer c.onBranchesMoved(ctx, repo, updates, authorOf(pusher), domain.PusherOf(pusher), domain.TrustedActor(pusher), !pusher.ViaJobToken()) | ||
| @@ -72,6 +75,7 @@ func (c *CI) OnPush(ctx context.Context, repo domain.RepoName, updates []domain. | |||||
| 72 | // merging. For the same reason the run is trusted (gets secrets) only when the | 75 | // merging. For the same reason the run is trusted (gets secrets) only when the | ||
| 73 | // merger is: the displayed actor is not the one that decides. | 76 | // merger is: the displayed actor is not the one that decides. | ||
| 74 | func (c *CI) OnMerge(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, merger domain.Principal) { | 77 | func (c *CI) OnMerge(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, merger domain.Principal) { | ||
| 78 | + | c.markSchedulesDirty(repo, updates) | |||
| 75 | // The merge moved a target branch: the test merges of the other requests | 79 | // The merge moved a target branch: the test merges of the other requests | ||
| 76 | // into it are computed again (a merge starts no synchronize run), after the | 80 | // into it are computed again (a merge starts no synchronize run), after the | ||
| 77 | // push workflows are queued. | 81 | // push workflows are queued. | ||
| @@ -181,13 +185,19 @@ type pushFacts struct { | |||||
| 181 | // refs/pull/<n>/merge (or /head), commit the test merge commit (or the | 185 | // refs/pull/<n>/merge (or /head), commit the test merge commit (or the | ||
| 182 | // head) and head the head commit. | 186 | // head) and head the head commit. | ||
| 183 | pull *pullFacts | 187 | pull *pullFacts | ||
| 188 | + | // custom is set for the events that are not a push or a merge request: | |||
| 189 | + | // workflow_dispatch, schedule and workflow_run (ci_events.go). | |||
| 190 | + | custom *customEvent | |||
| 184 | } | 191 | } | ||
| 185 | 192 | ||||
| 186 | // eventName is the event of the run, as the run records and github.event_name | 193 | // eventName is the event of the run, as the run records and github.event_name | ||
| 187 | // show it. | 194 | // show it. | ||
| 188 | func (f pushFacts) eventName() string { | 195 | func (f pushFacts) eventName() string { | ||
| 189 | - | if f.pull != nil { | 196 | + | switch { |
| 197 | + | case f.pull != nil: | |||
| 190 | return "pull_request" | 198 | return "pull_request" | ||
| 199 | + | case f.custom != nil: | |||
| 200 | + | return f.custom.name | |||
| 191 | } | 201 | } | ||
| 192 | return "push" | 202 | return "push" | ||
| 193 | } | 203 | } | ||
| @@ -195,8 +205,11 @@ func (f pushFacts) eventName() string { | |||||
| 195 | // runTitle is the title of a run without run-name: the commit message of a | 205 | // runTitle is the title of a run without run-name: the commit message of a | ||
| 196 | // push, the title of the merge request for a pull_request. | 206 | // push, the title of the merge request for a pull_request. | ||
| 197 | func (f pushFacts) runTitle() string { | 207 | func (f pushFacts) runTitle() string { | ||
| 198 | - | if f.pull != nil { | 208 | + | switch { |
| 209 | + | case f.pull != nil: | |||
| 199 | return firstLine(f.pull.mr.Title.String()) | 210 | return firstLine(f.pull.mr.Title.String()) | ||
| 211 | + | case f.custom != nil && f.custom.title != "": | |||
| 212 | + | return firstLine(f.custom.title) | |||
| 200 | } | 213 | } | ||
| 201 | return firstLine(f.head.Message) | 214 | return firstLine(f.head.Message) | ||
| 202 | } | 215 | } | ||
| @@ -469,6 +482,8 @@ func (c *CI) startInvalid(ctx context.Context, repo domain.Repo, t ciTrigger, f | |||||
| 469 | return err | 482 | return err | ||
| 470 | } | 483 | } | ||
| 471 | c.publishRun(ctx, repo, draft.Run) | 484 | c.publishRun(ctx, repo, draft.Run) | ||
| 485 | + | c.onRunEvent(ctx, draft.Run, runRequested) | |||
| 486 | + | c.onRunEvent(ctx, draft.Run, runCompleted) | |||
| 472 | log := logRun(ctx, draft.Run) | 487 | log := logRun(ctx, draft.Run) | ||
| 473 | log.Info().Str("workflow", wfPath).Msg("ci run failed: invalid workflow file") | 488 | log.Info().Str("workflow", wfPath).Msg("ci run failed: invalid workflow file") | ||
| 474 | return nil | 489 | return nil | ||
| @@ -481,6 +496,13 @@ func (c *CI) publishRun(ctx context.Context, repo domain.Repo, run domain.CIRun) | |||||
| 481 | // startRun creates the run of a matching workflow with one waiting record per | 496 | // startRun creates the run of a matching workflow with one waiting record per | ||
| 482 | // job of the file, then plans the jobs that can start. | 497 | // job of the file, then plans the jobs that can start. | ||
| 483 | func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f pushFacts, wfPath, yaml string, wf *workflow.Workflow) error { | 498 | func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f pushFacts, wfPath, yaml string, wf *workflow.Workflow) error { | ||
| 499 | + | _, err := c.launchRun(ctx, repo, t, f, wfPath, yaml, wf) | |||
| 500 | + | return err | |||
| 501 | + | } | |||
| 502 | + | ||||
| 503 | + | // launchRun is startRun that also returns the run it created, as it was when | |||
| 504 | + | // created (the zero run when the repository or the merge request is gone). | |||
| 505 | + | func (c *CI) launchRun(ctx context.Context, repo domain.Repo, t ciTrigger, f pushFacts, wfPath, yaml string, wf *workflow.Workflow) (domain.CIRun, error) { | |||
| 484 | now := c.now() | 506 | now := c.now() | ||
| 485 | name := workflowName(wf, wfPath) | 507 | name := workflowName(wf, wfPath) | ||
| 486 | // The title of a run without run-name is the commit message for a branch. | 508 | // The title of a run without run-name is the commit message for a branch. | ||
| @@ -498,7 +520,7 @@ func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f push | |||||
| 498 | } | 520 | } | ||
| 499 | vars, err := c.varsContext(ctx, repo.Name) | 521 | vars, err := c.varsContext(ctx, repo.Name) | ||
| 500 | if err != nil { | 522 | if err != nil { | ||
| 501 | - | return err | 523 | + | return domain.CIRun{}, err |
| 502 | } | 524 | } | ||
| 503 | draft, err := c.createRun(ctx, f, repo.Name, wfPath, func(id, number int64) ports.CIRunDraft { | 525 | draft, err := c.createRun(ctx, f, repo.Name, wfPath, func(id, number int64) ports.CIRunDraft { | ||
| 504 | run := domain.CIRun{ | 526 | run := domain.CIRun{ | ||
| @@ -527,18 +549,19 @@ func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f push | |||||
| 527 | return d | 549 | return d | ||
| 528 | }) | 550 | }) | ||
| 529 | if errors.Is(err, domain.ErrNotFound) || errors.Is(err, errPullClosed) { | 551 | if errors.Is(err, domain.ErrNotFound) || errors.Is(err, errPullClosed) { | ||
| 530 | - | return nil | 552 | + | return domain.CIRun{}, nil |
| 531 | } | 553 | } | ||
| 532 | if err != nil { | 554 | if err != nil { | ||
| 533 | - | return err | 555 | + | return domain.CIRun{}, err |
| 534 | } | 556 | } | ||
| 535 | c.publishRun(ctx, repo, draft.Run) | 557 | c.publishRun(ctx, repo, draft.Run) | ||
| 536 | for _, j := range draft.Jobs { | 558 | for _, j := range draft.Jobs { | ||
| 537 | c.cfg.Bus.Publish(ctx, domain.JobUpdated{Job: j, At: now, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt}) | 559 | c.cfg.Bus.Publish(ctx, domain.JobUpdated{Job: j, At: now, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt}) | ||
| 538 | } | 560 | } | ||
| 561 | + | c.onRunEvent(ctx, draft.Run, runRequested) | |||
| 539 | log := logRun(ctx, draft.Run) | 562 | log := logRun(ctx, draft.Run) | ||
| 540 | - | log.Info().Str("workflow", wfPath).Str("ref", f.update.Ref).Int("jobs", len(draft.Jobs)).Msg("ci run created") | 563 | + | log.Info().Str("workflow", wfPath).Str("ref", f.update.Ref).Str("event", draft.Run.Event).Int("jobs", len(draft.Jobs)).Msg("ci run created") |
| 541 | - | return c.advanceRun(ctx, draft.Run.ID) | 564 | + | return draft.Run, c.advanceRun(ctx, draft.Run.ID) |
| 542 | } | 565 | } | ||
| 543 | 566 | ||||
| 544 | // workflowName is the name of a workflow, the path of its file when it has no | 567 | // workflowName is the name of a workflow, the path of its file when it has no | ||
| @@ -620,7 +643,7 @@ func (c *CI) evalRunName(ctx context.Context, wf *workflow.Workflow, githubJSON | |||||
| 620 | return "", false | 643 | return "", false | ||
| 621 | } | 644 | } | ||
| 622 | env := template.Env{Contexts: map[string]expr.Value{ | 645 | env := template.Env{Contexts: map[string]expr.Value{ | ||
| 623 | - | "github": gh, "inputs": expr.NewDictionary(), "vars": vars, | 646 | + | "github": gh, "inputs": eventInputs(wf, githubJSON), "vars": vars, |
| 624 | }} | 647 | }} | ||
| 625 | name, err := template.EvaluateString(ctx, wf.RunName, env) | 648 | name, err := template.EvaluateString(ctx, wf.RunName, env) | ||
| 626 | if err != nil { | 649 | if err != nil { | ||
| @@ -652,7 +675,8 @@ func (c *CI) matrixProblem(ctx context.Context, repo domain.Repo, t ciTrigger, f | |||||
| 652 | return matrixIssue{} // nothing to expand: no context to build | 675 | return matrixIssue{} // nothing to expand: no context to build | ||
| 653 | } | 676 | } | ||
| 654 | probe := domain.CIRun{Repo: repo.Name, Ref: f.update.Ref, Sha: f.commit.String(), Actor: t.actor, WorkflowPath: wfPath, WorkflowName: wf.Name, CreatedAt: c.now()} | 677 | probe := domain.CIRun{Repo: repo.Name, Ref: f.update.Ref, Sha: f.commit.String(), Actor: t.actor, WorkflowPath: wfPath, WorkflowName: wf.Name, CreatedAt: c.now()} | ||
| 655 | - | gh, err := expr.FromJSON([]byte(c.githubJSON(ctx, repo, probe, f))) | 678 | + | ghJSON := c.githubJSON(ctx, repo, probe, f) |
| 679 | + | gh, err := expr.FromJSON([]byte(ghJSON)) | |||
| 656 | if err != nil { | 680 | if err != nil { | ||
| 657 | return matrixIssue{} | 681 | return matrixIssue{} | ||
| 658 | } | 682 | } | ||
| @@ -663,7 +687,7 @@ func (c *CI) matrixProblem(ctx context.Context, repo domain.Repo, t ciTrigger, f | |||||
| 663 | vars = expr.NewDictionary() | 687 | vars = expr.NewDictionary() | ||
| 664 | } | 688 | } | ||
| 665 | env := template.Env{Contexts: map[string]expr.Value{ | 689 | env := template.Env{Contexts: map[string]expr.Value{ | ||
| 666 | - | "github": gh, "needs": expr.NewDictionary(), "inputs": expr.NewDictionary(), "vars": vars, | 690 | + | "github": gh, "needs": expr.NewDictionary(), "inputs": eventInputs(wf, ghJSON), "vars": vars, |
| 667 | }} | 691 | }} | ||
| 668 | for _, j := range wf.Jobs { | 692 | for _, j := range wf.Jobs { | ||
| 669 | _, err := j.EvaluateStrategy(ctx, env) | 693 | _, err := j.EvaluateStrategy(ctx, env) | ||
M internal/usecase/ci_watch.go +7 -0
| @@ -307,6 +307,13 @@ func (c *CI) RunIDs(ctx context.Context, repo domain.RepoName) ([]int64, error) | |||
| 307 | 307 | return c.cfg.Store.RunIDs(ctx, repo) | |
| 308 | 308 | } | |
| 309 | 309 | ||
| 310 | + | // ForgetRepo drops what CI holds in memory for a repository that was deleted: | |
| 311 | + | // its cached schedules, so that it is never scheduled again (D12). | |
| 312 | + | func (c *CI) ForgetRepo(repo domain.RepoName) { | |
| 313 | + | c.sched.drop(repo) | |
| 314 | + | c.sched.forgetDirty(repo) | |
| 315 | + | } | |
| 316 | + | ||
| 310 | 317 | // RemoveLogs deletes the logs and the summaries of runs of a repository that | |
| 311 | 318 | // was deleted. | |
| 312 | 319 | func (c *CI) RemoveLogs(ctx context.Context, runIDs []int64) error { | |
| @@ -307,6 +307,13 @@ func (c *CI) RunIDs(ctx context.Context, repo domain.RepoName) ([]int64, error) | |||||
| 307 | return c.cfg.Store.RunIDs(ctx, repo) | 307 | return c.cfg.Store.RunIDs(ctx, repo) | ||
| 308 | } | 308 | } | ||
| 309 | 309 | ||||
| 310 | + | // ForgetRepo drops what CI holds in memory for a repository that was deleted: | |||
| 311 | + | // its cached schedules, so that it is never scheduled again (D12). | |||
| 312 | + | func (c *CI) ForgetRepo(repo domain.RepoName) { | |||
| 313 | + | c.sched.drop(repo) | |||
| 314 | + | c.sched.forgetDirty(repo) | |||
| 315 | + | } | |||
| 316 | + | ||||
| 310 | // RemoveLogs deletes the logs and the summaries of runs of a repository that | 317 | // RemoveLogs deletes the logs and the summaries of runs of a repository that | ||
| 311 | // was deleted. | 318 | // was deleted. | ||
| 312 | func (c *CI) RemoveLogs(ctx context.Context, runIDs []int64) error { | 319 | func (c *CI) RemoveLogs(ctx context.Context, runIDs []int64) error { | ||
A internal/usecase/ci_workflowrun.go +252 -0
| @@ -0,0 +1,252 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package usecase | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "encoding/json" | |
| 8 | + | "errors" | |
| 9 | + | "fmt" | |
| 10 | + | "runtime/debug" | |
| 11 | + | "slices" | |
| 12 | + | "strings" | |
| 13 | + | ||
| 14 | + | "github.com/bnema/zerowrap" | |
| 15 | + | ||
| 16 | + | "bnema.dev/ermite-actions/workflow" | |
| 17 | + | "bnema.dev/ermite/internal/domain" | |
| 18 | + | ) | |
| 19 | + | ||
| 20 | + | // The workflow_run event (CI-3, D5). A run that is created ("requested") or | |
| 21 | + | // completed ("completed") triggers the workflows of the default branch that list | |
| 22 | + | // its workflow by name under workflow_run.workflows. | |
| 23 | + | // | |
| 24 | + | // The triggered run is read from the default branch's latest commit and runs on | |
| 25 | + | // it. It inherits the actor, the pusher and the trusted flag of the triggering | |
| 26 | + | // run, and never gets more: an agent's untrusted run cannot launder secrets | |
| 27 | + | // through a workflow_run (D1). A job token's push starts no run, but a run | |
| 28 | + | // started by a push of anyone else triggers workflow_run workflows even when the | |
| 29 | + | // triggering run was itself a workflow_run, up to the chain limit. | |
| 30 | + | const ( | |
| 31 | + | // maxWorkflowRunDepth is the number of workflow_run levels in a chain: | |
| 32 | + | // docs.github.com ("Events that trigger workflows", workflow_run): "You can't | |
| 33 | + | // use workflow_run to chain together more than three levels of workflows" | |
| 34 | + | // (A -> B -> C -> D runs, E does not). Recalled from memory, not fetched. | |
| 35 | + | maxWorkflowRunDepth = 3 | |
| 36 | + | runRequested = "requested" | |
| 37 | + | runCompleted = "completed" | |
| 38 | + | ) | |
| 39 | + | ||
| 40 | + | // runEvent is a change of a run that workflow_run workflows may be waiting for. | |
| 41 | + | type runEvent struct { | |
| 42 | + | run domain.CIRun | |
| 43 | + | action string | |
| 44 | + | log zerowrap.Logger | |
| 45 | + | } | |
| 46 | + | ||
| 47 | + | // onRunEvent queues the "requested" or "completed" event of a run and returns: | |
| 48 | + | // a worker handles it, so the run's lock is never held while workflows are read. | |
| 49 | + | // A full queue drops it, logged, like the queue of pushes. | |
| 50 | + | func (c *CI) onRunEvent(ctx context.Context, run domain.CIRun, action string) { | |
| 51 | + | c.mu.Lock() | |
| 52 | + | closed := c.closed | |
| 53 | + | c.mu.Unlock() | |
| 54 | + | if closed { | |
| 55 | + | return | |
| 56 | + | } | |
| 57 | + | e := runEvent{run: run, action: action, log: zerowrap.FromCtx(ctx)} | |
| 58 | + | select { | |
| 59 | + | case c.runEvents <- e: | |
| 60 | + | default: | |
| 61 | + | e.log.Error().Int64("run_id", run.ID).Str("action", action).Msg("ci workflow_run queue is full: the workflows waiting for this run are not run") | |
| 62 | + | } | |
| 63 | + | } | |
| 64 | + | ||
| 65 | + | func (c *CI) runEventLoop() { | |
| 66 | + | defer c.wg.Done() | |
| 67 | + | for { | |
| 68 | + | select { | |
| 69 | + | case <-c.ctx.Done(): | |
| 70 | + | return | |
| 71 | + | case e := <-c.runEvents: | |
| 72 | + | c.planRunEvent(e) | |
| 73 | + | } | |
| 74 | + | } | |
| 75 | + | } | |
| 76 | + | ||
| 77 | + | // planRunEvent handles one event; a panic or an error is logged and never | |
| 78 | + | // escapes. | |
| 79 | + | func (c *CI) planRunEvent(e runEvent) { | |
| 80 | + | ctx := zerowrap.WithCtx(c.ctx, e.log) | |
| 81 | + | defer func() { | |
| 82 | + | if r := recover(); r != nil { | |
| 83 | + | e.log.Error().Interface("panic", r).Bytes("stack", debug.Stack()).Int64("run_id", e.run.ID).Msg("ci workflow_run panicked") | |
| 84 | + | } | |
| 85 | + | if c.cfg.RunEventHandled != nil { | |
| 86 | + | c.cfg.RunEventHandled(e.run.ID, e.action) | |
| 87 | + | } | |
| 88 | + | }() | |
| 89 | + | if err := c.handleRunEvent(ctx, e); err != nil && ctx.Err() == nil { | |
| 90 | + | e.log.Error().Err(err).Int64("run_id", e.run.ID).Str("action", e.action).Msg("ci workflow_run failed") | |
| 91 | + | } | |
| 92 | + | } | |
| 93 | + | ||
| 94 | + | // runLinks are the two facts of the github context of a run that workflow_run | |
| 95 | + | // reads back: the head branch of a merge request, and the run that triggered a | |
| 96 | + | // workflow_run run. | |
| 97 | + | type runLinks struct { | |
| 98 | + | HeadRef string `json:"head_ref"` | |
| 99 | + | Event struct { | |
| 100 | + | WorkflowRun struct { | |
| 101 | + | ID int64 `json:"id"` | |
| 102 | + | } `json:"workflow_run"` | |
| 103 | + | } `json:"event"` | |
| 104 | + | } | |
| 105 | + | ||
| 106 | + | func (c *CI) runLinks(ctx context.Context, runID int64) (runLinks, domain.CIRunSource, error) { | |
| 107 | + | src, err := c.cfg.Store.GetRunSource(ctx, runID) | |
| 108 | + | if err != nil { | |
| 109 | + | return runLinks{}, src, err | |
| 110 | + | } | |
| 111 | + | var l runLinks | |
| 112 | + | if err := json.Unmarshal([]byte(src.GithubJSON), &l); err != nil { | |
| 113 | + | return runLinks{}, src, nil // a context that does not read: no link | |
| 114 | + | } | |
| 115 | + | return l, src, nil | |
| 116 | + | } | |
| 117 | + | ||
| 118 | + | // headBranchOf is workflow_run.head_branch: the source branch of a merge | |
| 119 | + | // request, else the branch or tag of the run. | |
| 120 | + | func headBranchOf(run domain.CIRun, l runLinks) string { | |
| 121 | + | if l.HeadRef != "" { | |
| 122 | + | return l.HeadRef | |
| 123 | + | } | |
| 124 | + | return strings.TrimPrefix(strings.TrimPrefix(run.Ref, "refs/heads/"), "refs/tags/") | |
| 125 | + | } | |
| 126 | + | ||
| 127 | + | // workflowRunDepth is the number of workflow_run levels above and including run: | |
| 128 | + | // 0 for a run that is not a workflow_run run. The walk stops at a triggering | |
| 129 | + | // run that cannot be found, which counts for nothing more. | |
| 130 | + | func (c *CI) workflowRunDepth(ctx context.Context, run domain.CIRun) int { | |
| 131 | + | depth := 0 | |
| 132 | + | for cur := run; cur.Event == eventWorkflowRun && depth <= maxWorkflowRunDepth; { | |
| 133 | + | depth++ | |
| 134 | + | l, _, err := c.runLinks(ctx, cur.ID) | |
| 135 | + | if err != nil || l.Event.WorkflowRun.ID == 0 { | |
| 136 | + | break | |
| 137 | + | } | |
| 138 | + | parent, err := c.cfg.Store.GetRun(ctx, cur.Repo, l.Event.WorkflowRun.ID) | |
| 139 | + | if err != nil { | |
| 140 | + | break | |
| 141 | + | } | |
| 142 | + | cur = parent | |
| 143 | + | } | |
| 144 | + | return depth | |
| 145 | + | } | |
| 146 | + | ||
| 147 | + | // matchWorkflowRun reports whether the workflow runs for the action of a run of | |
| 148 | + | // the workflow called name on headBranch: the event is listed, the name is in | |
| 149 | + | // "workflows", the action is one of its "types" (requested and completed when it | |
| 150 | + | // lists none; ermite has no in_progress activity), and the branches or | |
| 151 | + | // branches-ignore filter, which applies to the head branch of the triggering | |
| 152 | + | // run, lets it through (the filters of the push event, under another name). | |
| 153 | + | func matchWorkflowRun(wf *workflow.Workflow, name, action, headBranch string) bool { | |
| 154 | + | if wf == nil || wf.On == nil { | |
| 155 | + | return false | |
| 156 | + | } | |
| 157 | + | ev := wf.On.Event(eventWorkflowRun) | |
| 158 | + | if ev == nil || !slices.Contains(ev.Workflows, name) { | |
| 159 | + | return false | |
| 160 | + | } | |
| 161 | + | types := ev.Types | |
| 162 | + | if types == nil { | |
| 163 | + | types = []string{runRequested, runCompleted} | |
| 164 | + | } | |
| 165 | + | if !slices.Contains(types, action) { | |
| 166 | + | return false | |
| 167 | + | } | |
| 168 | + | as := *ev | |
| 169 | + | as.Name = "push" | |
| 170 | + | probe := &workflow.Workflow{On: &workflow.On{Events: []*workflow.Event{&as}}} | |
| 171 | + | return probe.MatchPush(workflow.PushInfo{Ref: "refs/heads/" + headBranch}) | |
| 172 | + | } | |
| 173 | + | ||
| 174 | + | // handleRunEvent starts the workflow_run workflows of the default branch that | |
| 175 | + | // match the event. | |
| 176 | + | func (c *CI) handleRunEvent(ctx context.Context, e runEvent) error { | |
| 177 | + | runners, err := c.cfg.Store.ListRunners(ctx) | |
| 178 | + | if err != nil { | |
| 179 | + | return fmt.Errorf("list the runners: %w", err) | |
| 180 | + | } | |
| 181 | + | if len(runners) == 0 { | |
| 182 | + | return nil | |
| 183 | + | } | |
| 184 | + | if c.workflowRunDepth(ctx, e.run) >= maxWorkflowRunDepth { | |
| 185 | + | e.log.Debug().Int64("run_id", e.run.ID).Msg("workflow_run chain limit reached: no run is triggered") | |
| 186 | + | return nil | |
| 187 | + | } | |
| 188 | + | repo, err := c.cfg.Meta.Get(ctx, e.run.Repo) | |
| 189 | + | if err != nil { | |
| 190 | + | return nil // deleted meanwhile | |
| 191 | + | } | |
| 192 | + | def, exists, err := c.cfg.Repos.DefaultBranch(ctx, repo.Name) | |
| 193 | + | if err != nil || !exists { | |
| 194 | + | return nil | |
| 195 | + | } | |
| 196 | + | tip, err := c.cfg.Git.BranchHead(ctx, repo.Name, def) | |
| 197 | + | if err != nil { | |
| 198 | + | return nil | |
| 199 | + | } | |
| 200 | + | files, err := c.workflowFiles(ctx, repo.Name, tip) | |
| 201 | + | if err != nil || len(files) == 0 { | |
| 202 | + | return err | |
| 203 | + | } | |
| 204 | + | links, src, err := c.runLinks(ctx, e.run.ID) | |
| 205 | + | if err != nil { | |
| 206 | + | return nil // the run is gone | |
| 207 | + | } | |
| 208 | + | head := headBranchOf(e.run, links) | |
| 209 | + | ||
| 210 | + | var matching []workflowFile | |
| 211 | + | var parsed []*workflow.Workflow | |
| 212 | + | for _, f := range files { | |
| 213 | + | if f.problem != "" { | |
| 214 | + | continue | |
| 215 | + | } | |
| 216 | + | wf, perrs := workflow.Parse(f.path, f.content) | |
| 217 | + | if len(perrs) > 0 || !matchWorkflowRun(wf, e.run.WorkflowName, e.action, head) { | |
| 218 | + | continue | |
| 219 | + | } | |
| 220 | + | matching, parsed = append(matching, f), append(parsed, wf) | |
| 221 | + | } | |
| 222 | + | if len(matching) == 0 { | |
| 223 | + | return nil | |
| 224 | + | } | |
| 225 | + | commits, _, err := c.cfg.Git.ListRange(ctx, repo.Name, domain.ObjectID{}, tip, 1) | |
| 226 | + | if err != nil || len(commits) == 0 { | |
| 227 | + | return fmt.Errorf("read the head commit of %s: %w", def, err) | |
| 228 | + | } | |
| 229 | + | facts := pushFacts{ | |
| 230 | + | update: domain.RefUpdate{Ref: "refs/heads/" + def, Old: domain.ZeroOID, New: tip.String()}, commit: tip, head: commits[0], | |
| 231 | + | defaultBranch: def, | |
| 232 | + | custom: &customEvent{name: eventWorkflowRun, triggered: &workflowRunFacts{run: e.run, action: e.action, headBranch: head}}, | |
| 233 | + | } | |
| 234 | + | // The actor, the pusher and the trust of the triggering run, never more. | |
| 235 | + | t := ciTrigger{repo: repo.Name, actor: e.run.Actor, pusher: src.Pusher, trusted: e.run.Trusted, at: c.now(), log: e.log} | |
| 236 | + | var errs []error | |
| 237 | + | for i, f := range matching { | |
| 238 | + | wf := parsed[i] | |
| 239 | + | if problem := c.matrixProblem(ctx, repo, t, facts, f.path, wf); problem.text != "" { | |
| 240 | + | name := f.path | |
| 241 | + | if problem.named { | |
| 242 | + | name = workflowName(wf, f.path) | |
| 243 | + | } | |
| 244 | + | errs = append(errs, c.startInvalid(ctx, repo, t, facts, f.path, name, problem.text)) | |
| 245 | + | continue | |
| 246 | + | } | |
| 247 | + | if _, err := c.launchRun(ctx, repo, t, facts, f.path, string(f.content), wf); err != nil && !errors.Is(err, context.Canceled) { | |
| 248 | + | errs = append(errs, err) | |
| 249 | + | } | |
| 250 | + | } | |
| 251 | + | return errors.Join(errs...) | |
| 252 | + | } | |
| @@ -0,0 +1,252 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package usecase | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "context" | |||
| 7 | + | "encoding/json" | |||
| 8 | + | "errors" | |||
| 9 | + | "fmt" | |||
| 10 | + | "runtime/debug" | |||
| 11 | + | "slices" | |||
| 12 | + | "strings" | |||
| 13 | + | ||||
| 14 | + | "github.com/bnema/zerowrap" | |||
| 15 | + | ||||
| 16 | + | "bnema.dev/ermite-actions/workflow" | |||
| 17 | + | "bnema.dev/ermite/internal/domain" | |||
| 18 | + | ) | |||
| 19 | + | ||||
| 20 | + | // The workflow_run event (CI-3, D5). A run that is created ("requested") or | |||
| 21 | + | // completed ("completed") triggers the workflows of the default branch that list | |||
| 22 | + | // its workflow by name under workflow_run.workflows. | |||
| 23 | + | // | |||
| 24 | + | // The triggered run is read from the default branch's latest commit and runs on | |||
| 25 | + | // it. It inherits the actor, the pusher and the trusted flag of the triggering | |||
| 26 | + | // run, and never gets more: an agent's untrusted run cannot launder secrets | |||
| 27 | + | // through a workflow_run (D1). A job token's push starts no run, but a run | |||
| 28 | + | // started by a push of anyone else triggers workflow_run workflows even when the | |||
| 29 | + | // triggering run was itself a workflow_run, up to the chain limit. | |||
| 30 | + | const ( | |||
| 31 | + | // maxWorkflowRunDepth is the number of workflow_run levels in a chain: | |||
| 32 | + | // docs.github.com ("Events that trigger workflows", workflow_run): "You can't | |||
| 33 | + | // use workflow_run to chain together more than three levels of workflows" | |||
| 34 | + | // (A -> B -> C -> D runs, E does not). Recalled from memory, not fetched. | |||
| 35 | + | maxWorkflowRunDepth = 3 | |||
| 36 | + | runRequested = "requested" | |||
| 37 | + | runCompleted = "completed" | |||
| 38 | + | ) | |||
| 39 | + | ||||
| 40 | + | // runEvent is a change of a run that workflow_run workflows may be waiting for. | |||
| 41 | + | type runEvent struct { | |||
| 42 | + | run domain.CIRun | |||
| 43 | + | action string | |||
| 44 | + | log zerowrap.Logger | |||
| 45 | + | } | |||
| 46 | + | ||||
| 47 | + | // onRunEvent queues the "requested" or "completed" event of a run and returns: | |||
| 48 | + | // a worker handles it, so the run's lock is never held while workflows are read. | |||
| 49 | + | // A full queue drops it, logged, like the queue of pushes. | |||
| 50 | + | func (c *CI) onRunEvent(ctx context.Context, run domain.CIRun, action string) { | |||
| 51 | + | c.mu.Lock() | |||
| 52 | + | closed := c.closed | |||
| 53 | + | c.mu.Unlock() | |||
| 54 | + | if closed { | |||
| 55 | + | return | |||
| 56 | + | } | |||
| 57 | + | e := runEvent{run: run, action: action, log: zerowrap.FromCtx(ctx)} | |||
| 58 | + | select { | |||
| 59 | + | case c.runEvents <- e: | |||
| 60 | + | default: | |||
| 61 | + | e.log.Error().Int64("run_id", run.ID).Str("action", action).Msg("ci workflow_run queue is full: the workflows waiting for this run are not run") | |||
| 62 | + | } | |||
| 63 | + | } | |||
| 64 | + | ||||
| 65 | + | func (c *CI) runEventLoop() { | |||
| 66 | + | defer c.wg.Done() | |||
| 67 | + | for { | |||
| 68 | + | select { | |||
| 69 | + | case <-c.ctx.Done(): | |||
| 70 | + | return | |||
| 71 | + | case e := <-c.runEvents: | |||
| 72 | + | c.planRunEvent(e) | |||
| 73 | + | } | |||
| 74 | + | } | |||
| 75 | + | } | |||
| 76 | + | ||||
| 77 | + | // planRunEvent handles one event; a panic or an error is logged and never | |||
| 78 | + | // escapes. | |||
| 79 | + | func (c *CI) planRunEvent(e runEvent) { | |||
| 80 | + | ctx := zerowrap.WithCtx(c.ctx, e.log) | |||
| 81 | + | defer func() { | |||
| 82 | + | if r := recover(); r != nil { | |||
| 83 | + | e.log.Error().Interface("panic", r).Bytes("stack", debug.Stack()).Int64("run_id", e.run.ID).Msg("ci workflow_run panicked") | |||
| 84 | + | } | |||
| 85 | + | if c.cfg.RunEventHandled != nil { | |||
| 86 | + | c.cfg.RunEventHandled(e.run.ID, e.action) | |||
| 87 | + | } | |||
| 88 | + | }() | |||
| 89 | + | if err := c.handleRunEvent(ctx, e); err != nil && ctx.Err() == nil { | |||
| 90 | + | e.log.Error().Err(err).Int64("run_id", e.run.ID).Str("action", e.action).Msg("ci workflow_run failed") | |||
| 91 | + | } | |||
| 92 | + | } | |||
| 93 | + | ||||
| 94 | + | // runLinks are the two facts of the github context of a run that workflow_run | |||
| 95 | + | // reads back: the head branch of a merge request, and the run that triggered a | |||
| 96 | + | // workflow_run run. | |||
| 97 | + | type runLinks struct { | |||
| 98 | + | HeadRef string `json:"head_ref"` | |||
| 99 | + | Event struct { | |||
| 100 | + | WorkflowRun struct { | |||
| 101 | + | ID int64 `json:"id"` | |||
| 102 | + | } `json:"workflow_run"` | |||
| 103 | + | } `json:"event"` | |||
| 104 | + | } | |||
| 105 | + | ||||
| 106 | + | func (c *CI) runLinks(ctx context.Context, runID int64) (runLinks, domain.CIRunSource, error) { | |||
| 107 | + | src, err := c.cfg.Store.GetRunSource(ctx, runID) | |||
| 108 | + | if err != nil { | |||
| 109 | + | return runLinks{}, src, err | |||
| 110 | + | } | |||
| 111 | + | var l runLinks | |||
| 112 | + | if err := json.Unmarshal([]byte(src.GithubJSON), &l); err != nil { | |||
| 113 | + | return runLinks{}, src, nil // a context that does not read: no link | |||
| 114 | + | } | |||
| 115 | + | return l, src, nil | |||
| 116 | + | } | |||
| 117 | + | ||||
| 118 | + | // headBranchOf is workflow_run.head_branch: the source branch of a merge | |||
| 119 | + | // request, else the branch or tag of the run. | |||
| 120 | + | func headBranchOf(run domain.CIRun, l runLinks) string { | |||
| 121 | + | if l.HeadRef != "" { | |||
| 122 | + | return l.HeadRef | |||
| 123 | + | } | |||
| 124 | + | return strings.TrimPrefix(strings.TrimPrefix(run.Ref, "refs/heads/"), "refs/tags/") | |||
| 125 | + | } | |||
| 126 | + | ||||
| 127 | + | // workflowRunDepth is the number of workflow_run levels above and including run: | |||
| 128 | + | // 0 for a run that is not a workflow_run run. The walk stops at a triggering | |||
| 129 | + | // run that cannot be found, which counts for nothing more. | |||
| 130 | + | func (c *CI) workflowRunDepth(ctx context.Context, run domain.CIRun) int { | |||
| 131 | + | depth := 0 | |||
| 132 | + | for cur := run; cur.Event == eventWorkflowRun && depth <= maxWorkflowRunDepth; { | |||
| 133 | + | depth++ | |||
| 134 | + | l, _, err := c.runLinks(ctx, cur.ID) | |||
| 135 | + | if err != nil || l.Event.WorkflowRun.ID == 0 { | |||
| 136 | + | break | |||
| 137 | + | } | |||
| 138 | + | parent, err := c.cfg.Store.GetRun(ctx, cur.Repo, l.Event.WorkflowRun.ID) | |||
| 139 | + | if err != nil { | |||
| 140 | + | break | |||
| 141 | + | } | |||
| 142 | + | cur = parent | |||
| 143 | + | } | |||
| 144 | + | return depth | |||
| 145 | + | } | |||
| 146 | + | ||||
| 147 | + | // matchWorkflowRun reports whether the workflow runs for the action of a run of | |||
| 148 | + | // the workflow called name on headBranch: the event is listed, the name is in | |||
| 149 | + | // "workflows", the action is one of its "types" (requested and completed when it | |||
| 150 | + | // lists none; ermite has no in_progress activity), and the branches or | |||
| 151 | + | // branches-ignore filter, which applies to the head branch of the triggering | |||
| 152 | + | // run, lets it through (the filters of the push event, under another name). | |||
| 153 | + | func matchWorkflowRun(wf *workflow.Workflow, name, action, headBranch string) bool { | |||
| 154 | + | if wf == nil || wf.On == nil { | |||
| 155 | + | return false | |||
| 156 | + | } | |||
| 157 | + | ev := wf.On.Event(eventWorkflowRun) | |||
| 158 | + | if ev == nil || !slices.Contains(ev.Workflows, name) { | |||
| 159 | + | return false | |||
| 160 | + | } | |||
| 161 | + | types := ev.Types | |||
| 162 | + | if types == nil { | |||
| 163 | + | types = []string{runRequested, runCompleted} | |||
| 164 | + | } | |||
| 165 | + | if !slices.Contains(types, action) { | |||
| 166 | + | return false | |||
| 167 | + | } | |||
| 168 | + | as := *ev | |||
| 169 | + | as.Name = "push" | |||
| 170 | + | probe := &workflow.Workflow{On: &workflow.On{Events: []*workflow.Event{&as}}} | |||
| 171 | + | return probe.MatchPush(workflow.PushInfo{Ref: "refs/heads/" + headBranch}) | |||
| 172 | + | } | |||
| 173 | + | ||||
| 174 | + | // handleRunEvent starts the workflow_run workflows of the default branch that | |||
| 175 | + | // match the event. | |||
| 176 | + | func (c *CI) handleRunEvent(ctx context.Context, e runEvent) error { | |||
| 177 | + | runners, err := c.cfg.Store.ListRunners(ctx) | |||
| 178 | + | if err != nil { | |||
| 179 | + | return fmt.Errorf("list the runners: %w", err) | |||
| 180 | + | } | |||
| 181 | + | if len(runners) == 0 { | |||
| 182 | + | return nil | |||
| 183 | + | } | |||
| 184 | + | if c.workflowRunDepth(ctx, e.run) >= maxWorkflowRunDepth { | |||
| 185 | + | e.log.Debug().Int64("run_id", e.run.ID).Msg("workflow_run chain limit reached: no run is triggered") | |||
| 186 | + | return nil | |||
| 187 | + | } | |||
| 188 | + | repo, err := c.cfg.Meta.Get(ctx, e.run.Repo) | |||
| 189 | + | if err != nil { | |||
| 190 | + | return nil // deleted meanwhile | |||
| 191 | + | } | |||
| 192 | + | def, exists, err := c.cfg.Repos.DefaultBranch(ctx, repo.Name) | |||
| 193 | + | if err != nil || !exists { | |||
| 194 | + | return nil | |||
| 195 | + | } | |||
| 196 | + | tip, err := c.cfg.Git.BranchHead(ctx, repo.Name, def) | |||
| 197 | + | if err != nil { | |||
| 198 | + | return nil | |||
| 199 | + | } | |||
| 200 | + | files, err := c.workflowFiles(ctx, repo.Name, tip) | |||
| 201 | + | if err != nil || len(files) == 0 { | |||
| 202 | + | return err | |||
| 203 | + | } | |||
| 204 | + | links, src, err := c.runLinks(ctx, e.run.ID) | |||
| 205 | + | if err != nil { | |||
| 206 | + | return nil // the run is gone | |||
| 207 | + | } | |||
| 208 | + | head := headBranchOf(e.run, links) | |||
| 209 | + | ||||
| 210 | + | var matching []workflowFile | |||
| 211 | + | var parsed []*workflow.Workflow | |||
| 212 | + | for _, f := range files { | |||
| 213 | + | if f.problem != "" { | |||
| 214 | + | continue | |||
| 215 | + | } | |||
| 216 | + | wf, perrs := workflow.Parse(f.path, f.content) | |||
| 217 | + | if len(perrs) > 0 || !matchWorkflowRun(wf, e.run.WorkflowName, e.action, head) { | |||
| 218 | + | continue | |||
| 219 | + | } | |||
| 220 | + | matching, parsed = append(matching, f), append(parsed, wf) | |||
| 221 | + | } | |||
| 222 | + | if len(matching) == 0 { | |||
| 223 | + | return nil | |||
| 224 | + | } | |||
| 225 | + | commits, _, err := c.cfg.Git.ListRange(ctx, repo.Name, domain.ObjectID{}, tip, 1) | |||
| 226 | + | if err != nil || len(commits) == 0 { | |||
| 227 | + | return fmt.Errorf("read the head commit of %s: %w", def, err) | |||
| 228 | + | } | |||
| 229 | + | facts := pushFacts{ | |||
| 230 | + | update: domain.RefUpdate{Ref: "refs/heads/" + def, Old: domain.ZeroOID, New: tip.String()}, commit: tip, head: commits[0], | |||
| 231 | + | defaultBranch: def, | |||
| 232 | + | custom: &customEvent{name: eventWorkflowRun, triggered: &workflowRunFacts{run: e.run, action: e.action, headBranch: head}}, | |||
| 233 | + | } | |||
| 234 | + | // The actor, the pusher and the trust of the triggering run, never more. | |||
| 235 | + | t := ciTrigger{repo: repo.Name, actor: e.run.Actor, pusher: src.Pusher, trusted: e.run.Trusted, at: c.now(), log: e.log} | |||
| 236 | + | var errs []error | |||
| 237 | + | for i, f := range matching { | |||
| 238 | + | wf := parsed[i] | |||
| 239 | + | if problem := c.matrixProblem(ctx, repo, t, facts, f.path, wf); problem.text != "" { | |||
| 240 | + | name := f.path | |||
| 241 | + | if problem.named { | |||
| 242 | + | name = workflowName(wf, f.path) | |||
| 243 | + | } | |||
| 244 | + | errs = append(errs, c.startInvalid(ctx, repo, t, facts, f.path, name, problem.text)) | |||
| 245 | + | continue | |||
| 246 | + | } | |||
| 247 | + | if _, err := c.launchRun(ctx, repo, t, facts, f.path, string(f.content), wf); err != nil && !errors.Is(err, context.Canceled) { | |||
| 248 | + | errs = append(errs, err) | |||
| 249 | + | } | |||
| 250 | + | } | |||
| 251 | + | return errors.Join(errs...) | |||
| 252 | + | } | |||
A internal/usecase/ci_workflowrun_test.go +34 -0
| @@ -0,0 +1,34 @@ | |||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |
| 2 | + | ||
| 3 | + | package usecase_test | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "testing" | |
| 7 | + | ||
| 8 | + | "github.com/stretchr/testify/assert" | |
| 9 | + | ||
| 10 | + | "bnema.dev/ermite/internal/usecase" | |
| 11 | + | ) | |
| 12 | + | ||
| 13 | + | func TestMatchWorkflowRun(t *testing.T) { | |
| 14 | + | def := parsePR(t, " workflow_run:\n workflows: [ci, 'Build and test']\n") | |
| 15 | + | assert.True(t, usecase.MatchWorkflowRun(def, "ci", "completed", "main")) | |
| 16 | + | assert.True(t, usecase.MatchWorkflowRun(def, "Build and test", "requested", "x/y"), "no types: both activities") | |
| 17 | + | assert.False(t, usecase.MatchWorkflowRun(def, "other", "completed", "main"), "the name must be listed") | |
| 18 | + | assert.False(t, usecase.MatchWorkflowRun(def, "CI", "completed", "main"), "names are exact") | |
| 19 | + | assert.False(t, usecase.MatchWorkflowRun(def, "ci", "in_progress", "main"), "ermite has no in_progress activity") | |
| 20 | + | assert.False(t, usecase.MatchWorkflowRun(parsePR(t, " push:\n"), "ci", "completed", "main")) | |
| 21 | + | ||
| 22 | + | typed := parsePR(t, " workflow_run:\n workflows: [ci]\n types: [completed]\n") | |
| 23 | + | assert.True(t, usecase.MatchWorkflowRun(typed, "ci", "completed", "main")) | |
| 24 | + | assert.False(t, usecase.MatchWorkflowRun(typed, "ci", "requested", "main")) | |
| 25 | + | ||
| 26 | + | branches := parsePR(t, " workflow_run:\n workflows: [ci]\n branches: [main, 'release/**']\n") | |
| 27 | + | assert.True(t, usecase.MatchWorkflowRun(branches, "ci", "completed", "main")) | |
| 28 | + | assert.True(t, usecase.MatchWorkflowRun(branches, "ci", "completed", "release/1.0")) | |
| 29 | + | assert.False(t, usecase.MatchWorkflowRun(branches, "ci", "completed", "feature"), "the head branch of the triggering run is filtered") | |
| 30 | + | ||
| 31 | + | ignore := parsePR(t, " workflow_run:\n workflows: [ci]\n branches-ignore: [wip/**]\n") | |
| 32 | + | assert.True(t, usecase.MatchWorkflowRun(ignore, "ci", "completed", "main")) | |
| 33 | + | assert.False(t, usecase.MatchWorkflowRun(ignore, "ci", "completed", "wip/x")) | |
| 34 | + | } | |
| @@ -0,0 +1,34 @@ | |||||
| 1 | + | // SPDX-License-Identifier: AGPL-3.0-or-later | |||
| 2 | + | ||||
| 3 | + | package usecase_test | |||
| 4 | + | ||||
| 5 | + | import ( | |||
| 6 | + | "testing" | |||
| 7 | + | ||||
| 8 | + | "github.com/stretchr/testify/assert" | |||
| 9 | + | ||||
| 10 | + | "bnema.dev/ermite/internal/usecase" | |||
| 11 | + | ) | |||
| 12 | + | ||||
| 13 | + | func TestMatchWorkflowRun(t *testing.T) { | |||
| 14 | + | def := parsePR(t, " workflow_run:\n workflows: [ci, 'Build and test']\n") | |||
| 15 | + | assert.True(t, usecase.MatchWorkflowRun(def, "ci", "completed", "main")) | |||
| 16 | + | assert.True(t, usecase.MatchWorkflowRun(def, "Build and test", "requested", "x/y"), "no types: both activities") | |||
| 17 | + | assert.False(t, usecase.MatchWorkflowRun(def, "other", "completed", "main"), "the name must be listed") | |||
| 18 | + | assert.False(t, usecase.MatchWorkflowRun(def, "CI", "completed", "main"), "names are exact") | |||
| 19 | + | assert.False(t, usecase.MatchWorkflowRun(def, "ci", "in_progress", "main"), "ermite has no in_progress activity") | |||
| 20 | + | assert.False(t, usecase.MatchWorkflowRun(parsePR(t, " push:\n"), "ci", "completed", "main")) | |||
| 21 | + | ||||
| 22 | + | typed := parsePR(t, " workflow_run:\n workflows: [ci]\n types: [completed]\n") | |||
| 23 | + | assert.True(t, usecase.MatchWorkflowRun(typed, "ci", "completed", "main")) | |||
| 24 | + | assert.False(t, usecase.MatchWorkflowRun(typed, "ci", "requested", "main")) | |||
| 25 | + | ||||
| 26 | + | branches := parsePR(t, " workflow_run:\n workflows: [ci]\n branches: [main, 'release/**']\n") | |||
| 27 | + | assert.True(t, usecase.MatchWorkflowRun(branches, "ci", "completed", "main")) | |||
| 28 | + | assert.True(t, usecase.MatchWorkflowRun(branches, "ci", "completed", "release/1.0")) | |||
| 29 | + | assert.False(t, usecase.MatchWorkflowRun(branches, "ci", "completed", "feature"), "the head branch of the triggering run is filtered") | |||
| 30 | + | ||||
| 31 | + | ignore := parsePR(t, " workflow_run:\n workflows: [ci]\n branches-ignore: [wip/**]\n") | |||
| 32 | + | assert.True(t, usecase.MatchWorkflowRun(ignore, "ci", "completed", "main")) | |||
| 33 | + | assert.False(t, usecase.MatchWorkflowRun(ignore, "ci", "completed", "wip/x")) | |||
| 34 | + | } | |||
M internal/usecase/export_test.go +10 -0
| @@ -66,6 +66,16 @@ func MatchPullRequest(wf *workflow.Workflow, action, base string, files []string | |||
| 66 | 66 | return matchPullRequest(wf, action, base, files, truncated) | |
| 67 | 67 | } | |
| 68 | 68 | ||
| 69 | + | // MatchWorkflowRun exposes the workflow_run filter to the tests. | |
| 70 | + | func MatchWorkflowRun(wf *workflow.Workflow, name, action, headBranch string) bool { | |
| 71 | + | return matchWorkflowRun(wf, name, action, headBranch) | |
| 72 | + | } | |
| 73 | + | ||
| 74 | + | // ResolveDispatchInputs exposes resolveDispatchInputs to the tests. | |
| 75 | + | func ResolveDispatchInputs(defs []workflow.Input, given map[string]string) (map[string]string, error) { | |
| 76 | + | return resolveDispatchInputs(defs, given) | |
| 77 | + | } | |
| 78 | + | ||
| 69 | 79 | // PullFilesLink exposes the Link header of the files endpoint to the tests. | |
| 70 | 80 | func PullFilesLink(base string, page, perPage, total int) string { | |
| 71 | 81 | return pullFilesLink(base, page, perPage, total) | |
| @@ -66,6 +66,16 @@ func MatchPullRequest(wf *workflow.Workflow, action, base string, files []string | |||||
| 66 | return matchPullRequest(wf, action, base, files, truncated) | 66 | return matchPullRequest(wf, action, base, files, truncated) | ||
| 67 | } | 67 | } | ||
| 68 | 68 | ||||
| 69 | + | // MatchWorkflowRun exposes the workflow_run filter to the tests. | |||
| 70 | + | func MatchWorkflowRun(wf *workflow.Workflow, name, action, headBranch string) bool { | |||
| 71 | + | return matchWorkflowRun(wf, name, action, headBranch) | |||
| 72 | + | } | |||
| 73 | + | ||||
| 74 | + | // ResolveDispatchInputs exposes resolveDispatchInputs to the tests. | |||
| 75 | + | func ResolveDispatchInputs(defs []workflow.Input, given map[string]string) (map[string]string, error) { | |||
| 76 | + | return resolveDispatchInputs(defs, given) | |||
| 77 | + | } | |||
| 78 | + | ||||
| 69 | // PullFilesLink exposes the Link header of the files endpoint to the tests. | 79 | // PullFilesLink exposes the Link header of the files endpoint to the tests. | ||
| 70 | func PullFilesLink(base string, page, perPage, total int) string { | 80 | func PullFilesLink(base string, page, perPage, total int) string { | ||
| 71 | return pullFilesLink(base, page, perPage, total) | 81 | return pullFilesLink(base, page, perPage, total) | ||
M internal/usecase/mergerequests.go +22 -5
| @@ -52,6 +52,7 @@ type MergeRequests struct { | |||
| 52 | 52 | ci ports.WorkflowTrigger // nil: no CI | |
| 53 | 53 | checks ports.CIChecksReader // nil: no CI, so no check blocks a merge | |
| 54 | 54 | pulls ports.PullRequestTrigger // nil: no pull_request workflows | |
| 55 | + | tips ports.CIRefTipStore // nil: who merges is not recorded | |
| 55 | 56 | ||
| 56 | 57 | mu sync.Mutex | |
| 57 | 58 | locks map[domain.RepoName]chan struct{} | |
| @@ -87,13 +88,17 @@ type MergeRequestsConfig struct { | |||
| 87 | 88 | // closed or merged (the push of its source branch reaches CI instead). | |
| 88 | 89 | // Optional. | |
| 89 | 90 | PullRequests ports.PullRequestTrigger | |
| 91 | + | // RefTips records who moves the target branch (and deletes the record of a | |
| 92 | + | // deleted source branch) before the merge does, as a push does | |
| 93 | + | // (domain.CIRefTip). Optional. | |
| 94 | + | RefTips ports.CIRefTipStore | |
| 90 | 95 | } | |
| 91 | 96 | ||
| 92 | 97 | // NewMergeRequests builds the merge request use cases. | |
| 93 | 98 | func NewMergeRequests(cfg MergeRequestsConfig) *MergeRequests { | |
| 94 | 99 | return &MergeRequests{ | |
| 95 | 100 | meta: cfg.Meta, store: cfg.Store, repos: cfg.Repos, git: cfg.Git, signer: cfg.Signer, | |
| 96 | - | committer: cfg.Committer, bus: cfg.Bus, clock: cfg.Clock, maint: cfg.Maint, ci: cfg.CI, checks: cfg.Checks, pulls: cfg.PullRequests, | |
| 101 | + | committer: cfg.Committer, bus: cfg.Bus, clock: cfg.Clock, maint: cfg.Maint, ci: cfg.CI, checks: cfg.Checks, pulls: cfg.PullRequests, tips: cfg.RefTips, | |
| 97 | 102 | locks: make(map[domain.RepoName]chan struct{}), | |
| 98 | 103 | } | |
| 99 | 104 | } | |
| @@ -796,7 +801,7 @@ func (m *MergeRequests) merge(ctx context.Context, p domain.Principal, name doma | |||
| 796 | 801 | } | |
| 797 | 802 | } | |
| 798 | 803 | ||
| 799 | - | merged, err := m.moveTarget(ctx, mr, mode, tip, head) | |
| 804 | + | merged, err := m.moveTarget(ctx, mr, mode, tip, head, p) | |
| 800 | 805 | if err != nil { | |
| 801 | 806 | return domain.MergeRequest{}, false, err | |
| 802 | 807 | } | |
| @@ -812,7 +817,12 @@ func (m *MergeRequests) merge(ctx context.Context, p domain.Principal, name doma | |||
| 812 | 817 | updates := []domain.RefUpdate{{Old: tip.String(), New: merged.String(), Ref: "refs/heads/" + mr.Target}} | |
| 813 | 818 | deleted := false | |
| 814 | 819 | if params.DeleteSource { | |
| 815 | - | // The merge is done and recorded: a failure here keeps the branch. | |
| 820 | + | // The merge is done and recorded: a failure here keeps the branch. Who | |
| 821 | + | // pushed the source goes first (the branch is gone either way for a | |
| 822 | + | // schedule; a kept branch merely has no record: not trusted). | |
| 823 | + | if err := recordRefTips(ctx, m.tips, name, []domain.RefUpdate{{Old: head.String(), New: domain.ZeroOID, Ref: "refs/heads/" + mr.Source}}, "", domain.CIPusher{}, false); err != nil { | |
| 824 | + | log.Warn().Err(err).Str("repo", name.String()).Str("branch", mr.Source).Msg("the record of the source branch was kept") | |
| 825 | + | } | |
| 816 | 826 | switch err := m.git.DeleteBranch(context.WithoutCancel(ctx), name, mr.Source, head); { | |
| 817 | 827 | case err != nil: | |
| 818 | 828 | log.Warn().Err(err).Str("repo", name.String()).Int64("number", number).Str("branch", mr.Source). | |
| @@ -835,7 +845,7 @@ func (m *MergeRequests) merge(ctx context.Context, p domain.Principal, name doma | |||
| 835 | 845 | if m.ci != nil { | |
| 836 | 846 | // GitHub runs the "push" workflows of a merge, with the owner as actor; | |
| 837 | 847 | // the run is trusted with what the merging principal may read. | |
| 838 | - | m.ci.OnMerge(ctx, name, updates[:1], p) | |
| 848 | + | m.ci.OnMerge(ctx, name, updates, p) | |
| 839 | 849 | } | |
| 840 | 850 | return mr, deleted, nil | |
| 841 | 851 | } | |
| @@ -931,7 +941,7 @@ func (m *MergeRequests) recordPush(ctx context.Context, name domain.RepoName) { | |||
| 931 | 941 | ||
| 932 | 942 | // moveTarget moves the target branch from tip to the head (fast-forward) or to | |
| 933 | 943 | // a new signed squash commit, and returns the new tip. | |
| 934 | - | func (m *MergeRequests) moveTarget(ctx context.Context, mr domain.MergeRequest, mode domain.MergeMode, tip, head domain.ObjectID) (domain.ObjectID, error) { | |
| 944 | + | func (m *MergeRequests) moveTarget(ctx context.Context, mr domain.MergeRequest, mode domain.MergeMode, tip, head domain.ObjectID, p domain.Principal) (domain.ObjectID, error) { | |
| 935 | 945 | newTip := head | |
| 936 | 946 | if mode == domain.MergeModeSquash { | |
| 937 | 947 | commits, more, err := m.git.ListRange(ctx, mr.Repo, tip, head, domain.MaxSquashCommits) | |
| @@ -954,6 +964,13 @@ func (m *MergeRequests) moveTarget(ctx context.Context, mr domain.MergeRequest, | |||
| 954 | 964 | return domain.ObjectID{}, err | |
| 955 | 965 | } | |
| 956 | 966 | } | |
| 967 | + | // The target is the merger's from the moment it moves: recorded before it | |
| 968 | + | // does, under the merge lock (domain.CIRefTip). The run shows the owner, as a | |
| 969 | + | // merge's push does on GitHub. | |
| 970 | + | if err := recordRefTips(ctx, m.tips, mr.Repo, []domain.RefUpdate{{Old: tip.String(), New: newTip.String(), Ref: "refs/heads/" + mr.Target}}, | |
| 971 | + | authorOf(domain.Owner()), domain.PusherOf(p), domain.TrustedActor(p)); err != nil { | |
| 972 | + | return domain.ObjectID{}, err | |
| 973 | + | } | |
| 957 | 974 | err := m.git.UpdateBranch(ctx, mr.Repo, mr.Target, tip, newTip) | |
| 958 | 975 | if errors.Is(err, domain.ErrConflict) { | |
| 959 | 976 | return domain.ObjectID{}, fmt.Errorf("%w: the target branch %q moved during the merge, try again", domain.ErrPreconditionFailed, mr.Target) | |
| @@ -52,6 +52,7 @@ type MergeRequests struct { | |||||
| 52 | ci ports.WorkflowTrigger // nil: no CI | 52 | ci ports.WorkflowTrigger // nil: no CI | ||
| 53 | checks ports.CIChecksReader // nil: no CI, so no check blocks a merge | 53 | checks ports.CIChecksReader // nil: no CI, so no check blocks a merge | ||
| 54 | pulls ports.PullRequestTrigger // nil: no pull_request workflows | 54 | pulls ports.PullRequestTrigger // nil: no pull_request workflows | ||
| 55 | + | tips ports.CIRefTipStore // nil: who merges is not recorded | |||
| 55 | 56 | ||||
| 56 | mu sync.Mutex | 57 | mu sync.Mutex | ||
| 57 | locks map[domain.RepoName]chan struct{} | 58 | locks map[domain.RepoName]chan struct{} | ||
| @@ -87,13 +88,17 @@ type MergeRequestsConfig struct { | |||||
| 87 | // closed or merged (the push of its source branch reaches CI instead). | 88 | // closed or merged (the push of its source branch reaches CI instead). | ||
| 88 | // Optional. | 89 | // Optional. | ||
| 89 | PullRequests ports.PullRequestTrigger | 90 | PullRequests ports.PullRequestTrigger | ||
| 91 | + | // RefTips records who moves the target branch (and deletes the record of a | |||
| 92 | + | // deleted source branch) before the merge does, as a push does | |||
| 93 | + | // (domain.CIRefTip). Optional. | |||
| 94 | + | RefTips ports.CIRefTipStore | |||
| 90 | } | 95 | } | ||
| 91 | 96 | ||||
| 92 | // NewMergeRequests builds the merge request use cases. | 97 | // NewMergeRequests builds the merge request use cases. | ||
| 93 | func NewMergeRequests(cfg MergeRequestsConfig) *MergeRequests { | 98 | func NewMergeRequests(cfg MergeRequestsConfig) *MergeRequests { | ||
| 94 | return &MergeRequests{ | 99 | return &MergeRequests{ | ||
| 95 | meta: cfg.Meta, store: cfg.Store, repos: cfg.Repos, git: cfg.Git, signer: cfg.Signer, | 100 | meta: cfg.Meta, store: cfg.Store, repos: cfg.Repos, git: cfg.Git, signer: cfg.Signer, | ||
| 96 | - | committer: cfg.Committer, bus: cfg.Bus, clock: cfg.Clock, maint: cfg.Maint, ci: cfg.CI, checks: cfg.Checks, pulls: cfg.PullRequests, | 101 | + | committer: cfg.Committer, bus: cfg.Bus, clock: cfg.Clock, maint: cfg.Maint, ci: cfg.CI, checks: cfg.Checks, pulls: cfg.PullRequests, tips: cfg.RefTips, |
| 97 | locks: make(map[domain.RepoName]chan struct{}), | 102 | locks: make(map[domain.RepoName]chan struct{}), | ||
| 98 | } | 103 | } | ||
| 99 | } | 104 | } | ||
| @@ -796,7 +801,7 @@ func (m *MergeRequests) merge(ctx context.Context, p domain.Principal, name doma | |||||
| 796 | } | 801 | } | ||
| 797 | } | 802 | } | ||
| 798 | 803 | ||||
| 799 | - | merged, err := m.moveTarget(ctx, mr, mode, tip, head) | 804 | + | merged, err := m.moveTarget(ctx, mr, mode, tip, head, p) |
| 800 | if err != nil { | 805 | if err != nil { | ||
| 801 | return domain.MergeRequest{}, false, err | 806 | return domain.MergeRequest{}, false, err | ||
| 802 | } | 807 | } | ||
| @@ -812,7 +817,12 @@ func (m *MergeRequests) merge(ctx context.Context, p domain.Principal, name doma | |||||
| 812 | updates := []domain.RefUpdate{{Old: tip.String(), New: merged.String(), Ref: "refs/heads/" + mr.Target}} | 817 | updates := []domain.RefUpdate{{Old: tip.String(), New: merged.String(), Ref: "refs/heads/" + mr.Target}} | ||
| 813 | deleted := false | 818 | deleted := false | ||
| 814 | if params.DeleteSource { | 819 | if params.DeleteSource { | ||
| 815 | - | // The merge is done and recorded: a failure here keeps the branch. | 820 | + | // The merge is done and recorded: a failure here keeps the branch. Who |
| 821 | + | // pushed the source goes first (the branch is gone either way for a | |||
| 822 | + | // schedule; a kept branch merely has no record: not trusted). | |||
| 823 | + | if err := recordRefTips(ctx, m.tips, name, []domain.RefUpdate{{Old: head.String(), New: domain.ZeroOID, Ref: "refs/heads/" + mr.Source}}, "", domain.CIPusher{}, false); err != nil { | |||
| 824 | + | log.Warn().Err(err).Str("repo", name.String()).Str("branch", mr.Source).Msg("the record of the source branch was kept") | |||
| 825 | + | } | |||
| 816 | switch err := m.git.DeleteBranch(context.WithoutCancel(ctx), name, mr.Source, head); { | 826 | switch err := m.git.DeleteBranch(context.WithoutCancel(ctx), name, mr.Source, head); { | ||
| 817 | case err != nil: | 827 | case err != nil: | ||
| 818 | log.Warn().Err(err).Str("repo", name.String()).Int64("number", number).Str("branch", mr.Source). | 828 | log.Warn().Err(err).Str("repo", name.String()).Int64("number", number).Str("branch", mr.Source). | ||
| @@ -835,7 +845,7 @@ func (m *MergeRequests) merge(ctx context.Context, p domain.Principal, name doma | |||||
| 835 | if m.ci != nil { | 845 | if m.ci != nil { | ||
| 836 | // GitHub runs the "push" workflows of a merge, with the owner as actor; | 846 | // GitHub runs the "push" workflows of a merge, with the owner as actor; | ||
| 837 | // the run is trusted with what the merging principal may read. | 847 | // the run is trusted with what the merging principal may read. | ||
| 838 | - | m.ci.OnMerge(ctx, name, updates[:1], p) | 848 | + | m.ci.OnMerge(ctx, name, updates, p) |
| 839 | } | 849 | } | ||
| 840 | return mr, deleted, nil | 850 | return mr, deleted, nil | ||
| 841 | } | 851 | } | ||
| @@ -931,7 +941,7 @@ func (m *MergeRequests) recordPush(ctx context.Context, name domain.RepoName) { | |||||
| 931 | 941 | ||||
| 932 | // moveTarget moves the target branch from tip to the head (fast-forward) or to | 942 | // moveTarget moves the target branch from tip to the head (fast-forward) or to | ||
| 933 | // a new signed squash commit, and returns the new tip. | 943 | // a new signed squash commit, and returns the new tip. | ||
| 934 | - | func (m *MergeRequests) moveTarget(ctx context.Context, mr domain.MergeRequest, mode domain.MergeMode, tip, head domain.ObjectID) (domain.ObjectID, error) { | 944 | + | func (m *MergeRequests) moveTarget(ctx context.Context, mr domain.MergeRequest, mode domain.MergeMode, tip, head domain.ObjectID, p domain.Principal) (domain.ObjectID, error) { |
| 935 | newTip := head | 945 | newTip := head | ||
| 936 | if mode == domain.MergeModeSquash { | 946 | if mode == domain.MergeModeSquash { | ||
| 937 | commits, more, err := m.git.ListRange(ctx, mr.Repo, tip, head, domain.MaxSquashCommits) | 947 | commits, more, err := m.git.ListRange(ctx, mr.Repo, tip, head, domain.MaxSquashCommits) | ||
| @@ -954,6 +964,13 @@ func (m *MergeRequests) moveTarget(ctx context.Context, mr domain.MergeRequest, | |||||
| 954 | return domain.ObjectID{}, err | 964 | return domain.ObjectID{}, err | ||
| 955 | } | 965 | } | ||
| 956 | } | 966 | } | ||
| 967 | + | // The target is the merger's from the moment it moves: recorded before it | |||
| 968 | + | // does, under the merge lock (domain.CIRefTip). The run shows the owner, as a | |||
| 969 | + | // merge's push does on GitHub. | |||
| 970 | + | if err := recordRefTips(ctx, m.tips, mr.Repo, []domain.RefUpdate{{Old: tip.String(), New: newTip.String(), Ref: "refs/heads/" + mr.Target}}, | |||
| 971 | + | authorOf(domain.Owner()), domain.PusherOf(p), domain.TrustedActor(p)); err != nil { | |||
| 972 | + | return domain.ObjectID{}, err | |||
| 973 | + | } | |||
| 957 | err := m.git.UpdateBranch(ctx, mr.Repo, mr.Target, tip, newTip) | 974 | err := m.git.UpdateBranch(ctx, mr.Repo, mr.Target, tip, newTip) | ||
| 958 | if errors.Is(err, domain.ErrConflict) { | 975 | if errors.Is(err, domain.ErrConflict) { | ||
| 959 | return domain.ObjectID{}, fmt.Errorf("%w: the target branch %q moved during the merge, try again", domain.ErrPreconditionFailed, mr.Target) | 976 | return domain.ObjectID{}, fmt.Errorf("%w: the target branch %q moved during the merge, try again", domain.ErrPreconditionFailed, mr.Target) | ||
M internal/usecase/pushes.go +16 -0
| @@ -30,6 +30,7 @@ type Pushes struct { | |||
| 30 | 30 | fresh ports.RepoFreshener // nil: the reader needs no telling | |
| 31 | 31 | ci ports.WorkflowTrigger // nil: no CI | |
| 32 | 32 | wf ports.WorkflowGuard // nil: pushes by job tokens are refused | |
| 33 | + | tips ports.CIRefTipStore // nil: who pushes a ref is not recorded | |
| 33 | 34 | } | |
| 34 | 35 | ||
| 35 | 36 | var ( | |
| @@ -58,6 +59,14 @@ func (u *Pushes) WithWorkflowTrigger(t ports.WorkflowTrigger) *Pushes { | |||
| 58 | 59 | return u | |
| 59 | 60 | } | |
| 60 | 61 | ||
| 62 | + | // WithRefTips makes Check record who is about to move each branch and tag | |
| 63 | + | // (domain.CIRefTip), before Git moves it. Call it before the first push; it | |
| 64 | + | // returns u. | |
| 65 | + | func (u *Pushes) WithRefTips(t ports.CIRefTipStore) *Pushes { | |
| 66 | + | u.tips = t | |
| 67 | + | return u | |
| 68 | + | } | |
| 69 | + | ||
| 61 | 70 | // WithWorkflowGuard lets Check tell whether a push changes workflow files, which | |
| 62 | 71 | // a push by a job token may not do. Without it a push by a job token is | |
| 63 | 72 | // refused, as it cannot be judged. Call it before the first push; it returns u. | |
| @@ -166,6 +175,13 @@ func (u *Pushes) Check(ctx context.Context, p domain.Principal, name domain.Repo | |||
| 166 | 175 | return err | |
| 167 | 176 | } | |
| 168 | 177 | } | |
| 178 | + | // Last, once the push is accepted: who pushes the branches and tags is | |
| 179 | + | // recorded before Git moves them, so that a tip never has the record of an | |
| 180 | + | // earlier pusher at the same commit (domain.CIRefTip). A record that cannot | |
| 181 | + | // be written refuses the push, as it cannot be run later with a known trust. | |
| 182 | + | if err := recordRefTips(ctx, u.tips, name, updates, authorOf(p), domain.PusherOf(p), domain.TrustedActor(p)); err != nil { | |
| 183 | + | return err | |
| 184 | + | } | |
| 169 | 185 | log := zerowrap.FromCtx(ctx) | |
| 170 | 186 | log.Debug().Str("repo", name.String()).Int("refs", len(updates)).Msg("push checked") | |
| 171 | 187 | return nil | |
| @@ -30,6 +30,7 @@ type Pushes struct { | |||||
| 30 | fresh ports.RepoFreshener // nil: the reader needs no telling | 30 | fresh ports.RepoFreshener // nil: the reader needs no telling | ||
| 31 | ci ports.WorkflowTrigger // nil: no CI | 31 | ci ports.WorkflowTrigger // nil: no CI | ||
| 32 | wf ports.WorkflowGuard // nil: pushes by job tokens are refused | 32 | wf ports.WorkflowGuard // nil: pushes by job tokens are refused | ||
| 33 | + | tips ports.CIRefTipStore // nil: who pushes a ref is not recorded | |||
| 33 | } | 34 | } | ||
| 34 | 35 | ||||
| 35 | var ( | 36 | var ( | ||
| @@ -58,6 +59,14 @@ func (u *Pushes) WithWorkflowTrigger(t ports.WorkflowTrigger) *Pushes { | |||||
| 58 | return u | 59 | return u | ||
| 59 | } | 60 | } | ||
| 60 | 61 | ||||
| 62 | + | // WithRefTips makes Check record who is about to move each branch and tag | |||
| 63 | + | // (domain.CIRefTip), before Git moves it. Call it before the first push; it | |||
| 64 | + | // returns u. | |||
| 65 | + | func (u *Pushes) WithRefTips(t ports.CIRefTipStore) *Pushes { | |||
| 66 | + | u.tips = t | |||
| 67 | + | return u | |||
| 68 | + | } | |||
| 69 | + | ||||
| 61 | // WithWorkflowGuard lets Check tell whether a push changes workflow files, which | 70 | // WithWorkflowGuard lets Check tell whether a push changes workflow files, which | ||
| 62 | // a push by a job token may not do. Without it a push by a job token is | 71 | // a push by a job token may not do. Without it a push by a job token is | ||
| 63 | // refused, as it cannot be judged. Call it before the first push; it returns u. | 72 | // refused, as it cannot be judged. Call it before the first push; it returns u. | ||
| @@ -166,6 +175,13 @@ func (u *Pushes) Check(ctx context.Context, p domain.Principal, name domain.Repo | |||||
| 166 | return err | 175 | return err | ||
| 167 | } | 176 | } | ||
| 168 | } | 177 | } | ||
| 178 | + | // Last, once the push is accepted: who pushes the branches and tags is | |||
| 179 | + | // recorded before Git moves them, so that a tip never has the record of an | |||
| 180 | + | // earlier pusher at the same commit (domain.CIRefTip). A record that cannot | |||
| 181 | + | // be written refuses the push, as it cannot be run later with a known trust. | |||
| 182 | + | if err := recordRefTips(ctx, u.tips, name, updates, authorOf(p), domain.PusherOf(p), domain.TrustedActor(p)); err != nil { | |||
| 183 | + | return err | |||
| 184 | + | } | |||
| 169 | log := zerowrap.FromCtx(ctx) | 185 | log := zerowrap.FromCtx(ctx) | ||
| 170 | log.Debug().Str("repo", name.String()).Int("refs", len(updates)).Msg("push checked") | 186 | log.Debug().Str("repo", name.String()).Int("refs", len(updates)).Msg("push checked") | ||
| 171 | return nil | 187 | return nil | ||
M internal/usecase/repos_write.go +3 -0
| @@ -203,6 +203,9 @@ func (r *Repos) Delete(ctx context.Context, p domain.Principal, name domain.Repo | |||
| 203 | 203 | if err := r.meta.Delete(ctx, name); err != nil { | |
| 204 | 204 | return fmt.Errorf("delete metadata of %q: %w", name, err) | |
| 205 | 205 | } | |
| 206 | + | if r.ci != nil { | |
| 207 | + | r.ci.ForgetRepo(name) | |
| 208 | + | } | |
| 206 | 209 | if len(runIDs) > 0 { | |
| 207 | 210 | if err := r.ci.RemoveLogs(context.WithoutCancel(ctx), runIDs); err != nil { | |
| 208 | 211 | log.Warn().Err(err).Str("repo", name.String()).Msg("remove the CI logs of the repository; the sweep will") | |
| @@ -203,6 +203,9 @@ func (r *Repos) Delete(ctx context.Context, p domain.Principal, name domain.Repo | |||||
| 203 | if err := r.meta.Delete(ctx, name); err != nil { | 203 | if err := r.meta.Delete(ctx, name); err != nil { | ||
| 204 | return fmt.Errorf("delete metadata of %q: %w", name, err) | 204 | return fmt.Errorf("delete metadata of %q: %w", name, err) | ||
| 205 | } | 205 | } | ||
| 206 | + | if r.ci != nil { | |||
| 207 | + | r.ci.ForgetRepo(name) | |||
| 208 | + | } | |||
| 206 | if len(runIDs) > 0 { | 209 | if len(runIDs) > 0 { | ||
| 207 | if err := r.ci.RemoveLogs(context.WithoutCancel(ctx), runIDs); err != nil { | 210 | if err := r.ci.RemoveLogs(context.WithoutCancel(ctx), runIDs); err != nil { | ||
| 208 | log.Warn().Err(err).Str("repo", name.String()).Msg("remove the CI logs of the repository; the sweep will") | 211 | log.Warn().Err(err).Str("repo", name.String()).Msg("remove the CI logs of the repository; the sweep will") | ||
M proto/ermite/storage/ci.proto +15 -0
| @@ -416,6 +416,21 @@ message CIConcurrencyRecord { | |||
| 416 | 416 | CIConcurrencyEntry pending = 2; | |
| 417 | 417 | } | |
| 418 | 418 | ||
| 419 | + | // CIRefTip is stored under "cireftip/<repo>/<full ref name>" (a branch or a | |
| 420 | + | // tag): who moves the ref to the object sha, so that a scheduled or dispatched | |
| 421 | + | // run on it takes their actor and trust (domain.CIRefTip). It is written before | |
| 422 | + | // the ref moves, and names the object it is for: a record of another object | |
| 423 | + | // than the ref's is not valid. | |
| 424 | + | message CIRefTip { | |
| 425 | + | string sha = 1; | |
| 426 | + | // The actor the run shows ("owner", "root" or a token label). | |
| 427 | + | string actor = 2; | |
| 428 | + | bool trusted = 3; | |
| 429 | + | // As CIRunSource: "root", "owner", "token" or empty. | |
| 430 | + | string pusher_kind = 4; | |
| 431 | + | string pusher_token_id = 5; | |
| 432 | + | } | |
| 433 | + | ||
| 419 | 434 | // CIConcurrencyMark is stored under | |
| 420 | 435 | // "concurmark/<repo>/<escaped key(group)>/<run:020d>/<attempt:010d>/<job id or ->": | |
| 421 | 436 | // where an entry stands in its group. It is written in the same batch as the | |
| @@ -416,6 +416,21 @@ message CIConcurrencyRecord { | |||||
| 416 | CIConcurrencyEntry pending = 2; | 416 | CIConcurrencyEntry pending = 2; | ||
| 417 | } | 417 | } | ||
| 418 | 418 | ||||
| 419 | + | // CIRefTip is stored under "cireftip/<repo>/<full ref name>" (a branch or a | |||
| 420 | + | // tag): who moves the ref to the object sha, so that a scheduled or dispatched | |||
| 421 | + | // run on it takes their actor and trust (domain.CIRefTip). It is written before | |||
| 422 | + | // the ref moves, and names the object it is for: a record of another object | |||
| 423 | + | // than the ref's is not valid. | |||
| 424 | + | message CIRefTip { | |||
| 425 | + | string sha = 1; | |||
| 426 | + | // The actor the run shows ("owner", "root" or a token label). | |||
| 427 | + | string actor = 2; | |||
| 428 | + | bool trusted = 3; | |||
| 429 | + | // As CIRunSource: "root", "owner", "token" or empty. | |||
| 430 | + | string pusher_kind = 4; | |||
| 431 | + | string pusher_token_id = 5; | |||
| 432 | + | } | |||
| 433 | + | ||||
| 419 | // CIConcurrencyMark is stored under | 434 | // CIConcurrencyMark is stored under | ||
| 420 | // "concurmark/<repo>/<escaped key(group)>/<run:020d>/<attempt:010d>/<job id or ->": | 435 | // "concurmark/<repo>/<escaped key(group)>/<run:020d>/<attempt:010d>/<job id or ->": | ||
| 421 | // where an entry stands in its group. It is written in the same batch as the | 436 | // where an entry stands in its group. It is written in the same batch as the | ||