feat(ci): pull_request runs from merge requests (#32)

commit: 815fb98f762bebc9c6462822129af230835ec768
author: brice · 7d ago · 2026-10-15 07:46 UTC
committer: bnema · 7d ago · 2026-10-15 07:46 UTC
parent: c11af01
signature: SSH UnverifiedSHA256:VOCA5lYARHcbKDZ7GhTV45VmcjWnJ6ZQIBazj17LivA The key is unknown
refs: tag: v0.14.0

44 files · +4630 -49

M README.md +5 -3


@@ -107,7 +107,7 @@ For an AI agent that should propose changes, not publish them: `--branch-prefix
107	107		- It opens, comments on and closes merge requests as usual; with `mr:merge` it can merge only into a branch under its prefixes, and delete only a source branch under them. You review and merge the agent's request.
108	108		- CI: it reads runs, jobs, logs and checks as any token that can read the repository. With `ci:write` it can cancel and re-run only the runs of branches under its prefixes, never those of `main`, other branches or tags. It never manages runners (`runner list\|remove`, registration), even with `runner:register`: a runner runs every branch of every repository.
109	109		- `ermite token list` shows a `branches` column (`all` when unlimited); `--json` has `branch_prefixes`.
110		-	- This build stores tokens in storage schema 2. Once it has started on your data, an older ermite refuses to start on it (it would read limited tokens as unrestricted): to go back, restore the backup taken before the upgrade.
	110	+	- This build uses storage schema 3 (2 added the branch prefixes of tokens; 3 the jobs of reusable workflows and the head commit of the runs of merge requests). Once it has started on your data, an older ermite refuses to start on it (it would read limited tokens as unrestricted, run the jobs of a called workflow as the caller's, and attach the checks of merge request runs to the wrong commit): to go back, restore the backup taken before the upgrade.
111	111
112	112		## Merge requests
113	113

@@ -115,7 +115,7 @@ A merge request proposes a branch for its target (the default branch unless set)
115	115
116	116		## CI
117	117
118		-	ermite runs the `.github/workflows/*.yml` workflows of a repository on `push` (a branch or a tag, and the merge of a merge request) and shows each job as a check on the commit, the branch list and the merge request. CI is on once a runner is registered; there is no switch. Before that a push makes no run and no check.
	118	+	ermite runs the `.github/workflows/*.yml` workflows of a repository on `push` (a branch or a tag, and the merge of a merge request) and on `pull_request` (the merge requests, below) and shows each job as a check on the commit, the branch list and the merge request. CI is on once a runner is registered; there is no switch. Before that a push makes no run and no check.
119	119
120	120		Runs execute on [ermite-runner](https://git.bnema.dev/bnema/ermite-runner) hosts, which call ermite over its API. Create a token for the runner host with `ermite token create <label> --preset runner` (the `runner:register` scope, which also gives access to the code and the secrets of every repository: give it only to runners you operate), then register the runner with it. `ermite runner list` shows the runners and whether they are online; `ermite runner remove <id>` stops one. A job goes to a runner that has every label of its `runs-on`; a job no runner takes within 24 hours fails.
121	121

@@ -123,7 +123,9 @@ Every job gets a repository token (`github.token`, `ermitegh_…`) when a ru
123	123
124	124		An action of `ERMITE_OWNER/<repo>` that exists on this server wins over the same name on github.com. It is resolved on behalf of whoever pushed the run: ermite records the pusher with the run (the admin token, the owner's session, or the stored token by its id) and, at every answer, asks whether that pusher can read the action's repository, with the same rule as everywhere else: a public repository, or `repo:read` on a repository of the token's list. The token is looked up again each time, so a token revoked or expired since reads nothing for the run. A repository the pusher cannot read answers like one that does not exist (`repository not found`), whatever the ref, so it does not reveal itself. A job can therefore use an action from another private repository only when its pusher could read it. The runner gets an archive token for each such action: it is bound to that repository and that commit and downloads that one archive (`/-/github/api/repos/<owner>/<repo>/tarball/<sha>`) and nothing else. Git over HTTP, the repository endpoint and the rest of the API see it as a caller who can read nothing, not even a public repository. It goes with its job, and with the repository it reads if that is deleted. At most eight archives are written at once for one client address (`429` beyond: a runner host runs several jobs, and all come from one address) and sixteen on the server (`503`); both answers carry `Retry-After: 5`. A client that stops reading is cut after two minutes without progress.
125	125
126		-	The Actions that run here call a small part of GitHub's REST API, served at `github.api_url` (`/-/github/api`): `GET /repos/{owner}/{repo}`, `/repos/{owner}/{repo}/hash-algorithm` and `/repos/{owner}/{repo}/tarball[/{ref}]` (streamed with 200, not redirected; a `HEAD` answers the headers without building the archive). The owner in these paths and in the Git URLs is matched without regard to case, as on GitHub. Any other path is GitHub's `404 {"message":"Not Found",…}`, so an action that needs more fails as on a GitHub instance without that feature.
	126	+	`pull_request`. A merge request is a pull request for the workflows. ermite keeps `refs/pull/<n>/head` at the head of its source branch and, while that head merges cleanly into the target, `refs/pull/<n>/merge` at a test merge commit (parents: the target tip, then the head; message `Merge <head> into <base>`). Both are served to `git fetch`, never listed as branches or tags, and refused on every push (for any token, a job's included). The activities are `opened` (the request is created), `synchronize` (its source branch is pushed; each push runs for the commit it pushed, with the rights of whoever pushed it) and `closed` (closed or merged, with `merged` true or false; it uses the last test merge commit, or the head when there is none, and is never computed again, even when the source branch is gone); `reopened` is accepted in `types:` but merge requests cannot be reopened. Without `types:` a workflow gets `opened`, `synchronize` and `reopened`. The run has `github.ref` `refs/pull/<n>/merge`, `github.sha` the merge commit (the workflow file is read from it, so `actions/checkout` gets the merged tree), `github.head_ref` and `github.base_ref` set, and a payload shaped like GitHub's `pull_request` webhook with the fields ermite has. `branches` and `paths` (and their `-ignore` forms) apply to the target branch and to the files changed between the merge base and the head. A head that conflicts with the target makes no run for that update. The checks of the run are on the head commit. The run is trusted (secrets, write token) only when whoever caused the event is the owner, like a push; a job token starts no run.
	127	+
	128	+	The Actions that run here call a small part of GitHub's REST API, served at `github.api_url` (`/-/github/api`): `GET /repos/{owner}/{repo}`, `/repos/{owner}/{repo}/hash-algorithm`, `/repos/{owner}/{repo}/tarball[/{ref}]` (streamed with 200, not redirected; a `HEAD` answers the headers without building the archive) and `GET /repos/{owner}/{repo}/pulls/{number}/files` (what `dorny/paths-filter` reads on a `pull_request`: up to 3000 files, `page` and `per_page` of at most 100, 30 by default, with the `Link` header; a job token needs `pull-requests: read`, else `403 Resource not accessible by integration`). The owner in these paths and in the Git URLs is matched without regard to case, as on GitHub. Any other path is GitHub's `404 {"message":"Not Found",…}`, so an action that needs more fails as on a GitHub instance without that feature.
127	129
128	130		Actions from github.com are resolved with `git ls-remote https://github.com/<owner>/<repo>` (no credentials, 20 s, 16 MiB of output, at most four at once, 60 s for a whole call). ermite hands git only the proxy settings of its own environment (`HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`, `ALL_PROXY` and their lower-case forms), so a server that reaches the Internet through a proxy sets them for `ermite serve`; nothing else of its environment, no credential, reaches git.
129	131

M gen/ermite/storage/ci.pb.go +14 -3 generated file: collapsed

load diff

M internal/adapters/gitcli/backend.go +5 -0


@@ -10,6 +10,7 @@ import (
10	10		"strconv"
11	11		"strings"
12	12
	13	+	"bnema.dev/ermite/internal/domain"
13	14		"bnema.dev/ermite/internal/ports"
14	15		)
15	16

@@ -35,6 +36,7 @@ func HooksDir(dataDir string) string { return filepath.Join(dataDir, "run", "hoo
35	36		// pack.threads=1
36	37		// pack.windowMemory=256m
37	38		// http.receivepack=true (push only)
	39	+	// receive.hideRefs=refs/pull/ (push only: ermite writes them, git refuses updates)
38	40		// receive.maxInputSize=<max> (push only, when configured)
39	41		//
40	42		// A push also gets REMOTE_USER=owner.

@@ -55,6 +57,9 @@ func (g *Git) HTTPBackend(ctx context.Context, push bool) (ports.HTTPBackend, er
55	57		}
56	58		if push {
57	59		conf = append(conf, [2]string{"http.receivepack", "true"})
	60	+	// refs/pull/* (the merge requests' head and test merge) are served to
	61	+	// fetch and cannot be updated by a push: git itself refuses a hidden ref.
	62	+	conf = append(conf, [2]string{"receive.hideRefs", domain.PullRefPrefix})
58	63		if g.maxRecv > 0 {
59	64		conf = append(conf, [2]string{"receive.maxInputSize", strconv.FormatInt(g.maxRecv, 10)})
60	65		}

M internal/adapters/gitcli/backend_test.go +1 -0


@@ -101,6 +101,7 @@ func TestHTTPBackendEnvironment(t *testing.T) {
101	101		"pack.threads": "1",
102	102		"pack.windowMemory": "256m",
103	103		"http.receivepack": "true",
	104	+	"receive.hideRefs": "refs/pull/",
104	105		}, backendConf(pe), "http.receivepack only for a push; no size limit unless configured")
105	106
106	107		// Nothing from the caller's environment leaks.

A internal/adapters/gitcli/pull.go +91 -0

@@ -0,0 +1,91 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package gitcli
	4	+
	5	+	import (
	6	+	"context"
	7	+	"errors"
	8	+	"fmt"
	9	+	"strings"
	10	+
	11	+	"bnema.dev/ermite/internal/domain"
	12	+	"bnema.dev/ermite/internal/ports"
	13	+	)
	14	+
	15	+	var _ ports.PullGit = (*Git)(nil)
	16	+
	17	+	// TestMerge runs `git merge-tree --write-tree`, which merges in the object
	18	+	// database and touches no ref or index: it prints the tree of the result and
	19	+	// exits 0, or exits 1 when the merge conflicts. Commits with no common history
	20	+	// do not merge (GitHub makes no test merge for them either).
	21	+	//
	22	+	//hexcheck:ignore suspicious-business-logic-in-adapter maps git exit codes to the clean/conflict answer of the port
	23	+	func (g *Git) TestMerge(ctx context.Context, repo domain.RepoName, base, head domain.ObjectID) (domain.ObjectID, bool, error) {
	24	+	dir, err := g.repoDir(repo)
	25	+	if err != nil {
	26	+	return domain.ObjectID{}, false, err
	27	+	}
	28	+	if _, err := g.MergeBase(ctx, repo, base, head); errors.Is(err, domain.ErrNotFound) {
	29	+	return domain.ObjectID{}, false, nil
	30	+	} else if err != nil {
	31	+	return domain.ObjectID{}, false, err
	32	+	}
	33	+	out, err := g.run(ctx, readArgs(dir, "merge-tree", "--write-tree", "--no-messages", "--end-of-options", base.String(), head.String())...)
	34	+	switch {
	35	+	case exitCode(err) == 1:
	36	+	return domain.ObjectID{}, false, nil
	37	+	case err != nil:
	38	+	return domain.ObjectID{}, false, fmt.Errorf("test merge %s into %s in %q: %w", head, base, repo, err)
	39	+	}
	40	+	line, _, _ := strings.Cut(out, "\n")
	41	+	tree, err := parseID(line)
	42	+	if err != nil {
	43	+	return domain.ObjectID{}, false, err
	44	+	}
	45	+	return tree, true, nil
	46	+	}
	47	+
	48	+	// SetPullRef runs `git update-ref` on a ref that domain.IsPullRef accepts, or
	49	+	// `update-ref -d` for a zero id (deleting a ref that is not there is fine).
	50	+	//
	51	+	//hexcheck:ignore suspicious-business-logic-in-adapter checks the ref name with the domain rule before handing it to git
	52	+	func (g *Git) SetPullRef(ctx context.Context, repo domain.RepoName, ref string, id domain.ObjectID) error {
	53	+	if !domain.IsPullRef(ref) {
	54	+	return fmt.Errorf("pull ref %q of %q: %w", ref, repo, domain.ErrInvalid)
	55	+	}
	56	+	dir, err := g.repoDir(repo)
	57	+	if err != nil {
	58	+	return err
	59	+	}
	60	+	args := []string{"--git-dir=" + dir, "update-ref"}
	61	+	if id.IsZero() {
	62	+	args = append(args, "-d", ref)
	63	+	} else {
	64	+	args = append(args, ref, id.String())
	65	+	}
	66	+	if _, err := g.run(ctx, args...); err != nil {
	67	+	return fmt.Errorf("update pull ref %q of %q: %w", ref, repo, err)
	68	+	}
	69	+	return nil
	70	+	}
	71	+
	72	+	// PullRef reads a ref that domain.IsPullRef accepts with `git rev-parse`.
	73	+	//
	74	+	//hexcheck:ignore suspicious-business-logic-in-adapter checks the ref name with the domain rule before handing it to git
	75	+	func (g *Git) PullRef(ctx context.Context, repo domain.RepoName, ref string) (domain.ObjectID, error) {
	76	+	if !domain.IsPullRef(ref) {
	77	+	return domain.ObjectID{}, fmt.Errorf("pull ref %q of %q: %w", ref, repo, domain.ErrNotFound)
	78	+	}
	79	+	dir, err := g.repoDir(repo)
	80	+	if err != nil {
	81	+	return domain.ObjectID{}, err
	82	+	}
	83	+	out, err := g.run(ctx, readArgs(dir, "rev-parse", "--verify", "--quiet", ref+"^{commit}")...)
	84	+	if exitCode(err) == 1 {
	85	+	return domain.ObjectID{}, fmt.Errorf("pull ref %q of %q: %w", ref, repo, domain.ErrNotFound)
	86	+	}
	87	+	if err != nil {
	88	+	return domain.ObjectID{}, fmt.Errorf("read pull ref %q of %q: %w", ref, repo, err)
	89	+	}
	90	+	return parseID(out)
	91	+	}

M internal/adapters/githubapi/handler.go +40 -1


@@ -6,6 +6,7 @@
6	6		// GET /repos/{owner}/{repo}
7	7		// GET /repos/{owner}/{repo}/hash-algorithm
8	8		// GET /repos/{owner}/{repo}/tarball[/{ref}]
	9	+	// GET /repos/{owner}/{repo}/pulls/{number}/files (dorny/paths-filter on pull_request)
9	10		//
10	11		// Everything else under the prefix is GitHub's 404 JSON, so that an action
11	12		// calling an endpoint of a feature ermite lacks fails as it does on a GitHub

@@ -115,6 +116,10 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r http.Request) {
115	116		w.Header().Set("X-Content-Type-Options", "nosniff")
116	117		rest := strings.TrimPrefix(r.URL.Path, Prefix)
117	118		kind, owner, repo, ref, ok := route(rest)
	119	+	var number int64
	120	+	if kind == "pullfiles" {
	121	+	number, _ = strconv.ParseInt(ref, 10, 64)
	122	+	}
118	123		if !ok {
119	124		notFound(w)
120	125		return

@@ -160,10 +165,12 @@ func (h Handler) ServeHTTP(w http.ResponseWriter, r http.Request) {
160	165		writeJSON(w, r, []byte(`{"hash_algorithm":`+strconv.Quote(alg)+`}`))
161	166		case "tarball":
162	167		h.tarball(w, r, p, owner, repo, ref, clientlimit.Key(addr))
	168	+	case "pullfiles":
	169	+	h.pullFiles(w, r, p, owner, repo, number)
163	170		}
164	171		}
165	172
166		-	// route recognises the three endpoints. ref is "" for /tarball.
	173	+	// route recognises the endpoints. ref is "" for /tarball.
167	174		func route(rest string) (kind, owner, repo, ref string, ok bool) {
168	175		parts := strings.Split(strings.TrimPrefix(rest, "/"), "/")
169	176		if len(parts) < 3 \|\| parts[0] != "repos" \|\| parts[1] == "" \|\| parts[2] == "" {

@@ -175,6 +182,9 @@ func route(rest string) (kind, owner, repo, ref string, ok bool) {
175	182		return "repo", owner, repo, "", true
176	183		case len(parts) == 4 && parts[3] == "hash-algorithm":
177	184		return "hash", owner, repo, "", true
	185	+	case len(parts) == 6 && parts[3] == "pulls" && parts[5] == "files" && isNumber(parts[4]):
	186	+	// ref carries the number of the pull request.
	187	+	return "pullfiles", owner, repo, parts[4], true
178	188		case len(parts) >= 4 && parts[3] == "tarball":
179	189		// The ref may hold slashes (feature/x): everything after tarball/.
180	190		return "tarball", owner, repo, strings.Join(parts[4:], "/"), true

@@ -182,6 +192,35 @@ func route(rest string) (kind, owner, repo, ref string, ok bool) {
182	192		return "", "", "", "", false
183	193		}
184	194
	195	+	// isNumber reports whether s is a positive decimal number.
	196	+	func isNumber(s string) bool {
	197	+	n, err := strconv.ParseInt(s, 10, 64)
	198	+	return err == nil && n > 0 && strconv.FormatInt(n, 10) == s
	199	+	}
	200	+
	201	+	// pullFiles answers the files of a pull request, a page at a time as GitHub
	202	+	// does: page and per_page in the query (30 and 100 at most), the other pages
	203	+	// in the Link header. A parameter that is not a positive number is the default,
	204	+	// as GitHub treats it.
	205	+	func (h Handler) pullFiles(w http.ResponseWriter, r http.Request, p domain.Principal, owner, repo string, number int64) {
	206	+	q := r.URL.Query()
	207	+	page, _ := strconv.Atoi(q.Get("page"))
	208	+	perPage, _ := strconv.Atoi(q.Get("per_page"))
	209	+	body, link, err := h.cfg.API.ListPullFiles(r.Context(), p, owner, repo, number, page, perPage)
	210	+	if err != nil {
	211	+	if errors.Is(err, domain.ErrForbidden) {
	212	+	jsonError(w, http.StatusForbidden, "Resource not accessible by integration")
	213	+	return
	214	+	}
	215	+	h.fail(r.Context(), w, err)
	216	+	return
	217	+	}
	218	+	if link != "" {
	219	+	w.Header().Set("Link", link)
	220	+	}
	221	+	writeJSON(w, r, body)
	222	+	}
	223	+
185	224		// tarball answers an archive request. A HEAD resolves the ref and sends the
186	225		// headers of the archive, and runs no git. A GET streams the archive; it is not
187	226		// a 302 to a codeload: ermite serves the bytes itself, which the runner follows

M internal/adapters/kvstore/ci.go +3 -3


@@ -277,7 +277,7 @@ func encodeRun(r domain.CIRun) ([]byte, error) {
277	277		return mustMarshal(&storagepb.CIRun{
278	278		Repo: r.Repo.String(), Id: r.ID, Number: r.Number, Attempt: r.Attempt,
279	279		WorkflowPath: r.WorkflowPath, WorkflowName: r.WorkflowName, Name: r.Name, Title: r.Title,
280		-	Event: r.Event, Ref: r.Ref, Sha: r.Sha, Actor: r.Actor,
	280	+	Event: r.Event, Ref: r.Ref, Sha: r.Sha, HeadSha: r.HeadSha, Actor: r.Actor,
281	281		Status: encodeStatus(r.Status), Conclusion: encodeConclusion(r.Conclusion), Error: r.Error,
282	282		CreatedAt: encodeTime(r.CreatedAt), StartedAt: encodeTime(r.StartedAt), CompletedAt: encodeTime(r.CompletedAt),
283	283		CancelRequested: r.CancelRequested, CancelRequestedAt: encodeTime(r.CancelRequestedAt), Trusted: r.Trusted,

@@ -301,7 +301,7 @@ func decodeRun(item kv.Item) (domain.CIRun, error) {
301	301		return domain.CIRun{
302	302		Repo: repo, ID: pb.GetId(), Number: pb.GetNumber(), Attempt: pb.GetAttempt(),
303	303		WorkflowPath: pb.GetWorkflowPath(), WorkflowName: pb.GetWorkflowName(), Name: name, Title: pb.GetTitle(),
304		-	Event: pb.GetEvent(), Ref: pb.GetRef(), Sha: pb.GetSha(), Actor: pb.GetActor(),
	304	+	Event: pb.GetEvent(), Ref: pb.GetRef(), Sha: pb.GetSha(), HeadSha: pb.GetHeadSha(), Actor: pb.GetActor(),
305	305		Status: decodeStatus(pb.GetStatus()), Conclusion: decodeConclusion(pb.GetConclusion()), Error: pb.GetError(),
306	306		CreatedAt: decodeTime(pb.GetCreatedAt()), StartedAt: decodeTime(pb.GetStartedAt()), CompletedAt: decodeTime(pb.GetCompletedAt()),
307	307		CancelRequested: pb.GetCancelRequested(), CancelRequestedAt: decodeTime(pb.GetCancelRequestedAt()),

@@ -913,7 +913,7 @@ func (c *CI) Sweep(ctx context.Context, before time.Time) ([]int64, error) {
913	913		}
914	914		ids = append(ids, r.ID)
915	915		keys = append(keys, c.runKeys(ctx, snap, r.Repo, r.ID)...)
916		-	for ck := range snap.Prefix(ctx, []byte(checkPrefix+r.Repo.String()+"/"+r.Sha+"/")) {
	916	+	for ck := range snap.Prefix(ctx, []byte(checkPrefix+r.Repo.String()+"/"+r.CheckSha()+"/")) {
917	917		// A check belongs to the run only while it is the latest one
918	918		// for its job: leave it when another run replaced it.
919	919		if it, ok := snap.Get(ctx, ck); ok {

M internal/adapters/kvstore/mergerequests.go +203 -4


@@ -6,10 +6,14 @@ import (
6	6		"context"
7	7		"errors"
8	8		"fmt"
	9	+	"net/url"
9	10		"slices"
	11	+	"strconv"
	12	+	"strings"
10	13		"unicode/utf8"
11	14
12	15		kv "bnema.dev/ermite-kv"
	16	+	"github.com/bnema/zerowrap"
13	17		"google.golang.org/protobuf/proto"
14	18
15	19		storagepb "bnema.dev/ermite/gen/ermite/storage"

@@ -25,8 +29,16 @@ const (
25	29		commentPrefix = "mrc/" // mrc/<repo>/<number:010d>/<id:010d>
26	30		readPrefix = "mrread/" // mrread/<repo>/<number:010d>
27	31		seqPrefix = "mrseq/" // mrseq/<repo>
28		-	numberFormat = "%010d"
29		-	createTries = 10
	32	+	// The indexes of the open merge requests by source branch and by target
	33	+	// branch, which a push looks up (the values are not read). They are written in
	34	+	// the same batch as the merge request, so they are never out of step with it,
	35	+	// and built once, together, for the data that predates them (BuildOpenIndex,
	36	+	// mrOpenIndexMarker).
	37	+	mrOpenPrefix = "mropen/" // mropen/<repo>/<escaped source>/<number:010d>
	38	+	mrTargetPrefix = "mrtarget/" // mrtarget/<repo>/<escaped target>/<number:010d>
	39	+	mrOpenIndexMarker = "meta/mropen" // present once the indexes cover every merge request
	40	+	numberFormat = "%010d"
	41	+	createTries = 10
30	42		)
31	43
32	44		// MergeRequests is the ermite-kv backed ports.MergeRequestStore. It shares the

@@ -56,11 +68,111 @@ func readKey(repo domain.RepoName, n int64) []byte {
56	68		}
57	69		func seqKey(repo domain.RepoName) []byte { return []byte(seqPrefix + repo.String()) }
58	70
	71	+	// mrOpenRepoPrefix is where the open merge requests of a repository are indexed
	72	+	// by source branch.
	73	+	func mrOpenRepoPrefix(repo domain.RepoName) string { return mrOpenPrefix + repo.String() + "/" }
	74	+
	75	+	// indexBranchPrefix is where the open merge requests of one branch are indexed
	76	+	// under prefix (mrOpenPrefix or mrTargetPrefix): the branch is escaped, so it
	77	+	// holds no '/' and the prefix matches that branch only.
	78	+	func indexBranchPrefix(prefix string, repo domain.RepoName, branch string) string {
	79	+	return prefix + repo.String() + "/" + url.PathEscape(branch) + "/"
	80	+	}
	81	+
	82	+	func indexKey(prefix string, repo domain.RepoName, branch string, n int64) []byte {
	83	+	return []byte(indexBranchPrefix(prefix, repo, branch) + fmt.Sprintf(numberFormat, n))
	84	+	}
	85	+
	86	+	// indexOpen adds to b the writes that keep the indexes of the open merge
	87	+	// requests (by source and by target) in step with a write of mr over old (nil
	88	+	// when old is not known, as for a new request or a comment: the keys of mr are
	89	+	// then put or deleted, whatever they were). Deleting a key that is not there is
	90	+	// not an error.
	91	+	func indexOpen(b kv.Batch, old domain.MergeRequest, mr domain.MergeRequest) {
	92	+	if old != nil && old.IsOpen() {
	93	+	if !mr.IsOpen() \|\| old.Source != mr.Source {
	94	+	b.Delete(indexKey(mrOpenPrefix, old.Repo, old.Source, old.Number))
	95	+	}
	96	+	if !mr.IsOpen() \|\| old.Target != mr.Target {
	97	+	b.Delete(indexKey(mrTargetPrefix, old.Repo, old.Target, old.Number))
	98	+	}
	99	+	}
	100	+	switch {
	101	+	case mr.IsOpen():
	102	+	b.Put(indexKey(mrOpenPrefix, mr.Repo, mr.Source, mr.Number), []byte{1})
	103	+	b.Put(indexKey(mrTargetPrefix, mr.Repo, mr.Target, mr.Number), []byte{1})
	104	+	case old == nil:
	105	+	b.Delete(indexKey(mrOpenPrefix, mr.Repo, mr.Source, mr.Number))
	106	+	b.Delete(indexKey(mrTargetPrefix, mr.Repo, mr.Target, mr.Number))
	107	+	}
	108	+	}
	109	+
	110	+	// BuildOpenIndex builds the indexes of the open merge requests (by source and
	111	+	// by target) when the data does not have them yet (it predates them): one pass
	112	+	// over the merge requests, in batches, then the marker. It runs when the store is opened and is idempotent: a pass
	113	+	// cut short by a crash is simply done again, writing the same keys.
	114	+	//
	115	+	//hexcheck:ignore suspicious-business-logic-in-adapter one-time storage backfill of an index: a storage concern, no policy
	116	+	func (s *Store) BuildOpenIndex(ctx context.Context) error {
	117	+	if _, ok := s.db.Get(ctx, []byte(mrOpenIndexMarker)); ok {
	118	+	return nil
	119	+	}
	120	+	var b kv.Batch
	121	+	n := 0
	122	+	flush := func() error {
	123	+	if b.Len() == 0 {
	124	+	return nil
	125	+	}
	126	+	if _, err := s.db.Apply(ctx, &b); err != nil {
	127	+	return fmt.Errorf("build the index of open merge requests: %w", err)
	128	+	}
	129	+	b.Reset()
	130	+	return nil
	131	+	}
	132	+	for key, item := range s.db.Snapshot(ctx).Prefix(ctx, []byte(mrPrefix)) {
	133	+	repoName, _, ok := strings.Cut(strings.TrimPrefix(string(key), mrPrefix), "/")
	134	+	if !ok {
	135	+	continue
	136	+	}
	137	+	repo, err := domain.ParseRepoName(repoName)
	138	+	if err != nil {
	139	+	continue
	140	+	}
	141	+	mr, err := decodeMergeRequest(repo, item)
	142	+	if err != nil {
	143	+	return fmt.Errorf("build the index of open merge requests: decode %s: %w", key, err)
	144	+	}
	145	+	if !mr.IsOpen() {
	146	+	continue
	147	+	}
	148	+	b.Put(indexKey(mrOpenPrefix, mr.Repo, mr.Source, mr.Number), []byte{1})
	149	+	b.Put(indexKey(mrTargetPrefix, mr.Repo, mr.Target, mr.Number), []byte{1})
	150	+	if n++; n%(deleteChunk/2) == 0 {
	151	+	if err := flush(); err != nil {
	152	+	return err
	153	+	}
	154	+	}
	155	+	}
	156	+	if err := flush(); err != nil {
	157	+	return err
	158	+	}
	159	+	b.Put([]byte(mrOpenIndexMarker), []byte{1})
	160	+	if _, err := s.db.Apply(ctx, &b); err != nil {
	161	+	return fmt.Errorf("build the index of open merge requests: %w", err)
	162	+	}
	163	+	log := zerowrap.FromCtx(ctx)
	164	+	log.Info().Int("open", n).Msg("index of open merge requests built")
	165	+	return nil
	166	+	}
	167	+
59	168		// mergeRequestKeys lists every key of the snapshot that belongs to the
60	169		// repository's merge requests.
61	170		func mergeRequestKeys(ctx context.Context, snap *kv.Snapshot, repo domain.RepoName) [][]byte {
62	171		var keys [][]byte
63		-	for _, p := range []string{mrRepoPrefix(repo), commentPrefix + repo.String() + "/", readPrefix + repo.String() + "/"} {
	172	+	for _, p := range []string{
	173	+	mrRepoPrefix(repo), commentPrefix + repo.String() + "/", readPrefix + repo.String() + "/",
	174	+	mrOpenRepoPrefix(repo), mrTargetPrefix + repo.String() + "/",
	175	+	} {
64	176		for k := range snap.Prefix(ctx, []byte(p)) {
65	177		keys = append(keys, slices.Clone(k))
66	178		}

@@ -116,6 +228,7 @@ func (m *MergeRequests) Create(ctx context.Context, mr domain.MergeRequest) (dom
116	228		b.Put(seqKey(mr.Repo), mustMarshal(&storagepb.Sequence{Last: mr.Number}))
117	229		b.RequireAbsent(mrKey(mr.Repo, mr.Number))
118	230		b.Put(mrKey(mr.Repo, mr.Number), val)
	231	+	indexOpen(&b, nil, mr)
119	232		_, err = m.db.Apply(ctx, &b)
120	233		if errors.Is(err, kv.ErrConflict) {
121	234		continue

@@ -154,6 +267,83 @@ func (m *MergeRequests) List(ctx context.Context, repo domain.RepoName) ([]domai
154	267		return out, nil
155	268		}
156	269
	270	+	// scanIndex reads the numbers under the prefixes of an index, in key order. It
	271	+	// reads keys only: no merge request is decoded. With a limit above 0 it stops as
	272	+	// soon as it has seen limit+1 keys and returns the first limit with truncated
	273	+	// set, so the work does not grow with the number of open requests on a branch.
	274	+	// The context is checked before every prefix and every key (the scan of ermite-kv
	275	+	// itself ignores it): on cancellation it stops and returns what it has with the
	276	+	// error of the context.
	277	+	func (m *MergeRequests) scanIndex(ctx context.Context, prefixes []string, limit int) (numbers []int64, truncated bool, err error) {
	278	+	snap := m.db.Snapshot(ctx)
	279	+	for _, prefix := range prefixes {
	280	+	if err := ctx.Err(); err != nil {
	281	+	return numbers, true, err
	282	+	}
	283	+	for key := range snap.Prefix(ctx, []byte(prefix)) {
	284	+	if err := ctx.Err(); err != nil {
	285	+	return numbers, true, err
	286	+	}
	287	+	if limit > 0 && len(numbers) == limit {
	288	+	return numbers, true, nil // the limit+1th key: more exist
	289	+	}
	290	+	n, perr := strconv.ParseInt(string(key[strings.LastIndexByte(string(key), '/')+1:]), 10, 64)
	291	+	if perr != nil {
	292	+	return numbers, false, fmt.Errorf("index key %q: %w", key, perr)
	293	+	}
	294	+	numbers = append(numbers, n)
	295	+	}
	296	+	}
	297	+	return numbers, false, nil
	298	+	}
	299	+
	300	+	func (m *MergeRequests) byBranch(ctx context.Context, index string, repo domain.RepoName, branches []string, limit int) ([]int64, bool, error) {
	301	+	prefixes := make([]string, 0, len(branches))
	302	+	for _, b := range branches {
	303	+	prefixes = append(prefixes, indexBranchPrefix(index, repo, b))
	304	+	}
	305	+	return m.scanIndex(ctx, prefixes, limit)
	306	+	}
	307	+
	308	+	// OpenBySource returns the numbers of the open merge requests of the repository
	309	+	// whose source branch is one of branches, from the index: a scan of the keys of
	310	+	// those branches only, whatever the number of merge requests the repository
	311	+	// ever had, stopping after limit of them (truncated).
	312	+	func (m *MergeRequests) OpenBySource(ctx context.Context, repo domain.RepoName, branches []string, limit int) ([]int64, bool, error) {
	313	+	return m.byBranch(ctx, mrOpenPrefix, repo, branches, limit)
	314	+	}
	315	+
	316	+	// OpenByTarget is OpenBySource for the target branch.
	317	+	func (m *MergeRequests) OpenByTarget(ctx context.Context, repo domain.RepoName, branches []string, limit int) ([]int64, bool, error) {
	318	+	return m.byBranch(ctx, mrTargetPrefix, repo, branches, limit)
	319	+	}
	320	+
	321	+	// ListOpen returns the open merge requests of the repository, highest number
	322	+	// first, through the index: only the open ones are read.
	323	+	//
	324	+	//hexcheck:ignore suspicious-business-logic-in-adapter reads the open merge requests through their index: a storage concern, no policy
	325	+	func (m *MergeRequests) ListOpen(ctx context.Context, repo domain.RepoName) ([]domain.MergeRequest, error) {
	326	+	numbers, _, err := m.scanIndex(ctx, []string{mrOpenRepoPrefix(repo)}, 0)
	327	+	if err != nil {
	328	+	return nil, err
	329	+	}
	330	+	slices.Sort(numbers)
	331	+	var out []domain.MergeRequest
	332	+	for _, n := range slices.Backward(numbers) {
	333	+	mr, err := m.Get(ctx, repo, n)
	334	+	if errors.Is(err, domain.ErrNotFound) {
	335	+	continue // closed or deleted since the snapshot of the index
	336	+	}
	337	+	if err != nil {
	338	+	return nil, err
	339	+	}
	340	+	if mr.IsOpen() {
	341	+	out = append(out, mr)
	342	+	}
	343	+	}
	344	+	return out, nil
	345	+	}
	346	+
157	347		// Update replaces the merge request with a version precondition.
158	348		//
159	349		//hexcheck:ignore suspicious-business-logic-in-adapter port method name only

@@ -163,7 +353,8 @@ func (m *MergeRequests) Update(ctx context.Context, mr domain.MergeRequest) erro
163	353		return err
164	354		}
165	355		key := mrKey(mr.Repo, mr.Number)
166		-	if _, ok := m.db.Get(ctx, key); !ok {
	356	+	cur, ok := m.db.Get(ctx, key)
	357	+	if !ok {
167	358		return domain.ErrNotFound
168	359		}
169	360		if mr.Version == 0 {

@@ -172,6 +363,13 @@ func (m *MergeRequests) Update(ctx context.Context, mr domain.MergeRequest) erro
172	363		var b kv.Batch
173	364		b.RequireVersion(key, mr.Version)
174	365		b.Put(key, val)
	366	+	// The stored request is the one mr was read at (the version is required
	367	+	// above): its index key moves with the write.
	368	+	if old, err := decodeMergeRequest(mr.Repo, cur); err == nil {
	369	+	indexOpen(&b, &old, mr)
	370	+	} else {
	371	+	indexOpen(&b, nil, mr)
	372	+	}
175	373		if _, err := m.db.Apply(ctx, &b); err != nil {
176	374		if errors.Is(err, kv.ErrConflict) {
177	375		return domain.ErrConflict

@@ -227,6 +425,7 @@ func (m *MergeRequests) SaveComment(ctx context.Context, mr domain.MergeRequest,
227	425		}
228	426		b.RequireVersion(mrKey(mr.Repo, mr.Number), mr.Version)
229	427		b.Put(mrKey(mr.Repo, mr.Number), mrVal)
	428	+	indexOpen(&b, nil, mr)
230	429		if c.Version == 0 {
231	430		b.RequireAbsent(commentKey(mr.Repo, mr.Number, c.ID))
232	431		} else {

A internal/adapters/kvstore/mergerequests_open_test.go +286 -0

@@ -0,0 +1,286 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package kvstore_test
	4	+
	5	+	import (
	6	+	"context"
	7	+	"testing"
	8	+
	9	+	"github.com/stretchr/testify/assert"
	10	+	"github.com/stretchr/testify/require"
	11	+
	12	+	storagepb "bnema.dev/ermite/gen/ermite/storage"
	13	+	"bnema.dev/ermite/internal/adapters/kvstore"
	14	+	"bnema.dev/ermite/internal/domain"
	15	+	)
	16	+
	17	+	func bySource(t testing.T, mrs kvstore.MergeRequests, repo domain.RepoName, branches ...string) []int64 {
	18	+	t.Helper()
	19	+	got, truncated, err := mrs.OpenBySource(context.Background(), repo, branches, 0)
	20	+	require.NoError(t, err)
	21	+	require.False(t, truncated, "no limit: never truncated")
	22	+	return got
	23	+	}
	24	+
	25	+	func byTarget(t testing.T, mrs kvstore.MergeRequests, repo domain.RepoName, branches ...string) []int64 {
	26	+	t.Helper()
	27	+	got, truncated, err := mrs.OpenByTarget(context.Background(), repo, branches, 0)
	28	+	require.NoError(t, err)
	29	+	require.False(t, truncated)
	30	+	return got
	31	+	}
	32	+
	33	+	func mrFromTo(t *testing.T, repo domain.RepoName, source, target string) domain.MergeRequest {
	34	+	mr := mrOn(t, repo, source)
	35	+	mr.Target = target
	36	+	return mr
	37	+	}
	38	+
	39	+	func openNumbers(t testing.T, mrs kvstore.MergeRequests, repo domain.RepoName) []int64 {
	40	+	t.Helper()
	41	+	list, err := mrs.ListOpen(context.Background(), repo)
	42	+	require.NoError(t, err)
	43	+	var out []int64
	44	+	for _, mr := range list {
	45	+	out = append(out, mr.Number)
	46	+	}
	47	+	return out
	48	+	}
	49	+
	50	+	func mrOn(t *testing.T, repo domain.RepoName, source string) domain.MergeRequest {
	51	+	mr := newMR(t, repo, "t")
	52	+	mr.Source = source
	53	+	return mr
	54	+	}
	55	+
	56	+	// The index follows every write of a merge request: create, comment, source
	57	+	// change, close and merge.
	58	+	func TestOpenMergeRequestIndexFollowsTheWrites(t *testing.T) {
	59	+	ctx := context.Background()
	60	+	s, mrs, n := repoWithMRs(t)
	61	+
	62	+	a, err := mrs.Create(ctx, mrOn(t, n, "feat/a"))
	63	+	require.NoError(t, err)
	64	+	b, err := mrs.Create(ctx, mrOn(t, n, "feat/b"))
	65	+	require.NoError(t, err)
	66	+	c, err := mrs.Create(ctx, mrOn(t, n, "feat/a")) // a second open request on the same source
	67	+	require.NoError(t, err)
	68	+	assert.Equal(t, []int64{a.Number, c.Number}, bySource(t, mrs, n, "feat/a"))
	69	+	assert.Equal(t, []int64{b.Number}, bySource(t, mrs, n, "feat/b"))
	70	+	assert.Empty(t, bySource(t, mrs, n, "feat"), "a prefix of a branch is not the branch")
	71	+	assert.Empty(t, bySource(t, mrs, n, "feat/a/x"))
	72	+	assert.Equal(t, []int64{c.Number, b.Number, a.Number}, openNumbers(t, mrs, n), "highest number first")
	73	+
	74	+	// A comment rewrites the request: the index stays.
	75	+	text, err := domain.ParsePlainText("hi")
	76	+	require.NoError(t, err)
	77	+	cm := domain.Comment{ID: 1, Author: "x", Text: text, CreatedAt: created}
	78	+	a.CommentSeq = 1
	79	+	require.NoError(t, mrs.SaveComment(ctx, a, cm))
	80	+	assert.Equal(t, []int64{a.Number, c.Number}, bySource(t, mrs, n, "feat/a"))
	81	+
	82	+	// The source changes (a rewrite of the request): the key moves.
	83	+	b, err = mrs.Get(ctx, n, b.Number)
	84	+	require.NoError(t, err)
	85	+	b.Source = "feat/moved"
	86	+	require.NoError(t, mrs.Update(ctx, b))
	87	+	assert.Empty(t, bySource(t, mrs, n, "feat/b"))
	88	+	assert.Equal(t, []int64{b.Number}, bySource(t, mrs, n, "feat/moved"))
	89	+
	90	+	// Close and merge remove the key.
	91	+	c, err = mrs.Get(ctx, n, c.Number)
	92	+	require.NoError(t, err)
	93	+	c.State = domain.MRStateClosed
	94	+	require.NoError(t, mrs.Update(ctx, c))
	95	+	assert.Equal(t, []int64{a.Number}, bySource(t, mrs, n, "feat/a"))
	96	+	a, err = mrs.Get(ctx, n, a.Number)
	97	+	require.NoError(t, err)
	98	+	a.State, a.MergeMode = domain.MRStateMerged, domain.MergeModeFastForward
	99	+	require.NoError(t, mrs.Update(ctx, a))
	100	+	assert.Empty(t, bySource(t, mrs, n, "feat/a"))
	101	+	assert.Equal(t, []int64{b.Number}, openNumbers(t, mrs, n))
	102	+
	103	+	// Several branches in one lookup, each scanned on its own.
	104	+	assert.Equal(t, []int64{b.Number}, bySource(t, mrs, n, "feat/a", "feat/moved", "nothing"))
	105	+
	106	+	// Another repository has its own.
	107	+	other := name(t, "other")
	108	+	require.NoError(t, s.Create(ctx, domain.Repo{Name: other, CreatedAt: created, Visibility: domain.VisibilityPublic}))
	109	+	_, err = mrs.Create(ctx, mrOn(t, other, "feat/moved"))
	110	+	require.NoError(t, err)
	111	+	assert.Equal(t, []int64{b.Number}, bySource(t, mrs, n, "feat/moved"))
	112	+	}
	113	+
	114	+	// The target index follows the same writes as the source one: create, comment,
	115	+	// target change, close and merge.
	116	+	func TestOpenMergeRequestTargetIndexFollowsTheWrites(t *testing.T) {
	117	+	ctx := context.Background()
	118	+	s, mrs, n := repoWithMRs(t)
	119	+
	120	+	a, err := mrs.Create(ctx, mrFromTo(t, n, "feat/a", "main"))
	121	+	require.NoError(t, err)
	122	+	b, err := mrs.Create(ctx, mrFromTo(t, n, "feat/b", "release/1"))
	123	+	require.NoError(t, err)
	124	+	c, err := mrs.Create(ctx, mrFromTo(t, n, "feat/c", "main"))
	125	+	require.NoError(t, err)
	126	+	assert.Equal(t, []int64{a.Number, c.Number}, byTarget(t, mrs, n, "main"))
	127	+	assert.Equal(t, []int64{b.Number}, byTarget(t, mrs, n, "release/1"))
	128	+	assert.Empty(t, byTarget(t, mrs, n, "release"), "a prefix of a branch is not the branch")
	129	+	assert.Empty(t, byTarget(t, mrs, n, "feat/a"), "a source is not a target")
	130	+	assert.Equal(t, []int64{a.Number}, bySource(t, mrs, n, "feat/a"), "and the other way round")
	131	+
	132	+	text, err := domain.ParsePlainText("hi")
	133	+	require.NoError(t, err)
	134	+	a.CommentSeq = 1
	135	+	require.NoError(t, mrs.SaveComment(ctx, a, domain.Comment{ID: 1, Author: "x", Text: text, CreatedAt: created}))
	136	+	assert.Equal(t, []int64{a.Number, c.Number}, byTarget(t, mrs, n, "main"), "a comment keeps the index")
	137	+
	138	+	b, err = mrs.Get(ctx, n, b.Number)
	139	+	require.NoError(t, err)
	140	+	b.Target = "main"
	141	+	require.NoError(t, mrs.Update(ctx, b))
	142	+	assert.Empty(t, byTarget(t, mrs, n, "release/1"), "a changed target moves the key")
	143	+	assert.Equal(t, []int64{a.Number, b.Number, c.Number}, byTarget(t, mrs, n, "main"))
	144	+
	145	+	c, err = mrs.Get(ctx, n, c.Number)
	146	+	require.NoError(t, err)
	147	+	c.State = domain.MRStateClosed
	148	+	require.NoError(t, mrs.Update(ctx, c))
	149	+	a, err = mrs.Get(ctx, n, a.Number)
	150	+	require.NoError(t, err)
	151	+	a.State, a.MergeMode = domain.MRStateMerged, domain.MergeModeFastForward
	152	+	require.NoError(t, mrs.Update(ctx, a))
	153	+	assert.Equal(t, []int64{b.Number}, byTarget(t, mrs, n, "main"), "close and merge remove the key")
	154	+
	155	+	// The keys of a deleted repository go with it, both indexes.
	156	+	require.NoError(t, s.Delete(ctx, n))
	157	+	assert.Empty(t, byTarget(t, mrs, n, "main"))
	158	+	assert.Empty(t, bySource(t, mrs, n, "feat/b"))
	159	+	}
	160	+
	161	+	// The lookup stops at limit+1 keys, whatever the number of open requests on the
	162	+	// branch, and says it was cut without counting the rest.
	163	+	func TestOpenLookupStopsAtTheLimit(t *testing.T) {
	164	+	ctx := context.Background()
	165	+	_, mrs, n := repoWithMRs(t)
	166	+	for range 40 {
	167	+	_, err := mrs.Create(ctx, mrFromTo(t, n, "busy", "main"))
	168	+	require.NoError(t, err)
	169	+	}
	170	+	_, err := mrs.Create(ctx, mrFromTo(t, n, "calm", "main"))
	171	+	require.NoError(t, err)
	172	+
	173	+	got, truncated, err := mrs.OpenBySource(ctx, n, []string{"busy"}, 5)
	174	+	require.NoError(t, err)
	175	+	assert.True(t, truncated)
	176	+	assert.Equal(t, []int64{1, 2, 3, 4, 5}, got, "the first limit numbers")
	177	+
	178	+	got, truncated, err = mrs.OpenByTarget(ctx, n, []string{"main"}, 7)
	179	+	require.NoError(t, err)
	180	+	assert.True(t, truncated)
	181	+	assert.Len(t, got, 7)
	182	+
	183	+	// Exactly limit keys is not truncated: only a limit+1th key is.
	184	+	got, truncated, err = mrs.OpenBySource(ctx, n, []string{"calm"}, 1)
	185	+	require.NoError(t, err)
	186	+	assert.False(t, truncated)
	187	+	assert.Len(t, got, 1)
	188	+	got, truncated, err = mrs.OpenBySource(ctx, n, []string{"busy"}, 40)
	189	+	require.NoError(t, err)
	190	+	assert.False(t, truncated)
	191	+	assert.Len(t, got, 40)
	192	+
	193	+	// The limit spans the branches of one lookup.
	194	+	got, truncated, err = mrs.OpenBySource(ctx, n, []string{"calm", "busy"}, 3)
	195	+	require.NoError(t, err)
	196	+	assert.True(t, truncated)
	197	+	assert.Len(t, got, 3)
	198	+	}
	199	+
	200	+	// A cancelled context stops the lookup: nothing is scanned, and the caller
	201	+	// sees the error of the context and that the result is partial.
	202	+	func TestOpenLookupStopsOnCancel(t *testing.T) {
	203	+	_, mrs, n := repoWithMRs(t)
	204	+	for range 10 {
	205	+	_, err := mrs.Create(context.Background(), mrFromTo(t, n, "busy", "main"))
	206	+	require.NoError(t, err)
	207	+	}
	208	+	ctx, cancel := context.WithCancel(context.Background())
	209	+	cancel()
	210	+	got, truncated, err := mrs.OpenBySource(ctx, n, []string{"busy"}, 0)
	211	+	assert.ErrorIs(t, err, context.Canceled)
	212	+	assert.True(t, truncated)
	213	+	assert.Empty(t, got)
	214	+	_, truncated, err = mrs.OpenByTarget(ctx, n, []string{"main"}, 5)
	215	+	assert.ErrorIs(t, err, context.Canceled)
	216	+	assert.True(t, truncated)
	217	+	}
	218	+
	219	+	// A branch name with the characters of a key separator stays in its own prefix.
	220	+	func TestOpenMergeRequestIndexEscapesBranchNames(t *testing.T) {
	221	+	ctx := context.Background()
	222	+	_, mrs, n := repoWithMRs(t)
	223	+	x, err := mrs.Create(ctx, mrOn(t, n, "a/b"))
	224	+	require.NoError(t, err)
	225	+	y, err := mrs.Create(ctx, mrOn(t, n, "a%2Fb"))
	226	+	require.NoError(t, err)
	227	+	assert.Equal(t, []int64{x.Number}, bySource(t, mrs, n, "a/b"))
	228	+	assert.Equal(t, []int64{y.Number}, bySource(t, mrs, n, "a%2Fb"))
	229	+	}
	230	+
	231	+	// Many historical merge requests are not read by a lookup of a push: the index
	232	+	// holds the keys of the open ones, and only those are fetched.
	233	+	func TestOpenBySourceDoesNotReadHistoricalMergeRequests(t *testing.T) {
	234	+	ctx := context.Background()
	235	+	_, mrs, n := repoWithMRs(t)
	236	+	for range 200 {
	237	+	mr, err := mrs.Create(ctx, mrOn(t, n, "old/closed"))
	238	+	require.NoError(t, err)
	239	+	mr.State = domain.MRStateClosed
	240	+	require.NoError(t, mrs.Update(ctx, mr))
	241	+	}
	242	+	open, err := mrs.Create(ctx, mrOn(t, n, "feat/live"))
	243	+	require.NoError(t, err)
	244	+	assert.Empty(t, bySource(t, mrs, n, "old/closed"), "none of the 200 closed ones is indexed")
	245	+	assert.Equal(t, []int64{open.Number}, bySource(t, mrs, n, "feat/live"))
	246	+	assert.Equal(t, []int64{open.Number}, openNumbers(t, mrs, n))
	247	+	}
	248	+
	249	+	// Data that predates the index (a schema 2 database, or one an older build wrote
	250	+	// to) is indexed once at open, and the index is not built twice.
	251	+	func TestOpenIndexIsBackfilledOnce(t *testing.T) {
	252	+	ctx := context.Background()
	253	+	dir := t.TempDir()
	254	+	putRaw(t, dir, "meta/schema", mustProto(t, &storagepb.SchemaVersion{Version: 2}))
	255	+	putRaw(t, dir, "repo/old", mustProto(t, &storagepb.Repo{Name: "old", Visibility: storagepb.Visibility_VISIBILITY_PUBLIC}))
	256	+	mrRaw := func(number int64, source string, state storagepb.MergeRequestState) []byte {
	257	+	return mustProto(t, &storagepb.MergeRequest{
	258	+	Number: number, Title: "t", SourceBranch: source, TargetBranch: "main", State: state, Author: "a",
	259	+	})
	260	+	}
	261	+	putRaw(t, dir, "mr/old/0000000001", mrRaw(1, "feat/one", storagepb.MergeRequestState_MERGE_REQUEST_STATE_OPEN))
	262	+	putRaw(t, dir, "mr/old/0000000002", mrRaw(2, "feat/one", storagepb.MergeRequestState_MERGE_REQUEST_STATE_CLOSED))
	263	+	putRaw(t, dir, "mr/old/0000000003", mrRaw(3, "feat/two", storagepb.MergeRequestState_MERGE_REQUEST_STATE_MERGED))
	264	+	putRaw(t, dir, "mr/old/0000000004", mrRaw(4, "feat/two", storagepb.MergeRequestState_MERGE_REQUEST_STATE_OPEN))
	265	+	putRaw(t, dir, "mrseq/old", mustProto(t, &storagepb.Sequence{Last: 4}))
	266	+
	267	+	s := openStore(t, dir)
	268	+	mrs := s.MergeRequests()
	269	+	old := name(t, "old")
	270	+	assert.Equal(t, []int64{1}, bySource(t, mrs, old, "feat/one"))
	271	+	assert.Equal(t, []int64{4}, bySource(t, mrs, old, "feat/two"))
	272	+	assert.Equal(t, []int64{4, 1}, openNumbers(t, mrs, old))
	273	+	assert.Equal(t, []int64{1, 4}, byTarget(t, mrs, old, "main"), "the target index is built by the same pass")
	274	+	require.NoError(t, s.Close(ctx))
	275	+
	276	+	// Opened again: the marker is there, nothing is built, and the index kept
	277	+	// following the writes in between.
	278	+	s2, err := kvstore.Open(ctx, dir)
	279	+	require.NoError(t, err)
	280	+	t.Cleanup(func() { _ = s2.Close(ctx) })
	281	+	m2 := s2.MergeRequests()
	282	+	five, err := m2.Create(ctx, mrOn(t, old, "feat/one"))
	283	+	require.NoError(t, err)
	284	+	assert.Equal(t, int64(5), five.Number)
	285	+	assert.Equal(t, []int64{1, 5}, bySource(t, m2, old, "feat/one"))
	286	+	}

M internal/adapters/kvstore/store.go +10 -1


@@ -23,7 +23,11 @@ import (
23	23		// (ErrSchemaTooNew) instead. Version 3 adds the records of CI-3 that an older
24	24		// build would misread: the call and scoped jobs of reusable workflows (it would
25	25		// run the jobs of a called workflow as jobs of the caller's file, and plan the
26		-	// call itself).
	26	+	// call itself), the head commit of the runs of merge requests
	27	+	// (CIRun.head_sha: it would attach their checks to the merge commit, where
	28	+	// nothing looks for them), and the index of the open merge requests by source
	29	+	// branch (mropen/: an older build would not keep it up to date; the upgrade
	30	+	// builds it once, see BuildOpenIndex).
27	31		const SchemaVersion = 3
28	32
29	33		const schemaKey = "meta/schema"

@@ -51,6 +55,11 @@ func Open(ctx context.Context, dir string) (*Store, error) {
51	55		_ = db.Close(context.WithoutCancel(ctx))
52	56		return nil, err
53	57		}
	58	+	// Part of the schema 3 upgrade: the index of the open merge requests.
	59	+	if err := s.BuildOpenIndex(ctx); err != nil {
	60	+	_ = db.Close(context.WithoutCancel(ctx))
	61	+	return nil, err
	62	+	}
54	63		return s, nil
55	64		}
56	65

M internal/adapters/kvstore/store_test.go +34 -0


@@ -223,6 +223,40 @@ func TestSchema(t *testing.T) {
223	223		assert.Equal(t, uint32(kvstore.SchemaVersion), readSchema(t, dir))
224	224		})
225	225
	226	+	t.Run("version 2 is upgraded to 3, its runs kept", func(t *testing.T) {
	227	+	dir := t.TempDir()
	228	+	putRaw(t, dir, "meta/schema", mustProto(t, &storagepb.SchemaVersion{Version: 2}))
	229	+	putRaw(t, dir, "repo/old", mustProto(t, &storagepb.Repo{Name: "old", Visibility: storagepb.Visibility_VISIBILITY_PUBLIC}))
	230	+	// A run written by a schema 2 build (no head sha), and a retained pull_request
	231	+	// run that has one: both must read back after the upgrade.
	232	+	putRaw(t, dir, "run/old/00000000000000000001", mustProto(t, &storagepb.CIRun{
	233	+	Repo: "old", Id: 1, Number: 1, Attempt: 1, Event: "push", Ref: "refs/heads/main", Sha: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
	234	+	Status: storagepb.CIStatus_CI_STATUS_COMPLETED,
	235	+	}))
	236	+	putRaw(t, dir, "run/old/00000000000000000002", mustProto(t, &storagepb.CIRun{
	237	+	Repo: "old", Id: 2, Number: 2, Attempt: 1, Event: "pull_request", Ref: "refs/pull/1/merge",
	238	+	Sha: "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", HeadSha: "cccccccccccccccccccccccccccccccccccccccc",
	239	+	Status: storagepb.CIStatus_CI_STATUS_COMPLETED,
	240	+	}))
	241	+	s, err := kvstore.Open(ctx, dir)
	242	+	require.NoError(t, err)
	243	+	got, err := s.Get(ctx, name(t, "old"))
	244	+	require.NoError(t, err)
	245	+	assert.Equal(t, "old", got.Name.String())
	246	+	old, err := s.CI().GetRun(ctx, name(t, "old"), 1)
	247	+	require.NoError(t, err)
	248	+	assert.Equal(t, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", old.Sha)
	249	+	assert.Empty(t, old.HeadSha)
	250	+	assert.Equal(t, old.Sha, old.CheckSha(), "a run without a head sha keeps its checks on its commit")
	251	+	pr, err := s.CI().GetRun(ctx, name(t, "old"), 2)
	252	+	require.NoError(t, err)
	253	+	assert.Equal(t, "pull_request", pr.Event)
	254	+	assert.Equal(t, "cccccccccccccccccccccccccccccccccccccccc", pr.HeadSha)
	255	+	assert.Equal(t, pr.HeadSha, pr.CheckSha(), "the checks of a pull_request run are on the head")
	256	+	require.NoError(t, s.Close(ctx))
	257	+	assert.Equal(t, uint32(3), readSchema(t, dir))
	258	+	})
	259	+
226	260		t.Run("version 2 is upgraded to 3, its data kept", func(t *testing.T) {
227	261		dir := t.TempDir()
228	262		putRaw(t, dir, "meta/schema", mustProto(t, &storagepb.SchemaVersion{Version: 2}))

M internal/adapters/web/actions_pages.go +14 -0


@@ -234,6 +234,10 @@ func (s *Server) runRow(repo string, run domain.CIRun, now time.Time) RunRow {
234	234		Duration: duration(run.StartedAt, run.CompletedAt, now), Number: "#" + strconv.FormatInt(run.Number, 10), Attempt: run.Attempt,
235	235		}
236	236		row.RefHref = repoHref(repo, "tree", strings.TrimPrefix(strings.TrimPrefix(run.Ref, "refs/heads/"), "refs/tags/"))
	237	+	if n, ok := pullNumber(run.Ref); ok {
	238	+	// A pull_request run is for refs/pull/<n>/merge: it shows the merge request.
	239	+	row.Ref, row.RefHref = "!"+strconv.FormatInt(n, 10), mrHref(repo, n)
	240	+	}
237	241		row.When, row.WhenAt = whenOf(now, run.CreatedAt)
238	242		return row
239	243		}

@@ -491,3 +495,13 @@ func runnerText(j JobRow) string {
491	495		}
492	496
493	497		func joinWords(ws []string) string { return strings.Join(ws, ", ") }
	498	+
	499	+	// pullNumber reads the merge request number of refs/pull/<n>/merge or /head.
	500	+	func pullNumber(ref string) (int64, bool) {
	501	+	if !domain.IsPullRef(ref) {
	502	+	return 0, false
	503	+	}
	504	+	num, _, _ := strings.Cut(strings.TrimPrefix(ref, domain.PullRefPrefix), "/")
	505	+	n, err := strconv.ParseInt(num, 10, 64)
	506	+	return n, err == nil
	507	+	}

A internal/app/ci_pull_race_test.go +620 -0

@@ -0,0 +1,620 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package app
	4	+
	5	+	import (
	6	+	"context"
	7	+	"net/http"
	8	+	"strings"
	9	+	"sync"
	10	+	"sync/atomic"
	11	+	"testing"
	12	+	"time"
	13	+
	14	+	"connectrpc.com/connect"
	15	+	"github.com/stretchr/testify/assert"
	16	+	"github.com/stretchr/testify/require"
	17	+
	18	+	pb "bnema.dev/ermite-api/gen/ermite"
	19	+	"bnema.dev/ermite/internal/domain"
	20	+	"bnema.dev/ermite/internal/ports"
	21	+	"bnema.dev/ermite/internal/usecase"
	22	+	)
	23	+
	24	+	// pullHook is CIConfig.PullStage for the tests that interleave other changes
	25	+	// with the workers of the merge requests: armed for one stage, it runs fn (or
	26	+	// waits to be released) the next time a worker reaches it, once.
	27	+	type pullHook struct {
	28	+	mu sync.Mutex
	29	+	stage string
	30	+	fn func()
	31	+	reached chan struct{}
	32	+	release chan struct{}
	33	+	}
	34	+
	35	+	func (h *pullHook) call(stage string) {
	36	+	h.mu.Lock()
	37	+	if stage != h.stage {
	38	+	h.mu.Unlock()
	39	+	return
	40	+	}
	41	+	h.stage = ""
	42	+	fn, reached, release := h.fn, h.reached, h.release
	43	+	h.mu.Unlock()
	44	+	if fn != nil {
	45	+	fn()
	46	+	return
	47	+	}
	48	+	close(reached)
	49	+	<-release // held until the test releases it; its cleanup always does
	50	+	}
	51	+
	52	+	// block arms the hook: the worker stops at stage and stays there until release
	53	+	// is called (the cleanup of the test calls it too, so a failed test never leaves
	54	+	// a worker stuck). reached is closed when the worker has stopped.
	55	+	func (h pullHook) block(t testing.T, stage string) (reached <-chan struct{}, release func()) {
	56	+	t.Helper()
	57	+	r, rel := make(chan struct{}), make(chan struct{})
	58	+	h.mu.Lock()
	59	+	h.stage, h.fn, h.reached, h.release = stage, nil, r, rel
	60	+	h.mu.Unlock()
	61	+	var once sync.Once
	62	+	release = func() { once.Do(func() { close(rel) }) }
	63	+	t.Cleanup(release)
	64	+	return r, release
	65	+	}
	66	+
	67	+	// await waits for the worker to reach a blocked stage, failing the test when it
	68	+	// does not within the bound.
	69	+	func await(t *testing.T, reached <-chan struct{}, what string) {
	70	+	t.Helper()
	71	+	select {
	72	+	case <-reached:
	73	+	case <-time.After(15 * time.Second):
	74	+	t.Fatalf("the worker never reached %s", what)
	75	+	}
	76	+	}
	77	+
	78	+	// run arms the hook with a function the worker runs at stage.
	79	+	func (h *pullHook) run(stage string, fn func()) {
	80	+	h.mu.Lock()
	81	+	h.stage, h.fn = stage, fn
	82	+	h.mu.Unlock()
	83	+	}
	84	+
	85	+	func newHookedHarness(t testing.T) (ciHarness, *pullHook) {
	86	+	t.Helper()
	87	+	h := &pullHook{}
	88	+	return newCIHarness(t, func(cfg *usecase.CIConfig) { cfg.PullStage = h.call }), h
	89	+	}
	90	+
	91	+	func (c *ciHarness) parents(sha string) string {
	92	+	c.t.Helper()
	93	+	return strings.TrimSpace(c.git2("log", "-1", "--format=%P", sha))
	94	+	}
	95	+
	96	+	// S1: an event runs for the commits it was made for, with the trust of whoever
	97	+	// caused it, however the branch moves before the event is handled.
	98	+	func TestPullEventKeepsItsCommitsAndTrustWhenALimitedTokenPushesNext(t *testing.T) {
	99	+	c, hook := newHookedHarness(t)
	100	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	101	+	main := strings.TrimSpace(c.push("main", pullWF("", ""), ""))
	102	+	c.mustGit(c.wt, "checkout", "-q", "-b", "agent/x")
	103	+	x1 := c.push("agent/x", map[string]string{"x1.txt": "1\n"}, "")
	104	+	secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), []string{"agent/"}, "proj")
	105	+
	106	+	reached, release := hook.block(t, "event")
	107	+	n := c.openMR(c.root, "agent/x") // the owner opens it: trusted
	108	+	await(t, reached, "the opening event")
	109	+	// While the owner's event waits, the agent pushes a new commit.
	110	+	c.writeFile(c.wt, "x2.txt", []byte("2\n"))
	111	+	x2 := strings.TrimSpace(c.mustGit(c.wt, "rev-parse", "HEAD"))
	112	+	c.mustGit(c.wt, "push", "-q", c.urlAuth(secret, "/proj.git"), "agent/x")
	113	+	release()
	114	+
	115	+	c.waitFor("both runs", func() bool { return len(c.runs()) == 2 })
	116	+	runs := c.runs() // newest first: the synchronize, then the opened
	117	+	opened, sync := runs[1], runs[0]
	118	+	assert.True(t, opened.GetTrusted(), "the owner's event keeps the owner's trust")
	119	+	assert.Equal(t, main+" "+x1, c.parents(opened.GetSha()), "and its own commits, not the agent's later one")
	120	+	assert.False(t, sync.GetTrusted(), "the agent's push is the agent's")
	121	+	assert.Equal(t, main+" "+x2, c.parents(sync.GetSha()))
	122	+	assert.Equal(t, "claude", sync.GetActor())
	123	+	// The refs are the latest ones.
	124	+	c.waitFor("the refs follow the latest push", func() bool {
	125	+	return c.ref(domain.PullHeadRef(n)) == x2 && strings.HasSuffix(c.parents(c.ref(domain.PullMergeRef(n))), " "+x2)
	126	+	})
	127	+	_ = runner
	128	+	}
	129	+
	130	+	// S1: a push made with a job token starts no pull_request run, not even through
	131	+	// the event of an earlier opening.
	132	+	func TestPullEventIsNotRunForAJobTokenPush(t *testing.T) {
	133	+	c, hook := newHookedHarness(t)
	134	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	135	+	files := pullWF("", "")
	136	+	files[".github/workflows/push.yml"] = "on:\n push:\n branches: [feat]\npermissions:\n contents: write\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo push\n"
	137	+	main := strings.TrimSpace(c.push("main", files, ""))
	138	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	139	+	f1 := c.push("feat", map[string]string{"f1.txt": "1\n"}, "")
	140	+	pushJob := runner.await() // the push workflow's job holds a token with contents: write
	141	+
	142	+	reached, release := hook.block(t, "event")
	143	+	c.openMR(c.root, "feat")
	144	+	await(t, reached, "the opening event")
	145	+	c.writeFile(c.wt, "f2.txt", []byte("2\n"))
	146	+	out, err := c.tokenPush(c.wt, pushJob.GetGithubToken(), "HEAD:refs/heads/feat")
	147	+	require.NoError(t, err, out)
	148	+	release()
	149	+
	150	+	prJob := runner.await()
	151	+	runner.runJob(prJob, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	152	+	runner.runJob(pushJob, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	153	+	var pulls []*pb.Run
	154	+	for _, r := range c.runs() {
	155	+	if r.GetEvent() == "pull_request" {
	156	+	pulls = append(pulls, r)
	157	+	}
	158	+	}
	159	+	require.Len(t, pulls, 1, "the job's push started no run")
	160	+	assert.Equal(t, main+" "+f1, c.parents(pulls[0].GetSha()))
	161	+	time.Sleep(200 * time.Millisecond)
	162	+	for _, r := range c.runs() {
	163	+	assert.NotEqual(t, "synchronize", r.GetTitle())
	164	+	}
	165	+	assert.Len(t, c.runs(), 2, "the push run and the opened run")
	166	+	}
	167	+
	168	+	// S4: a request closed while its merge is being computed gets neither the refs
	169	+	// nor the run of the opening, and the closed run does not use that merge.
	170	+	func TestPullCloseDuringComputeDiscardsTheOpening(t *testing.T) {
	171	+	c, hook := newHookedHarness(t)
	172	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	173	+	c.push("main", pullWF(" types: [opened, closed]", ""), "")
	174	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	175	+	f1 := c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	176	+
	177	+	hook.run("computed", func() {
	178	+	_, err := c.root.mr.CloseMergeRequest(c.ctx, connect.NewRequest(&pb.CloseMergeRequestRequest{Repo: "proj", Number: 1}))
	179	+	assert.NoError(t, err)
	180	+	})
	181	+	n := c.openMR(c.root, "feat")
	182	+	job := runner.await()
	183	+	runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	184	+	gh := c.ghContext(job)
	185	+	assert.Equal(t, "closed", gh["event"].(map[string]any)["action"])
	186	+	assert.Equal(t, f1, gh["sha"], "no test merge was published: the closed run uses the head")
	187	+	assert.Empty(t, c.ref(domain.PullMergeRef(n)), "the merge ref of a closed request is never published")
	188	+	assert.Empty(t, c.ref(domain.PullHeadRef(n)))
	189	+	time.Sleep(200 * time.Millisecond)
	190	+	assert.Len(t, c.runs(), 1, "no run for the opening")
	191	+	}
	192	+
	193	+	// E1: a slow refresh of a merge request's refs does not delay the push
	194	+	// workflows.
	195	+	func TestPullSlowRefreshDoesNotDelayPushRuns(t *testing.T) {
	196	+	c, hook := newHookedHarness(t)
	197	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	198	+	both := "name: both\non:\n push:\n branches: [main]\n pull_request:\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"
	199	+	c.push("main", map[string]string{".github/workflows/both.yml": both}, "")
	200	+	runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	201	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	202	+	c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	203	+	n := c.openMR(c.root, "feat")
	204	+	runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	205	+	c.waitFor("the merge ref", func() bool { return c.ref(domain.PullMergeRef(n)) != "" })
	206	+
	207	+	reached, release := hook.block(t, "refresh")
	208	+	c.mustGit(c.wt, "checkout", "-q", "main")
	209	+	m2 := c.push("main", map[string]string{"m2.txt": "m\n"}, "") // the target moves
	210	+	await(t, reached, "the refresh") // the refresh worker is stuck
	211	+	c.waitFor("the push run of main, while the refresh waits", func() bool {
	212	+	for _, r := range c.runs() {
	213	+	if r.GetEvent() == "push" && r.GetSha() == m2 {
	214	+	return true
	215	+	}
	216	+	}
	217	+	return false
	218	+	})
	219	+	release()
	220	+	c.waitFor("the refresh then follows", func() bool {
	221	+	return strings.HasPrefix(c.parents(c.ref(domain.PullMergeRef(n))), m2+" ")
	222	+	})
	223	+	}
	224	+
	225	+	// pushRuns counts the push runs, without failing the test (it runs in the hook
	226	+	// of a worker).
	227	+	func (c *ciHarness) pushRuns() int {
	228	+	res, err := c.root.run.ListRuns(c.ctx, connect.NewRequest(&pb.ListRunsRequest{Repo: c.repo}))
	229	+	if err != nil {
	230	+	return -1
	231	+	}
	232	+	n := 0
	233	+	for _, r := range res.Msg.GetRuns() {
	234	+	if r.GetEvent() == "push" {
	235	+	n++
	236	+	}
	237	+	}
	238	+	return n
	239	+	}
	240	+
	241	+	// A push's workflows are queued before its merge requests are looked at: when
	242	+	// the discovery starts, the push run is already being created, so a slow
	243	+	// discovery never delays it.
	244	+	func TestPullDiscoveryDoesNotDelayPushRuns(t *testing.T) {
	245	+	c, hook := newHookedHarness(t)
	246	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	247	+	both := "name: both\non:\n push:\n branches: [main]\n pull_request:\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo hi\n"
	248	+	c.push("main", map[string]string{".github/workflows/both.yml": both}, "")
	249	+	runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	250	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	251	+	c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	252	+	n := c.openMR(c.root, "feat")
	253	+	runner.runJob(runner.await(), pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	254	+	c.waitFor("the merge ref", func() bool { return c.ref(domain.PullMergeRef(n)) != "" })
	255	+	c.waitFor("the workers are idle", func() bool { return len(c.runs()) == 2 })
	256	+
	257	+	before := c.pushRuns()
	258	+	var sawRun atomic.Bool
	259	+	hook.run("discover", func() {
	260	+	// Inside the discovery of the push: its push run must come into being
	261	+	// while it goes on (the push handler waits here, up to the bound).
	262	+	deadline := time.Now().Add(10 * time.Second)
	263	+	for time.Now().Before(deadline) {
	264	+	if c.pushRuns() > before {
	265	+	sawRun.Store(true)
	266	+	return
	267	+	}
	268	+	time.Sleep(10 * time.Millisecond)
	269	+	}
	270	+	})
	271	+	c.mustGit(c.wt, "checkout", "-q", "main")
	272	+	m2 := c.push("main", map[string]string{"m2.txt": "m\n"}, "")
	273	+	assert.True(t, sawRun.Load(), "the push run was created while the discovery ran")
	274	+	c.waitFor("the merge ref follows the target", func() bool {
	275	+	return strings.HasPrefix(c.parents(c.ref(domain.PullMergeRef(n))), m2+" ")
	276	+	})
	277	+	}
	278	+
	279	+	// Past the bound of the refresh queue the refs are repaired by a rescan, and
	280	+	// the activities are never lost: every synchronize still runs.
	281	+	func TestPullSaturatedRefreshQueueKeepsEventsAndRepairsRefs(t *testing.T) {
	282	+	h := &pullHook{}
	283	+	c := newCIHarness(t, func(cfg *usecase.CIConfig) { cfg.PullStage = h.call; cfg.PullRefreshes = 1 })
	284	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	285	+	_ = runner
	286	+	c.push("main", pullWF("", ""), "")
	287	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	288	+	c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	289	+	n1 := c.openMR(c.root, "feat")
	290	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat2", "main")
	291	+	c.push("feat2", map[string]string{"g.txt": "g\n"}, "")
	292	+	n2 := c.openMR(c.root, "feat2")
	293	+	c.waitFor("both opened runs and both merge refs", func() bool {
	294	+	return len(c.runs()) == 2 && c.ref(domain.PullMergeRef(n1)) != "" && c.ref(domain.PullMergeRef(n2)) != ""
	295	+	})
	296	+
	297	+	reached, release := h.block(t, "refresh")
	298	+	c.mustGit(c.wt, "checkout", "-q", "feat")
	299	+	c.push("feat", map[string]string{"f2.txt": "2\n"}, "") // its refresh is taken by the worker, which stops
	300	+	await(t, reached, "the first refresh")
	301	+	c.mustGit(c.wt, "checkout", "-q", "feat2")
	302	+	g2 := c.push("feat2", map[string]string{"g2.txt": "2\n"}, "") // waits: the queue is at its bound of one
	303	+	c.mustGit(c.wt, "checkout", "-q", "feat")
	304	+	f3 := c.push("feat", map[string]string{"f3.txt": "3\n"}, "") // over the bound: the repository is marked dirty
	305	+	release()
	306	+
	307	+	c.waitFor("the three synchronize runs, none lost to the cap", func() bool { return len(c.runs()) == 5 })
	308	+	c.waitFor("the refs end at the latest heads", func() bool {
	309	+	return c.ref(domain.PullHeadRef(n1)) == f3 && strings.HasSuffix(c.parents(c.ref(domain.PullMergeRef(n1))), " "+f3) &&
	310	+	c.ref(domain.PullHeadRef(n2)) == g2 && strings.HasSuffix(c.parents(c.ref(domain.PullMergeRef(n2))), " "+g2)
	311	+	})
	312	+	}
	313	+
	314	+	// GO-010: the target tip an event is tested against is read when the push
	315	+	// happens: a target that moves before the event is handled changes nothing for
	316	+	// the run, which keeps the merge of the commits it was made for.
	317	+	func TestPullEventKeepsTheTargetTipOfItsPush(t *testing.T) {
	318	+	c, hook := newHookedHarness(t)
	319	+	c.registerRunner("alpha", "ubuntu-latest")
	320	+	m1 := c.push("main", pullWF("", ""), "")
	321	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	322	+	c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	323	+	c.openMR(c.root, "feat")
	324	+	c.waitFor("the opened run", func() bool { return len(c.runs()) == 1 })
	325	+
	326	+	reached, release := hook.block(t, "event")
	327	+	f2 := c.push("feat", map[string]string{"f2.txt": "2\n"}, "") // the synchronize event is taken and stops
	328	+	await(t, reached, "the synchronize event")
	329	+	c.mustGit(c.wt, "checkout", "-q", "main")
	330	+	m2 := c.push("main", map[string]string{"m2.txt": "m\n"}, "") // the target moves meanwhile
	331	+	release()
	332	+
	333	+	c.waitFor("the synchronize run", func() bool { return len(c.runs()) == 2 })
	334	+	assert.Equal(t, m1+" "+f2, c.parents(c.runs()[0].GetSha()), "the run merges the target as it was at the push, not as it is now")
	335	+	assert.NotEqual(t, m2, strings.Fields(c.parents(c.runs()[0].GetSha()))[0])
	336	+	c.waitFor("the merge ref follows the target", func() bool {
	337	+	return c.parents(c.ref(domain.PullMergeRef(1))) == m2+" "+f2
	338	+	})
	339	+	}
	340	+
	341	+	// countingPull counts the test merges the CI asks of Git.
	342	+	type countingPull struct {
	343	+	ports.PullGit
	344	+	merges atomic.Int32
	345	+	}
	346	+
	347	+	func (p *countingPull) TestMerge(ctx context.Context, repo domain.RepoName, base, head domain.ObjectID) (domain.ObjectID, bool, error) {
	348	+	p.merges.Add(1)
	349	+	return p.PullGit.TestMerge(ctx, repo, base, head)
	350	+	}
	351	+
	352	+	// GO-009: conflicting merge requests are not tried again by a rescan until a
	353	+	// branch moves, even when there are more of them than the memory holds, and the
	354	+	// rescan neither loops nor marks its repository dirty again.
	355	+	func TestPullRescanDoesNotRetryConflicts(t *testing.T) {
	356	+	p := &countingPull{}
	357	+	var rescans atomic.Int32
	358	+	c := newCIHarness(t, func(cfg *usecase.CIConfig) {
	359	+	p.PullGit = cfg.Pull
	360	+	cfg.Pull = p
	361	+	cfg.PullAttempts = 2
	362	+	cfg.PullStage = func(stage string) {
	363	+	if stage == "rescan" {
	364	+	rescans.Add(1)
	365	+	}
	366	+	}
	367	+	})
	368	+	c.push("main", map[string]string{"a.txt": "base\n"}, "")
	369	+	const conflicting = 4
	370	+	for i := range conflicting {
	371	+	b := "c" + string(rune('a'+i))
	372	+	c.mustGit(c.wt, "checkout", "-q", "-b", b, "main")
	373	+	c.push(b, map[string]string{"a.txt": b + "\n"}, "")
	374	+	}
	375	+	c.mustGit(c.wt, "checkout", "-q", "main")
	376	+	c.push("main", map[string]string{"a.txt": "main\n"}, "") // every branch now conflicts with main
	377	+	for i := range conflicting {
	378	+	c.openMR(c.root, "c"+string(rune('a'+i)))
	379	+	}
	380	+	c.waitFor("every conflict was tried", func() bool { return p.merges.Load() >= conflicting })
	381	+	time.Sleep(300 * time.Millisecond)
	382	+	for i := range conflicting {
	383	+	assert.Empty(t, c.ref(domain.PullMergeRef(int64(i+1))), "a conflict has no merge ref")
	384	+	}
	385	+
	386	+	// A rescan (what a saturated queue or a restart leads to) tries again only
	387	+	// what the memory forgot: at most the number of merge requests, and it ends.
	388	+	before, scans := p.merges.Load(), rescans.Load()
	389	+	require.NoError(t, c.forge.CI.RescanPulls(c.ctx, c.name("proj")))
	390	+	assert.LessOrEqual(t, p.merges.Load()-before, int32(conflicting))
	391	+	time.Sleep(300 * time.Millisecond)
	392	+	after := p.merges.Load()
	393	+	assert.Equal(t, scans+1, rescans.Load(), "the rescan was not marked dirty again")
	394	+	assert.Equal(t, after, p.merges.Load(), "and nothing goes on trying")
	395	+	}
	396	+
	397	+	// readingMRs counts what the CI asks of the merge request store: the full
	398	+	// listings and the single reads of requests.
	399	+	type readingMRs struct {
	400	+	ports.MergeRequestStore
	401	+	lists, opens, gets atomic.Int32
	402	+	}
	403	+
	404	+	func (s *readingMRs) List(ctx context.Context, repo domain.RepoName) ([]domain.MergeRequest, error) {
	405	+	s.lists.Add(1)
	406	+	return s.MergeRequestStore.List(ctx, repo)
	407	+	}
	408	+
	409	+	func (s *readingMRs) ListOpen(ctx context.Context, repo domain.RepoName) ([]domain.MergeRequest, error) {
	410	+	s.opens.Add(1)
	411	+	return s.MergeRequestStore.ListOpen(ctx, repo)
	412	+	}
	413	+
	414	+	func (s *readingMRs) Get(ctx context.Context, repo domain.RepoName, n int64) (domain.MergeRequest, error) {
	415	+	s.gets.Add(1)
	416	+	return s.MergeRequestStore.Get(ctx, repo, n)
	417	+	}
	418	+
	419	+	// GO-011: the snapshot of a push looks up the open merge requests of the pushed
	420	+	// branches in the index and reads those only: the many closed ones of the
	421	+	// repository are not read, and the repository's merge requests are not listed.
	422	+	func TestPullPushDoesNotReadHistoricalMergeRequests(t *testing.T) {
	423	+	store := &readingMRs{}
	424	+	c := newCIHarness(t, func(cfg *usecase.CIConfig) {
	425	+	store.MergeRequestStore = cfg.MRs
	426	+	cfg.MRs = store
	427	+	})
	428	+	c.registerRunner("alpha", "ubuntu-latest")
	429	+	c.push("main", pullWF("", ""), "")
	430	+	const closed = 30
	431	+	for i := range closed {
	432	+	b := "old" + string(rune('a'+i%26)) + string(rune('a'+i/26))
	433	+	c.mustGit(c.wt, "checkout", "-q", "-b", b, "main")
	434	+	c.push(b, map[string]string{b + ".txt": "x\n"}, "")
	435	+	n := c.openMR(c.root, b)
	436	+	_, err := c.root.mr.CloseMergeRequest(c.ctx, connect.NewRequest(&pb.CloseMergeRequestRequest{Repo: "proj", Number: n}))
	437	+	require.NoError(t, err)
	438	+	}
	439	+	c.mustGit(c.wt, "checkout", "-q", "-b", "live", "main")
	440	+	c.push("live", map[string]string{"l.txt": "l\n"}, "")
	441	+	live := c.openMR(c.root, "live")
	442	+	runsOf := func() int {
	443	+	n := 0
	444	+	for _, r := range c.runs() {
	445	+	if r.GetRef() == domain.PullMergeRef(live) {
	446	+	n++
	447	+	}
	448	+	}
	449	+	return n
	450	+	}
	451	+	// (The closed requests may or may not have had their opened run: they were
	452	+	// closed at once.) The live one has its own.
	453	+	c.waitFor("the opened run of the live request", func() bool { return runsOf() == 1 })
	454	+	time.Sleep(300 * time.Millisecond)
	455	+
	456	+	lists, gets := store.lists.Load(), store.gets.Load()
	457	+	c.push("live", map[string]string{"l2.txt": "2\n"}, "")
	458	+	c.waitFor("the synchronize run of the live request", func() bool { return runsOf() == 2 })
	459	+	time.Sleep(300 * time.Millisecond)
	460	+	assert.Zero(t, store.lists.Load()-lists, "the merge requests of the repository are not listed for a push")
	461	+	// The synchronize snapshot reads the one open request on the pushed branch (and
	462	+	// the workers read it again to handle the event and refresh the refs): a
	463	+	// small number of reads, nothing like the thirty closed ones.
	464	+	assert.Less(t, store.gets.Load()-gets, int32(closed), "the closed requests are not read")
	465	+	}
	466	+
	467	+	// GO-011: past the bound of one push's snapshot the rest is skipped, an error
	468	+	// naming the repository and the number skipped is logged, and the refs of the
	469	+	// skipped requests are repaired by the rescan.
	470	+	func TestPullSnapshotBoundSkipsLogsAndMarksDirty(t *testing.T) {
	471	+	c := newCIHarness(t, func(cfg *usecase.CIConfig) { cfg.PullSnapshot = 2 })
	472	+	c.registerRunner("alpha", "ubuntu-latest")
	473	+	c.push("main", pullWF("", ""), "")
	474	+	c.mustGit(c.wt, "checkout", "-q", "-b", "shared")
	475	+	c.push("shared", map[string]string{"s.txt": "s\n"}, "")
	476	+	const requests = 5
	477	+	var numbers []int64
	478	+	for i := range requests {
	479	+	// Several open requests on the same source branch (distinct targets).
	480	+	t0 := "t" + string(rune('a'+i))
	481	+	c.mustGit(c.wt, "checkout", "-q", "-b", t0, "main")
	482	+	c.push(t0, map[string]string{t0 + ".txt": "t\n"}, "")
	483	+	res, err := c.root.mr.CreateMergeRequest(c.ctx, connect.NewRequest(&pb.CreateMergeRequestRequest{
	484	+	Repo: "proj", Title: "t" + t0, SourceBranch: "shared", TargetBranch: t0,
	485	+	}))
	486	+	require.NoError(t, err)
	487	+	numbers = append(numbers, res.Msg.GetMergeRequest().GetNumber())
	488	+	}
	489	+	c.waitFor("the opened runs", func() bool { return len(c.runs()) == requests })
	490	+
	491	+	c.mustGit(c.wt, "checkout", "-q", "shared")
	492	+	latest := c.push("shared", map[string]string{"s2.txt": "2\n"}, "")
	493	+
	494	+	c.waitFor("the error naming the repository and the skipped requests", func() bool {
	495	+	logs := c.logs.String()
	496	+	return strings.Contains(logs, "snapshot of a push is cut short") && strings.Contains(logs, "more than the bound") &&
	497	+	strings.Contains(logs, `"bound":2`) && strings.Contains(logs, `"repo":"proj"`) && !strings.Contains(logs, `"skipped"`)
	498	+	})
	499	+	c.waitFor("only the bounded part got a synchronize run", func() bool { return len(c.runs()) == requests+2 })
	500	+	time.Sleep(300 * time.Millisecond)
	501	+	assert.Len(t, c.runs(), requests+2, "the three others were skipped")
	502	+	// The refs of every request still end at the pushed head: the repository was
	503	+	// marked dirty and the rescan refreshed them.
	504	+	for _, n := range numbers {
	505	+	c.waitFor("the head ref of a skipped request is repaired", func() bool {
	506	+	return c.ref(domain.PullHeadRef(n)) == latest
	507	+	})
	508	+	}
	509	+	}
	510	+
	511	+	// GO-013: a push that touches no merge request (not a source, not a target)
	512	+	// queues no refresh and marks nothing for a rescan; a push of a target queues the
	513	+	// refresh of exactly the requests that target it.
	514	+	func TestPullUnrelatedPushQueuesNoRefreshAndMarksNothing(t *testing.T) {
	515	+	var refreshes, rescans atomic.Int32
	516	+	c := newCIHarness(t, func(cfg *usecase.CIConfig) {
	517	+	cfg.PullStage = func(stage string) {
	518	+	switch stage {
	519	+	case "refresh":
	520	+	refreshes.Add(1)
	521	+	case "rescan":
	522	+	rescans.Add(1)
	523	+	}
	524	+	}
	525	+	})
	526	+	c.registerRunner("alpha", "ubuntu-latest")
	527	+	c.push("main", pullWF("", ""), "")
	528	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	529	+	c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	530	+	n := c.openMR(c.root, "feat")
	531	+	c.waitFor("the merge ref", func() bool { return c.ref(domain.PullMergeRef(n)) != "" })
	532	+	time.Sleep(300 * time.Millisecond)
	533	+
	534	+	// Pushes of branches that are neither the source nor the target of any open
	535	+	// merge request: nothing is queued, nothing is marked.
	536	+	beforeRefresh, beforeRescan := refreshes.Load(), rescans.Load()
	537	+	for _, b := range []string{"unrelated", "other/topic", "another"} {
	538	+	c.mustGit(c.wt, "checkout", "-q", "-b", b, "main")
	539	+	c.push(b, map[string]string{b + ".txt": "x\n"}, "")
	540	+	}
	541	+	time.Sleep(500 * time.Millisecond)
	542	+	assert.Equal(t, beforeRefresh, refreshes.Load(), "no refresh for pushes that touch no merge request")
	543	+	assert.Equal(t, beforeRescan, rescans.Load(), "and no repository marked for a rescan")
	544	+
	545	+	// A push of the target queues the refresh of the request that targets it.
	546	+	c.mustGit(c.wt, "checkout", "-q", "main")
	547	+	m2 := c.push("main", map[string]string{"m2.txt": "m\n"}, "")
	548	+	c.waitFor("the merge ref follows the target", func() bool {
	549	+	return strings.HasPrefix(c.parents(c.ref(domain.PullMergeRef(n))), m2+" ")
	550	+	})
	551	+	assert.Greater(t, refreshes.Load(), beforeRefresh, "a refresh was queued for the target move")
	552	+	assert.Equal(t, beforeRescan, rescans.Load(), "through the index, not a rescan")
	553	+	}
	554	+
	555	+	// GO-013: more merge requests target a pushed branch than the bound: the lookup
	556	+	// is cut, an error is logged, and the repository is rescanned, so that every
	557	+	// request ends with a merge ref on the new target.
	558	+	func TestPullTargetLookupTruncationMarksDirty(t *testing.T) {
	559	+	c := newCIHarness(t, func(cfg *usecase.CIConfig) { cfg.PullSnapshot = 2 })
	560	+	c.registerRunner("alpha", "ubuntu-latest")
	561	+	c.push("main", pullWF("", ""), "")
	562	+	const requests = 5
	563	+	var numbers []int64
	564	+	for i := range requests {
	565	+	b := "f" + string(rune('a'+i))
	566	+	c.mustGit(c.wt, "checkout", "-q", "-b", b, "main")
	567	+	c.push(b, map[string]string{b + ".txt": "x\n"}, "")
	568	+	numbers = append(numbers, c.openMR(c.root, b))
	569	+	}
	570	+	c.waitFor("the opened runs", func() bool { return len(c.runs()) == requests })
	571	+	c.mustGit(c.wt, "checkout", "-q", "main")
	572	+	m2 := c.push("main", map[string]string{"m2.txt": "m\n"}, "")
	573	+	c.waitFor("the truncation is logged", func() bool {
	574	+	logs := c.logs.String()
	575	+	return strings.Contains(logs, "refresh of a push is cut short") && strings.Contains(logs, "more than the bound") && strings.Contains(logs, `"repo":"proj"`)
	576	+	})
	577	+	for _, n := range numbers {
	578	+	c.waitFor("the merge ref of a request is on the new target", func() bool {
	579	+	return strings.HasPrefix(c.parents(c.ref(domain.PullMergeRef(n))), m2+" ")
	580	+	})
	581	+	}
	582	+	}
	583	+
	584	+	// countingGit counts the diffs the CI asks of Git.
	585	+	type countingGit struct {
	586	+	ports.MergeGit
	587	+	diffs atomic.Int32
	588	+	}
	589	+
	590	+	func (g *countingGit) DiffCommits(ctx context.Context, repo domain.RepoName, base, head domain.ObjectID, l domain.DiffLimits) (domain.DiffStats, domain.Diff, error) {
	591	+	g.diffs.Add(1)
	592	+	return g.MergeGit.DiffCommits(ctx, repo, base, head, l)
	593	+	}
	594	+
	595	+	// E2: the pages of one merge request share the diff.
	596	+	func TestPullFilesPagesShareOneDiff(t *testing.T) {
	597	+	g := &countingGit{}
	598	+	c := newCIHarness(t, func(cfg *usecase.CIConfig) {
	599	+	g.MergeGit = cfg.Git
	600	+	cfg.Git = g
	601	+	})
	602	+	c.push("main", pullWF("", ""), "")
	603	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	604	+	for i := range 5 {
	605	+	c.writeFile(c.wt, "d/"+string(rune('a'+i))+".txt", []byte("x\n"))
	606	+	}
	607	+	c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "feat")
	608	+	c.openMR(c.root, "feat")
	609	+	for _, q := range []string{"?per_page=2&page=1", "?per_page=2&page=2", "?per_page=2&page=3", ""} {
	610	+	code, _, _ := c.apiGet(adminToken, "/repos/owner/proj/pulls/1/files"+q)
	611	+	require.Equal(t, http.StatusOK, code)
	612	+	}
	613	+	assert.EqualValues(t, 1, g.diffs.Load(), "one diff for four pages")
	614	+	// A push to the branch is another pair of commits: another diff.
	615	+	c.writeFile(c.wt, "d/z.txt", []byte("z\n"))
	616	+	c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "feat")
	617	+	code, _, _ := c.apiGet(adminToken, "/repos/owner/proj/pulls/1/files")
	618	+	require.Equal(t, http.StatusOK, code)
	619	+	assert.EqualValues(t, 2, g.diffs.Load())
	620	+	}

A internal/app/ci_pull_test.go +509 -0

@@ -0,0 +1,509 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package app
	4	+
	5	+	import (
	6	+	"encoding/json"
	7	+	"fmt"
	8	+	"io"
	9	+	"net/http"
	10	+	"strings"
	11	+	"testing"
	12	+
	13	+	"connectrpc.com/connect"
	14	+	"github.com/stretchr/testify/assert"
	15	+	"github.com/stretchr/testify/require"
	16	+
	17	+	pb "bnema.dev/ermite-api/gen/ermite"
	18	+	"bnema.dev/ermite/internal/domain"
	19	+	"bnema.dev/ermite/internal/ports"
	20	+	)
	21	+
	22	+	// CI-3 D5/D6, AC5: the pull_request event of merge requests, the refs
	23	+	// refs/pull/<n>/{head,merge} and the REST files endpoint paths-filter reads.
	24	+
	25	+	const pullWorkflow = `name: pr
	26	+	on:
	27	+	pull_request:
	28	+	%s
	29	+	jobs:
	30	+	j:
	31	+	runs-on: ubuntu-latest
	32	+	%s
	33	+	steps:
	34	+	- run: echo hi
	35	+	`
	36	+
	37	+	func pullWF(on, jobExtra string) map[string]string {
	38	+	return map[string]string{".github/workflows/pr.yml": fmt.Sprintf(pullWorkflow, on, jobExtra)}
	39	+	}
	40	+
	41	+	// openMR opens a merge request from source into main as client.
	42	+	func (c ciHarness) openMR(cl clients, source string) int64 {
	43	+	c.t.Helper()
	44	+	res, err := cl.mr.CreateMergeRequest(c.ctx, connect.NewRequest(&pb.CreateMergeRequestRequest{
	45	+	Repo: c.repo, Title: "Title of " + source, Description: "About " + source, SourceBranch: source, TargetBranch: "main",
	46	+	}))
	47	+	require.NoError(c.t, err)
	48	+	return res.Msg.GetMergeRequest().GetNumber()
	49	+	}
	50	+
	51	+	func (c ciHarness) ghContext(j pb.Job) map[string]any {
	52	+	c.t.Helper()
	53	+	var ctxs map[string]map[string]any
	54	+	require.NoError(c.t, json.Unmarshal([]byte(j.GetContexts()), &ctxs))
	55	+	return ctxs["github"]
	56	+	}
	57	+
	58	+	func (c *ciHarness) ref(name string) string {
	59	+	c.t.Helper()
	60	+	out, err := c.git(c.repoDirOf(), "rev-parse", "--verify", "-q", name)
	61	+	if err != nil {
	62	+	return ""
	63	+	}
	64	+	return strings.TrimSpace(out)
	65	+	}
	66	+
	67	+	func (c *ciHarness) repoDirOf() string { return c.repoDir("proj") }
	68	+
	69	+	// finish runs the job of the next run to success.
	70	+	func (c ciHarness) finish(r fakeRunner) *pb.Job {
	71	+	c.t.Helper()
	72	+	j := r.await()
	73	+	r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	74	+	return j
	75	+	}
	76	+
	77	+	func TestPullRequestOpenedSynchronizeRuns(t *testing.T) {
	78	+	c := newCIHarness(t, nil)
	79	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	80	+	c.push("main", pullWF("", ""), "")
	81	+	c.push("main", map[string]string{"a.txt": "a\n"}, "")
	82	+	main := strings.TrimSpace(c.mustGit(c.wt, "rev-parse", "HEAD"))
	83	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	84	+	feat1 := c.push("feat", map[string]string{"b.txt": "b\n"}, "")
	85	+
	86	+	n := c.openMR(c.root, "feat")
	87	+	job := c.finish(runner)
	88	+
	89	+	// The refs: head is the branch, merge a test merge commit of main and feat.
	90	+	assert.Equal(t, feat1, c.ref(domain.PullHeadRef(n)))
	91	+	merge := c.ref(domain.PullMergeRef(n))
	92	+	require.NotEmpty(t, merge)
	93	+	assert.Equal(t, main+" "+feat1, strings.TrimSpace(c.git2("log", "-1", "--format=%P", merge)), "parents: base then head")
	94	+	assert.Equal(t, "Merge "+feat1+" into "+main, strings.TrimSpace(c.git2("log", "-1", "--format=%s", merge)))
	95	+	assert.Contains(t, c.git2("ls-tree", "--name-only", merge), "a.txt", "the tree has main's file")
	96	+	assert.Contains(t, c.git2("ls-tree", "--name-only", merge), "b.txt", "and feat's")
	97	+
	98	+	runs := c.runs()
	99	+	require.Len(t, runs, 1)
	100	+	assert.Equal(t, "pull_request", runs[0].GetEvent())
	101	+	assert.Equal(t, "refs/pull/1/merge", runs[0].GetRef())
	102	+	assert.Equal(t, merge, runs[0].GetSha())
	103	+	assert.True(t, runs[0].GetTrusted())
	104	+	assert.Equal(t, "Title of feat", runs[0].GetTitle())
	105	+
	106	+	// Checks attach to the head, not to the merge commit.
	107	+	require.Len(t, c.checks(feat1), 1)
	108	+	assert.Equal(t, "j", c.checks(feat1)[0].GetName())
	109	+	assert.Empty(t, c.checks(merge))
	110	+
	111	+	gh := c.ghContext(job)
	112	+	assert.Equal(t, "pull_request", gh["event_name"])
	113	+	assert.Equal(t, "refs/pull/1/merge", gh["ref"])
	114	+	assert.Equal(t, "1/merge", gh["ref_name"])
	115	+	assert.Equal(t, merge, gh["sha"])
	116	+	assert.Equal(t, "feat", gh["head_ref"])
	117	+	assert.Equal(t, "main", gh["base_ref"])
	118	+	assert.Equal(t, merge, job.GetSha())
	119	+	ev := gh["event"].(map[string]any)
	120	+	assert.Equal(t, "opened", ev["action"])
	121	+	assert.EqualValues(t, 1, ev["number"])
	122	+	pr := ev["pull_request"].(map[string]any)
	123	+	assert.EqualValues(t, 1, pr["number"])
	124	+	assert.Equal(t, "open", pr["state"])
	125	+	assert.Equal(t, "Title of feat", pr["title"])
	126	+	assert.Equal(t, "About feat", pr["body"])
	127	+	assert.Equal(t, false, pr["merged"])
	128	+	assert.Equal(t, merge, pr["merge_commit_sha"])
	129	+	assert.Equal(t, publicOrigin+"/owner/proj/merge-requests/1", pr["html_url"])
	130	+	head, base := pr["head"].(map[string]any), pr["base"].(map[string]any)
	131	+	assert.Equal(t, "feat", head["ref"])
	132	+	assert.Equal(t, feat1, head["sha"])
	133	+	assert.Equal(t, "main", base["ref"])
	134	+	assert.Equal(t, main, base["sha"])
	135	+	assert.Equal(t, "owner/proj", head["repo"].(map[string]any)["full_name"])
	136	+	assert.Equal(t, "owner/proj", ev["repository"].(map[string]any)["full_name"])
	137	+	assert.NotEmpty(t, ev["sender"].(map[string]any)["login"])
	138	+
	139	+	// A push to the source branch is a synchronize run, with the new merge.
	140	+	feat2 := c.push("feat", map[string]string{"c.txt": "c\n"}, "")
	141	+	job = c.finish(runner)
	142	+	c.waitFor("two runs", func() bool { return len(c.runs()) == 2 })
	143	+	assert.Equal(t, feat2, c.ref(domain.PullHeadRef(n)))
	144	+	merge2 := c.ref(domain.PullMergeRef(n))
	145	+	assert.NotEqual(t, merge, merge2)
	146	+	gh = c.ghContext(job)
	147	+	assert.Equal(t, merge2, gh["sha"])
	148	+	ev = gh["event"].(map[string]any)
	149	+	assert.Equal(t, "synchronize", ev["action"])
	150	+	assert.Equal(t, feat1, ev["before"])
	151	+	assert.Equal(t, feat2, ev["after"])
	152	+	assert.Len(t, c.checks(feat2), 1)
	153	+	}
	154	+
	155	+	// git2 runs git in the bare repository.
	156	+	func (c *ciHarness) git2(args ...string) string { return c.bareGit("proj", args...) }
	157	+
	158	+	func TestPullRequestConflictingMergeRunsNothing(t *testing.T) {
	159	+	c := newCIHarness(t, nil)
	160	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	161	+	files := pullWF("", "")
	162	+	files["a.txt"] = "base\n"
	163	+	c.push("main", files, "")
	164	+	c.mustGit(c.wt, "checkout", "-q", "-b", "conflict")
	165	+	c.push("conflict", map[string]string{"a.txt": "feature\n"}, "")
	166	+	c.mustGit(c.wt, "checkout", "-q", "main")
	167	+	c.push("main", map[string]string{"a.txt": "main\n"}, "")
	168	+	c.mustGit(c.wt, "checkout", "-q", "-b", "clean", "main")
	169	+	c.push("clean", map[string]string{"z.txt": "z\n"}, "")
	170	+
	171	+	n1 := c.openMR(c.root, "conflict")
	172	+	n2 := c.openMR(c.root, "clean")
	173	+	job := c.finish(runner) // the one run is the clean request's
	174	+	assert.EqualValues(t, 2, c.ghContext(job)["event"].(map[string]any)["number"])
	175	+	assert.NotEmpty(t, c.ref(domain.PullHeadRef(n1)), "the head ref is kept")
	176	+	assert.Empty(t, c.ref(domain.PullMergeRef(n1)), "no merge ref when the merge conflicts")
	177	+	assert.NotEmpty(t, c.ref(domain.PullMergeRef(n2)))
	178	+	require.Len(t, c.runs(), 1)
	179	+
	180	+	// A push that does not fix the conflict makes no run either.
	181	+	c.mustGit(c.wt, "checkout", "-q", "conflict")
	182	+	c.push("conflict", map[string]string{"other.txt": "o\n"}, "")
	183	+	c.mustGit(c.wt, "checkout", "-q", "-b", "later", "main")
	184	+	c.push("later", map[string]string{"l.txt": "l\n"}, "")
	185	+	c.openMR(c.root, "later")
	186	+	job = c.finish(runner)
	187	+	assert.EqualValues(t, 3, c.ghContext(job)["event"].(map[string]any)["number"])
	188	+	assert.Len(t, c.runs(), 2)
	189	+	}
	190	+
	191	+	func TestPullRequestTypesAndClosed(t *testing.T) {
	192	+	c := newCIHarness(t, nil)
	193	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	194	+	// closed only: opened and synchronize start nothing.
	195	+	c.push("main", pullWF(" types: [closed]", ""), "")
	196	+	c.mustGit(c.wt, "checkout", "-q", "-b", "f1")
	197	+	c.push("f1", map[string]string{"1.txt": "1\n"}, "")
	198	+	n1 := c.openMR(c.root, "f1")
	199	+	f1 := c.push("f1", map[string]string{"1b.txt": "1\n"}, "")
	200	+	c.mustGit(c.wt, "checkout", "-q", "main")
	201	+	c.mustGit(c.wt, "checkout", "-q", "-b", "f2")
	202	+	f2 := c.push("f2", map[string]string{"2.txt": "2\n"}, "")
	203	+	n2 := c.openMR(c.root, "f2")
	204	+	// The refs follow the MR creation and the push asynchronously: wait for
	205	+	// both of each request, at the latest source sha.
	206	+	c.waitFor("the refs of both requests", func() bool {
	207	+	mergesHead := func(n int64, head string) bool {
	208	+	m := c.ref(domain.PullMergeRef(n))
	209	+	parents, err := c.git(c.repoDirOf(), "log", "-1", "--format=%P", m) // no require: runs in the poller
	210	+	return m != "" && err == nil && strings.HasSuffix(strings.TrimSpace(parents), " "+head)
	211	+	}
	212	+	return c.ref(domain.PullHeadRef(n1)) == f1 && mergesHead(n1, f1) &&
	213	+	c.ref(domain.PullHeadRef(n2)) == f2 && mergesHead(n2, f2)
	214	+	})
	215	+	require.Empty(t, c.runs(), "no run for opened and synchronize")
	216	+	head1 := c.ref(domain.PullHeadRef(n1))
	217	+	merge1 := c.ref(domain.PullMergeRef(n1))
	218	+	require.Equal(t, f1, head1)
	219	+
	220	+	// Closed without merging: merged false, the last computed merge commit.
	221	+	_, err := c.root.mr.CloseMergeRequest(c.ctx, connect.NewRequest(&pb.CloseMergeRequestRequest{Repo: "proj", Number: n1}))
	222	+	require.NoError(t, err)
	223	+	job := c.finish(runner)
	224	+	gh := c.ghContext(job)
	225	+	ev := gh["event"].(map[string]any)
	226	+	assert.Equal(t, "closed", ev["action"])
	227	+	pr := ev["pull_request"].(map[string]any)
	228	+	assert.Equal(t, false, pr["merged"])
	229	+	assert.Equal(t, "closed", pr["state"])
	230	+	assert.Equal(t, merge1, gh["sha"], "the last computed merge commit")
	231	+	assert.Equal(t, merge1, pr["merge_commit_sha"])
	232	+	assert.Equal(t, "refs/pull/1/merge", gh["ref"])
	233	+
	234	+	// Merged: merged true, merge_commit_sha is the commit the merge made; the
	235	+	// source branch is deleted and the refs are not computed again.
	236	+	head2 := c.ref(domain.PullHeadRef(n2))
	237	+	merge2 := c.ref(domain.PullMergeRef(n2))
	238	+	_, err = c.root.mr.MergeMergeRequest(c.ctx, connect.NewRequest(&pb.MergeMergeRequestRequest{
	239	+	Repo: "proj", Number: n2, HeadId: head2, DeleteSourceBranch: true,
	240	+	}))
	241	+	require.NoError(t, err)
	242	+	job = c.finish(runner)
	243	+	gh = c.ghContext(job)
	244	+	ev = gh["event"].(map[string]any)
	245	+	pr = ev["pull_request"].(map[string]any)
	246	+	assert.Equal(t, "closed", ev["action"])
	247	+	assert.Equal(t, true, pr["merged"])
	248	+	assert.Equal(t, head2, pr["merge_commit_sha"], "fast-forward: the merged commit is the head")
	249	+	assert.Equal(t, merge2, gh["sha"], "github.sha stays the last test merge")
	250	+	assert.Equal(t, head2, pr["head"].(map[string]any)["sha"])
	251	+	assert.Equal(t, head2, c.ref(domain.PullHeadRef(n2)))
	252	+	assert.Equal(t, merge2, c.ref(domain.PullMergeRef(n2)))
	253	+	assert.Len(t, c.checks(head2), 1)
	254	+	}
	255	+
	256	+	func TestPullRequestPathsAndBranchesFilters(t *testing.T) {
	257	+	c := newCIHarness(t, nil)
	258	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	259	+	c.push("main", pullWF(" branches: [main]\n paths: ['docs/**']", ""), "")
	260	+	c.mustGit(c.wt, "checkout", "-q", "-b", "f1")
	261	+	c.push("f1", map[string]string{"src/x.go": "x\n"}, "")
	262	+	n := c.openMR(c.root, "f1")
	263	+	// Another request whose changes match: the only run is its.
	264	+	c.mustGit(c.wt, "checkout", "-q", "-b", "f2", "main")
	265	+	c.push("f2", map[string]string{"docs/readme.md": "d\n"}, "")
	266	+	n2 := c.openMR(c.root, "f2")
	267	+	job := c.finish(runner)
	268	+	assert.EqualValues(t, n2, c.ghContext(job)["event"].(map[string]any)["number"])
	269	+	require.Len(t, c.runs(), 1)
	270	+
	271	+	// A later push that adds a docs file to the first request: the files are
	272	+	// those between the merge base and the head, so it runs now.
	273	+	c.mustGit(c.wt, "checkout", "-q", "f1")
	274	+	c.push("f1", map[string]string{"docs/new.md": "n\n"}, "")
	275	+	job = c.finish(runner)
	276	+	assert.EqualValues(t, n, c.ghContext(job)["event"].(map[string]any)["number"])
	277	+	assert.Equal(t, "synchronize", c.ghContext(job)["event"].(map[string]any)["action"])
	278	+	}
	279	+
	280	+	func TestPullRequestTrustFollowsTheActor(t *testing.T) {
	281	+	c := newCIHarness(t, nil)
	282	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	283	+	c.push("main", pullWF("", ""), "")
	284	+	c.mustGit(c.wt, "checkout", "-q", "-b", "agent/x")
	285	+	c.push("agent/x", map[string]string{"x.txt": "x\n"}, "")
	286	+	secret, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite, domain.ScopeMRWrite), []string{"agent/"}, "proj")
	287	+
	288	+	// Opened by a branch-limited token: untrusted, no secrets, read-only token.
	289	+	c.openMR(c.api(secret), "agent/x")
	290	+	job := c.finish(runner)
	291	+	assert.False(t, job.GetTrusted())
	292	+	runs := c.runs()
	293	+	require.Len(t, runs, 1)
	294	+	assert.False(t, runs[0].GetTrusted())
	295	+	assert.Equal(t, "claude", runs[0].GetActor())
	296	+
	297	+	// A synchronize pushed by the owner (root token) is trusted.
	298	+	c.push("agent/x", map[string]string{"y.txt": "y\n"}, "")
	299	+	job = c.finish(runner)
	300	+	assert.True(t, job.GetTrusted())
	301	+	assert.Equal(t, "root", c.runs()[0].GetActor())
	302	+
	303	+	// A synchronize pushed by the limited token is not.
	304	+	c.writeFile(c.wt, "z.txt", []byte("z\n"))
	305	+	c.mustGit(c.wt, "push", "-q", c.urlAuth(secret, "/proj.git"), "agent/x")
	306	+	job = c.finish(runner)
	307	+	assert.False(t, job.GetTrusted())
	308	+	}
	309	+
	310	+	func TestPullRefsAreServedHiddenAndNotPushable(t *testing.T) {
	311	+	c := newCIHarness(t, nil)
	312	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	313	+	c.push("main", pullWF("", " permissions:\n contents: write"), "")
	314	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	315	+	feat := c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	316	+	n := c.openMR(c.root, "feat")
	317	+	job := runner.await()
	318	+	defer runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	319	+	merge := c.ref(domain.PullMergeRef(n))
	320	+
	321	+	// Fetchable, as checkout does.
	322	+	dir := c.t.TempDir()
	323	+	c.mustGit(dir, "init", "-q")
	324	+	c.mustGit(dir, "fetch", "-q", c.urlAuth(adminToken, "/proj.git"), "refs/pull/1/merge:refs/remotes/pull/1/merge", "refs/pull/1/head:refs/remotes/pull/1/head")
	325	+	assert.Equal(t, merge, strings.TrimSpace(c.mustGit(dir, "rev-parse", "refs/remotes/pull/1/merge")))
	326	+	assert.Equal(t, feat, strings.TrimSpace(c.mustGit(dir, "rev-parse", "refs/remotes/pull/1/head")))
	327	+
	328	+	// Hidden from the branch and tag lists.
	329	+	res, err := c.root.git.ListRefs(c.ctx, connect.NewRequest(&pb.ListRefsRequest{Repo: "proj"}))
	330	+	require.NoError(t, err)
	331	+	for _, r := range res.Msg.GetRefs() {
	332	+	assert.NotContains(t, r.GetName(), "refs/pull/")
	333	+	}
	334	+
	335	+	// Refused on push: the owner, a stored token with repo:write, and the job
	336	+	// token with contents: write (the workflow guard does not reach it first).
	337	+	other, _ := c.branchToken(scopes(domain.ScopeRepoRead, domain.ScopeRepoWrite), nil, "proj")
	338	+	for _, tc := range []struct{ name, url string }{
	339	+	{"owner", c.urlAuth(adminToken, "/proj.git")},
	340	+	{"token", c.urlAuth(other, "/proj.git")},
	341	+	} {
	342	+	for _, ref := range []string{"refs/pull/1/head", "refs/pull/9/head", "refs/pull/1/merge", "refs/pull/x"} {
	343	+	out, err := c.git(c.wt, "push", tc.url, "HEAD:"+ref)
	344	+	require.Error(t, err, "%s pushes %s: %s", tc.name, ref, out)
	345	+	}
	346	+	out, err := c.git(c.wt, "push", "--delete", tc.url, "refs/pull/1/merge")
	347	+	require.Error(t, err, out)
	348	+	}
	349	+	out, err := c.tokenPush(c.wt, job.GetGithubToken(), "HEAD:refs/pull/1/head")
	350	+	require.Error(t, err, out)
	351	+	assert.Equal(t, merge, c.ref(domain.PullMergeRef(n)))
	352	+	assert.Equal(t, feat, c.ref(domain.PullHeadRef(n)))
	353	+	assert.Empty(t, c.ref("refs/pull/9/head"))
	354	+	}
	355	+
	356	+	func (c *ciHarness) apiGet(token, path string) (int, http.Header, string) {
	357	+	c.t.Helper()
	358	+	req, err := http.NewRequestWithContext(c.ctx, http.MethodGet, c.url("/-/github/api"+path), nil)
	359	+	require.NoError(c.t, err)
	360	+	if token != "" {
	361	+	req.Header.Set("Authorization", "Bearer "+token)
	362	+	}
	363	+	res, err := http.DefaultClient.Do(req)
	364	+	require.NoError(c.t, err)
	365	+	defer func() { _ = res.Body.Close() }()
	366	+	b, err := io.ReadAll(res.Body)
	367	+	require.NoError(c.t, err)
	368	+	return res.StatusCode, res.Header, string(b)
	369	+	}
	370	+
	371	+	func TestPullFilesEndpoint(t *testing.T) {
	372	+	c := newCIHarness(t, nil)
	373	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	374	+	c.push("main", pullWF("", " permissions:\n contents: read\n pull-requests: read"), "")
	375	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	376	+	files := map[string]string{}
	377	+	for i := range 5 {
	378	+	files[fmt.Sprintf("d/f%d.txt", i)] = fmt.Sprintf("line %d\n", i)
	379	+	}
	380	+	c.writeFile(c.wt, "d/gone.txt", []byte("x\n"))
	381	+	for name, content := range files {
	382	+	c.writeFile(c.wt, name, []byte(content))
	383	+	}
	384	+	c.mustGit(c.wt, "push", "-q", c.urlAuth(adminToken, "/proj.git"), "feat")
	385	+	c.openMR(c.root, "feat")
	386	+	job := runner.await() // the job token lives while the job runs
	387	+	defer runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	388	+
	389	+	type entry struct {
	390	+	Filename, Status, Sha, Patch string
	391	+	Additions, Deletions, Changes int
	392	+	}
	393	+	list := func(token, q string) (int, http.Header, []entry) {
	394	+	code, hdr, body := c.apiGet(token, "/repos/owner/proj/pulls/1/files"+q)
	395	+	var es []entry
	396	+	if code == http.StatusOK {
	397	+	require.NoError(t, json.Unmarshal([]byte(body), &es), body)
	398	+	}
	399	+	return code, hdr, es
	400	+	}
	401	+
	402	+	code, hdr, es := list(adminToken, "?per_page=4&page=1")
	403	+	require.Equal(t, http.StatusOK, code)
	404	+	require.Len(t, es, 4)
	405	+	assert.Equal(t, "d/f0.txt", es[0].Filename)
	406	+	assert.Equal(t, "added", es[0].Status)
	407	+	assert.Equal(t, 1, es[0].Additions)
	408	+	assert.Equal(t, 1, es[0].Changes)
	409	+	assert.Equal(t, "@@ -0,0 +1 @@\n+line 0", es[0].Patch)
	410	+	assert.Len(t, es[0].Sha, 40)
	411	+	link := hdr.Get("Link")
	412	+	assert.Contains(t, link, `page=2>; rel="next"`)
	413	+	assert.Contains(t, link, `page=2>; rel="last"`)
	414	+	assert.NotContains(t, link, `rel="prev"`)
	415	+
	416	+	code, hdr, es = list(adminToken, "?per_page=4&page=2")
	417	+	require.Equal(t, http.StatusOK, code)
	418	+	require.Len(t, es, 2, "6 files in all")
	419	+	assert.NotContains(t, hdr.Get("Link"), `rel="next"`)
	420	+	assert.Contains(t, hdr.Get("Link"), `rel="prev"`)
	421	+	assert.Contains(t, hdr.Get("Link"), `rel="first"`)
	422	+
	423	+	code, _, es = list(adminToken, "?page=9")
	424	+	require.Equal(t, http.StatusOK, code)
	425	+	assert.Empty(t, es)
	426	+	for _, q := range []string{"?page=9223372036854775807", "?per_page=100&page=9223372036854775807", "?page=2305843009213693953&per_page=4"} {
	427	+	code, _, es = list(adminToken, q)
	428	+	require.Equal(t, http.StatusOK, code, q)
	429	+	assert.Empty(t, es, q)
	430	+	}
	431	+	code, hdr, es = list(adminToken, "")
	432	+	require.Equal(t, http.StatusOK, code)
	433	+	assert.Len(t, es, 6, "30 per page by default")
	434	+	assert.Empty(t, hdr.Get("Link"))
	435	+	code, _, es = list(adminToken, "?per_page=1000")
	436	+	require.Equal(t, http.StatusOK, code)
	437	+	assert.Len(t, es, 6, "100 at most")
	438	+
	439	+	// The job token with pull-requests: read; the answer is 404 for a request
	440	+	// that does not exist and for anonymous callers on a private repository.
	441	+	code, _, es = list(job.GetGithubToken(), "")
	442	+	require.Equal(t, http.StatusOK, code)
	443	+	assert.Len(t, es, 6)
	444	+	code, _, _ = c.apiGet(job.GetGithubToken(), "/repos/owner/proj/pulls/7/files")
	445	+	assert.Equal(t, http.StatusNotFound, code)
	446	+	code, _, _ = c.apiGet("", "/repos/owner/proj/pulls/1/files")
	447	+	assert.Equal(t, http.StatusNotFound, code)
	448	+	code, _, _ = c.apiGet(adminToken, "/repos/owner/proj/pulls/1")
	449	+	assert.Equal(t, http.StatusNotFound, code, "GET /pulls/{n} is not served: paths-filter does not call it")
	450	+	}
	451	+
	452	+	// A job token with pull-requests: read and contents: none reads the merge
	453	+	// requests of its own private repository, and only of that one.
	454	+	func TestPullFilesWithPullRequestsPermissionOnly(t *testing.T) {
	455	+	c := newCIHarness(t, nil)
	456	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	457	+	c.push("main", pullWF("", " permissions:\n contents: none\n pull-requests: read"), "")
	458	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	459	+	c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	460	+	c.openMR(c.root, "feat")
	461	+	job := runner.await()
	462	+	defer runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	463	+	code, _, body := c.apiGet(job.GetGithubToken(), "/repos/owner/proj/pulls/1/files")
	464	+	assert.Equal(t, http.StatusOK, code, body)
	465	+	assert.Contains(t, body, "f.txt")
	466	+	// Not the repository itself (contents: none) and not another repository.
	467	+	code, _, _ = c.apiGet(job.GetGithubToken(), "/repos/owner/proj")
	468	+	assert.Equal(t, http.StatusNotFound, code)
	469	+	_, err := c.forge.Repos.Create(c.ctx, domain.Root(), ports.CreateRepoParams{Name: c.name("other"), Visibility: domain.VisibilityPrivate})
	470	+	require.NoError(t, err)
	471	+	code, _, _ = c.apiGet(job.GetGithubToken(), "/repos/owner/other/pulls/1/files")
	472	+	assert.Equal(t, http.StatusNotFound, code)
	473	+	}
	474	+
	475	+	// The merge requests of a public repository are read by anyone, except by the
	476	+	// holder of an archive token, which reads one archive and nothing else.
	477	+	func TestPullFilesOfAPublicRepositoryAreNotReadByAnArchiveToken(t *testing.T) {
	478	+	c := newCIHarness(t, nil)
	479	+	c.push("main", pullWF("", ""), "")
	480	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	481	+	c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	482	+	c.openMR(c.root, "feat")
	483	+	public := domain.VisibilityPublic
	484	+	_, err := c.forge.Repos.Update(c.ctx, domain.Root(), c.name("proj"), ports.UpdateRepoParams{Visibility: &public})
	485	+	require.NoError(t, err)
	486	+
	487	+	code, _, body := c.apiGet("", "/repos/owner/proj/pulls/1/files")
	488	+	require.Equal(t, http.StatusOK, code, body)
	489	+	assert.Contains(t, body, "f.txt", "anyone reads a public repository")
	490	+
	491	+	archive := domain.RepoTokenPrincipal(domain.CIRepoToken{ID: "a", Repo: c.name("proj"), Archive: true, Sha: strings.Repeat("a", 40)})
	492	+	require.True(t, archive.ViaJobToken(), "the archive token passes ViaJobToken")
	493	+	_, _, err = c.forge.CI.ListPullFiles(c.ctx, archive, "owner", "proj", 1, 1, 30)
	494	+	assert.ErrorIs(t, err, domain.ErrNotFound, "but it reads no merge request")
	495	+	}
	496	+
	497	+	func TestPullFilesNeedsPullRequestsPermission(t *testing.T) {
	498	+	c := newCIHarness(t, nil)
	499	+	runner := c.registerRunner("alpha", "ubuntu-latest")
	500	+	c.push("main", pullWF("", ""), "") // the default permissions: contents read only
	501	+	c.mustGit(c.wt, "checkout", "-q", "-b", "feat")
	502	+	c.push("feat", map[string]string{"f.txt": "f\n"}, "")
	503	+	c.openMR(c.root, "feat")
	504	+	job := runner.await()
	505	+	defer runner.runJob(job, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil, "ok")
	506	+	code, _, body := c.apiGet(job.GetGithubToken(), "/repos/owner/proj/pulls/1/files")
	507	+	assert.Equal(t, http.StatusForbidden, code, body)
	508	+	assert.Contains(t, body, "Resource not accessible by integration")
	509	+	}

M internal/app/cli_run.go +12 -1


@@ -254,7 +254,7 @@ func newRunListCmd(env cliEnv) cobra.Command {
254	254		for _, r := range runs {
255	255		rows = append(rows, []string{
256	256		strconv.FormatInt(r.GetId(), 10), runResult(r.GetStatus(), r.GetConclusion()), r.GetWorkflowName(),
257		-	strings.TrimPrefix(strings.TrimPrefix(r.GetRef(), "refs/heads/"), "refs/tags/"), shortSha(r.GetSha()),
	257	+	refText(r.GetRef()), shortSha(r.GetSha()),
258	258		r.GetActor(), shortTime(r.GetCreatedAt(), "-"), r.GetTitle(),
259	259		})
260	260		}

@@ -767,3 +767,14 @@ func newRunRerunCmd(env cliEnv) cobra.Command {
767	767		}
768	768		return cmd
769	769		}
	770	+
	771	+	// refText is a ref as the run list shows it: the short name of a branch or a
	772	+	// tag, "!<n>" for the merge ref of a merge request (a pull_request run).
	773	+	func refText(ref string) string {
	774	+	if rest, ok := strings.CutPrefix(ref, "refs/pull/"); ok {
	775	+	if n, _, ok := strings.Cut(rest, "/"); ok {
	776	+	return "!" + n
	777	+	}
	778	+	}
	779	+	return strings.TrimPrefix(strings.TrimPrefix(ref, "refs/heads/"), "refs/tags/")
	780	+	}

M internal/app/cli_run_test.go +6 -0


@@ -130,3 +130,9 @@ func TestCLIRunsAndRunners(t *testing.T) {
130	130		assert.NoError(t, h.cli(t, created.Secret, "runner", "list").err)
131	131		assert.Error(t, h.cli(t, created.Secret, "run", "cancel", "proj", id).err)
132	132		}
	133	+
	134	+	func TestRefTextShowsMergeRequests(t *testing.T) {
	135	+	assert.Equal(t, "!12", refText("refs/pull/12/merge"))
	136	+	assert.Equal(t, "main", refText("refs/heads/main"))
	137	+	assert.Equal(t, "v1", refText("refs/tags/v1"))
	138	+	}

M internal/app/forge.go +13 -2


@@ -245,6 +245,7 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e
245	245
246	246		// CI (ADR 0025): the engine reads workflows through the reader, and plans
247	247		// them in the background; a push only queues its refs.
	248	+	committer := domain.Identity{Name: cfg.CommitterName, Email: cfg.CommitterEmail}
248	249		ciCfg := usecase.CIConfig{
249	250		Store: store.CI(), Logs: cilogs.New(cilogs.Dir(cfg.DataDir)), Summaries: cisummaries.New(cisummaries.Dir(cfg.DataDir)), Meta: store, Repos: git, Reader: reader,
250	251		Git: git, Diff: git, Bus: f.Bus, Clock: clock, Auth: f.Auth,

@@ -258,6 +259,7 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e
258	259		Settings: f.Settings,
259	260		Workflows: opts.workflows(githubraw.New(githubraw.Config{})),
260	261		}
	262	+	f.pullRequests(&ciCfg, store, git, committer)
261	263		if opts.ci != nil {
262	264		opts.ci(&ciCfg)
263	265		}

@@ -278,13 +280,13 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e
278	280		if err != nil {
279	281		return nil, err
280	282		}
281		-	committer := domain.Identity{Name: cfg.CommitterName, Email: cfg.CommitterEmail}
282	283		f.SigningKey = usecase.NewSigningKey(signer, committer)
283	284		f.MergeRequests = usecase.NewMergeRequests(usecase.MergeRequestsConfig{
284	285		Meta: store, Store: store.MergeRequests(), Repos: git, Git: git, Signer: signer,
285	286		Committer: committer, Bus: f.Bus, Clock: clock, Maint: f.maint,
286		-	CI: f.CI, Checks: f.CI,
	287	+	CI: f.CI, Checks: f.CI, PullRequests: f.CI,
287	288		})
	289	+	f.CI.SetPullGate(f.MergeRequests)
288	290		log.Info().Str("fingerprint", signer.PublicKey().Fingerprint).Msg("commit signing key ready")
289	291
290	292		f.Sessions, err = usecase.NewSessions(ctx, store.Sessions(), clock, cfg.PublicURL)

@@ -442,3 +444,12 @@ func (f *forge) Close(ctx context.Context) {
442	444		f.store = nil
443	445		}
444	446		}
	447	+
	448	+	// pullRequests gives the CI the merge requests, the refs/pull refs and the
	449	+	// identity of the test merge commits (CI-3, D5).
	450	+	func (f forge) pullRequests(cfg usecase.CIConfig, store kvstore.Store, git gitcli.Git, committer domain.Identity) {
	451	+	cfg.MRs, cfg.Pull, cfg.Committer = store.MergeRequests(), git, committer
	452	+	if f.reader != nil {
	453	+	cfg.Fresh = f.reader // a nil *ermitegit.Adapter must not become a non-nil interface
	454	+	}
	455	+	}

M internal/domain/ci.go +19 -5


@@ -167,11 +167,15 @@ type CIRun struct {
167	167		// Name is GitHub's name of the run: the evaluated run-name when the
168	168		// workflow has one, else WorkflowName. WorkflowName stays the name of the
169	169		// workflow (github.workflow, the checks).
170		-	Name string
171		-	Title string
172		-	Event string
173		-	Ref string
174		-	Sha string
	170	+	Name string
	171	+	Title string
	172	+	Event string
	173	+	Ref string
	174	+	Sha string
	175	+	// HeadSha is the head of the merge request of a pull_request run, whose Sha
	176	+	// is the test merge commit github.sha shows; empty for the other events. The
	177	+	// checks of the run attach to it (CheckSha).
	178	+	HeadSha string
175	179		Actor string
176	180		Status CIStatus
177	181		Conclusion CIConclusion

@@ -193,6 +197,16 @@ type CIRun struct {
193	197		Version uint64
194	198		}
195	199
	200	+	// CheckSha is the commit the checks of the run attach to: the head of the merge
	201	+	// request for a pull_request run (as on GitHub, where the checks of the test
	202	+	// merge show on the head), the commit of the run otherwise.
	203	+	func (r CIRun) CheckSha() string {
	204	+	if r.HeadSha != "" {
	205	+	return r.HeadSha
	206	+	}
	207	+	return r.Sha
	208	+	}
	209	+
196	210		// CIRunSource is what a run was planned from: the workflow file at the sha of
197	211		// the run, and the "github" context without the parts that depend on the job
198	212		// and the attempt.

M internal/domain/ci_token.go +7 -1


@@ -3,6 +3,7 @@
3	3		package domain
4	4
5	5		import (
	6	+	"maps"
6	7		"strings"
7	8		"time"
8	9		)

@@ -80,10 +81,15 @@ func RepoTokenPrincipal(t CIRepoToken) Principal {
80	81		archiveRepo: t.Repo, archiveSha: t.Sha,
81	82		}
82	83		}
83		-	return TokenPrincipal(Token{
	84	+	p := TokenPrincipal(Token{
84	85		ID: repoTokenIDPrefix + t.ID, Label: "ci job", Scopes: RepoTokenScopes(t.Permissions),
85	86		Repos: []RepoName{t.Repo}, ExpiresAt: t.ExpiresAt, CreatedAt: t.CreatedAt,
86	87		})
	88	+	p.jobPerms = maps.Clone(t.Permissions)
	89	+	if len(p.jobPerms) == 0 {
	90	+	p.jobPerms = DefaultTokenPermissions()
	91	+	}
	92	+	return p
87	93		}
88	94
89	95		// CIPusherKind tells who pushed the commit of a run.

M internal/domain/principal.go +16 -0


@@ -38,6 +38,9 @@ type Principal struct {
38	38		// archiveRepo and archiveSha are the grant of a kindArchive principal.
39	39		archiveRepo RepoName
40	40		archiveSha string
	41	+	// jobPerms are the permissions (key to level) of a job's own token; nil for
	42	+	// every other principal.
	43	+	jobPerms map[string]string
41	44		}
42	45
43	46		// Anonymous returns the principal of a caller without credentials.

@@ -166,6 +169,19 @@ func (p Principal) AllowsRef(ref string) bool {
166	169		return p.CoversBranchPrefix(name)
167	170		}
168	171
	172	+	// JobPermits reports whether p is the job's own token, for the repository, and
	173	+	// its workflow permissions give key at least the level (write includes read).
	174	+	// It is how an endpoint that GitHub gates on a permission with no ermite scope
	175	+	// (pull-requests: read) checks a job token; every other principal answers
	176	+	// false and is judged by its scopes.
	177	+	func (p Principal) JobPermits(key, level string, repo RepoName) bool {
	178	+	if p.kind != kindToken \|\| !IsRepoTokenID(p.tokenID) \|\| !slices.Contains(p.repos, repo) {
	179	+	return false
	180	+	}
	181	+	have := p.jobPerms[key]
	182	+	return have == PermissionWrite \|\| (have == PermissionRead && level == PermissionRead)
	183	+	}
	184	+
169	185		// ViaJobToken reports whether p is the holder of a job repository token (the
170	186		// GITHUB_TOKEN of a CI job, or an action archive token): what it does must not
171	187		// start CI runs, and it may not change workflows (CI-3, D3).

A internal/domain/pullrequest.go +77 -0

@@ -0,0 +1,77 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package domain
	4	+
	5	+	import (
	6	+	"strconv"
	7	+	"strings"
	8	+	"time"
	9	+	)
	10	+
	11	+	// A merge request is a "pull request" for the workflows that run on it (CI-3,
	12	+	// D5): ermite keeps two refs per request, as GitHub does.
	13	+	//
	14	+	// refs/pull/<n>/head the head of the source branch
	15	+	// refs/pull/<n>/merge a test merge of the head into the target, when it merges cleanly
	16	+	//
	17	+	// They are written by ermite only: a push to them is refused (see IsPullRef),
	18	+	// and they are not branches nor tags, so no listing shows them. Git serves them
	19	+	// to fetch like any ref.
	20	+
	21	+	// PullRefPrefix starts the refs of the merge requests.
	22	+	const PullRefPrefix = "refs/pull/"
	23	+
	24	+	// PullHeadRef is the ref that holds the head of merge request number.
	25	+	func PullHeadRef(number int64) string { return PullRefPrefix + strconv.FormatInt(number, 10) + "/head" }
	26	+
	27	+	// PullMergeRef is the ref that holds the test merge of merge request number.
	28	+	func PullMergeRef(number int64) string {
	29	+	return PullRefPrefix + strconv.FormatInt(number, 10) + "/merge"
	30	+	}
	31	+
	32	+	// IsPullRefName reports whether ref is under refs/pull/, whatever follows.
	33	+	// Pushes are refused on all of that namespace.
	34	+	func IsPullRefName(ref string) bool { return strings.HasPrefix(ref, PullRefPrefix) }
	35	+
	36	+	// IsPullRef reports whether ref is exactly refs/pull/<n>/head or
	37	+	// refs/pull/<n>/merge, the two refs ermite maintains.
	38	+	func IsPullRef(ref string) bool {
	39	+	rest, ok := strings.CutPrefix(ref, PullRefPrefix)
	40	+	if !ok {
	41	+	return false
	42	+	}
	43	+	num, kind, ok := strings.Cut(rest, "/")
	44	+	if !ok \|\| (kind != "head" && kind != "merge") {
	45	+	return false
	46	+	}
	47	+	n, err := strconv.ParseInt(num, 10, 64)
	48	+	return err == nil && n > 0 && strconv.FormatInt(n, 10) == num
	49	+	}
	50	+
	51	+	// PullRequestActions are the activity types of the pull_request event that
	52	+	// ermite can produce. A merge request cannot be reopened yet, so "reopened"
	53	+	// never occurs, but a workflow may list it.
	54	+	const (
	55	+	PullOpened = "opened"
	56	+	PullSynchronize = "synchronize"
	57	+	PullReopened = "reopened"
	58	+	PullClosed = "closed"
	59	+	)
	60	+
	61	+	// DefaultPullRequestTypes are the types of a pull_request that lists none.
	62	+	var DefaultPullRequestTypes = []string{PullOpened, PullSynchronize, PullReopened}
	63	+
	64	+	// TestMergeCommit returns the text of the commit that tests the merge of head
	65	+	// into base: the tree the merge produced, parents base then head, and GitHub's
	66	+	// message. It is not signed.
	67	+	func TestMergeCommit(tree, base, head ObjectID, who Identity, at time.Time) []byte {
	68	+	sig := signatureLine(Signature{Name: who.Name, Email: who.Email, Time: at.UTC()})
	69	+	var b strings.Builder
	70	+	b.WriteString("tree " + tree.String() + "\n")
	71	+	b.WriteString("parent " + base.String() + "\n")
	72	+	b.WriteString("parent " + head.String() + "\n")
	73	+	b.WriteString("author " + sig + "\n")
	74	+	b.WriteString("committer " + sig + "\n")
	75	+	b.WriteString("\nMerge " + head.String() + " into " + base.String() + "\n")
	76	+	return []byte(b.String())
	77	+	}

A internal/domain/pullrequest_test.go +66 -0

@@ -0,0 +1,66 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package domain_test
	4	+
	5	+	import (
	6	+	"strings"
	7	+	"testing"
	8	+	"time"
	9	+
	10	+	"github.com/stretchr/testify/assert"
	11	+	"github.com/stretchr/testify/require"
	12	+
	13	+	"bnema.dev/ermite/internal/domain"
	14	+	)
	15	+
	16	+	func TestPullRefNames(t *testing.T) {
	17	+	assert.Equal(t, "refs/pull/7/head", domain.PullHeadRef(7))
	18	+	assert.Equal(t, "refs/pull/7/merge", domain.PullMergeRef(7))
	19	+	for _, ok := range []string{"refs/pull/1/head", "refs/pull/12/merge"} {
	20	+	assert.True(t, domain.IsPullRef(ok), ok)
	21	+	}
	22	+	for _, no := range []string{"refs/pull/0/head", "refs/pull/01/head", "refs/pull/1/other", "refs/pull/1", "refs/pull/x/head", "refs/heads/pull/1/head", "refs/pull/1/head/x", "refs/pull/-1/merge"} {
	23	+	assert.False(t, domain.IsPullRef(no), no)
	24	+	}
	25	+	assert.True(t, domain.IsPullRefName("refs/pull/anything"))
	26	+	assert.False(t, domain.IsPullRefName("refs/pullx/1/head"))
	27	+	}
	28	+
	29	+	func TestTestMergeCommit(t *testing.T) {
	30	+	tree := mustID(t, "1")
	31	+	base, head := mustID(t, "2"), mustID(t, "3")
	32	+	got := string(domain.TestMergeCommit(tree, base, head, domain.Identity{Name: "ermite", Email: "e@x"}, time.Unix(1700000000, 0)))
	33	+	assert.Equal(t, "tree "+tree.String()+"\nparent "+base.String()+"\nparent "+head.String()+
	34	+	"\nauthor ermite <e@x> 1700000000 +0000\ncommitter ermite <e@x> 1700000000 +0000\n\nMerge "+head.String()+" into "+base.String()+"\n", got)
	35	+	}
	36	+
	37	+	func mustID(t *testing.T, c string) domain.ObjectID {
	38	+	t.Helper()
	39	+	id, err := domain.ParseObjectID(strings.Repeat(c, 40))
	40	+	require.NoError(t, err)
	41	+	return id
	42	+	}
	43	+
	44	+	func TestJobPermits(t *testing.T) {
	45	+	repo, other := mustRepo(t, "proj"), mustRepo(t, "other")
	46	+	tok := func(perms map[string]string) domain.Principal {
	47	+	return domain.RepoTokenPrincipal(domain.CIRepoToken{ID: "a", Repo: repo, Permissions: perms})
	48	+	}
	49	+	read := map[string]string{domain.PermPullRequests: domain.PermissionRead}
	50	+	write := map[string]string{domain.PermPullRequests: domain.PermissionWrite}
	51	+	assert.True(t, tok(read).JobPermits(domain.PermPullRequests, domain.PermissionRead, repo))
	52	+	assert.False(t, tok(read).JobPermits(domain.PermPullRequests, domain.PermissionWrite, repo))
	53	+	assert.True(t, tok(write).JobPermits(domain.PermPullRequests, domain.PermissionRead, repo), "write includes read")
	54	+	assert.False(t, tok(read).JobPermits(domain.PermPullRequests, domain.PermissionRead, other), "another repository")
	55	+	assert.False(t, tok(nil).JobPermits(domain.PermPullRequests, domain.PermissionRead, repo), "the default has no pull-requests")
	56	+	assert.False(t, tok(map[string]string{domain.PermContents: domain.PermissionWrite}).JobPermits(domain.PermPullRequests, domain.PermissionRead, repo))
	57	+	assert.False(t, domain.Root().JobPermits(domain.PermPullRequests, domain.PermissionRead, repo), "only job tokens")
	58	+	assert.False(t, domain.Anonymous().JobPermits(domain.PermPullRequests, domain.PermissionRead, repo))
	59	+	}
	60	+
	61	+	func mustRepo(t *testing.T, name string) domain.RepoName {
	62	+	t.Helper()
	63	+	r, err := domain.ParseRepoName(name)
	64	+	require.NoError(t, err)
	65	+	return r
	66	+	}

M internal/ports/ci.go +51 -0


@@ -267,6 +267,57 @@ type WorkflowTrigger interface {
267	267		OnMerge(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, merger domain.Principal)
268	268		}
269	269
	270	+	// PullRequestTrigger starts the pull_request workflows of the merge requests
	271	+	// (CI-3, D5), like WorkflowTrigger does for pushes. It only queues the work.
	272	+	// The synchronize activity needs no call here: the push of the source branch
	273	+	// reaches WorkflowTrigger.OnPush.
	274	+	type PullRequestTrigger interface {
	275	+	// OnPullRequest queues the activity ev of a merge request. p is who caused
	276	+	// it: its trust is the run's, whatever happens to the branches before the
	277	+	// event is handled. A job token starts no run.
	278	+	OnPullRequest(ctx context.Context, repo domain.RepoName, ev PullRequestEvent, p domain.Principal)
	279	+	}
	280	+
	281	+	// PullRequestEvent is one activity of a merge request, with the commits it is
	282	+	// for fixed when it happened: it runs for exactly these, with the trust of
	283	+	// whoever caused it, even when the branches have moved on by the time it is
	284	+	// handled.
	285	+	type PullRequestEvent struct {
	286	+	Number int64
	287	+	// Action is domain.PullOpened or domain.PullClosed.
	288	+	Action string
	289	+	// Head is the head of the source branch and Base the tip of the target at
	290	+	// that moment; either may be zero when unknown (the current one is read).
	291	+	Head, Base domain.ObjectID
	292	+	}
	293	+
	294	+	// PullGate lets the CI change what depends on a merge request staying open
	295	+	// (its refs/pull refs, the runs of its opening and updates) without racing a
	296	+	// close or a merge. *usecase.MergeRequests implements it.
	297	+	type PullGate interface {
	298	+	// WhileOpen runs fn holding the merge lock of the repository (the one
	299	+	// Close and Merge take) when merge request number is open, and returns
	300	+	// domain.ErrPreconditionFailed without running fn when it is not. fn must
	301	+	// be short and must not call the merge request use cases.
	302	+	WhileOpen(ctx context.Context, repo domain.RepoName, number int64, fn func() error) error
	303	+	}
	304	+
	305	+	// PullGit is what the refs/pull/<n>/{head,merge} of the merge requests need
	306	+	// from Git. Every method returns domain.ErrNotFound when the repository has no
	307	+	// directory.
	308	+	type PullGit interface {
	309	+	// TestMerge merges head into base without touching a ref (`git merge-tree
	310	+	// --write-tree`) and returns the tree of the result. clean is false when
	311	+	// the merge conflicts, or when the commits share no history.
	312	+	TestMerge(ctx context.Context, repo domain.RepoName, base, head domain.ObjectID) (tree domain.ObjectID, clean bool, err error)
	313	+	// SetPullRef points a ref of domain.IsPullRef to the commit. Any other ref
	314	+	// name is domain.ErrInvalid.
	315	+	SetPullRef(ctx context.Context, repo domain.RepoName, ref string, id domain.ObjectID) error
	316	+	// PullRef returns the commit a ref of domain.IsPullRef points to, or
	317	+	// domain.ErrNotFound.
	318	+	PullRef(ctx context.Context, repo domain.RepoName, ref string) (domain.ObjectID, error)
	319	+	}
	320	+
270	321		// CIGit is what planning needs from Git beyond RepoReader and MergeGit.
271	322		type CIGit interface {
272	323		// ChangedPaths returns the paths changed from base to head (a zero base

M internal/ports/mocks/GitHubAPIService.go +104 -0


@@ -206,6 +206,110 @@ func (_c *GitHubAPIService_HashAlgorithm_Call) RunAndReturn(run func(ctx context
206	206		return _c
207	207		}
208	208
	209	+	// ListPullFiles provides a mock function for the type GitHubAPIService
	210	+	func (_mock *GitHubAPIService) ListPullFiles(ctx context.Context, p domain.Principal, owner string, repo string, number int64, page int, perPage int) ([]byte, string, error) {
	211	+	ret := _mock.Called(ctx, p, owner, repo, number, page, perPage)
	212	+
	213	+	if len(ret) == 0 {
	214	+	panic("no return value specified for ListPullFiles")
	215	+	}
	216	+
	217	+	var r0 []byte
	218	+	var r1 string
	219	+	var r2 error
	220	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.Principal, string, string, int64, int, int) ([]byte, string, error)); ok {
	221	+	return returnFunc(ctx, p, owner, repo, number, page, perPage)
	222	+	}
	223	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.Principal, string, string, int64, int, int) []byte); ok {
	224	+	r0 = returnFunc(ctx, p, owner, repo, number, page, perPage)
	225	+	} else {
	226	+	if ret.Get(0) != nil {
	227	+	r0 = ret.Get(0).([]byte)
	228	+	}
	229	+	}
	230	+	if returnFunc, ok := ret.Get(1).(func(context.Context, domain.Principal, string, string, int64, int, int) string); ok {
	231	+	r1 = returnFunc(ctx, p, owner, repo, number, page, perPage)
	232	+	} else {
	233	+	r1 = ret.Get(1).(string)
	234	+	}
	235	+	if returnFunc, ok := ret.Get(2).(func(context.Context, domain.Principal, string, string, int64, int, int) error); ok {
	236	+	r2 = returnFunc(ctx, p, owner, repo, number, page, perPage)
	237	+	} else {
	238	+	r2 = ret.Error(2)
	239	+	}
	240	+	return r0, r1, r2
	241	+	}
	242	+
	243	+	// GitHubAPIService_ListPullFiles_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ListPullFiles'
	244	+	type GitHubAPIService_ListPullFiles_Call struct {
	245	+	*mock.Call
	246	+	}
	247	+
	248	+	// ListPullFiles is a helper method to define mock.On call
	249	+	// - ctx context.Context
	250	+	// - p domain.Principal
	251	+	// - owner string
	252	+	// - repo string
	253	+	// - number int64
	254	+	// - page int
	255	+	// - perPage int
	256	+	func (_e GitHubAPIService_Expecter) ListPullFiles(ctx any, p any, owner any, repo any, number any, page any, perPage any) GitHubAPIService_ListPullFiles_Call {
	257	+	return &GitHubAPIService_ListPullFiles_Call{Call: _e.mock.On("ListPullFiles", ctx, p, owner, repo, number, page, perPage)}
	258	+	}
	259	+
	260	+	func (_c GitHubAPIService_ListPullFiles_Call) Run(run func(ctx context.Context, p domain.Principal, owner string, repo string, number int64, page int, perPage int)) GitHubAPIService_ListPullFiles_Call {
	261	+	_c.Call.Run(func(args mock.Arguments) {
	262	+	var arg0 context.Context
	263	+	if args[0] != nil {
	264	+	arg0 = args[0].(context.Context)
	265	+	}
	266	+	var arg1 domain.Principal
	267	+	if args[1] != nil {
	268	+	arg1 = args[1].(domain.Principal)
	269	+	}
	270	+	var arg2 string
	271	+	if args[2] != nil {
	272	+	arg2 = args[2].(string)
	273	+	}
	274	+	var arg3 string
	275	+	if args[3] != nil {
	276	+	arg3 = args[3].(string)
	277	+	}
	278	+	var arg4 int64
	279	+	if args[4] != nil {
	280	+	arg4 = args[4].(int64)
	281	+	}
	282	+	var arg5 int
	283	+	if args[5] != nil {
	284	+	arg5 = args[5].(int)
	285	+	}
	286	+	var arg6 int
	287	+	if args[6] != nil {
	288	+	arg6 = args[6].(int)
	289	+	}
	290	+	run(
	291	+	arg0,
	292	+	arg1,
	293	+	arg2,
	294	+	arg3,
	295	+	arg4,
	296	+	arg5,
	297	+	arg6,
	298	+	)
	299	+	})
	300	+	return _c
	301	+	}
	302	+
	303	+	func (_c GitHubAPIService_ListPullFiles_Call) Return(body []byte, link string, err error) GitHubAPIService_ListPullFiles_Call {
	304	+	_c.Call.Return(body, link, err)
	305	+	return _c
	306	+	}
	307	+
	308	+	func (_c GitHubAPIService_ListPullFiles_Call) RunAndReturn(run func(ctx context.Context, p domain.Principal, owner string, repo string, number int64, page int, perPage int) ([]byte, string, error)) GitHubAPIService_ListPullFiles_Call {
	309	+	_c.Call.Return(run)
	310	+	return _c
	311	+	}
	312	+
209	313		// Tarball provides a mock function for the type GitHubAPIService
210	314		func (_mock *GitHubAPIService) Tarball(ctx context.Context, p domain.Principal, owner string, repo string, ref string, begin func(filename string) io.Writer) error {
211	315		ret := _mock.Called(ctx, p, owner, repo, ref, begin)

M internal/ports/mocks/MergeRequestStore.go +240 -0


@@ -477,6 +477,246 @@ func (_c *MergeRequestStore_ListComments_Call) RunAndReturn(run func(ctx context
477	477		return _c
478	478		}
479	479
	480	+	// ListOpen provides a mock function for the type MergeRequestStore
	481	+	func (_mock *MergeRequestStore) ListOpen(ctx context.Context, repo domain.RepoName) ([]domain.MergeRequest, error) {
	482	+	ret := _mock.Called(ctx, repo)
	483	+
	484	+	if len(ret) == 0 {
	485	+	panic("no return value specified for ListOpen")
	486	+	}
	487	+
	488	+	var r0 []domain.MergeRequest
	489	+	var r1 error
	490	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName) ([]domain.MergeRequest, error)); ok {
	491	+	return returnFunc(ctx, repo)
	492	+	}
	493	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName) []domain.MergeRequest); ok {
	494	+	r0 = returnFunc(ctx, repo)
	495	+	} else {
	496	+	if ret.Get(0) != nil {
	497	+	r0 = ret.Get(0).([]domain.MergeRequest)
	498	+	}
	499	+	}
	500	+	if returnFunc, ok := ret.Get(1).(func(context.Context, domain.RepoName) error); ok {
	501	+	r1 = returnFunc(ctx, repo)
	502	+	} else {
	503	+	r1 = ret.Error(1)
	504	+	}
	505	+	return r0, r1
	506	+	}
	507	+
	508	+	// MergeRequestStore_ListOpen_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'ListOpen'
	509	+	type MergeRequestStore_ListOpen_Call struct {
	510	+	*mock.Call
	511	+	}
	512	+
	513	+	// ListOpen is a helper method to define mock.On call
	514	+	// - ctx context.Context
	515	+	// - repo domain.RepoName
	516	+	func (_e MergeRequestStore_Expecter) ListOpen(ctx any, repo any) MergeRequestStore_ListOpen_Call {
	517	+	return &MergeRequestStore_ListOpen_Call{Call: _e.mock.On("ListOpen", ctx, repo)}
	518	+	}
	519	+
	520	+	func (_c MergeRequestStore_ListOpen_Call) Run(run func(ctx context.Context, repo domain.RepoName)) MergeRequestStore_ListOpen_Call {
	521	+	_c.Call.Run(func(args mock.Arguments) {
	522	+	var arg0 context.Context
	523	+	if args[0] != nil {
	524	+	arg0 = args[0].(context.Context)
	525	+	}
	526	+	var arg1 domain.RepoName
	527	+	if args[1] != nil {
	528	+	arg1 = args[1].(domain.RepoName)
	529	+	}
	530	+	run(
	531	+	arg0,
	532	+	arg1,
	533	+	)
	534	+	})
	535	+	return _c
	536	+	}
	537	+
	538	+	func (_c MergeRequestStore_ListOpen_Call) Return(mergeRequests []domain.MergeRequest, err error) MergeRequestStore_ListOpen_Call {
	539	+	_c.Call.Return(mergeRequests, err)
	540	+	return _c
	541	+	}
	542	+
	543	+	func (_c MergeRequestStore_ListOpen_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName) ([]domain.MergeRequest, error)) MergeRequestStore_ListOpen_Call {
	544	+	_c.Call.Return(run)
	545	+	return _c
	546	+	}
	547	+
	548	+	// OpenBySource provides a mock function for the type MergeRequestStore
	549	+	func (_mock *MergeRequestStore) OpenBySource(ctx context.Context, repo domain.RepoName, branches []string, limit int) ([]int64, bool, error) {
	550	+	ret := _mock.Called(ctx, repo, branches, limit)
	551	+
	552	+	if len(ret) == 0 {
	553	+	panic("no return value specified for OpenBySource")
	554	+	}
	555	+
	556	+	var r0 []int64
	557	+	var r1 bool
	558	+	var r2 error
	559	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, []string, int) ([]int64, bool, error)); ok {
	560	+	return returnFunc(ctx, repo, branches, limit)
	561	+	}
	562	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, []string, int) []int64); ok {
	563	+	r0 = returnFunc(ctx, repo, branches, limit)
	564	+	} else {
	565	+	if ret.Get(0) != nil {
	566	+	r0 = ret.Get(0).([]int64)
	567	+	}
	568	+	}
	569	+	if returnFunc, ok := ret.Get(1).(func(context.Context, domain.RepoName, []string, int) bool); ok {
	570	+	r1 = returnFunc(ctx, repo, branches, limit)
	571	+	} else {
	572	+	r1 = ret.Get(1).(bool)
	573	+	}
	574	+	if returnFunc, ok := ret.Get(2).(func(context.Context, domain.RepoName, []string, int) error); ok {
	575	+	r2 = returnFunc(ctx, repo, branches, limit)
	576	+	} else {
	577	+	r2 = ret.Error(2)
	578	+	}
	579	+	return r0, r1, r2
	580	+	}
	581	+
	582	+	// MergeRequestStore_OpenBySource_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OpenBySource'
	583	+	type MergeRequestStore_OpenBySource_Call struct {
	584	+	*mock.Call
	585	+	}
	586	+
	587	+	// OpenBySource is a helper method to define mock.On call
	588	+	// - ctx context.Context
	589	+	// - repo domain.RepoName
	590	+	// - branches []string
	591	+	// - limit int
	592	+	func (_e MergeRequestStore_Expecter) OpenBySource(ctx any, repo any, branches any, limit any) MergeRequestStore_OpenBySource_Call {
	593	+	return &MergeRequestStore_OpenBySource_Call{Call: _e.mock.On("OpenBySource", ctx, repo, branches, limit)}
	594	+	}
	595	+
	596	+	func (_c MergeRequestStore_OpenBySource_Call) Run(run func(ctx context.Context, repo domain.RepoName, branches []string, limit int)) MergeRequestStore_OpenBySource_Call {
	597	+	_c.Call.Run(func(args mock.Arguments) {
	598	+	var arg0 context.Context
	599	+	if args[0] != nil {
	600	+	arg0 = args[0].(context.Context)
	601	+	}
	602	+	var arg1 domain.RepoName
	603	+	if args[1] != nil {
	604	+	arg1 = args[1].(domain.RepoName)
	605	+	}
	606	+	var arg2 []string
	607	+	if args[2] != nil {
	608	+	arg2 = args[2].([]string)
	609	+	}
	610	+	var arg3 int
	611	+	if args[3] != nil {
	612	+	arg3 = args[3].(int)
	613	+	}
	614	+	run(
	615	+	arg0,
	616	+	arg1,
	617	+	arg2,
	618	+	arg3,
	619	+	)
	620	+	})
	621	+	return _c
	622	+	}
	623	+
	624	+	func (_c MergeRequestStore_OpenBySource_Call) Return(numbers []int64, truncated bool, err error) MergeRequestStore_OpenBySource_Call {
	625	+	_c.Call.Return(numbers, truncated, err)
	626	+	return _c
	627	+	}
	628	+
	629	+	func (_c MergeRequestStore_OpenBySource_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, branches []string, limit int) ([]int64, bool, error)) MergeRequestStore_OpenBySource_Call {
	630	+	_c.Call.Return(run)
	631	+	return _c
	632	+	}
	633	+
	634	+	// OpenByTarget provides a mock function for the type MergeRequestStore
	635	+	func (_mock *MergeRequestStore) OpenByTarget(ctx context.Context, repo domain.RepoName, branches []string, limit int) ([]int64, bool, error) {
	636	+	ret := _mock.Called(ctx, repo, branches, limit)
	637	+
	638	+	if len(ret) == 0 {
	639	+	panic("no return value specified for OpenByTarget")
	640	+	}
	641	+
	642	+	var r0 []int64
	643	+	var r1 bool
	644	+	var r2 error
	645	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, []string, int) ([]int64, bool, error)); ok {
	646	+	return returnFunc(ctx, repo, branches, limit)
	647	+	}
	648	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, []string, int) []int64); ok {
	649	+	r0 = returnFunc(ctx, repo, branches, limit)
	650	+	} else {
	651	+	if ret.Get(0) != nil {
	652	+	r0 = ret.Get(0).([]int64)
	653	+	}
	654	+	}
	655	+	if returnFunc, ok := ret.Get(1).(func(context.Context, domain.RepoName, []string, int) bool); ok {
	656	+	r1 = returnFunc(ctx, repo, branches, limit)
	657	+	} else {
	658	+	r1 = ret.Get(1).(bool)
	659	+	}
	660	+	if returnFunc, ok := ret.Get(2).(func(context.Context, domain.RepoName, []string, int) error); ok {
	661	+	r2 = returnFunc(ctx, repo, branches, limit)
	662	+	} else {
	663	+	r2 = ret.Error(2)
	664	+	}
	665	+	return r0, r1, r2
	666	+	}
	667	+
	668	+	// MergeRequestStore_OpenByTarget_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OpenByTarget'
	669	+	type MergeRequestStore_OpenByTarget_Call struct {
	670	+	*mock.Call
	671	+	}
	672	+
	673	+	// OpenByTarget is a helper method to define mock.On call
	674	+	// - ctx context.Context
	675	+	// - repo domain.RepoName
	676	+	// - branches []string
	677	+	// - limit int
	678	+	func (_e MergeRequestStore_Expecter) OpenByTarget(ctx any, repo any, branches any, limit any) MergeRequestStore_OpenByTarget_Call {
	679	+	return &MergeRequestStore_OpenByTarget_Call{Call: _e.mock.On("OpenByTarget", ctx, repo, branches, limit)}
	680	+	}
	681	+
	682	+	func (_c MergeRequestStore_OpenByTarget_Call) Run(run func(ctx context.Context, repo domain.RepoName, branches []string, limit int)) MergeRequestStore_OpenByTarget_Call {
	683	+	_c.Call.Run(func(args mock.Arguments) {
	684	+	var arg0 context.Context
	685	+	if args[0] != nil {
	686	+	arg0 = args[0].(context.Context)
	687	+	}
	688	+	var arg1 domain.RepoName
	689	+	if args[1] != nil {
	690	+	arg1 = args[1].(domain.RepoName)
	691	+	}
	692	+	var arg2 []string
	693	+	if args[2] != nil {
	694	+	arg2 = args[2].([]string)
	695	+	}
	696	+	var arg3 int
	697	+	if args[3] != nil {
	698	+	arg3 = args[3].(int)
	699	+	}
	700	+	run(
	701	+	arg0,
	702	+	arg1,
	703	+	arg2,
	704	+	arg3,
	705	+	)
	706	+	})
	707	+	return _c
	708	+	}
	709	+
	710	+	func (_c MergeRequestStore_OpenByTarget_Call) Return(numbers []int64, truncated bool, err error) MergeRequestStore_OpenByTarget_Call {
	711	+	_c.Call.Return(numbers, truncated, err)
	712	+	return _c
	713	+	}
	714	+
	715	+	func (_c MergeRequestStore_OpenByTarget_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, branches []string, limit int) ([]int64, bool, error)) MergeRequestStore_OpenByTarget_Call {
	716	+	_c.Call.Return(run)
	717	+	return _c
	718	+	}
	719	+
480	720		// SaveComment provides a mock function for the type MergeRequestStore
481	721		func (_mock *MergeRequestStore) SaveComment(ctx context.Context, mr domain.MergeRequest, c domain.Comment) error {
482	722		ret := _mock.Called(ctx, mr, c)

A internal/ports/mocks/PullGate.go +117 -0

@@ -0,0 +1,117 @@
	1	+	// Code generated by mockery; DO NOT EDIT.
	2	+	// github.com/vektra/mockery
	3	+	// template: testify
	4	+
	5	+	package mocks
	6	+
	7	+	import (
	8	+	"context"
	9	+
	10	+	"bnema.dev/ermite/internal/domain"
	11	+	mock "github.com/stretchr/testify/mock"
	12	+	)
	13	+
	14	+	// NewPullGate creates a new instance of PullGate. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
	15	+	// The first argument is typically a *testing.T value.
	16	+	func NewPullGate(t interface {
	17	+	mock.TestingT
	18	+	Cleanup(func())
	19	+	}) *PullGate {
	20	+	if helper, ok := t.(interface{ Helper() }); ok {
	21	+	helper.Helper()
	22	+	}
	23	+
	24	+	mock := &PullGate{}
	25	+	mock.Mock.Test(t)
	26	+
	27	+	t.Cleanup(func() {
	28	+	if helper, ok := t.(interface{ Helper() }); ok {
	29	+	helper.Helper()
	30	+	}
	31	+	mock.AssertExpectations(t)
	32	+	})
	33	+
	34	+	return mock
	35	+	}
	36	+
	37	+	// PullGate is an autogenerated mock type for the PullGate type
	38	+	type PullGate struct {
	39	+	mock.Mock
	40	+	}
	41	+
	42	+	type PullGate_Expecter struct {
	43	+	mock *mock.Mock
	44	+	}
	45	+
	46	+	func (_m PullGate) EXPECT() PullGate_Expecter {
	47	+	return &PullGate_Expecter{mock: &_m.Mock}
	48	+	}
	49	+
	50	+	// WhileOpen provides a mock function for the type PullGate
	51	+	func (_mock *PullGate) WhileOpen(ctx context.Context, repo domain.RepoName, number int64, fn func() error) error {
	52	+	ret := _mock.Called(ctx, repo, number, fn)
	53	+
	54	+	if len(ret) == 0 {
	55	+	panic("no return value specified for WhileOpen")
	56	+	}
	57	+
	58	+	var r0 error
	59	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, int64, func() error) error); ok {
	60	+	r0 = returnFunc(ctx, repo, number, fn)
	61	+	} else {
	62	+	r0 = ret.Error(0)
	63	+	}
	64	+	return r0
	65	+	}
	66	+
	67	+	// PullGate_WhileOpen_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'WhileOpen'
	68	+	type PullGate_WhileOpen_Call struct {
	69	+	*mock.Call
	70	+	}
	71	+
	72	+	// WhileOpen is a helper method to define mock.On call
	73	+	// - ctx context.Context
	74	+	// - repo domain.RepoName
	75	+	// - number int64
	76	+	// - fn func() error
	77	+	func (_e PullGate_Expecter) WhileOpen(ctx any, repo any, number any, fn any) PullGate_WhileOpen_Call {
	78	+	return &PullGate_WhileOpen_Call{Call: _e.mock.On("WhileOpen", ctx, repo, number, fn)}
	79	+	}
	80	+
	81	+	func (_c PullGate_WhileOpen_Call) Run(run func(ctx context.Context, repo domain.RepoName, number int64, fn func() error)) PullGate_WhileOpen_Call {
	82	+	_c.Call.Run(func(args mock.Arguments) {
	83	+	var arg0 context.Context
	84	+	if args[0] != nil {
	85	+	arg0 = args[0].(context.Context)
	86	+	}
	87	+	var arg1 domain.RepoName
	88	+	if args[1] != nil {
	89	+	arg1 = args[1].(domain.RepoName)
	90	+	}
	91	+	var arg2 int64
	92	+	if args[2] != nil {
	93	+	arg2 = args[2].(int64)
	94	+	}
	95	+	var arg3 func() error
	96	+	if args[3] != nil {
	97	+	arg3 = args[3].(func() error)
	98	+	}
	99	+	run(
	100	+	arg0,
	101	+	arg1,
	102	+	arg2,
	103	+	arg3,
	104	+	)
	105	+	})
	106	+	return _c
	107	+	}
	108	+
	109	+	func (_c PullGate_WhileOpen_Call) Return(err error) PullGate_WhileOpen_Call {
	110	+	_c.Call.Return(err)
	111	+	return _c
	112	+	}
	113	+
	114	+	func (_c PullGate_WhileOpen_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, number int64, fn func() error) error) PullGate_WhileOpen_Call {
	115	+	_c.Call.Return(run)
	116	+	return _c
	117	+	}

A internal/ports/mocks/PullGit.go +273 -0

@@ -0,0 +1,273 @@
	1	+	// Code generated by mockery; DO NOT EDIT.
	2	+	// github.com/vektra/mockery
	3	+	// template: testify
	4	+
	5	+	package mocks
	6	+
	7	+	import (
	8	+	"context"
	9	+
	10	+	"bnema.dev/ermite/internal/domain"
	11	+	mock "github.com/stretchr/testify/mock"
	12	+	)
	13	+
	14	+	// NewPullGit creates a new instance of PullGit. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
	15	+	// The first argument is typically a *testing.T value.
	16	+	func NewPullGit(t interface {
	17	+	mock.TestingT
	18	+	Cleanup(func())
	19	+	}) *PullGit {
	20	+	if helper, ok := t.(interface{ Helper() }); ok {
	21	+	helper.Helper()
	22	+	}
	23	+
	24	+	mock := &PullGit{}
	25	+	mock.Mock.Test(t)
	26	+
	27	+	t.Cleanup(func() {
	28	+	if helper, ok := t.(interface{ Helper() }); ok {
	29	+	helper.Helper()
	30	+	}
	31	+	mock.AssertExpectations(t)
	32	+	})
	33	+
	34	+	return mock
	35	+	}
	36	+
	37	+	// PullGit is an autogenerated mock type for the PullGit type
	38	+	type PullGit struct {
	39	+	mock.Mock
	40	+	}
	41	+
	42	+	type PullGit_Expecter struct {
	43	+	mock *mock.Mock
	44	+	}
	45	+
	46	+	func (_m PullGit) EXPECT() PullGit_Expecter {
	47	+	return &PullGit_Expecter{mock: &_m.Mock}
	48	+	}
	49	+
	50	+	// PullRef provides a mock function for the type PullGit
	51	+	func (_mock *PullGit) PullRef(ctx context.Context, repo domain.RepoName, ref string) (domain.ObjectID, error) {
	52	+	ret := _mock.Called(ctx, repo, ref)
	53	+
	54	+	if len(ret) == 0 {
	55	+	panic("no return value specified for PullRef")
	56	+	}
	57	+
	58	+	var r0 domain.ObjectID
	59	+	var r1 error
	60	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string) (domain.ObjectID, error)); ok {
	61	+	return returnFunc(ctx, repo, ref)
	62	+	}
	63	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string) domain.ObjectID); ok {
	64	+	r0 = returnFunc(ctx, repo, ref)
	65	+	} else {
	66	+	r0 = ret.Get(0).(domain.ObjectID)
	67	+	}
	68	+	if returnFunc, ok := ret.Get(1).(func(context.Context, domain.RepoName, string) error); ok {
	69	+	r1 = returnFunc(ctx, repo, ref)
	70	+	} else {
	71	+	r1 = ret.Error(1)
	72	+	}
	73	+	return r0, r1
	74	+	}
	75	+
	76	+	// PullGit_PullRef_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'PullRef'
	77	+	type PullGit_PullRef_Call struct {
	78	+	*mock.Call
	79	+	}
	80	+
	81	+	// PullRef is a helper method to define mock.On call
	82	+	// - ctx context.Context
	83	+	// - repo domain.RepoName
	84	+	// - ref string
	85	+	func (_e PullGit_Expecter) PullRef(ctx any, repo any, ref any) PullGit_PullRef_Call {
	86	+	return &PullGit_PullRef_Call{Call: _e.mock.On("PullRef", ctx, repo, ref)}
	87	+	}
	88	+
	89	+	func (_c PullGit_PullRef_Call) Run(run func(ctx context.Context, repo domain.RepoName, ref string)) PullGit_PullRef_Call {
	90	+	_c.Call.Run(func(args mock.Arguments) {
	91	+	var arg0 context.Context
	92	+	if args[0] != nil {
	93	+	arg0 = args[0].(context.Context)
	94	+	}
	95	+	var arg1 domain.RepoName
	96	+	if args[1] != nil {
	97	+	arg1 = args[1].(domain.RepoName)
	98	+	}
	99	+	var arg2 string
	100	+	if args[2] != nil {
	101	+	arg2 = args[2].(string)
	102	+	}
	103	+	run(
	104	+	arg0,
	105	+	arg1,
	106	+	arg2,
	107	+	)
	108	+	})
	109	+	return _c
	110	+	}
	111	+
	112	+	func (_c PullGit_PullRef_Call) Return(objectID domain.ObjectID, err error) PullGit_PullRef_Call {
	113	+	_c.Call.Return(objectID, err)
	114	+	return _c
	115	+	}
	116	+
	117	+	func (_c PullGit_PullRef_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, ref string) (domain.ObjectID, error)) PullGit_PullRef_Call {
	118	+	_c.Call.Return(run)
	119	+	return _c
	120	+	}
	121	+
	122	+	// SetPullRef provides a mock function for the type PullGit
	123	+	func (_mock *PullGit) SetPullRef(ctx context.Context, repo domain.RepoName, ref string, id domain.ObjectID) error {
	124	+	ret := _mock.Called(ctx, repo, ref, id)
	125	+
	126	+	if len(ret) == 0 {
	127	+	panic("no return value specified for SetPullRef")
	128	+	}
	129	+
	130	+	var r0 error
	131	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, string, domain.ObjectID) error); ok {
	132	+	r0 = returnFunc(ctx, repo, ref, id)
	133	+	} else {
	134	+	r0 = ret.Error(0)
	135	+	}
	136	+	return r0
	137	+	}
	138	+
	139	+	// PullGit_SetPullRef_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'SetPullRef'
	140	+	type PullGit_SetPullRef_Call struct {
	141	+	*mock.Call
	142	+	}
	143	+
	144	+	// SetPullRef is a helper method to define mock.On call
	145	+	// - ctx context.Context
	146	+	// - repo domain.RepoName
	147	+	// - ref string
	148	+	// - id domain.ObjectID
	149	+	func (_e PullGit_Expecter) SetPullRef(ctx any, repo any, ref any, id any) PullGit_SetPullRef_Call {
	150	+	return &PullGit_SetPullRef_Call{Call: _e.mock.On("SetPullRef", ctx, repo, ref, id)}
	151	+	}
	152	+
	153	+	func (_c PullGit_SetPullRef_Call) Run(run func(ctx context.Context, repo domain.RepoName, ref string, id domain.ObjectID)) PullGit_SetPullRef_Call {
	154	+	_c.Call.Run(func(args mock.Arguments) {
	155	+	var arg0 context.Context
	156	+	if args[0] != nil {
	157	+	arg0 = args[0].(context.Context)
	158	+	}
	159	+	var arg1 domain.RepoName
	160	+	if args[1] != nil {
	161	+	arg1 = args[1].(domain.RepoName)
	162	+	}
	163	+	var arg2 string
	164	+	if args[2] != nil {
	165	+	arg2 = args[2].(string)
	166	+	}
	167	+	var arg3 domain.ObjectID
	168	+	if args[3] != nil {
	169	+	arg3 = args[3].(domain.ObjectID)
	170	+	}
	171	+	run(
	172	+	arg0,
	173	+	arg1,
	174	+	arg2,
	175	+	arg3,
	176	+	)
	177	+	})
	178	+	return _c
	179	+	}
	180	+
	181	+	func (_c PullGit_SetPullRef_Call) Return(err error) PullGit_SetPullRef_Call {
	182	+	_c.Call.Return(err)
	183	+	return _c
	184	+	}
	185	+
	186	+	func (_c PullGit_SetPullRef_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, ref string, id domain.ObjectID) error) PullGit_SetPullRef_Call {
	187	+	_c.Call.Return(run)
	188	+	return _c
	189	+	}
	190	+
	191	+	// TestMerge provides a mock function for the type PullGit
	192	+	func (_mock *PullGit) TestMerge(ctx context.Context, repo domain.RepoName, base domain.ObjectID, head domain.ObjectID) (domain.ObjectID, bool, error) {
	193	+	ret := _mock.Called(ctx, repo, base, head)
	194	+
	195	+	if len(ret) == 0 {
	196	+	panic("no return value specified for TestMerge")
	197	+	}
	198	+
	199	+	var r0 domain.ObjectID
	200	+	var r1 bool
	201	+	var r2 error
	202	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, domain.ObjectID, domain.ObjectID) (domain.ObjectID, bool, error)); ok {
	203	+	return returnFunc(ctx, repo, base, head)
	204	+	}
	205	+	if returnFunc, ok := ret.Get(0).(func(context.Context, domain.RepoName, domain.ObjectID, domain.ObjectID) domain.ObjectID); ok {
	206	+	r0 = returnFunc(ctx, repo, base, head)
	207	+	} else {
	208	+	r0 = ret.Get(0).(domain.ObjectID)
	209	+	}
	210	+	if returnFunc, ok := ret.Get(1).(func(context.Context, domain.RepoName, domain.ObjectID, domain.ObjectID) bool); ok {
	211	+	r1 = returnFunc(ctx, repo, base, head)
	212	+	} else {
	213	+	r1 = ret.Get(1).(bool)
	214	+	}
	215	+	if returnFunc, ok := ret.Get(2).(func(context.Context, domain.RepoName, domain.ObjectID, domain.ObjectID) error); ok {
	216	+	r2 = returnFunc(ctx, repo, base, head)
	217	+	} else {
	218	+	r2 = ret.Error(2)
	219	+	}
	220	+	return r0, r1, r2
	221	+	}
	222	+
	223	+	// PullGit_TestMerge_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'TestMerge'
	224	+	type PullGit_TestMerge_Call struct {
	225	+	*mock.Call
	226	+	}
	227	+
	228	+	// TestMerge is a helper method to define mock.On call
	229	+	// - ctx context.Context
	230	+	// - repo domain.RepoName
	231	+	// - base domain.ObjectID
	232	+	// - head domain.ObjectID
	233	+	func (_e PullGit_Expecter) TestMerge(ctx any, repo any, base any, head any) PullGit_TestMerge_Call {
	234	+	return &PullGit_TestMerge_Call{Call: _e.mock.On("TestMerge", ctx, repo, base, head)}
	235	+	}
	236	+
	237	+	func (_c PullGit_TestMerge_Call) Run(run func(ctx context.Context, repo domain.RepoName, base domain.ObjectID, head domain.ObjectID)) PullGit_TestMerge_Call {
	238	+	_c.Call.Run(func(args mock.Arguments) {
	239	+	var arg0 context.Context
	240	+	if args[0] != nil {
	241	+	arg0 = args[0].(context.Context)
	242	+	}
	243	+	var arg1 domain.RepoName
	244	+	if args[1] != nil {
	245	+	arg1 = args[1].(domain.RepoName)
	246	+	}
	247	+	var arg2 domain.ObjectID
	248	+	if args[2] != nil {
	249	+	arg2 = args[2].(domain.ObjectID)
	250	+	}
	251	+	var arg3 domain.ObjectID
	252	+	if args[3] != nil {
	253	+	arg3 = args[3].(domain.ObjectID)
	254	+	}
	255	+	run(
	256	+	arg0,
	257	+	arg1,
	258	+	arg2,
	259	+	arg3,
	260	+	)
	261	+	})
	262	+	return _c
	263	+	}
	264	+
	265	+	func (_c PullGit_TestMerge_Call) Return(tree domain.ObjectID, clean bool, err error) PullGit_TestMerge_Call {
	266	+	_c.Call.Return(tree, clean, err)
	267	+	return _c
	268	+	}
	269	+
	270	+	func (_c PullGit_TestMerge_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, base domain.ObjectID, head domain.ObjectID) (domain.ObjectID, bool, error)) PullGit_TestMerge_Call {
	271	+	_c.Call.Return(run)
	272	+	return _c
	273	+	}

A internal/ports/mocks/PullRequestTrigger.go +107 -0

@@ -0,0 +1,107 @@
	1	+	// Code generated by mockery; DO NOT EDIT.
	2	+	// github.com/vektra/mockery
	3	+	// template: testify
	4	+
	5	+	package mocks
	6	+
	7	+	import (
	8	+	"context"
	9	+
	10	+	"bnema.dev/ermite/internal/domain"
	11	+	"bnema.dev/ermite/internal/ports"
	12	+	mock "github.com/stretchr/testify/mock"
	13	+	)
	14	+
	15	+	// NewPullRequestTrigger creates a new instance of PullRequestTrigger. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
	16	+	// The first argument is typically a *testing.T value.
	17	+	func NewPullRequestTrigger(t interface {
	18	+	mock.TestingT
	19	+	Cleanup(func())
	20	+	}) *PullRequestTrigger {
	21	+	if helper, ok := t.(interface{ Helper() }); ok {
	22	+	helper.Helper()
	23	+	}
	24	+
	25	+	mock := &PullRequestTrigger{}
	26	+	mock.Mock.Test(t)
	27	+
	28	+	t.Cleanup(func() {
	29	+	if helper, ok := t.(interface{ Helper() }); ok {
	30	+	helper.Helper()
	31	+	}
	32	+	mock.AssertExpectations(t)
	33	+	})
	34	+
	35	+	return mock
	36	+	}
	37	+
	38	+	// PullRequestTrigger is an autogenerated mock type for the PullRequestTrigger type
	39	+	type PullRequestTrigger struct {
	40	+	mock.Mock
	41	+	}
	42	+
	43	+	type PullRequestTrigger_Expecter struct {
	44	+	mock *mock.Mock
	45	+	}
	46	+
	47	+	func (_m PullRequestTrigger) EXPECT() PullRequestTrigger_Expecter {
	48	+	return &PullRequestTrigger_Expecter{mock: &_m.Mock}
	49	+	}
	50	+
	51	+	// OnPullRequest provides a mock function for the type PullRequestTrigger
	52	+	func (_mock *PullRequestTrigger) OnPullRequest(ctx context.Context, repo domain.RepoName, ev ports.PullRequestEvent, p domain.Principal) {
	53	+	_mock.Called(ctx, repo, ev, p)
	54	+	return
	55	+	}
	56	+
	57	+	// PullRequestTrigger_OnPullRequest_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'OnPullRequest'
	58	+	type PullRequestTrigger_OnPullRequest_Call struct {
	59	+	*mock.Call
	60	+	}
	61	+
	62	+	// OnPullRequest is a helper method to define mock.On call
	63	+	// - ctx context.Context
	64	+	// - repo domain.RepoName
	65	+	// - ev ports.PullRequestEvent
	66	+	// - p domain.Principal
	67	+	func (_e PullRequestTrigger_Expecter) OnPullRequest(ctx any, repo any, ev any, p any) PullRequestTrigger_OnPullRequest_Call {
	68	+	return &PullRequestTrigger_OnPullRequest_Call{Call: _e.mock.On("OnPullRequest", ctx, repo, ev, p)}
	69	+	}
	70	+
	71	+	func (_c PullRequestTrigger_OnPullRequest_Call) Run(run func(ctx context.Context, repo domain.RepoName, ev ports.PullRequestEvent, p domain.Principal)) PullRequestTrigger_OnPullRequest_Call {
	72	+	_c.Call.Run(func(args mock.Arguments) {
	73	+	var arg0 context.Context
	74	+	if args[0] != nil {
	75	+	arg0 = args[0].(context.Context)
	76	+	}
	77	+	var arg1 domain.RepoName
	78	+	if args[1] != nil {
	79	+	arg1 = args[1].(domain.RepoName)
	80	+	}
	81	+	var arg2 ports.PullRequestEvent
	82	+	if args[2] != nil {
	83	+	arg2 = args[2].(ports.PullRequestEvent)
	84	+	}
	85	+	var arg3 domain.Principal
	86	+	if args[3] != nil {
	87	+	arg3 = args[3].(domain.Principal)
	88	+	}
	89	+	run(
	90	+	arg0,
	91	+	arg1,
	92	+	arg2,
	93	+	arg3,
	94	+	)
	95	+	})
	96	+	return _c
	97	+	}
	98	+
	99	+	func (_c PullRequestTrigger_OnPullRequest_Call) Return() PullRequestTrigger_OnPullRequest_Call {
	100	+	_c.Call.Return()
	101	+	return _c
	102	+	}
	103	+
	104	+	func (_c PullRequestTrigger_OnPullRequest_Call) RunAndReturn(run func(ctx context.Context, repo domain.RepoName, ev ports.PullRequestEvent, p domain.Principal)) PullRequestTrigger_OnPullRequest_Call {
	105	+	_c.Run(run)
	106	+	return _c
	107	+	}

M internal/ports/ports.go +14 -0


@@ -245,6 +245,20 @@ type MergeRequestStore interface {
245	245		// List returns every merge request of the repository, highest number
246	246		// first.
247	247		List(ctx context.Context, repo domain.RepoName) ([]domain.MergeRequest, error)
	248	+	// ListOpen returns the open merge requests of the repository, highest
	249	+	// number first, reading only the open ones (through an index).
	250	+	ListOpen(ctx context.Context, repo domain.RepoName) ([]domain.MergeRequest, error)
	251	+	// OpenBySource returns the numbers of the open merge requests whose source
	252	+	// branch is one of branches (short names), from an index of keys: a push
	253	+	// looks up the branches it moved, whatever the number of merge requests the
	254	+	// repository ever had. No request is read. It stops after limit numbers (a
	255	+	// limit of 0 or less means none) and reports truncated when there are more,
	256	+	// without counting them; the context is checked between keys, and a
	257	+	// cancelled one ends the lookup with its error and what was found so far.
	258	+	OpenBySource(ctx context.Context, repo domain.RepoName, branches []string, limit int) (numbers []int64, truncated bool, err error)
	259	+	// OpenByTarget is OpenBySource for the open merge requests whose target
	260	+	// branch is one of branches.
	261	+	OpenByTarget(ctx context.Context, repo domain.RepoName, branches []string, limit int) (numbers []int64, truncated bool, err error)
248	262		// Update replaces the stored merge request. It returns domain.ErrNotFound
249	263		// if there is none and domain.ErrConflict if another write changed it
250	264		// since mr was read.

M internal/ports/services.go +8 -0


@@ -76,6 +76,14 @@ type GitHubAPIService interface {
76	76		// TarballName is Tarball for a HEAD request: the same access and the same
77	77		// resolution of ref, then the file name, without producing the archive.
78	78		TarballName(ctx context.Context, p domain.Principal, owner, repo, ref string) (string, error)
	79	+	// ListPullFiles answers GET /repos/{owner}/{repo}/pulls/{number}/files: the
	80	+	// JSON array of the files of page (1-based) of perPage files (1 to 100) of
	81	+	// the merge request, and the Link header value ("" for a single page). A
	82	+	// public repository is read by anyone; a private one by a principal that
	83	+	// reads it. The job token of the repository needs pull-requests: read; one
	84	+	// that reads the repository without it gets domain.ErrForbidden (GitHub: 403
	85	+	// "Resource not accessible by integration").
	86	+	ListPullFiles(ctx context.Context, p domain.Principal, owner, repo string, number int64, page, perPage int) (body []byte, link string, err error)
79	87		}
80	88
81	89		// BrowseService is the read side of a repository's content. *usecase.Browse

M internal/usecase/ci.go +44 -5


@@ -62,8 +62,36 @@ type CIConfig struct {
62	62		Reader ports.RepoReader
63	63		Git ports.MergeGit
64	64		Diff ports.CIGit
65		-	Bus ports.EventBus
66		-	Clock ports.Clock
	65	+	// MRs, Pull and Committer give the workflows their pull_request event and
	66	+	// the refs/pull/<n>/{head,merge} refs of the merge requests (CI-3, D5);
	67	+	// without MRs or Pull the merge requests start no run and have no such
	68	+	// refs. Committer is the identity of the test merge commits (default
	69	+	// "ermite"). Fresh, when set, is told when the refs of a repository changed
	70	+	// so that the reader sees them at once.
	71	+	MRs ports.MergeRequestStore
	72	+	Pull ports.PullGit
	73	+	Committer domain.Identity
	74	+	Fresh ports.RepoFreshener
	75	+	// PullStage, when set, is called by the workers of the merge requests at
	76	+	// the stages "event" (an activity is taken), "computed" (its merge is
	77	+	// computed, before it is published) and "refresh" (a refresh is taken).
	78	+	// Tests use it to interleave other changes.
	79	+	PullStage func(stage string)
	80	+	// PullRefreshes bounds the merge requests waiting for a refresh of their refs
	81	+	// (default DefaultCIPullRefreshes); past it repositories are rescanned. It
	82	+	// does not bound the activities (synchronize runs), which have their own queue
	83	+	// with the overload policy of the push triggers.
	84	+	PullRefreshes int
	85	+	// PullAttempts bounds the merge requests whose last conflict is remembered
	86	+	// (default DefaultCIPullAttempts), so that a rescan does not try a conflicting
	87	+	// merge again until a branch moves.
	88	+	PullAttempts int
	89	+	// PullSnapshot bounds the merge requests one push takes the snapshot of, in the
	90	+	// push handler (default DefaultCIPullSnapshot): past it the others are skipped
	91	+	// with an error in the log and the repository is rescanned for its refs.
	92	+	PullSnapshot int
	93	+	Bus ports.EventBus
	94	+	Clock ports.Clock
67	95		// Auth re-validates the token of a log stream; nil skips it.
68	96		Auth ports.Revalidator
69	97		// Remote lists the refs of github.com repositories for ResolveActions; nil

@@ -134,6 +162,12 @@ func (c CIConfig) withDefaults() CIConfig {
134	162		c.RemoteConcurrency = DefaultCIRemoteConcurrency
135	163		}
136	164		def(&c.ResolveTimeout, DefaultCIResolveTimeout)
	165	+	if c.PullSnapshot <= 0 {
	166	+	c.PullSnapshot = DefaultCIPullSnapshot
	167	+	}
	168	+	if c.PullRefreshes <= 0 {
	169	+	c.PullRefreshes = DefaultCIPullRefreshes
	170	+	}
137	171		if c.TriggerQueue <= 0 {
138	172		c.TriggerQueue = DefaultCITriggerQueue
139	173		}

@@ -154,6 +188,9 @@ type CI struct {
154	188		cfg CIConfig
155	189		boot time.Time
156	190		queue chan ciTrigger
	191	+	pull *pullQueue // the work of the merge requests (ci_pull.go)
	192	+
	193	+	pullFileLists pullFilesCache // the files of the merge requests (ci_pullfiles.go)
157	194
158	195		stop context.CancelFunc
159	196		ctx context.Context

@@ -206,7 +243,7 @@ func NewCI(ctx context.Context, cfg CIConfig) *CI {
206	243		cfg = cfg.withDefaults()
207	244		run, stop := context.WithCancel(context.WithoutCancel(ctx))
208	245		return &CI{
209		-	cfg: cfg, boot: cfg.Clock.Now(), queue: make(chan ciTrigger, cfg.TriggerQueue),
	246	+	cfg: cfg, boot: cfg.Clock.Now(), queue: make(chan ciTrigger, cfg.TriggerQueue), pull: newPullQueue(cfg.TriggerQueue, cfg.PullRefreshes, cfg.PullAttempts),
210	247		ctx: run, stop: stop, flows: newWfCache(64), remoteSlots: make(chan struct{}, cfg.RemoteConcurrency),
211	248		heartbeats: map[string]time.Time{}, seen: map[string]time.Time{}, polls: map[string]pollRecord{},
212	249		archiveTokens: map[string]map[string]string{},

@@ -217,7 +254,9 @@ func NewCI(ctx context.Context, cfg CIConfig) *CI {
217	254		// Start runs the trigger worker and the watchdog, and plans again the runs
218	255		// that were open when the server stopped.
219	256		func (c *CI) Start() {
220		-	c.wg.Add(2)
	257	+	c.wg.Add(4)
	258	+	go c.pullLoop()
	259	+	go c.refreshLoop()
221	260		go c.triggerLoop()
222	261		go c.watchdog()
223	262		c.wg.Add(1)

@@ -374,7 +413,7 @@ func (rs runState) touch(j domain.CIJob) { rs.dirty[j.ID] = true }
374	413		// checkOf builds the check of a job.
375	414		func (rs *runState) checkOf(j domain.CIJob) domain.CICheck {
376	415		return domain.CICheck{
377		-	ID: j.ID, Repo: rs.run.Repo, Sha: rs.run.Sha, Name: j.Name, Status: j.Status(), Conclusion: j.Conclusion,
	416	+	ID: j.ID, Repo: rs.run.Repo, Sha: rs.run.CheckSha(), Name: j.Name, Status: j.Status(), Conclusion: j.Conclusion,
378	417		RunID: rs.run.ID, JobID: j.ID, Attempt: rs.run.Attempt, WorkflowName: rs.run.WorkflowName,
379	418		CreatedAt: j.CreatedAt, StartedAt: j.StartedAt, CompletedAt: j.CompletedAt,
380	419		}

M internal/usecase/ci_github.go +24 -4


@@ -264,9 +264,11 @@ func person(name, email string) map[string]any { return map[string]any{"name": n
264	264		// The commits of the event come from f.payload, built once per ref; without
265	265		// one they are read from Git here.
266	266		func (c *CI) githubJSON(ctx context.Context, repo domain.Repo, run domain.CIRun, f pushFacts) string {
	267	+	if f.pull != nil {
	268	+	return c.pullGithubJSON(repo, run, f)
	269	+	}
267	270		u := c.urls(repo)
268	271		id := c.identity(run.Actor)
269		-	owner := c.identity("owner")
270	272		refName, refType := strings.TrimPrefix(run.Ref, "refs/heads/"), "branch"
271	273		if f.isTag {
272	274		refName, refType = strings.TrimPrefix(run.Ref, "refs/tags/"), "tag"

@@ -293,6 +295,24 @@ func (c *CI) githubJSON(ctx context.Context, repo domain.Repo, run domain.CIRun,
293	295		"repository": c.repoObj(repo, u, f.defaultBranch, run.CreatedAt),
294	296		"sender": u.userObj(id, false),
295	297		}
	298	+	return c.githubContext(repo, run, ghEvent{name: "push", refName: refName, refType: refType, payload: event})
	299	+	}
	300	+
	301	+	// ghEvent is what differs between the github contexts of the events.
	302	+	type ghEvent struct {
	303	+	name, refName, refType string
	304	+	// headRef and baseRef are github.head_ref and github.base_ref: set for a
	305	+	// pull_request only.
	306	+	headRef, baseRef string
	307	+	payload map[string]any
	308	+	}
	309	+
	310	+	// githubContext assembles the "github" context of a run in GitHub's key order
	311	+	// around the payload of the event.
	312	+	func (c *CI) githubContext(repo domain.Repo, run domain.CIRun, e ghEvent) string {
	313	+	u := c.urls(repo)
	314	+	id := c.identity(run.Actor)
	315	+	owner := c.identity("owner")
296	316		repoID := strconv.FormatInt(stableID("repo", u.full), 10)
297	317		gh := ordered{
298	318		{"job", ""}, {"ref", run.Ref}, {"sha", run.Sha}, {"repository", u.full}, {"repository_owner", c.cfg.Owner},

@@ -301,9 +321,9 @@ func (c *CI) githubJSON(ctx context.Context, repo domain.Repo, run domain.CIRun,
301	321		{"retention_days", strconv.Itoa(c.retentionDays())}, {"run_attempt", "1"},
302	322		{"artifact_cache_size_limit", ghCacheLimit}, {"repository_visibility", string(repo.Visibility)},
303	323		{"actor_id", strconv.FormatInt(id.id, 10)}, {"actor", id.login}, {"workflow", run.WorkflowName},
304		-	{"head_ref", ""}, {"base_ref", ""}, {"event_name", "push"}, {"server_url", u.server},
305		-	{"api_url", u.api}, {"graphql_url", u.api + "/graphql"}, {"ref_name", refName}, {"ref_protected", false},
306		-	{"ref_type", refType}, {"secret_source", ghSecretSource}, {"event", event},
	324	+	{"head_ref", e.headRef}, {"base_ref", e.baseRef}, {"event_name", e.name}, {"server_url", u.server},
	325	+	{"api_url", u.api}, {"graphql_url", u.api + "/graphql"}, {"ref_name", e.refName}, {"ref_protected", false},
	326	+	{"ref_type", e.refType}, {"secret_source", ghSecretSource}, {"event", e.payload},
307	327		{"workflow_ref", u.full + "/" + run.WorkflowPath + "@" + run.Ref}, {"workflow_sha", run.Sha},
308	328		{"repository_id", repoID}, {"triggering_actor", id.login},
309	329		}

A internal/usecase/ci_pull.go +885 -0

@@ -0,0 +1,885 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package usecase
	4	+
	5	+	import (
	6	+	"context"
	7	+	"errors"
	8	+	"fmt"
	9	+	"runtime/debug"
	10	+	"slices"
	11	+	"strconv"
	12	+	"strings"
	13	+	"sync"
	14	+	"time"
	15	+
	16	+	"github.com/bnema/zerowrap"
	17	+
	18	+	"bnema.dev/ermite-actions/workflow"
	19	+	"bnema.dev/ermite/internal/domain"
	20	+	"bnema.dev/ermite/internal/ports"
	21	+	)
	22	+
	23	+	// The pull_request event (CI-3, D5). A merge request is what GitHub calls a pull
	24	+	// request: ermite keeps refs/pull/<n>/head at the head of its source branch and,
	25	+	// while the head merges cleanly into the target, refs/pull/<n>/merge at a test
	26	+	// merge commit (parents: target tip, then head; message "Merge <head> into
	27	+	// <base>"). A pull_request run is for that merge commit: github.ref is the merge
	28	+	// ref, github.sha the merge commit, the workflow is read from it, and its checks
	29	+	// attach to the head (domain.CIRun.CheckSha).
	30	+	//
	31	+	// Two workers, apart from the one that plans pushes (so a push never waits for
	32	+	// them):
	33	+	//
	34	+	// - the event worker handles the activities one at a time, in order: a merge
	35	+	// request opened (OnPullRequest), its source branch pushed (a synchronize),
	36	+	// closed or merged. An event carries the commits it is for, fixed when it
	37	+	// happened, and the actor and trust of whoever caused it: it runs for exactly
	38	+	// those commits, with that trust, even when the branches have moved on by the
	39	+	// time it is handled (GitHub runs each synchronize). The merge commit of a
	40	+	// run is computed from the head and base of its event, never from the branches
	41	+	// as they are now.
	42	+	// - the refresh worker keeps refs/pull/<n>/{head,merge} at the latest branches
	43	+	// when a source or a target branch moves; the refreshes of one merge request
	44	+	// coalesce, and the work is bounded.
	45	+	//
	46	+	// Whatever publishes a ref or creates a run does it through the PullGate, under
	47	+	// the merge lock of the repository and only while the request is open, so a close
	48	+	// or a merge that completes meanwhile discards it: a closed request gets no
	49	+	// refs and no run but its "closed" one, which uses the last merge commit that was
	50	+	// published, or the head, and is never computed again.
	51	+	//
	52	+	// A head that does not merge cleanly makes no run for that update, as on GitHub;
	53	+	// the merge ref then keeps the last commit that merged, as GitHub's does.
	54	+
	55	+	const (
	56	+	// DefaultCIPullRefreshes bounds the merge requests (and the pushes) waiting
	57	+	// for the refresh worker. Past it the repository is marked dirty instead and
	58	+	// the worker rescans its open merge requests when it has capacity, so that
	59	+	// nothing stays stale for good.
	60	+	DefaultCIPullRefreshes = 4096
	61	+	// DefaultCIPullSnapshot bounds the merge requests one push takes the snapshot
	62	+	// of (the open requests whose source branch it pushed). Past it the rest is
	63	+	// skipped, an error is logged, and the repository is rescanned for the refs.
	64	+	DefaultCIPullSnapshot = 256
	65	+	// Stages of the work of the merge requests, for the test hook
	66	+	// CIConfig.PullStage.
	67	+	pullStageDiscover = "discover"
	68	+	pullStageRescan = "rescan"
	69	+	pullStageEvent = "event"
	70	+	pullStageComputed = "computed"
	71	+	pullStageRefresh = "refresh"
	72	+	)
	73	+
	74	+	// errPullClosed means a merge request was closed or merged before the work for
	75	+	// it could be published.
	76	+	var errPullClosed = errors.New("merge request is not open")
	77	+
	78	+	// pullEvent is one activity of a merge request waiting to be handled.
	79	+	type pullEvent struct {
	80	+	repo domain.RepoName
	81	+	number int64
	82	+	action string
	83	+	// head and base are the commits the activity is for; before is the head
	84	+	// before a synchronize, zero otherwise.
	85	+	head, base, before domain.ObjectID
	86	+	actor string
	87	+	pusher domain.CIPusher
	88	+	trusted bool
	89	+	// run is false when the cause is a job token: no run starts.
	90	+	run bool
	91	+	log zerowrap.Logger
	92	+	}
	93	+
	94	+	func (e pullEvent) trigger(at time.Time) ciTrigger {
	95	+	return ciTrigger{repo: e.repo, actor: e.actor, pusher: e.pusher, trusted: e.trusted, at: at, log: e.log}
	96	+	}
	97	+
	98	+	type pullKey struct {
	99	+	repo domain.RepoName
	100	+	number int64
	101	+	}
	102	+
	103	+	// pullQueue is what the two workers read.
	104	+	//
	105	+	// Overload policy. The activities (events) are never coalesced and never
	106	+	// dropped by the cap of the refreshes: they go on their own queue, which behaves
	107	+	// as the queue of the push triggers (CI.queue) does when it is full: the item is
	108	+	// dropped and an error is logged, so a push is never slowed or failed by CI. Only
	109	+	// the refresh of refs/pull/<n>/{head,merge}, which is idempotent and recomputed
	110	+	// from the branches as they are, is coalesced (one waiting refresh per merge
	111	+	// request) and bounded (max); past the bound the repository is marked dirty and
	112	+	// its open merge requests are rescanned when the worker has capacity.
	113	+	type pullQueue struct {
	114	+	events chan pullEvent
	115	+
	116	+	mu sync.Mutex
	117	+	max int // bound of the waiting refreshes
	118	+	pending map[pullKey]bool
	119	+	order []pullKey
	120	+	// dirty are the repositories whose merge requests may have stale refs: the
	121	+	// bound is the number of repositories.
	122	+	dirty map[domain.RepoName]bool
	123	+	dirtyOrder []domain.RepoName
	124	+	// attempts remember the last state tried for each merge request, so that a
	125	+	// rescan does not try again what it already tried (a conflict).
	126	+	attempts *pullAttempts
	127	+	wake chan struct{}
	128	+	gate ports.PullGate // set by SetPullGate
	129	+	}
	130	+
	131	+	func newPullQueue(size, max, attempts int) *pullQueue {
	132	+	return &pullQueue{
	133	+	events: make(chan pullEvent, size), max: max, pending: map[pullKey]bool{},
	134	+	dirty: map[domain.RepoName]bool{}, attempts: newPullAttempts(attempts), wake: make(chan struct{}, 1),
	135	+	}
	136	+	}
	137	+
	138	+	// SetPullGate gives the CI the gate of the merge requests (see PullGate). Call
	139	+	// it before merge requests can be opened.
	140	+	func (c *CI) SetPullGate(g ports.PullGate) {
	141	+	c.pull.mu.Lock()
	142	+	c.pull.gate = g
	143	+	c.pull.mu.Unlock()
	144	+	}
	145	+
	146	+	func (c *CI) pullEnabled() bool { return c.cfg.MRs != nil && c.cfg.Pull != nil }
	147	+
	148	+	func (c *CI) stage(stage string) {
	149	+	if c.cfg.PullStage != nil {
	150	+	c.cfg.PullStage(stage)
	151	+	}
	152	+	}
	153	+
	154	+	// OnPullRequest queues an activity of a merge request.
	155	+	func (c *CI) OnPullRequest(ctx context.Context, repo domain.RepoName, ev ports.PullRequestEvent, p domain.Principal) {
	156	+	if !c.pullEnabled() {
	157	+	return
	158	+	}
	159	+	c.queuePullEvent(pullEvent{
	160	+	repo: repo, number: ev.Number, action: ev.Action, head: ev.Head, base: ev.Base,
	161	+	actor: authorOf(p), pusher: domain.PusherOf(p), trusted: domain.TrustedActor(p), run: !p.ViaJobToken(),
	162	+	log: zerowrap.FromCtx(ctx),
	163	+	})
	164	+	}
	165	+
	166	+	// onBranchesMoved is called for every push and merge, after its push
	167	+	// workflows are queued (so a push run never waits for this). It takes the
	168	+	// snapshot of the push for the merge requests it touches, here and now: the open
	169	+	// merge requests are listed (a scan), and for those whose source branch was
	170	+	// pushed a synchronize event is queued carrying the pushed head, the target tip
	171	+	// as it is at this moment (read once per target for the push), and the actor and
	172	+	// trust of the push. The event runs for exactly that snapshot, however the
	173	+	// branches move before it is handled: a later push to the target, possibly by
	174	+	// someone less trusted, changes nothing about what a trusted run executes.
	175	+	// The refresh of the refs of every merge request touched (those that have a
	176	+	// pushed branch as source or as target, from the two indexes) is queued for the
	177	+	// refresh worker, which is the only fully asynchronous part. A push that touches
	178	+	// no merge request queues nothing and marks nothing; the whole-repository rescan
	179	+	// is for overflow, truncation and startup.
	180	+	func (c *CI) onBranchesMoved(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, actor string, pusher domain.CIPusher, trusted, run bool) {
	181	+	if !c.pullEnabled() {
	182	+	return
	183	+	}
	184	+	moved := map[string]domain.RefUpdate{}
	185	+	for _, u := range updates {
	186	+	if b, ok := strings.CutPrefix(u.Ref, "refs/heads/"); ok && !u.IsDelete() {
	187	+	moved[b] = u
	188	+	}
	189	+	}
	190	+	if len(moved) == 0 {
	191	+	return
	192	+	}
	193	+	c.stage(pullStageDiscover)
	194	+	log := zerowrap.FromCtx(ctx)
	195	+	branches := make([]string, 0, len(moved))
	196	+	for b := range moved {
	197	+	branches = append(branches, b)
	198	+	}
	199	+
	200	+	// The requests whose branch is the target of a push (a target move): their
	201	+	// refs are queued for a refresh. Only these: an unrelated push queues nothing
	202	+	// and marks nothing. The lookup is bounded like the snapshot; past the bound
	203	+	// or on an error the repository is marked dirty, and its rescan finds the
	204	+	// stale ones.
	205	+	targets, tgtTruncated, err := c.cfg.MRs.OpenByTarget(ctx, repo, branches, c.cfg.PullSnapshot)
	206	+	switch {
	207	+	case err != nil:
	208	+	log.Error().Err(err).Str("repo", repo.String()).Msg("ci: look up the merge requests targeting a pushed branch: the repository is rescanned")
	209	+	c.markPullDirty(repo)
	210	+	case tgtTruncated:
	211	+	log.Error().Str("repo", repo.String()).Int("bound", c.cfg.PullSnapshot).
	212	+	Msg("ci pull request refresh of a push is cut short: more than the bound of merge requests target the pushed branches; the repository is rescanned")
	213	+	c.markPullDirty(repo)
	214	+	}
	215	+	for _, n := range targets {
	216	+	c.queuePullRefresh(pullKey{repo, n})
	217	+	}
	218	+
	219	+	// The requests whose source branch was pushed come from the index of the open
	220	+	// merge requests by source: a scan of the keys of those branches, bounded and
	221	+	// cancellable, never of every merge request the repository ever had. Past the
	222	+	// bound or on cancellation the rest is skipped and logged (the overload policy
	223	+	// of the event queue: those events are lost), and the repository is marked
	224	+	// dirty, so that the refs of the skipped requests are repaired by the rescan.
	225	+	numbers, truncated, err := c.cfg.MRs.OpenBySource(ctx, repo, branches, c.cfg.PullSnapshot)
	226	+	if err != nil {
	227	+	log.Error().Err(err).Str("repo", repo.String()).Msg("ci: look up the merge requests of a push: its pull_request runs are not started; the repository is rescanned")
	228	+	c.markPullDirty(repo)
	229	+	if len(numbers) == 0 {
	230	+	return
	231	+	}
	232	+	} else if truncated {
	233	+	log.Error().Str("repo", repo.String()).Int("bound", c.cfg.PullSnapshot).
	234	+	Msg("ci pull request snapshot of a push is cut short: more than the bound of merge requests have a pushed branch as source; the pull_request runs of the others are not started and the repository is rescanned")
	235	+	c.markPullDirty(repo)
	236	+	}
	237	+	tips := map[string]domain.ObjectID{}
	238	+	for _, n := range numbers {
	239	+	mr, err := c.cfg.MRs.Get(ctx, repo, n)
	240	+	if err != nil \|\| !mr.IsOpen() {
	241	+	continue // closed or gone since the index was read
	242	+	}
	243	+	u, srcMoved := moved[mr.Source]
	244	+	if !srcMoved {
	245	+	continue
	246	+	}
	247	+	if run {
	248	+	if head, herr := domain.ParseObjectID(u.New); herr == nil {
	249	+	base, seen := tips[mr.Target]
	250	+	if !seen {
	251	+	base, _ = c.cfg.Git.BranchHead(ctx, repo, mr.Target)
	252	+	tips[mr.Target] = base
	253	+	}
	254	+	before, _ := domain.ParseObjectID(u.Old) // zero for a new branch
	255	+	c.queuePullEvent(pullEvent{
	256	+	repo: repo, number: mr.Number, action: domain.PullSynchronize, head: head, base: base, before: before,
	257	+	actor: actor, pusher: pusher, trusted: trusted, run: true, log: log,
	258	+	})
	259	+	}
	260	+	}
	261	+	c.queuePullRefresh(pullKey{repo, mr.Number})
	262	+	}
	263	+	}
	264	+
	265	+	// markPullDirty marks a repository for a rescan of its merge requests.
	266	+	func (c *CI) markPullDirty(repo domain.RepoName) {
	267	+	c.pull.mu.Lock()
	268	+	c.pull.markDirty(repo)
	269	+	c.pull.mu.Unlock()
	270	+	c.wakePull()
	271	+	}
	272	+
	273	+	// markDirty records that the merge requests of a repository may be stale. The
	274	+	// caller holds mu.
	275	+	func (q *pullQueue) markDirty(repo domain.RepoName) {
	276	+	if !q.dirty[repo] {
	277	+	q.dirty[repo] = true
	278	+	q.dirtyOrder = append(q.dirtyOrder, repo)
	279	+	}
	280	+	}
	281	+
	282	+	func (c *CI) wakePull() {
	283	+	select {
	284	+	case c.pull.wake <- struct{}{}:
	285	+	default:
	286	+	}
	287	+	}
	288	+
	289	+	// queuePullEvent queues an activity. A full queue drops it and logs an error,
	290	+	// exactly as the queue of the push triggers does (CI.enqueue): CI never slows
	291	+	// or fails a push. The refresh cap does not apply to events.
	292	+	func (c *CI) queuePullEvent(e pullEvent) {
	293	+	c.mu.Lock()
	294	+	closed := c.closed
	295	+	c.mu.Unlock()
	296	+	if closed {
	297	+	return
	298	+	}
	299	+	select {
	300	+	case c.pull.events <- e:
	301	+	default:
	302	+	e.log.Error().Str("repo", e.repo.String()).Int64("number", e.number).Str("action", e.action).Msg("ci pull request queue is full: this activity of a merge request is not run")
	303	+	}
	304	+	}
	305	+
	306	+	// queuePullRefresh asks for the refs of a merge request to be brought up to
	307	+	// date; asking again while it waits changes nothing. When the queue is full the
	308	+	// repository is marked dirty instead: the worker rescans its open merge requests
	309	+	// once it has capacity.
	310	+	func (c *CI) queuePullRefresh(k pullKey) {
	311	+	q := c.pull
	312	+	q.mu.Lock()
	313	+	switch {
	314	+	case q.pending[k]:
	315	+	case len(q.pending) < q.max:
	316	+	q.pending[k] = true
	317	+	q.order = append(q.order, k)
	318	+	default:
	319	+	q.markDirty(k.repo)
	320	+	}
	321	+	q.mu.Unlock()
	322	+	c.wakePull()
	323	+	}
	324	+
	325	+	// nextPullWork takes what the refresh worker does next: a refresh, or a dirty
	326	+	// repository to rescan, in that order.
	327	+	func (c CI) nextPullWork() (key pullKey, dirty *domain.RepoName) {
	328	+	q := c.pull
	329	+	q.mu.Lock()
	330	+	defer q.mu.Unlock()
	331	+	switch {
	332	+	case len(q.order) > 0:
	333	+	k := q.order[0]
	334	+	q.order = q.order[1:]
	335	+	delete(q.pending, k)
	336	+	return &k, nil
	337	+	case len(q.dirtyOrder) > 0:
	338	+	r := q.dirtyOrder[0]
	339	+	q.dirtyOrder = q.dirtyOrder[1:]
	340	+	delete(q.dirty, r)
	341	+	return nil, &r
	342	+	}
	343	+	return nil, nil
	344	+	}
	345	+
	346	+	// RescanPulls rescans the merge requests of a repository now, on the calling
	347	+	// goroutine, as the refresh worker does for a repository it marked dirty. It is
	348	+	// for tests and operators' tooling; the server never needs to call it.
	349	+	func (c *CI) RescanPulls(ctx context.Context, repo domain.RepoName) error {
	350	+	return c.rescanPulls(ctx, repo)
	351	+	}
	352	+
	353	+	// rescanPulls checks the open merge requests of a repository against their refs
	354	+	// and refreshes, one after the other and here (not through the bounded queue, so
	355	+	// a rescan never marks its repository dirty again), each whose
	356	+	// refs/pull/<n>/head is not the tip of its source branch or whose merge ref is
	357	+	// not a merge of the two current tips. A merge request whose current state was
	358	+	// already tried (a head that conflicts has no merge ref, so it never looks
	359	+	// current) is skipped until one of its branches moves. It repairs what a
	360	+	// saturated queue, a failed discovery or a restart left behind.
	361	+	func (c *CI) rescanPulls(ctx context.Context, repo domain.RepoName) error {
	362	+	c.stage(pullStageRescan)
	363	+	mrs, err := c.cfg.MRs.ListOpen(ctx, repo)
	364	+	if err != nil {
	365	+	return fmt.Errorf("list the open merge requests: %w", err)
	366	+	}
	367	+	for _, mr := range mrs {
	368	+	if ctx.Err() != nil {
	369	+	return nil
	370	+	}
	371	+	head, herr := c.cfg.Git.BranchHead(ctx, repo, mr.Source)
	372	+	base, berr := c.cfg.Git.BranchHead(ctx, repo, mr.Target)
	373	+	if herr != nil \|\| berr != nil {
	374	+	continue // a branch is gone: nothing to publish
	375	+	}
	376	+	if c.pull.attempts.tried(pullKey{repo, mr.Number}, base, head) \|\| c.pullRefsCurrent(ctx, mr, head, base) {
	377	+	continue
	378	+	}
	379	+	if err := c.refreshPull(ctx, pullKey{repo, mr.Number}); err != nil && !errors.Is(err, errPullClosed) {
	380	+	ctxLog(ctx).Error().Err(err).Str("repo", repo.String()).Int64("number", mr.Number).Msg("ci: refresh the refs of a merge request in a rescan")
	381	+	}
	382	+	}
	383	+	return nil
	384	+	}
	385	+
	386	+	// pullRefsCurrent reports whether the head ref is head and the merge ref, if
	387	+	// there is one, merges base and head. A head that conflicts has no merge ref and
	388	+	// counts as current once the head ref is right: the next push or move tries again.
	389	+	func (c *CI) pullRefsCurrent(ctx context.Context, mr domain.MergeRequest, head, base domain.ObjectID) bool {
	390	+	if cur, err := c.cfg.Pull.PullRef(ctx, mr.Repo, domain.PullHeadRef(mr.Number)); err != nil \|\| cur != head {
	391	+	return false
	392	+	}
	393	+	merge, err := c.cfg.Pull.PullRef(ctx, mr.Repo, domain.PullMergeRef(mr.Number))
	394	+	if err != nil {
	395	+	return false // none: a refresh decides whether the merge is clean
	396	+	}
	397	+	cs, _, err := c.cfg.Git.ListRange(ctx, mr.Repo, domain.ObjectID{}, merge, 1)
	398	+	return err == nil && len(cs) == 1 && len(cs[0].Parents) == 2 && cs[0].Parents[0] == base && cs[0].Parents[1] == head
	399	+	}
	400	+
	401	+	// rescanAllPulls marks dirty every repository that has open merge requests, at
	402	+	// startup: the refs may be stale after a restart or a crash.
	403	+	func (c *CI) rescanAllPulls(ctx context.Context) {
	404	+	repos, err := c.cfg.Meta.List(ctx)
	405	+	if err != nil {
	406	+	ctxLog(ctx).Error().Err(err).Msg("ci: list the repositories to rescan their merge requests")
	407	+	return
	408	+	}
	409	+	q := c.pull
	410	+	q.mu.Lock()
	411	+	for _, r := range repos {
	412	+	q.markDirty(r.Name)
	413	+	}
	414	+	q.mu.Unlock()
	415	+	c.wakePull()
	416	+	}
	417	+
	418	+	// pullLoop is the event worker.
	419	+	func (c *CI) pullLoop() {
	420	+	defer c.wg.Done()
	421	+	for {
	422	+	select {
	423	+	case <-c.ctx.Done():
	424	+	return
	425	+	case e := <-c.pull.events:
	426	+	c.guardPull(e.log, e.repo, func(ctx context.Context) error { return c.handlePullEvent(ctx, e) })
	427	+	}
	428	+	}
	429	+	}
	430	+
	431	+	// refreshLoop is the refresh worker: it refreshes the refs of the merge
	432	+	// requests a push touched, and rescans the repositories that fell behind.
	433	+	func (c *CI) refreshLoop() {
	434	+	defer c.wg.Done()
	435	+	c.rescanAllPulls(c.ctx)
	436	+	for {
	437	+	key, dirty := c.nextPullWork()
	438	+	if c.ctx.Err() != nil {
	439	+	return
	440	+	}
	441	+	switch {
	442	+	case key != nil:
	443	+	c.guardPull(zerowrap.FromCtx(c.ctx), key.repo, func(ctx context.Context) error { return c.refreshPull(ctx, *key) })
	444	+	case dirty != nil:
	445	+	c.guardPull(zerowrap.FromCtx(c.ctx), dirty, func(ctx context.Context) error { return c.rescanPulls(ctx, dirty) })
	446	+	default:
	447	+	select {
	448	+	case <-c.ctx.Done():
	449	+	return
	450	+	case <-c.pull.wake:
	451	+	}
	452	+	}
	453	+	}
	454	+	}
	455	+
	456	+	// guardPull runs fn; a panic or an error is logged and never escapes.
	457	+	func (c *CI) guardPull(log zerowrap.Logger, repo domain.RepoName, fn func(context.Context) error) {
	458	+	ctx := zerowrap.WithCtx(c.ctx, log)
	459	+	defer func() {
	460	+	if r := recover(); r != nil {
	461	+	log.Error().Interface("panic", r).Bytes("stack", debug.Stack()).Str("repo", repo.String()).Msg("ci pull request work panicked")
	462	+	}
	463	+	}()
	464	+	if err := fn(ctx); err != nil && ctx.Err() == nil && !errors.Is(err, errPullClosed) {
	465	+	log.Error().Err(err).Str("repo", repo.String()).Msg("ci pull request work failed")
	466	+	}
	467	+	}
	468	+
	469	+	// whileOpen runs fn under the merge lock while the request is open; errPullClosed
	470	+	// when it is not. Without a gate (tests of the CI alone) it checks and runs.
	471	+	func (c *CI) whileOpen(ctx context.Context, repo domain.RepoName, number int64, fn func() error) error {
	472	+	c.pull.mu.Lock()
	473	+	gate := c.pull.gate
	474	+	c.pull.mu.Unlock()
	475	+	var err error
	476	+	if gate != nil {
	477	+	err = gate.WhileOpen(ctx, repo, number, fn)
	478	+	} else if mr, gerr := c.cfg.MRs.Get(ctx, repo, number); gerr != nil {
	479	+	err = gerr
	480	+	} else if !mr.IsOpen() {
	481	+	err = domain.ErrPreconditionFailed
	482	+	} else {
	483	+	err = fn()
	484	+	}
	485	+	if errors.Is(err, domain.ErrPreconditionFailed) \|\| errors.Is(err, domain.ErrNotFound) {
	486	+	return errPullClosed
	487	+	}
	488	+	return err
	489	+	}
	490	+
	491	+	// createRun stores a run. The run of an open merge request is stored under the
	492	+	// gate, so that one that is closed meanwhile gets none (errPullClosed).
	493	+	func (c *CI) createRun(ctx context.Context, f pushFacts, repo domain.RepoName, wfPath string, build func(id, number int64) ports.CIRunDraft) (ports.CIRunDraft, error) {
	494	+	if f.pull == nil \|\| f.pull.action == domain.PullClosed {
	495	+	return c.cfg.Store.CreateRun(ctx, repo, wfPath, build)
	496	+	}
	497	+	var d ports.CIRunDraft
	498	+	err := c.whileOpen(ctx, repo, f.pull.mr.Number, func() error {
	499	+	var err error
	500	+	d, err = c.cfg.Store.CreateRun(ctx, repo, wfPath, build)
	501	+	return err
	502	+	})
	503	+	return d, err
	504	+	}
	505	+
	506	+	// pullFacts is what a pull_request run is for.
	507	+	type pullFacts struct {
	508	+	mr domain.MergeRequest
	509	+	action string
	510	+	// head and base are the commits of the event; the merge commit of the run
	511	+	// has them as parents.
	512	+	head, base domain.ObjectID
	513	+	// before is the head before a synchronize.
	514	+	before domain.ObjectID
	515	+	merged bool
	516	+	}
	517	+
	518	+	// pullState is the merge of a head into a base.
	519	+	type pullState struct {
	520	+	head, base domain.ObjectID
	521	+	// merge is the test merge commit of head and base when clean; for a closed
	522	+	// request, the last one published, zero if none.
	523	+	merge domain.ObjectID
	524	+	clean bool
	525	+	}
	526	+
	527	+	func (c *CI) handlePullEvent(ctx context.Context, e pullEvent) error {
	528	+	c.stage(pullStageEvent)
	529	+	mr, err := c.cfg.MRs.Get(ctx, e.repo, e.number)
	530	+	if errors.Is(err, domain.ErrNotFound) {
	531	+	return nil
	532	+	}
	533	+	if err != nil {
	534	+	return err
	535	+	}
	536	+	switch e.action {
	537	+	case domain.PullOpened, domain.PullSynchronize:
	538	+	if !mr.IsOpen() {
	539	+	return nil // closed meanwhile: the closed event follows
	540	+	}
	541	+	st, err := c.mergeOf(ctx, mr, e.head, e.base)
	542	+	if err != nil {
	543	+	return err
	544	+	}
	545	+	c.stage(pullStageComputed)
	546	+	if err := c.publishPull(ctx, mr, st); err != nil \|\| !st.clean \|\| !e.run {
	547	+	return err
	548	+	}
	549	+	return c.startPullRuns(ctx, e, mr, st)
	550	+	case domain.PullClosed:
	551	+	if mr.IsOpen() \|\| !e.run {
	552	+	return nil
	553	+	}
	554	+	st, ok, err := c.closedPull(ctx, e, mr)
	555	+	if err != nil \|\| !ok {
	556	+	return err
	557	+	}
	558	+	return c.startPullRuns(ctx, e, mr, st)
	559	+	}
	560	+	return nil
	561	+	}
	562	+
	563	+	// refreshPull brings the refs of a merge request to its branches as they are.
	564	+	func (c *CI) refreshPull(ctx context.Context, k pullKey) error {
	565	+	c.stage(pullStageRefresh)
	566	+	mr, err := c.cfg.MRs.Get(ctx, k.repo, k.number)
	567	+	if errors.Is(err, domain.ErrNotFound) \|\| (err == nil && !mr.IsOpen()) {
	568	+	return nil
	569	+	}
	570	+	if err != nil {
	571	+	return err
	572	+	}
	573	+	head, err := c.cfg.Git.BranchHead(ctx, k.repo, mr.Source)
	574	+	if errors.Is(err, domain.ErrNotFound) {
	575	+	return nil
	576	+	}
	577	+	if err != nil {
	578	+	return err
	579	+	}
	580	+	base, err := c.cfg.Git.BranchHead(ctx, k.repo, mr.Target)
	581	+	if errors.Is(err, domain.ErrNotFound) {
	582	+	return nil
	583	+	}
	584	+	if err != nil {
	585	+	return err
	586	+	}
	587	+	st, err := c.mergeOf(ctx, mr, head, base)
	588	+	if err != nil {
	589	+	return err
	590	+	}
	591	+	return c.publishPull(ctx, mr, st)
	592	+	}
	593	+
	594	+	// mergeOf is the test merge of head into base. It computes the commit only; a
	595	+	// merge ref that is already that merge is reused.
	596	+	func (c *CI) mergeOf(ctx context.Context, mr domain.MergeRequest, head, base domain.ObjectID) (pullState, error) {
	597	+	st := pullState{head: head, base: base}
	598	+	if head.IsZero() \|\| base.IsZero() {
	599	+	return st, nil
	600	+	}
	601	+	if old, err := c.cfg.Pull.PullRef(ctx, mr.Repo, domain.PullMergeRef(mr.Number)); err == nil {
	602	+	cs, _, err := c.cfg.Git.ListRange(ctx, mr.Repo, domain.ObjectID{}, old, 1)
	603	+	if err == nil && len(cs) == 1 && len(cs[0].Parents) == 2 && cs[0].Parents[0] == base && cs[0].Parents[1] == head {
	604	+	st.merge, st.clean = old, true
	605	+	return st, nil
	606	+	}
	607	+	} else if !errors.Is(err, domain.ErrNotFound) {
	608	+	return st, err
	609	+	}
	610	+	tree, clean, err := c.cfg.Pull.TestMerge(ctx, mr.Repo, base, head)
	611	+	k := pullKey{mr.Repo, mr.Number}
	612	+	if err != nil {
	613	+	return st, err
	614	+	}
	615	+	if !clean {
	616	+	c.pull.attempts.conflicted(k, base, head) // a rescan does not try these tips again
	617	+	return st, nil
	618	+	}
	619	+	c.pull.attempts.merged(k)
	620	+	committer := c.cfg.Committer
	621	+	if committer.Name == "" {
	622	+	committer = domain.Identity{Name: "ermite", Email: "ermite@localhost"}
	623	+	}
	624	+	id, err := c.cfg.Git.WriteCommit(ctx, mr.Repo, domain.TestMergeCommit(tree, base, head, committer, c.now()))
	625	+	if err != nil {
	626	+	return st, err
	627	+	}
	628	+	st.merge, st.clean = id, true
	629	+	return st, nil
	630	+	}
	631	+
	632	+	// publishPull points refs/pull/<n>/head and /merge at st, under the gate. A ref
	633	+	// moves only to the latest: the head ref when st.head is still the head of the
	634	+	// source branch, the merge ref when the target tip is st.base too, so that an
	635	+	// older event never moves a ref back. errPullClosed when the request is closed.
	636	+	func (c *CI) publishPull(ctx context.Context, mr domain.MergeRequest, st pullState) error {
	637	+	if st.head.IsZero() {
	638	+	return nil
	639	+	}
	640	+	changed := false
	641	+	err := c.whileOpen(ctx, mr.Repo, mr.Number, func() error {
	642	+	curHead, err := c.cfg.Git.BranchHead(ctx, mr.Repo, mr.Source)
	643	+	if err != nil \|\| curHead != st.head {
	644	+	return ignoreNotFound(err)
	645	+	}
	646	+	if old, err := c.cfg.Pull.PullRef(ctx, mr.Repo, domain.PullHeadRef(mr.Number)); err != nil \|\| old != st.head {
	647	+	if err != nil && !errors.Is(err, domain.ErrNotFound) {
	648	+	return err
	649	+	}
	650	+	if err := c.cfg.Pull.SetPullRef(ctx, mr.Repo, domain.PullHeadRef(mr.Number), st.head); err != nil {
	651	+	return err
	652	+	}
	653	+	changed = true
	654	+	}
	655	+	if !st.clean {
	656	+	return nil
	657	+	}
	658	+	curBase, err := c.cfg.Git.BranchHead(ctx, mr.Repo, mr.Target)
	659	+	if err != nil \|\| curBase != st.base {
	660	+	return ignoreNotFound(err)
	661	+	}
	662	+	if old, err := c.cfg.Pull.PullRef(ctx, mr.Repo, domain.PullMergeRef(mr.Number)); err != nil \|\| old != st.merge {
	663	+	if err != nil && !errors.Is(err, domain.ErrNotFound) {
	664	+	return err
	665	+	}
	666	+	if err := c.cfg.Pull.SetPullRef(ctx, mr.Repo, domain.PullMergeRef(mr.Number), st.merge); err != nil {
	667	+	return err
	668	+	}
	669	+	changed = true
	670	+	}
	671	+	return nil
	672	+	})
	673	+	if changed && c.cfg.Fresh != nil {
	674	+	c.cfg.Fresh.Refresh(ctx, mr.Repo)
	675	+	}
	676	+	return err
	677	+	}
	678	+
	679	+	func ignoreNotFound(err error) error {
	680	+	if errors.Is(err, domain.ErrNotFound) {
	681	+	return nil
	682	+	}
	683	+	return err
	684	+	}
	685	+
	686	+	// closedPull is the state a closed or merged merge request runs with: the last
	687	+	// test merge commit published, or the head when there is none. Nothing is
	688	+	// computed again. ok is false when the request has neither (its branch never
	689	+	// existed).
	690	+	func (c *CI) closedPull(ctx context.Context, e pullEvent, mr domain.MergeRequest) (st pullState, ok bool, err error) {
	691	+	if st.merge, err = c.cfg.Pull.PullRef(ctx, mr.Repo, domain.PullMergeRef(mr.Number)); err != nil && !errors.Is(err, domain.ErrNotFound) {
	692	+	return st, false, err
	693	+	}
	694	+	st.head = e.head
	695	+	if st.head.IsZero() {
	696	+	st.head = mr.HeadID
	697	+	}
	698	+	if st.head.IsZero() {
	699	+	if st.head, err = c.cfg.Pull.PullRef(ctx, mr.Repo, domain.PullHeadRef(mr.Number)); err != nil && !errors.Is(err, domain.ErrNotFound) {
	700	+	return st, false, err
	701	+	}
	702	+	}
	703	+	st.base = e.base
	704	+	if !st.merge.IsZero() {
	705	+	if cs, _, err := c.cfg.Git.ListRange(ctx, mr.Repo, domain.ObjectID{}, st.merge, 1); err == nil && len(cs) == 1 && len(cs[0].Parents) == 2 {
	706	+	st.base = cs[0].Parents[0]
	707	+	if st.head.IsZero() {
	708	+	st.head = cs[0].Parents[1]
	709	+	}
	710	+	}
	711	+	}
	712	+	if st.base.IsZero() {
	713	+	st.base = mr.BaseID
	714	+	}
	715	+	st.clean = true
	716	+	return st, !st.head.IsZero(), nil
	717	+	}
	718	+
	719	+	// startPullRuns starts the pull_request workflows of the merge request that
	720	+	// match the activity and the files it changes.
	721	+	func (c *CI) startPullRuns(ctx context.Context, e pullEvent, mr domain.MergeRequest, st pullState) error {
	722	+	runners, err := c.cfg.Store.ListRunners(ctx)
	723	+	if err != nil {
	724	+	return fmt.Errorf("list the runners: %w", err)
	725	+	}
	726	+	if len(runners) == 0 {
	727	+	return nil
	728	+	}
	729	+	repo, err := c.cfg.Meta.Get(ctx, e.repo)
	730	+	if err != nil {
	731	+	return nil // deleted meanwhile
	732	+	}
	733	+	t := e.trigger(c.now())
	734	+	ref, sha := domain.PullMergeRef(mr.Number), st.merge
	735	+	if sha.IsZero() {
	736	+	ref, sha = domain.PullHeadRef(mr.Number), st.head
	737	+	}
	738	+	pf := &pullFacts{mr: mr, action: e.action, head: st.head, base: st.base, before: e.before, merged: mr.State == domain.MRStateMerged}
	739	+	f := pushFacts{
	740	+	update: domain.RefUpdate{Ref: ref, Old: domain.ZeroOID, New: sha.String()}, commit: sha, pull: pf,
	741	+	}
	742	+	if def, _, err := c.cfg.Repos.DefaultBranch(ctx, e.repo); err == nil {
	743	+	f.defaultBranch = def
	744	+	}
	745	+	head, _, err := c.cfg.Git.ListRange(ctx, e.repo, domain.ObjectID{}, st.head, 1)
	746	+	if err != nil \|\| len(head) == 0 {
	747	+	return fmt.Errorf("read the head commit of !%d: %w", mr.Number, err)
	748	+	}
	749	+	f.head = head[0]
	750	+	// The files of the merge request are those between the merge base of the
	751	+	// target and the head, and the head: what the review shows.
	752	+	f.filesTruncated = true // unavailable unless read below: a paths filter then passes
	753	+	if !st.base.IsZero() {
	754	+	if mb, err := c.cfg.Git.MergeBase(ctx, e.repo, st.base, st.head); err == nil {
	755	+	if paths, truncated, err := c.cfg.Diff.ChangedPaths(ctx, e.repo, mb, st.head, workflow.MaxFilesEvaluated); err == nil {
	756	+	f.files, f.filesTruncated = paths, truncated
	757	+	}
	758	+	}
	759	+	}
	760	+	files, err := c.workflowFiles(ctx, e.repo, sha)
	761	+	if err != nil \|\| len(files) == 0 {
	762	+	return err
	763	+	}
	764	+	if workflow.SkipCI(f.head.Message) {
	765	+	return nil
	766	+	}
	767	+	var errs []error
	768	+	for _, file := range files {
	769	+	if file.problem != "" {
	770	+	continue // reported by the push of the branch
	771	+	}
	772	+	wf, perrs := workflow.Parse(file.path, file.content)
	773	+	if len(perrs) > 0 \|\| !matchPullRequest(wf, e.action, mr.Target, f.files, f.filesTruncated) {
	774	+	continue
	775	+	}
	776	+	if problem := c.matrixProblem(ctx, repo, t, f, file.path, wf); problem.text != "" {
	777	+	name := file.path
	778	+	if problem.named {
	779	+	name = workflowName(wf, file.path)
	780	+	}
	781	+	errs = append(errs, c.startInvalid(ctx, repo, t, f, file.path, name, problem.text))
	782	+	continue
	783	+	}
	784	+	errs = append(errs, c.startRun(ctx, repo, t, f, file.path, string(file.content), wf))
	785	+	}
	786	+	return errors.Join(errs...)
	787	+	}
	788	+
	789	+	// matchPullRequest reports whether the workflow runs for the activity of a merge
	790	+	// request into base: the event is listed, the activity is one of its types
	791	+	// (opened, synchronize and reopened when it lists none), and the branches and
	792	+	// paths filters, which apply to the base branch and the changed files, let it
	793	+	// through. The filters are those of the push event (ermite-actions), run on the
	794	+	// pull_request event under another name.
	795	+	func matchPullRequest(wf *workflow.Workflow, action, base string, files []string, truncated bool) bool {
	796	+	if wf == nil \|\| wf.On == nil {
	797	+	return false
	798	+	}
	799	+	ev := wf.On.Event("pull_request")
	800	+	if ev == nil {
	801	+	return false
	802	+	}
	803	+	types := ev.Types
	804	+	if types == nil {
	805	+	types = domain.DefaultPullRequestTypes
	806	+	}
	807	+	if !slices.Contains(types, action) {
	808	+	return false
	809	+	}
	810	+	as := *ev
	811	+	as.Name = "push"
	812	+	probe := &workflow.Workflow{On: &workflow.On{Events: []*workflow.Event{&as}}}
	813	+	return probe.MatchPush(workflow.PushInfo{Ref: "refs/heads/" + base, ChangedFiles: files, FilesTruncated: truncated})
	814	+	}
	815	+
	816	+	// --- the github context ---
	817	+
	818	+	// pullGithubJSON builds the github context of a pull_request run: github.ref is
	819	+	// the merge ref, github.sha the merge commit (the head for a closed request
	820	+	// that never merged), and the event is shaped like GitHub's pull_request
	821	+	// webhook payload with the fields ermite can fill.
	822	+	func (c *CI) pullGithubJSON(repo domain.Repo, run domain.CIRun, f pushFacts) string {
	823	+	p := f.pull
	824	+	mr := p.mr
	825	+	u := c.urls(repo)
	826	+	id := c.identity(run.Actor)
	827	+	owner := c.identity("owner")
	828	+	repoObj := c.repoObj(repo, u, f.defaultBranch, run.CreatedAt)
	829	+	num := strconv.FormatInt(mr.Number, 10)
	830	+	prAPI := u.api + "/repos/" + u.full + "/pulls/" + num
	831	+	html := u.repo + "/merge-requests/" + num
	832	+	side := func(branch string, sha domain.ObjectID) map[string]any {
	833	+	return map[string]any{
	834	+	"label": u.owner + ":" + branch, "ref": branch, "sha": sha.String(),
	835	+	"user": u.userObj(owner, false), "repo": repoObj,
	836	+	}
	837	+	}
	838	+	tm := func(t time.Time) any {
	839	+	if t.IsZero() {
	840	+	return nil
	841	+	}
	842	+	return t.UTC().Format(time.RFC3339)
	843	+	}
	844	+	var body any
	845	+	if s := mr.Description.String(); s != "" {
	846	+	body = s
	847	+	}
	848	+	state, closedAt, mergedAt := "open", any(nil), any(nil)
	849	+	mergeSha := any(nil)
	850	+	if !mr.IsOpen() {
	851	+	state, closedAt = "closed", tm(mr.UpdatedAt)
	852	+	}
	853	+	switch {
	854	+	case p.merged:
	855	+	mergedAt = tm(mr.UpdatedAt)
	856	+	mergeSha = mr.MergedCommitID.String()
	857	+	case f.commit.String() != p.head.String():
	858	+	mergeSha = f.commit.String() // the test merge commit
	859	+	}
	860	+	prID := stableID("pull", u.full+"#"+num)
	861	+	pr := map[string]any{
	862	+	"url": prAPI, "id": prID, "node_id": nodeID("011", "PullRequest", prID), "html_url": html,
	863	+	"diff_url": html + ".diff", "patch_url": html + ".patch", "issue_url": u.api + "/repos/" + u.full + "/issues/" + num,
	864	+	"number": mr.Number, "state": state, "locked": false, "title": mr.Title.String(), "user": u.userObj(c.identity(mr.Author), false),
	865	+	"body": body, "created_at": tm(mr.CreatedAt), "updated_at": tm(mr.UpdatedAt), "closed_at": closedAt, "merged_at": mergedAt,
	866	+	"merge_commit_sha": mergeSha, "assignee": nil, "assignees": []any{}, "requested_reviewers": []any{},
	867	+	"requested_teams": []any{}, "labels": []any{}, "milestone": nil, "draft": false,
	868	+	"head": side(mr.Source, p.head), "base": side(mr.Target, p.base), "author_association": "OWNER",
	869	+	"auto_merge": nil, "active_lock_reason": nil, "merged": p.merged,
	870	+	}
	871	+	event := map[string]any{
	872	+	"action": p.action, "number": mr.Number, "pull_request": pr, "repository": repoObj, "sender": u.userObj(id, false),
	873	+	}
	874	+	if p.action == domain.PullSynchronize {
	875	+	before := p.before.String()
	876	+	if p.before.IsZero() {
	877	+	before = domain.ZeroOID
	878	+	}
	879	+	event["before"], event["after"] = before, p.head.String()
	880	+	}
	881	+	return c.githubContext(repo, run, ghEvent{
	882	+	name: "pull_request", refName: strings.TrimPrefix(run.Ref, domain.PullRefPrefix), refType: "branch",
	883	+	headRef: mr.Source, baseRef: mr.Target, payload: event,
	884	+	})
	885	+	}

A internal/usecase/ci_pull_test.go +87 -0

@@ -0,0 +1,87 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package usecase_test
	4	+
	5	+	import (
	6	+	"testing"
	7	+
	8	+	"github.com/stretchr/testify/assert"
	9	+	"github.com/stretchr/testify/require"
	10	+
	11	+	"bnema.dev/ermite-actions/workflow"
	12	+	"bnema.dev/ermite/internal/domain"
	13	+	"bnema.dev/ermite/internal/usecase"
	14	+	)
	15	+
	16	+	func parsePR(t testing.T, on string) workflow.Workflow {
	17	+	t.Helper()
	18	+	wf, errs := workflow.Parse("pr.yml", []byte("on:\n"+on+"\njobs:\n j:\n runs-on: x\n steps:\n - run: 'true'\n"))
	19	+	require.Empty(t, errs)
	20	+	return wf
	21	+	}
	22	+
	23	+	func TestMatchPullRequestTypes(t *testing.T) {
	24	+	def := parsePR(t, " pull_request:\n")
	25	+	for action, want := range map[string]bool{"opened": true, "synchronize": true, "reopened": true, "closed": false} {
	26	+	assert.Equal(t, want, usecase.MatchPullRequest(def, action, "main", nil, false), "default types, "+action)
	27	+	}
	28	+	closed := parsePR(t, " pull_request:\n types: [closed]\n")
	29	+	assert.True(t, usecase.MatchPullRequest(closed, "closed", "main", nil, false))
	30	+	assert.False(t, usecase.MatchPullRequest(closed, "opened", "main", nil, false))
	31	+	both := parsePR(t, " pull_request:\n types: [opened, closed]\n")
	32	+	assert.True(t, usecase.MatchPullRequest(both, "opened", "main", nil, false))
	33	+	assert.True(t, usecase.MatchPullRequest(both, "closed", "main", nil, false))
	34	+	assert.False(t, usecase.MatchPullRequest(both, "synchronize", "main", nil, false))
	35	+	assert.False(t, usecase.MatchPullRequest(parsePR(t, " push:\n"), "opened", "main", nil, false), "no pull_request event")
	36	+	assert.True(t, usecase.MatchPullRequest(parsePR(t, " [push, pull_request]\n"), "opened", "main", nil, false))
	37	+	}
	38	+
	39	+	func TestMatchPullRequestFilters(t *testing.T) {
	40	+	wf := parsePR(t, " pull_request:\n branches: [main, 'release/']\n paths: ['src/', '!src/gen/**']\n")
	41	+	files := func(f ...string) []string { return f }
	42	+	assert.True(t, usecase.MatchPullRequest(wf, "opened", "main", files("src/a.go"), false))
	43	+	assert.True(t, usecase.MatchPullRequest(wf, "opened", "release/1.2", files("README.md", "src/b.go"), false))
	44	+	assert.False(t, usecase.MatchPullRequest(wf, "opened", "dev", files("src/a.go"), false), "the base branch is filtered")
	45	+	assert.False(t, usecase.MatchPullRequest(wf, "opened", "main", files("README.md"), false), "the changed files are filtered")
	46	+	assert.False(t, usecase.MatchPullRequest(wf, "opened", "main", files("src/gen/x.go"), false))
	47	+	assert.True(t, usecase.MatchPullRequest(wf, "opened", "main", nil, true), "an unavailable list passes the paths filter")
	48	+
	49	+	ignore := parsePR(t, " pull_request:\n branches-ignore: [wip]\n paths-ignore: ['docs/**']\n")
	50	+	assert.True(t, usecase.MatchPullRequest(ignore, "synchronize", "main", files("docs/a.md", "x.go"), false))
	51	+	assert.False(t, usecase.MatchPullRequest(ignore, "synchronize", "main", files("docs/a.md"), false))
	52	+	assert.False(t, usecase.MatchPullRequest(ignore, "synchronize", "wip", files("x.go"), false))
	53	+	}
	54	+
	55	+	func TestPullFilesLink(t *testing.T) {
	56	+	const base = "http://x/pulls/1/files"
	57	+	assert.Empty(t, usecase.PullFilesLink(base, 1, 30, 0))
	58	+	assert.Empty(t, usecase.PullFilesLink(base, 1, 30, 30))
	59	+	assert.Equal(t, `<http://x/pulls/1/files?per_page=30&page=2>; rel="next", <http://x/pulls/1/files?per_page=30&page=3>; rel="last"`, usecase.PullFilesLink(base, 1, 30, 61))
	60	+	assert.Equal(t,
	61	+	`<http://x/pulls/1/files?per_page=30&page=3>; rel="next", <http://x/pulls/1/files?per_page=30&page=3>; rel="last", <http://x/pulls/1/files?per_page=30&page=1>; rel="first", <http://x/pulls/1/files?per_page=30&page=1>; rel="prev"`,
	62	+	usecase.PullFilesLink(base, 2, 30, 61))
	63	+	last := usecase.PullFilesLink(base, 3, 30, 61)
	64	+	assert.NotContains(t, last, "next")
	65	+	assert.Contains(t, last, `page=2>; rel="prev"`)
	66	+	beyond := usecase.PullFilesLink(base, 9, 30, 61)
	67	+	assert.NotContains(t, beyond, "next")
	68	+	assert.Contains(t, beyond, `page=3>; rel="prev"`)
	69	+	}
	70	+
	71	+	func TestFilePatch(t *testing.T) {
	72	+	f := domain.FileDiff{Hunks: []domain.Hunk{
	73	+	{OldStart: 1, OldLines: 2, NewStart: 1, NewLines: 2, Header: "func x()", Lines: []domain.Line{
	74	+	{Kind: domain.LineContext, Text: "a"}, {Kind: domain.LineDeletion, Text: "b"}, {Kind: domain.LineAddition, Text: "c", NoNewlineAtEOF: true},
	75	+	}},
	76	+	{OldStart: 9, OldLines: 1, NewStart: 9, NewLines: 1, Lines: []domain.Line{{Kind: domain.LineAddition, Text: "z"}}},
	77	+	}}
	78	+	got, ok := usecase.FilePatch(f)
	79	+	require.True(t, ok)
	80	+	assert.Equal(t, "@@ -1,2 +1,2 @@ func x()\n a\n-b\n+c\n\\ No newline at end of file\n@@ -9 +9 @@\n+z", got)
	81	+	_, ok = usecase.FilePatch(domain.FileDiff{Binary: true, Hunks: f.Hunks})
	82	+	assert.False(t, ok, "binary")
	83	+	_, ok = usecase.FilePatch(domain.FileDiff{Truncated: true})
	84	+	assert.False(t, ok, "truncated")
	85	+	_, ok = usecase.FilePatch(domain.FileDiff{Status: domain.FileRenamed})
	86	+	assert.False(t, ok, "a pure rename has no hunk")
	87	+	}

A internal/usecase/ci_pullattempts.go +89 -0

@@ -0,0 +1,89 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package usecase
	4	+
	5	+	import (
	6	+	"sync"
	7	+
	8	+	"bnema.dev/ermite/internal/domain"
	9	+	)
	10	+
	11	+	// DefaultCIPullAttempts bounds the merge requests whose last conflicting merge
	12	+	// is remembered.
	13	+	const DefaultCIPullAttempts = 4096
	14	+
	15	+	// pullAttempt is the pair of tips a merge was tried on.
	16	+	type pullAttempt struct{ base, head domain.ObjectID }
	17	+
	18	+	// pullAttempts remembers, per merge request, the tips of the last test merge
	19	+	// that conflicted. A merge request whose head conflicts has no merge ref, so it
	20	+	// never looks up to date to a rescan: without this memory every rescan would try
	21	+	// the same merge again, for ever. A rescan skips a merge request while its tips
	22	+	// are the remembered ones, and tries it again as soon as one moves. Only
	23	+	// conflicts are remembered: a merge that went through is judged by its refs.
	24	+	//
	25	+	// A least recently used cache bounded by max: a conflict that was forgotten is
	26	+	// tried once more by the next rescan, never more often than that.
	27	+	type pullAttempts struct {
	28	+	mu sync.Mutex
	29	+	max int
	30	+	items map[pullKey]pullAttempt
	31	+	order []pullKey // oldest use first
	32	+	}
	33	+
	34	+	func newPullAttempts(max int) *pullAttempts {
	35	+	if max <= 0 {
	36	+	max = DefaultCIPullAttempts
	37	+	}
	38	+	return &pullAttempts{max: max, items: map[pullKey]pullAttempt{}}
	39	+	}
	40	+
	41	+	// tried reports whether the last conflict of k was on these tips.
	42	+	func (a *pullAttempts) tried(k pullKey, base, head domain.ObjectID) bool {
	43	+	a.mu.Lock()
	44	+	defer a.mu.Unlock()
	45	+	at, ok := a.items[k]
	46	+	if ok {
	47	+	a.touch(k)
	48	+	}
	49	+	return ok && at.base == base && at.head == head
	50	+	}
	51	+
	52	+	// conflicted remembers that the merge of head into base conflicted.
	53	+	func (a *pullAttempts) conflicted(k pullKey, base, head domain.ObjectID) {
	54	+	a.mu.Lock()
	55	+	defer a.mu.Unlock()
	56	+	if _, ok := a.items[k]; !ok && len(a.items) >= a.max {
	57	+	delete(a.items, a.order[0])
	58	+	a.order = a.order[1:]
	59	+	}
	60	+	a.items[k] = pullAttempt{base: base, head: head}
	61	+	a.touch(k)
	62	+	}
	63	+
	64	+	// merged forgets k: its last merge went through.
	65	+	func (a *pullAttempts) merged(k pullKey) {
	66	+	a.mu.Lock()
	67	+	defer a.mu.Unlock()
	68	+	if _, ok := a.items[k]; !ok {
	69	+	return
	70	+	}
	71	+	delete(a.items, k)
	72	+	for i, o := range a.order {
	73	+	if o == k {
	74	+	a.order = append(a.order[:i], a.order[i+1:]...)
	75	+	break
	76	+	}
	77	+	}
	78	+	}
	79	+
	80	+	// touch makes k the most recently used. The caller holds mu.
	81	+	func (a *pullAttempts) touch(k pullKey) {
	82	+	for i, o := range a.order {
	83	+	if o == k {
	84	+	a.order = append(a.order[:i], a.order[i+1:]...)
	85	+	break
	86	+	}
	87	+	}
	88	+	a.order = append(a.order, k)
	89	+	}

A internal/usecase/ci_pullattempts_test.go +47 -0

@@ -0,0 +1,47 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package usecase
	4	+
	5	+	import (
	6	+	"strconv"
	7	+	"strings"
	8	+	"testing"
	9	+
	10	+	"github.com/stretchr/testify/assert"
	11	+	"github.com/stretchr/testify/require"
	12	+
	13	+	"bnema.dev/ermite/internal/domain"
	14	+	)
	15	+
	16	+	func TestPullAttemptsRememberConflictsByTipsAndAreBounded(t *testing.T) {
	17	+	id := func(c string) domain.ObjectID {
	18	+	o, err := domain.ParseObjectID(strings.Repeat(c, 40))
	19	+	require.NoError(t, err)
	20	+	return o
	21	+	}
	22	+	key := func(i int) pullKey {
	23	+	r, err := domain.ParseRepoName("r" + strconv.Itoa(i))
	24	+	require.NoError(t, err)
	25	+	return pullKey{repo: r, number: 1}
	26	+	}
	27	+	a := newPullAttempts(2)
	28	+	a.conflicted(key(0), id("a"), id("b"))
	29	+	assert.True(t, a.tried(key(0), id("a"), id("b")), "the same tips were tried")
	30	+	assert.False(t, a.tried(key(0), id("c"), id("b")), "the target moved: try again")
	31	+	assert.False(t, a.tried(key(0), id("a"), id("c")), "the source moved: try again")
	32	+	assert.False(t, a.tried(key(1), id("a"), id("b")), "another merge request")
	33	+
	34	+	a.conflicted(key(1), id("a"), id("b"))
	35	+	a.tried(key(0), id("a"), id("b")) // 0 is the most recently used; 1 the oldest
	36	+	a.conflicted(key(2), id("a"), id("b"))
	37	+	assert.False(t, a.tried(key(1), id("a"), id("b")), "the least recently used is forgotten")
	38	+	assert.True(t, a.tried(key(0), id("a"), id("b")))
	39	+	assert.True(t, a.tried(key(2), id("a"), id("b")))
	40	+	assert.Len(t, a.items, 2)
	41	+	assert.Len(t, a.order, 2)
	42	+
	43	+	a.merged(key(0))
	44	+	assert.False(t, a.tried(key(0), id("a"), id("b")), "a merge that went through is forgotten")
	45	+	assert.Len(t, a.items, 1)
	46	+	assert.Len(t, a.order, 1)
	47	+	}

A internal/usecase/ci_pullcache.go +65 -0

@@ -0,0 +1,65 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package usecase
	4	+
	5	+	import (
	6	+	"sync"
	7	+
	8	+	"bnema.dev/ermite/internal/domain"
	9	+	)
	10	+
	11	+	// maxPullFileLists bounds the file lists the files endpoint keeps.
	12	+	const maxPullFileLists = 16
	13	+
	14	+	// pullFilesKey names a diff by the commits it is between: it never changes, so
	15	+	// a cached list is never stale.
	16	+	type pullFilesKey struct {
	17	+	repo domain.RepoName
	18	+	base, head domain.ObjectID
	19	+	}
	20	+
	21	+	// pullFilesCache keeps the last file lists computed by the files endpoint, so
	22	+	// that the pages of one merge request (paths-filter reads them all, 100 files
	23	+	// at a time) do not each run the whole diff again. A small least recently used
	24	+	// cache: entries are the lists of at most 3000 files, and there are at most
	25	+	// maxPullFileLists of them.
	26	+	type pullFilesCache struct {
	27	+	mu sync.Mutex
	28	+	items map[pullFilesKey][]domain.FileDiff
	29	+	order []pullFilesKey // oldest use first
	30	+	}
	31	+
	32	+	func (c *pullFilesCache) get(k pullFilesKey) ([]domain.FileDiff, bool) {
	33	+	c.mu.Lock()
	34	+	defer c.mu.Unlock()
	35	+	files, ok := c.items[k]
	36	+	if ok {
	37	+	c.touch(k)
	38	+	}
	39	+	return files, ok
	40	+	}
	41	+
	42	+	func (c *pullFilesCache) put(k pullFilesKey, files []domain.FileDiff) {
	43	+	c.mu.Lock()
	44	+	defer c.mu.Unlock()
	45	+	if c.items == nil {
	46	+	c.items = map[pullFilesKey][]domain.FileDiff{}
	47	+	}
	48	+	if _, ok := c.items[k]; !ok && len(c.items) >= maxPullFileLists {
	49	+	delete(c.items, c.order[0])
	50	+	c.order = c.order[1:]
	51	+	}
	52	+	c.items[k] = files
	53	+	c.touch(k)
	54	+	}
	55	+
	56	+	// touch makes k the most recently used. The caller holds mu.
	57	+	func (c *pullFilesCache) touch(k pullFilesKey) {
	58	+	for i, o := range c.order {
	59	+	if o == k {
	60	+	c.order = append(c.order[:i], c.order[i+1:]...)
	61	+	break
	62	+	}
	63	+	}
	64	+	c.order = append(c.order, k)
	65	+	}

A internal/usecase/ci_pullcache_test.go +38 -0

@@ -0,0 +1,38 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package usecase
	4	+
	5	+	import (
	6	+	"strconv"
	7	+	"testing"
	8	+
	9	+	"github.com/stretchr/testify/assert"
	10	+	"github.com/stretchr/testify/require"
	11	+
	12	+	"bnema.dev/ermite/internal/domain"
	13	+	)
	14	+
	15	+	func TestPullFilesCacheIsBoundedAndLeastRecentlyUsed(t *testing.T) {
	16	+	var c pullFilesCache
	17	+	key := func(i int) pullFilesKey {
	18	+	repo, err := domain.ParseRepoName("r" + strconv.Itoa(i))
	19	+	require.NoError(t, err)
	20	+	return pullFilesKey{repo: repo}
	21	+	}
	22	+	for i := range maxPullFileLists {
	23	+	c.put(key(i), []domain.FileDiff{{NewPath: strconv.Itoa(i)}})
	24	+	}
	25	+	_, ok := c.get(key(0)) // 0 is now the most recently used; 1 the oldest
	26	+	require.True(t, ok)
	27	+	c.put(key(maxPullFileLists), nil)
	28	+
	29	+	_, oldest := c.get(key(1))
	30	+	assert.False(t, oldest, "the least recently used entry goes")
	31	+	files, kept := c.get(key(0))
	32	+	require.True(t, kept, "a used entry stays")
	33	+	assert.Equal(t, "0", files[0].NewPath)
	34	+	_, newest := c.get(key(maxPullFileLists))
	35	+	assert.True(t, newest)
	36	+	assert.Len(t, c.items, maxPullFileLists)
	37	+	assert.Len(t, c.order, maxPullFileLists)
	38	+	}

A internal/usecase/ci_pullfiles.go +271 -0

@@ -0,0 +1,271 @@
	1	+	// SPDX-License-Identifier: AGPL-3.0-or-later
	2	+
	3	+	package usecase
	4	+
	5	+	import (
	6	+	"context"
	7	+	"encoding/json"
	8	+	"errors"
	9	+	"fmt"
	10	+	"net/url"
	11	+	"strconv"
	12	+	"strings"
	13	+
	14	+	"bnema.dev/ermite/internal/domain"
	15	+	)
	16	+
	17	+	// GET /repos/{owner}/{repo}/pulls/{number}/files, the only REST call that
	18	+	// dorny/paths-filter@v3 makes on a pull_request (src/main.ts:181-215, `octokit
	19	+	// .rest.pulls.listFiles` with per_page 100 through paginate.iterator; it reads
	20	+	// `filename`, `status`, and `previous_filename` for renames, and stops when the
	21	+	// Link header has no rel="next"). It needs no GET /pulls/{n}: the number comes
	22	+	// from the event payload.
	23	+
	24	+	const (
	25	+	// DefaultPullFilesPerPage and MaxPullFilesPerPage are GitHub's page sizes.
	26	+	DefaultPullFilesPerPage = 30
	27	+	MaxPullFilesPerPage = 100
	28	+	// maxPullFiles is GitHub's cap: "responses include a maximum of 3000 files".
	29	+	maxPullFiles = 3000
	30	+	)
	31	+
	32	+	// ListPullFiles implements ports.GitHubAPIService.
	33	+	func (c *CI) ListPullFiles(ctx context.Context, p domain.Principal, owner, name string, number int64, page, perPage int) ([]byte, string, error) {
	34	+	repo, err := c.pullFilesRepo(ctx, p, owner, name)
	35	+	if err != nil {
	36	+	return nil, "", err
	37	+	}
	38	+	if c.cfg.MRs == nil \|\| number < 1 {
	39	+	return nil, "", domain.ErrNotFound
	40	+	}
	41	+	if page < 1 {
	42	+	page = 1
	43	+	}
	44	+	if perPage < 1 {
	45	+	perPage = DefaultPullFilesPerPage
	46	+	}
	47	+	perPage = min(perPage, MaxPullFilesPerPage)
	48	+	mr, err := c.cfg.MRs.Get(ctx, repo.Name, number)
	49	+	if err != nil {
	50	+	return nil, "", err
	51	+	}
	52	+	files, err := c.pullFiles(ctx, repo.Name, mr)
	53	+	if err != nil {
	54	+	return nil, "", err
	55	+	}
	56	+	total := len(files)
	57	+	// Bounds by division: page * perPage overflows for a huge page, which would
	58	+	// make a negative slice index. A page past the end is empty.
	59	+	lo, hi := total, total
	60	+	if page-1 <= total/perPage {
	61	+	lo = min((page-1)*perPage, total)
	62	+	hi = min(lo+perPage, total)
	63	+	}
	64	+	u := c.urls(repo)
	65	+	out := make([]map[string]any, 0, hi-lo)
	66	+	for _, f := range files[lo:hi] {
	67	+	out = append(out, c.pullFileJSON(u, f))
	68	+	}
	69	+	body, err := json.Marshal(out)
	70	+	if err != nil {
	71	+	return nil, "", err
	72	+	}
	73	+	return body, pullFilesLink(u.api+"/repos/"+u.full+"/pulls/"+strconv.FormatInt(number, 10)+"/files", page, perPage, total), nil
	74	+	}
	75	+
	76	+	// pullFilesRepo is the access rule of the endpoint. The job token of the
	77	+	// repository (which has repo:read only with contents: read) is judged by its
	78	+	// pull-requests permission: with pull-requests: read it reads the merge requests
	79	+	// of its own repository even when contents is none, as on GitHub; without it a
	80	+	// private repository answers 403, a public one is read by anyone. A job token
	81	+	// never reads another repository's merge requests. Every other principal is
	82	+	// judged by its reach on the repository (githubRepo).
	83	+	func (c *CI) pullFilesRepo(ctx context.Context, p domain.Principal, owner, name string) (domain.Repo, error) {
	84	+	if _, _, archive := p.ArchiveGrant(); archive {
	85	+	// An archive token downloads one archive and reads nothing else, not even
	86	+	// the merge requests of a public repository.
	87	+	return domain.Repo{}, domain.ErrNotFound
	88	+	}
	89	+	if !p.ViaJobToken() {
	90	+	return c.githubRepo(ctx, p, owner, name)
	91	+	}
	92	+	if !domain.OwnerMatches(c.cfg.Owner, owner) {
	93	+	return domain.Repo{}, domain.ErrNotFound
	94	+	}
	95	+	rn, err := domain.ParseRepoName(strings.ToLower(name))
	96	+	if err != nil {
	97	+	return domain.Repo{}, domain.ErrNotFound
	98	+	}
	99	+	repo, err := lookupRepo(ctx, c.cfg.Meta, rn)
	100	+	if err != nil {
	101	+	return domain.Repo{}, err
	102	+	}
	103	+	switch {
	104	+	case p.JobPermits(domain.PermPullRequests, domain.PermissionRead, rn):
	105	+	return repo, nil
	106	+	case repo.Visibility == domain.VisibilityPublic:
	107	+	return repo, nil
	108	+	case canRead(p, repo):
	109	+	return domain.Repo{}, fmt.Errorf("%w: the token lacks pull-requests: read", domain.ErrForbidden)
	110	+	}
	111	+	return domain.Repo{}, domain.ErrNotFound // another repository's token
	112	+	}
	113	+
	114	+	// pullFiles lists the files a merge request changes: between the merge base of
	115	+	// the target and the head, and the head, as its review shows them (GitHub's
	116	+	// three-dot diff). A closed or merged request uses the base and head it stored.
	117	+	func (c *CI) pullFiles(ctx context.Context, repo domain.RepoName, mr domain.MergeRequest) ([]domain.FileDiff, error) {
	118	+	head, base := mr.HeadID, mr.BaseID
	119	+	if mr.IsOpen() {
	120	+	var err error
	121	+	if head, err = c.cfg.Git.BranchHead(ctx, repo, mr.Source); err != nil {
	122	+	if errors.Is(err, domain.ErrNotFound) {
	123	+	return nil, nil // the source branch is gone: no files
	124	+	}
	125	+	return nil, err
	126	+	}
	127	+	tip, err := c.cfg.Git.BranchHead(ctx, repo, mr.Target)
	128	+	switch {
	129	+	case errors.Is(err, domain.ErrNotFound):
	130	+	base = domain.ObjectID{}
	131	+	case err != nil:
	132	+	return nil, err
	133	+	default:
	134	+	switch base, err = c.cfg.Git.MergeBase(ctx, repo, tip, head); {
	135	+	case errors.Is(err, domain.ErrNotFound):
	136	+	base = domain.ObjectID{}
	137	+	case err != nil:
	138	+	return nil, err
	139	+	}
	140	+	}
	141	+	}
	142	+	if head.IsZero() {
	143	+	return nil, nil
	144	+	}
	145	+	if ok, err := c.cfg.Git.HasCommit(ctx, repo, head); err != nil \|\| !ok {
	146	+	return nil, err // the reviewed commits are gone
	147	+	}
	148	+	if base.IsZero() {
	149	+	return nil, nil
	150	+	}
	151	+	// The diff between two commits never changes: the pages of one request (and
	152	+	// the requests that follow while it is not pushed) share one computation.
	153	+	key := pullFilesKey{repo: repo, base: base, head: head}
	154	+	if files, ok := c.pullFileLists.get(key); ok {
	155	+	return files, nil
	156	+	}
	157	+	limits := domain.DiffLimits{MaxFiles: maxPullFiles, MaxFileLines: 2000, MaxTotalLines: 50000, MaxTotalBytes: 8 << 20}
	158	+	_, diff, err := c.cfg.Git.DiffCommits(ctx, repo, base, head, limits)
	159	+	if err != nil {
	160	+	return nil, err
	161	+	}
	162	+	c.pullFileLists.put(key, diff.Files)
	163	+	return diff.Files, nil
	164	+	}
	165	+
	166	+	// pullFileJSON is one element of the answer, in GitHub's schema (the "diff-entry"
	167	+	// of the REST description).
	168	+	func (c *CI) pullFileJSON(u ghURLs, f domain.FileDiff) map[string]any {
	169	+	name, status := f.NewPath, "modified"
	170	+	switch f.Status {
	171	+	case domain.FileAdded:
	172	+	status = "added"
	173	+	case domain.FileDeleted:
	174	+	name, status = f.OldPath, "removed"
	175	+	case domain.FileRenamed:
	176	+	status = "renamed"
	177	+	case domain.FileCopied:
	178	+	status = "copied"
	179	+	case domain.FileTypeChanged:
	180	+	status = "changed"
	181	+	}
	182	+	sha := f.NewID.String()
	183	+	if f.NewID.IsZero() {
	184	+	sha = f.OldID.String()
	185	+	}
	186	+	esc := escapeURLPath(name)
	187	+	o := map[string]any{
	188	+	"sha": sha, "filename": name, "status": status,
	189	+	"additions": f.Additions, "deletions": f.Deletions, "changes": f.Additions + f.Deletions,
	190	+	"blob_url": u.repo + "/blob/" + sha + "/" + esc,
	191	+	"raw_url": u.repo + "/raw/" + sha + "/" + esc,
	192	+	"contents_url": u.api + "/repos/" + u.full + "/contents/" + esc,
	193	+	}
	194	+	if f.Status == domain.FileRenamed \|\| f.Status == domain.FileCopied {
	195	+	o["previous_filename"] = f.OldPath
	196	+	}
	197	+	if patch, ok := filePatch(f); ok {
	198	+	o["patch"] = patch
	199	+	}
	200	+	return o
	201	+	}
	202	+
	203	+	func escapeURLPath(p string) string {
	204	+	segs := strings.Split(p, "/")
	205	+	for i, s := range segs {
	206	+	segs[i] = url.PathEscape(s)
	207	+	}
	208	+	return strings.Join(segs, "/")
	209	+	}
	210	+
	211	+	// filePatch is the unified-diff hunks of a file, as GitHub's "patch" shows them
	212	+	// (hunk headers and lines, no file headers). Absent for a binary file, a pure
	213	+	// rename and a file too large for the diff limits, as on GitHub.
	214	+	func filePatch(f domain.FileDiff) (string, bool) {
	215	+	if f.Binary \|\| f.Truncated \|\| len(f.Hunks) == 0 {
	216	+	return "", false
	217	+	}
	218	+	var b strings.Builder
	219	+	for i, h := range f.Hunks {
	220	+	if i > 0 {
	221	+	b.WriteByte('\n')
	222	+	}
	223	+	fmt.Fprintf(&b, "@@ -%s +%s @@", hunkRange(h.OldStart, h.OldLines), hunkRange(h.NewStart, h.NewLines))
	224	+	if h.Header != "" {
	225	+	b.WriteString(" " + h.Header)
	226	+	}
	227	+	for _, l := range h.Lines {
	228	+	b.WriteByte('\n')
	229	+	switch l.Kind {
	230	+	case domain.LineAddition:
	231	+	b.WriteByte('+')
	232	+	case domain.LineDeletion:
	233	+	b.WriteByte('-')
	234	+	default:
	235	+	b.WriteByte(' ')
	236	+	}
	237	+	b.WriteString(l.Text)
	238	+	if l.NoNewlineAtEOF {
	239	+	b.WriteString("\n\\ No newline at end of file")
	240	+	}
	241	+	}
	242	+	}
	243	+	return b.String(), true
	244	+	}
	245	+
	246	+	func hunkRange(start, n int) string {
	247	+	if n == 1 {
	248	+	return strconv.Itoa(start)
	249	+	}
	250	+	return strconv.Itoa(start) + "," + strconv.Itoa(n)
	251	+	}
	252	+
	253	+	// pullFilesLink is the Link header of a page: rel next, last, first and prev as
	254	+	// GitHub writes them; empty when everything fits on one page.
	255	+	func pullFilesLink(base string, page, perPage, total int) string {
	256	+	last := max((total+perPage-1)/perPage, 1)
	257	+	if last == 1 {
	258	+	return ""
	259	+	}
	260	+	at := func(n int) string {
	261	+	return "<" + base + "?per_page=" + strconv.Itoa(perPage) + "&page=" + strconv.Itoa(n) + ">"
	262	+	}
	263	+	var parts []string
	264	+	if page < last {
	265	+	parts = append(parts, at(page+1)+`; rel="next"`, at(last)+`; rel="last"`)
	266	+	}
	267	+	if page > 1 {
	268	+	parts = append(parts, at(1)+`; rel="first"`, at(min(page-1, last))+`; rel="prev"`)
	269	+	}
	270	+	return strings.Join(parts, ", ")
	271	+	}

M internal/usecase/ci_trigger.go +51 -12


@@ -44,14 +44,19 @@ type ciTrigger struct {
44	44		log zerowrap.Logger
45	45		}
46	46
47		-	// OnPush queues the refs of a push or a merge for planning and returns at once;
48		-	// a worker plans them. A full queue drops the trigger, logged at error: a push
49		-	// is never slowed down or failed by CI.
	47	+	// OnPush queues the refs of a push or a merge for planning and returns; a
	48	+	// worker plans them. A full queue drops the trigger, logged at error: a push is
	49	+	// never slowed down or failed by CI. The push workflows are queued first; the
	50	+	// merge requests the push touches are then handed to their own worker, with a
	51	+	// short lock and no I/O, so nothing here waits for Git or the store.
50	52		//
51	53		// A push made with a job token starts nothing, as on GitHub (events created
52	54		// with GITHUB_TOKEN start no run): that is what stops a workflow that pushes
53	55		// from running itself again.
54	56		func (c *CI) OnPush(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, pusher domain.Principal) {
	57	+	// The merge requests follow the push whoever made it (their refs); only a
	58	+	// push that is not a job's starts pull_request runs.
	59	+	defer c.onBranchesMoved(ctx, repo, updates, authorOf(pusher), domain.PusherOf(pusher), domain.TrustedActor(pusher), !pusher.ViaJobToken())
55	60		if pusher.ViaJobToken() {
56	61		log := zerowrap.FromCtx(ctx)
57	62		log.Debug().Str("repo", repo.String()).Int("refs", len(updates)).Msg("push by a job token starts no run")

@@ -67,6 +72,10 @@ func (c *CI) OnPush(ctx context.Context, repo domain.RepoName, updates []domain.
67	72		// merging. For the same reason the run is trusted (gets secrets) only when the
68	73		// merger is: the displayed actor is not the one that decides.
69	74		func (c *CI) OnMerge(ctx context.Context, repo domain.RepoName, updates []domain.RefUpdate, merger domain.Principal) {
	75	+	// The merge moved a target branch: the test merges of the other requests
	76	+	// into it are computed again (a merge starts no synchronize run), after the
	77	+	// push workflows are queued.
	78	+	defer c.onBranchesMoved(ctx, repo, updates, authorOf(domain.Owner()), domain.PusherOf(merger), domain.TrustedActor(merger), false)
70	79		if merger.ViaJobToken() {
71	80		log := zerowrap.FromCtx(ctx)
72	81		log.Debug().Str("repo", repo.String()).Int("refs", len(updates)).Msg("merge by a job token starts no run")

@@ -168,6 +177,36 @@ type pushFacts struct {
168	177		// payload is the commit part of the push event, shared by the workflows
169	178		// of the ref; handleRef builds it once, before the first one that matches.
170	179		payload *pushPayload
	180	+	// pull is set for a pull_request run: update.Ref is then
	181	+	// refs/pull/<n>/merge (or /head), commit the test merge commit (or the
	182	+	// head) and head the head commit.
	183	+	pull *pullFacts
	184	+	}
	185	+
	186	+	// eventName is the event of the run, as the run records and github.event_name
	187	+	// show it.
	188	+	func (f pushFacts) eventName() string {
	189	+	if f.pull != nil {
	190	+	return "pull_request"
	191	+	}
	192	+	return "push"
	193	+	}
	194	+
	195	+	// runTitle is the title of a run without run-name: the commit message of a
	196	+	// push, the title of the merge request for a pull_request.
	197	+	func (f pushFacts) runTitle() string {
	198	+	if f.pull != nil {
	199	+	return firstLine(f.pull.mr.Title.String())
	200	+	}
	201	+	return firstLine(f.head.Message)
	202	+	}
	203	+
	204	+	// headSha is CIRun.HeadSha: the head of the merge request, "" for a push.
	205	+	func (f pushFacts) headSha() string {
	206	+	if f.pull != nil {
	207	+	return f.pull.head.String()
	208	+	}
	209	+	return ""
171	210		}
172	211
173	212		func (c *CI) handleRef(ctx context.Context, t ciTrigger, u domain.RefUpdate, tagCount int) error {

@@ -408,22 +447,22 @@ func firstLine(s string) string {
408	447		// parses but whose strategy GitHub refuses (recorded: matrix-over-256).
409	448		func (c *CI) startInvalid(ctx context.Context, repo domain.Repo, t ciTrigger, f pushFacts, wfPath, name, problem string) error {
410	449		now := c.now()
411		-	draft, err := c.cfg.Store.CreateRun(ctx, repo.Name, wfPath, func(id, number int64) ports.CIRunDraft {
	450	+	draft, err := c.createRun(ctx, f, repo.Name, wfPath, func(id, number int64) ports.CIRunDraft {
412	451		run := domain.CIRun{
413	452		Repo: repo.Name, ID: id, Number: number, Attempt: 1, WorkflowPath: wfPath, WorkflowName: name, Name: name,
414		-	Title: firstLine(f.head.Message), Event: "push", Ref: f.update.Ref, Sha: f.commit.String(), Actor: t.actor,
	453	+	Title: f.runTitle(), Event: f.eventName(), Ref: f.update.Ref, Sha: f.commit.String(), HeadSha: f.headSha(), Actor: t.actor,
415	454		Status: domain.CICompleted, Conclusion: domain.CIFailure, Error: cutText(problem, maxRunError), Trusted: t.trusted,
416	455		CreatedAt: now, StartedAt: now, CompletedAt: now,
417	456		}
418	457		return ports.CIRunDraft{
419	458		Run: run, Source: domain.CIRunSource{Repo: repo.Name, WorkflowYAML: "", GithubJSON: "{}", Pusher: t.pusher},
420	459		Checks: []domain.CICheck{{
421		-	ID: "run-" + strconv.FormatInt(id, 10), Repo: repo.Name, Sha: run.Sha, Name: wfPath, Status: domain.CICompleted,
	460	+	ID: "run-" + strconv.FormatInt(id, 10), Repo: repo.Name, Sha: run.CheckSha(), Name: wfPath, Status: domain.CICompleted,
422	461		Conclusion: domain.CIFailure, RunID: id, Attempt: 1, WorkflowName: name, CreatedAt: now, StartedAt: now, CompletedAt: now,
423	462		}},
424	463		}
425	464		})
426		-	if errors.Is(err, domain.ErrNotFound) {
	465	+	if errors.Is(err, domain.ErrNotFound) \|\| errors.Is(err, errPullClosed) {
427	466		return nil
428	467		}
429	468		if err != nil {

@@ -453,7 +492,7 @@ func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f push
453	492		// workflow has another name: 14 of 14 tag runs). Registered is read as the
454	493		// "name:" of the file on the default branch; the first version GitHub saw
455	494		// is the same file in the recording and cannot be told apart.
456		-	title := firstLine(f.head.Message)
	495	+	title := f.runTitle()
457	496		if f.isTag {
458	497		title = c.registeredName(ctx, repo.Name, wfPath, name)
459	498		}

@@ -461,10 +500,10 @@ func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f push
461	500		if err != nil {
462	501		return err
463	502		}
464		-	draft, err := c.cfg.Store.CreateRun(ctx, repo.Name, wfPath, func(id, number int64) ports.CIRunDraft {
	503	+	draft, err := c.createRun(ctx, f, repo.Name, wfPath, func(id, number int64) ports.CIRunDraft {
465	504		run := domain.CIRun{
466	505		Repo: repo.Name, ID: id, Number: number, Attempt: 1, WorkflowPath: wfPath, WorkflowName: name, Name: name,
467		-	Title: title, Event: "push", Ref: f.update.Ref, Sha: f.commit.String(), Actor: t.actor,
	506	+	Title: title, Event: f.eventName(), Ref: f.update.Ref, Sha: f.commit.String(), HeadSha: f.headSha(), Actor: t.actor,
468	507		Status: domain.CIQueued, Trusted: t.trusted, CreatedAt: now,
469	508		}
470	509		githubJSON := c.githubJSON(ctx, repo, run, f)

@@ -481,13 +520,13 @@ func (c *CI) startRun(ctx context.Context, repo domain.Repo, t ciTrigger, f push
481	520		}
482	521		d.Jobs = append(d.Jobs, job)
483	522		d.Checks = append(d.Checks, domain.CICheck{
484		-	ID: job.ID, Repo: repo.Name, Sha: run.Sha, Name: job.Name, Status: domain.CIQueued, RunID: id, JobID: job.ID,
	523	+	ID: job.ID, Repo: repo.Name, Sha: run.CheckSha(), Name: job.Name, Status: domain.CIQueued, RunID: id, JobID: job.ID,
485	524		Attempt: 1, WorkflowName: name, CreatedAt: now,
486	525		})
487	526		}
488	527		return d
489	528		})
490		-	if errors.Is(err, domain.ErrNotFound) {
	529	+	if errors.Is(err, domain.ErrNotFound) \|\| errors.Is(err, errPullClosed) {
491	530		return nil
492	531		}
493	532		if err != nil {

M internal/usecase/export_test.go +13 -0


@@ -60,3 +60,16 @@ func (s *SecretUseCache) Drop(runID int64) { s.c.drop(runID) }
60	60		func EvalConcurrency(ctx context.Context, tok template.Token, env template.Env) (domain.CIConcurrency, error) {
61	61		return evalConcurrency(ctx, tok, env)
62	62		}
	63	+
	64	+	// MatchPullRequest exposes the pull_request filter to the tests.
	65	+	func MatchPullRequest(wf *workflow.Workflow, action, base string, files []string, truncated bool) bool {
	66	+	return matchPullRequest(wf, action, base, files, truncated)
	67	+	}
	68	+
	69	+	// PullFilesLink exposes the Link header of the files endpoint to the tests.
	70	+	func PullFilesLink(base string, page, perPage, total int) string {
	71	+	return pullFilesLink(base, page, perPage, total)
	72	+	}
	73	+
	74	+	// FilePatch exposes the patch text of a file of the files endpoint to the tests.
	75	+	func FilePatch(f domain.FileDiff) (string, bool) { return filePatch(f) }

M internal/usecase/mergerequests.go +42 -4


@@ -49,14 +49,18 @@ type MergeRequests struct {
49	49		bus ports.EventBus
50	50		clock ports.Clock
51	51		maint ports.MaintenanceScheduler
52		-	ci ports.WorkflowTrigger // nil: no CI
53		-	checks ports.CIChecksReader // nil: no CI, so no check blocks a merge
	52	+	ci ports.WorkflowTrigger // nil: no CI
	53	+	checks ports.CIChecksReader // nil: no CI, so no check blocks a merge
	54	+	pulls ports.PullRequestTrigger // nil: no pull_request workflows
54	55
55	56		mu sync.Mutex
56	57		locks map[domain.RepoName]chan struct{}
57	58		}
58	59
59		-	var _ ports.MergeRequestService = (*MergeRequests)(nil)
	60	+	var (
	61	+	_ ports.MergeRequestService = (*MergeRequests)(nil)
	62	+	_ ports.PullGate = (*MergeRequests)(nil)
	63	+	)
60	64
61	65		// MergeRequestsConfig holds what the merge request use cases need. Every field
62	66		// is required.

@@ -79,13 +83,17 @@ type MergeRequestsConfig struct {
79	83		// optional: without them there is no CI.
80	84		CI ports.WorkflowTrigger
81	85		Checks ports.CIChecksReader
	86	+	// PullRequests starts the pull_request workflows when a request is opened,
	87	+	// closed or merged (the push of its source branch reaches CI instead).
	88	+	// Optional.
	89	+	PullRequests ports.PullRequestTrigger
82	90		}
83	91
84	92		// NewMergeRequests builds the merge request use cases.
85	93		func NewMergeRequests(cfg MergeRequestsConfig) *MergeRequests {
86	94		return &MergeRequests{
87	95		meta: cfg.Meta, store: cfg.Store, repos: cfg.Repos, git: cfg.Git, signer: cfg.Signer,
88		-	committer: cfg.Committer, bus: cfg.Bus, clock: cfg.Clock, maint: cfg.Maint, ci: cfg.CI, checks: cfg.Checks,
	96	+	committer: cfg.Committer, bus: cfg.Bus, clock: cfg.Clock, maint: cfg.Maint, ci: cfg.CI, checks: cfg.Checks, pulls: cfg.PullRequests,
89	97		locks: make(map[domain.RepoName]chan struct{}),
90	98		}
91	99		}

@@ -232,6 +240,11 @@ func (m *MergeRequests) Create(ctx context.Context, p domain.Principal, name dom
232	240		return domain.MergeRequest{}, err
233	241		}
234	242		m.bus.Publish(ctx, domain.MergeRequestOpened{MergeRequest: mr, At: now, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt})
	243	+	if m.pulls != nil {
	244	+	// The commits the opening is for are the ones of now.
	245	+	base, _ := m.branchOrZero(ctx, name, target)
	246	+	m.pulls.OnPullRequest(ctx, name, ports.PullRequestEvent{Number: mr.Number, Action: domain.PullOpened, Head: mr.HeadID, Base: base}, p)
	247	+	}
235	248		return mr, nil
236	249		}
237	250

@@ -619,6 +632,9 @@ func (m *MergeRequests) Close(ctx context.Context, p domain.Principal, name doma
619	632		log.Info().Str("repo", name.String()).Int64("number", number).
620	633		Str("principal", p.Label()).Str("token_id", p.TokenID()).Msg("merge request closed")
621	634		m.bus.Publish(ctx, domain.MergeRequestClosed{MergeRequest: mr, At: mr.UpdatedAt, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt})
	635	+	if m.pulls != nil {
	636	+	m.pulls.OnPullRequest(ctx, name, ports.PullRequestEvent{Number: number, Action: domain.PullClosed, Head: mr.HeadID, Base: mr.BaseID}, p)
	637	+	}
622	638		return mr, nil
623	639		}
624	640		return domain.MergeRequest{}, fmt.Errorf("close !%d of %q: %w", number, name, domain.ErrConflict)

@@ -813,6 +829,9 @@ func (m *MergeRequests) merge(ctx context.Context, p domain.Principal, name doma
813	829		m.bus.Publish(ctx, domain.PushEvent{
814	830		Repo: name, At: at, Visibility: repo.Visibility, CreatedAt: repo.CreatedAt, Updates: updates,
815	831		})
	832	+	if m.pulls != nil {
	833	+	m.pulls.OnPullRequest(ctx, name, ports.PullRequestEvent{Number: number, Action: domain.PullClosed, Head: mr.HeadID, Base: mr.BaseID}, p) // closed, merged: true
	834	+	}
816	835		if m.ci != nil {
817	836		// GitHub runs the "push" workflows of a merge, with the owner as actor;
818	837		// the run is trusted with what the merging principal may read.

@@ -821,6 +840,25 @@ func (m *MergeRequests) merge(ctx context.Context, p domain.Principal, name doma
821	840		return mr, deleted, nil
822	841		}
823	842
	843	+	// WhileOpen implements ports.PullGate: fn runs under the merge lock, so a
	844	+	// close or a merge of the request cannot complete in the middle of it, and not
	845	+	// at all when the request is no longer open.
	846	+	func (m *MergeRequests) WhileOpen(ctx context.Context, name domain.RepoName, number int64, fn func() error) error {
	847	+	unlock, err := m.lockRepo(ctx, name)
	848	+	if err != nil {
	849	+	return err
	850	+	}
	851	+	defer unlock()
	852	+	mr, err := m.store.Get(ctx, name, number)
	853	+	if err != nil {
	854	+	return err
	855	+	}
	856	+	if !mr.IsOpen() {
	857	+	return fmt.Errorf("%w: !%d is %s", domain.ErrPreconditionFailed, number, mr.State)
	858	+	}
	859	+	return fn()
	860	+	}
	861	+
824	862		// checksAllow is the merge rule of FailingChecksBlock: every check of the
825	863		// head must have ended as success, skipped or neutral. A check still queued or
826	864		// running blocks too (the merge could land code that then fails), and a head

M internal/usecase/pushes.go +6 -0


@@ -150,6 +150,12 @@ func (u *Pushes) Check(ctx context.Context, p domain.Principal, name domain.Repo
150	150		return fmt.Errorf("%w: no write access to %s", domain.ErrRejected, name)
151	151		}
152	152		for _, up := range updates {
	153	+	// refs/pull/* belong to ermite (the head and the test merge of the merge
	154	+	// requests): nobody pushes there, whatever the token, the workflow
	155	+	// guard included.
	156	+	if domain.IsPullRefName(up.Ref) {
	157	+	return fmt.Errorf("%w: %s is maintained by ermite: refs/pull/* cannot be pushed", domain.ErrRejected, up.Ref)
	158	+	}
153	159		if !p.AllowsRef(up.Ref) {
154	160		return fmt.Errorf("%w: %s is outside the branches this token may push (%s)",
155	161		domain.ErrRejected, up.Ref, strings.Join(p.BranchPrefixes(), ", "))

M proto/ermite/storage/ci.proto +3 -0


@@ -106,6 +106,9 @@ message CIRun {
106	106		bool trusted = 23;
107	107		// The evaluated workflow-level concurrency of the attempt.
108	108		CIConcurrency concurrency = 24;
	109	+	// head_sha: the head of the merge request of a pull_request run, where sha is
	110	+	// the test merge commit. Empty for the other events.
	111	+	string head_sha = 25;
109	112		}
110	113
111	114		// CIRunSource is stored under "runsrc/<run id:020d>": what a run was planned