detached c11af01

feat(ci): reusable workflows (#33)

commit
c11af01c6987ffc7e4db6cb9e7eddc64b035da09
author
brice · · 2026-10-15 05:33 UTC
committer
bnema · · 2026-10-15 05:33 UTC
parent
dc42452
signature
SSH UnverifiedSHA256:VOCA5lYARHcbKDZ7GhTV45VmcjWnJ6ZQIBazj17LivA The key is unknown
26 files · +3487 -172
Unified
UnifiedSplit
M README.md +1 -1
@@ -127,7 +127,7 @@ The Actions that run here call a small part of GitHub's REST API, served at `git
127127
128128 Actions from github.com are resolved with `git ls-remote https://github.com/<owner>/<repo>` (no credentials, 20 s, 16 MiB of output, at most four at once, 60 s for a whole call). ermite hands git only the proxy settings of its own environment (`HTTPS_PROXY`, `HTTP_PROXY`, `NO_PROXY`, `ALL_PROXY` and their lower-case forms), so a server that reaches the Internet through a proxy sets them for `ermite serve`; nothing else of its environment, no credential, reaches git.
129129
130-`ermite run list|view|watch|cancel|rerun` and the `Actions` tab of a repository read and control the runs; `cancel` and `rerun` need the `ci:write` scope. With `failing_checks_block` on (the default, `ermite repo edit --failing-checks-block=false`), a merge is refused while a check of the head failed or has not finished. `concurrency:` is supported at the workflow level (evaluated when the run starts) and at the job level (evaluated when the job is planned, once per matrix leg): a group is per repository and compared without regard to case, `queue: single` (the default) lets one run or job go and keeps one waiting, a newer waiting entry cancels the older one (`Canceling since a higher priority waiting request for '<group>' exists`), and `cancel-in-progress` (a boolean or an expression) also cancels the running one. A job held back by a group shows `pending` in the API, the web pages and the CLI. A group that evaluates to nothing fails the job, and `queue: max` is refused with `not supported yet: queue: max`. A workflow that uses another feature ermite does not support yet (reusable workflows) fails with `not supported yet: <feature>`, never silently. Logs are files under `$ERMITE_DATA_DIR/ci/logs`, and the job summaries (the Markdown a step writes to `GITHUB_STEP_SUMMARY`) under `$ERMITE_DATA_DIR/ci/summaries`; both go with their run (retention, repository deletion). Annotations (`::notice`, `::warning`, `::error`) are kept in the metadata store. The run page shows them in an Annotations box and the job page at their step; the summaries are rendered under the run with the README renderer (raw HTML dropped, `javascript:` links dropped). `ermite run view` prints an ANNOTATIONS section, and `ermite run view --summary` prints the summaries as Markdown.
130+`ermite run list|view|watch|cancel|rerun` and the `Actions` tab of a repository read and control the runs; `cancel` and `rerun` need the `ci:write` scope. With `failing_checks_block` on (the default, `ermite repo edit --failing-checks-block=false`), a merge is refused while a check of the head failed or has not finished. `concurrency:` is supported at the workflow level (evaluated when the run starts) and at the job level (evaluated when the job is planned, once per matrix leg): a group is per repository and compared without regard to case, `queue: single` (the default) lets one run or job go and keeps one waiting, a newer waiting entry cancels the older one (`Canceling since a higher priority waiting request for '<group>' exists`), and `cancel-in-progress` (a boolean or an expression) also cancels the running one. A job held back by a group shows `pending` in the API, the web pages and the CLI. A group that evaluates to nothing fails the job, and `queue: max` is refused with `not supported yet: queue: max`. Reusable workflows (`jobs.<id>.uses`) run: `./.github/workflows/x.yml` (the caller's repository, at the commit of the run), `owner/repo/.github/workflows/x.yml@ref` of a repository of this server when `owner` is `ERMITE_OWNER` and the actor of the run can read it (else the call fails as for a repository that does not exist), and otherwise a public repository on github.com, whose ref is resolved like the ref of an action and whose file is downloaded at that commit (at most 1 MiB). ermite expands the call when the job is planned: the called workflow must declare `on: workflow_call`, `with:` is checked against its inputs (types, `required`, defaults), `secrets:` passes the listed secrets, renamed as written, or all of them with `secrets: inherit` (a secret that is not passed is not there), and its `outputs` are available to the later jobs of the caller as `needs.<job>.outputs`. The jobs of the called workflow are named `<calling job> / <job>` and keyed `<calling job id>/<job id>`, wait for the `needs` of the calling job, and are repeated for each leg of its `strategy`; `permissions` can only be lowered by the called workflow. A call may nest four levels deep. `concurrency:` on the calling job and the workflow-level `concurrency:` of the called workflow (evaluated with the caller's `github` and the call's `inputs`) apply to the whole call, which holds the group until it ends, with `cancel-in-progress` as for any job. A workflow that uses another feature ermite does not support yet fails with `not supported yet: <feature>`, never silently. Logs are files under `$ERMITE_DATA_DIR/ci/logs`, and the job summaries (the Markdown a step writes to `GITHUB_STEP_SUMMARY`) under `$ERMITE_DATA_DIR/ci/summaries`; both go with their run (retention, repository deletion). Annotations (`::notice`, `::warning`, `::error`) are kept in the metadata store. The run page shows them in an Annotations box and the job page at their step; the summaries are rendered under the run with the README renderer (raw HTML dropped, `javascript:` links dropped). `ermite run view` prints an ANNOTATIONS section, and `ermite run view --summary` prints the summaries as Markdown.
131131
132132 ## Web sign-in
133133
M gen/ermite/storage/ci.pb.go +323 -90
@@ -973,7 +973,16 @@ type CIJob struct {
973973 Permissions map[string]string `protobuf:"bytes,42,rep,name=permissions,proto3" json:"permissions,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
974974 // The job's concurrency group (its own, else the one of its run) and where
975975 // it is in it.
976- Concurrency *CIConcurrency `protobuf:"bytes,43,opt,name=concurrency,proto3" json:"concurrency,omitempty"`
976+ Concurrency *CIConcurrency `protobuf:"bytes,43,opt,name=concurrency,proto3" json:"concurrency,omitempty"`
977+ // Reusable workflows (D7). scope is the id of the record of the job that
978+ // called the workflow this job belongs to, empty for a job of the run's own
979+ // workflow; call_id is the id of the job in that called workflow (key is
980+ // "<caller key>/<call id>"). call is set on the record that stands for a job
981+ // calling a reusable workflow: it never runs, it ends when the jobs of its
982+ // scope have.
983+ Scope string `protobuf:"bytes,44,opt,name=scope,proto3" json:"scope,omitempty"`
984+ CallId string `protobuf:"bytes,45,opt,name=call_id,json=callId,proto3" json:"call_id,omitempty"`
985+ Call *CICall `protobuf:"bytes,46,opt,name=call,proto3" json:"call,omitempty"`
977986 unknownFields protoimpl.UnknownFields
978987 sizeCache protoimpl.SizeCache
979988 }
@@ -1302,6 +1311,199 @@ func (x *CIJob) GetConcurrency() *CIConcurrency {
13021311 return nil
13031312 }
13041313
1314+func (x *CIJob) GetScope() string {
1315+ if x != nil {
1316+ return x.Scope
1317+ }
1318+ return ""
1319+}
1320+
1321+func (x *CIJob) GetCallId() string {
1322+ if x != nil {
1323+ return x.CallId
1324+ }
1325+ return ""
1326+}
1327+
1328+func (x *CIJob) GetCall() *CICall {
1329+ if x != nil {
1330+ return x.Call
1331+ }
1332+ return nil
1333+}
1334+
1335+// CICall is the call a reusable-workflow job makes, set when the calling job is
1336+// planned. The called workflow's file is kept here, so that the jobs of the call
1337+// are planned and run from the file of the commit that was resolved.
1338+type CICall struct {
1339+ state protoimpl.MessageState `protogen:"open.v1"`
1340+ Uses string `protobuf:"bytes,1,opt,name=uses,proto3" json:"uses,omitempty"`
1341+ // path of the file in its repository, repository as "owner/repo" (github.com)
1342+ // or the name of the repository on this server, ref as the calling workflow
1343+ // wrote it ("" for a local call), sha the commit it resolved to.
1344+ Path string `protobuf:"bytes,2,opt,name=path,proto3" json:"path,omitempty"`
1345+ Repository string `protobuf:"bytes,3,opt,name=repository,proto3" json:"repository,omitempty"`
1346+ Ref string `protobuf:"bytes,4,opt,name=ref,proto3" json:"ref,omitempty"`
1347+ Sha string `protobuf:"bytes,5,opt,name=sha,proto3" json:"sha,omitempty"`
1348+ Yaml string `protobuf:"bytes,6,opt,name=yaml,proto3" json:"yaml,omitempty"`
1349+ // inputs_json is the inputs context of the called workflow: the values given
1350+ // and the defaults, typed.
1351+ InputsJson string `protobuf:"bytes,7,opt,name=inputs_json,json=inputsJson,proto3" json:"inputs_json,omitempty"`
1352+ // secrets holds the names of the secrets of the called workflow that the
1353+ // calling workflow's "secrets:" gives (upper case), with an empty value: the
1354+ // expressions are not stored. They are read as template tokens from the pinned
1355+ // workflow file of the caller when a job is assigned, and evaluated once there.
1356+ Secrets map[string]string `protobuf:"bytes,8,rep,name=secrets,proto3" json:"secrets,omitempty" protobuf_key:"bytes,1,opt,name=key" protobuf_val:"bytes,2,opt,name=value"`
1357+ Inherit bool `protobuf:"varint,9,opt,name=inherit,proto3" json:"inherit,omitempty"`
1358+ Depth int32 `protobuf:"varint,10,opt,name=depth,proto3" json:"depth,omitempty"`
1359+ // expanded: the jobs of the called workflow exist.
1360+ Expanded bool `protobuf:"varint,11,opt,name=expanded,proto3" json:"expanded,omitempty"`
1361+ // concurrency is the workflow-level group of the called workflow, evaluated
1362+ // with the caller's github context and the call's inputs when the call is
1363+ // planned. The call record is its entry, like the group of the calling job
1364+ // (CIJob.concurrency): the jobs of the call run once the call holds both.
1365+ Concurrency *CIConcurrency `protobuf:"bytes,12,opt,name=concurrency,proto3" json:"concurrency,omitempty"`
1366+ // admitted: the call may start (the max-parallel of the calling job's
1367+ // strategy allows it); it asks for its groups and plans its jobs only then.
1368+ Admitted bool `protobuf:"varint,13,opt,name=admitted,proto3" json:"admitted,omitempty"`
1369+ // secret_ctx is the JSON of the contexts the "secrets:" expressions read apart
1370+ // from the secrets, as they were when the call was planned: needs, matrix,
1371+ // strategy, inputs and vars of the calling scope (none is secret). Plain data,
1372+ // never interpreted as a reference; the vars are the caller's, not those of
1373+ // the environment of the job that is assigned.
1374+ SecretCtx string `protobuf:"bytes,14,opt,name=secret_ctx,json=secretCtx,proto3" json:"secret_ctx,omitempty"`
1375+ unknownFields protoimpl.UnknownFields
1376+ sizeCache protoimpl.SizeCache
1377+}
1378+
1379+func (x *CICall) Reset() {
1380+ *x = CICall{}
1381+ mi := &file_ermite_storage_ci_proto_msgTypes[4]
1382+ ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
1383+ ms.StoreMessageInfo(mi)
1384+}
1385+
1386+func (x *CICall) String() string {
1387+ return protoimpl.X.MessageStringOf(x)
1388+}
1389+
1390+func (*CICall) ProtoMessage() {}
1391+
1392+func (x *CICall) ProtoReflect() protoreflect.Message {
1393+ mi := &file_ermite_storage_ci_proto_msgTypes[4]
1394+ if x != nil {
1395+ ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
1396+ if ms.LoadMessageInfo() == nil {
1397+ ms.StoreMessageInfo(mi)
1398+ }
1399+ return ms
1400+ }
1401+ return mi.MessageOf(x)
1402+}
1403+
1404+// Deprecated: Use CICall.ProtoReflect.Descriptor instead.
1405+func (*CICall) Descriptor() ([]byte, []int) {
1406+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{4}
1407+}
1408+
1409+func (x *CICall) GetUses() string {
1410+ if x != nil {
1411+ return x.Uses
1412+ }
1413+ return ""
1414+}
1415+
1416+func (x *CICall) GetPath() string {
1417+ if x != nil {
1418+ return x.Path
1419+ }
1420+ return ""
1421+}
1422+
1423+func (x *CICall) GetRepository() string {
1424+ if x != nil {
1425+ return x.Repository
1426+ }
1427+ return ""
1428+}
1429+
1430+func (x *CICall) GetRef() string {
1431+ if x != nil {
1432+ return x.Ref
1433+ }
1434+ return ""
1435+}
1436+
1437+func (x *CICall) GetSha() string {
1438+ if x != nil {
1439+ return x.Sha
1440+ }
1441+ return ""
1442+}
1443+
1444+func (x *CICall) GetYaml() string {
1445+ if x != nil {
1446+ return x.Yaml
1447+ }
1448+ return ""
1449+}
1450+
1451+func (x *CICall) GetInputsJson() string {
1452+ if x != nil {
1453+ return x.InputsJson
1454+ }
1455+ return ""
1456+}
1457+
1458+func (x *CICall) GetSecrets() map[string]string {
1459+ if x != nil {
1460+ return x.Secrets
1461+ }
1462+ return nil
1463+}
1464+
1465+func (x *CICall) GetInherit() bool {
1466+ if x != nil {
1467+ return x.Inherit
1468+ }
1469+ return false
1470+}
1471+
1472+func (x *CICall) GetDepth() int32 {
1473+ if x != nil {
1474+ return x.Depth
1475+ }
1476+ return 0
1477+}
1478+
1479+func (x *CICall) GetExpanded() bool {
1480+ if x != nil {
1481+ return x.Expanded
1482+ }
1483+ return false
1484+}
1485+
1486+func (x *CICall) GetConcurrency() *CIConcurrency {
1487+ if x != nil {
1488+ return x.Concurrency
1489+ }
1490+ return nil
1491+}
1492+
1493+func (x *CICall) GetAdmitted() bool {
1494+ if x != nil {
1495+ return x.Admitted
1496+ }
1497+ return false
1498+}
1499+
1500+func (x *CICall) GetSecretCtx() string {
1501+ if x != nil {
1502+ return x.SecretCtx
1503+ }
1504+ return ""
1505+}
1506+
13051507 // CICheck is stored under "check/<repo>/<sha>/<id>": the id of its job, or
13061508 // "run-<run id>" for a run that failed without a job (invalid workflow file).
13071509 type CICheck struct {
@@ -1325,7 +1527,7 @@ type CICheck struct {
13251527
13261528 func (x *CICheck) Reset() {
13271529 *x = CICheck{}
1328- mi := &file_ermite_storage_ci_proto_msgTypes[4]
1530+ mi := &file_ermite_storage_ci_proto_msgTypes[5]
13291531 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
13301532 ms.StoreMessageInfo(mi)
13311533 }
@@ -1337,7 +1539,7 @@ func (x *CICheck) String() string {
13371539 func (*CICheck) ProtoMessage() {}
13381540
13391541 func (x *CICheck) ProtoReflect() protoreflect.Message {
1340- mi := &file_ermite_storage_ci_proto_msgTypes[4]
1542+ mi := &file_ermite_storage_ci_proto_msgTypes[5]
13411543 if x != nil {
13421544 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
13431545 if ms.LoadMessageInfo() == nil {
@@ -1350,7 +1552,7 @@ func (x *CICheck) ProtoReflect() protoreflect.Message {
13501552
13511553 // Deprecated: Use CICheck.ProtoReflect.Descriptor instead.
13521554 func (*CICheck) Descriptor() ([]byte, []int) {
1353- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{4}
1555+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{5}
13541556 }
13551557
13561558 func (x *CICheck) GetId() string {
@@ -1462,7 +1664,7 @@ type CIRunner struct {
14621664
14631665 func (x *CIRunner) Reset() {
14641666 *x = CIRunner{}
1465- mi := &file_ermite_storage_ci_proto_msgTypes[5]
1667+ mi := &file_ermite_storage_ci_proto_msgTypes[6]
14661668 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
14671669 ms.StoreMessageInfo(mi)
14681670 }
@@ -1474,7 +1676,7 @@ func (x *CIRunner) String() string {
14741676 func (*CIRunner) ProtoMessage() {}
14751677
14761678 func (x *CIRunner) ProtoReflect() protoreflect.Message {
1477- mi := &file_ermite_storage_ci_proto_msgTypes[5]
1679+ mi := &file_ermite_storage_ci_proto_msgTypes[6]
14781680 if x != nil {
14791681 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
14801682 if ms.LoadMessageInfo() == nil {
@@ -1487,7 +1689,7 @@ func (x *CIRunner) ProtoReflect() protoreflect.Message {
14871689
14881690 // Deprecated: Use CIRunner.ProtoReflect.Descriptor instead.
14891691 func (*CIRunner) Descriptor() ([]byte, []int) {
1490- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{5}
1692+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{6}
14911693 }
14921694
14931695 func (x *CIRunner) GetId() string {
@@ -1586,7 +1788,7 @@ type CIRepoToken struct {
15861788
15871789 func (x *CIRepoToken) Reset() {
15881790 *x = CIRepoToken{}
1589- mi := &file_ermite_storage_ci_proto_msgTypes[6]
1791+ mi := &file_ermite_storage_ci_proto_msgTypes[7]
15901792 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
15911793 ms.StoreMessageInfo(mi)
15921794 }
@@ -1598,7 +1800,7 @@ func (x *CIRepoToken) String() string {
15981800 func (*CIRepoToken) ProtoMessage() {}
15991801
16001802 func (x *CIRepoToken) ProtoReflect() protoreflect.Message {
1601- mi := &file_ermite_storage_ci_proto_msgTypes[6]
1803+ mi := &file_ermite_storage_ci_proto_msgTypes[7]
16021804 if x != nil {
16031805 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
16041806 if ms.LoadMessageInfo() == nil {
@@ -1611,7 +1813,7 @@ func (x *CIRepoToken) ProtoReflect() protoreflect.Message {
16111813
16121814 // Deprecated: Use CIRepoToken.ProtoReflect.Descriptor instead.
16131815 func (*CIRepoToken) Descriptor() ([]byte, []int) {
1614- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{6}
1816+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{7}
16151817 }
16161818
16171819 func (x *CIRepoToken) GetId() string {
@@ -1708,7 +1910,7 @@ type CIActionResolution struct {
17081910
17091911 func (x *CIActionResolution) Reset() {
17101912 *x = CIActionResolution{}
1711- mi := &file_ermite_storage_ci_proto_msgTypes[7]
1913+ mi := &file_ermite_storage_ci_proto_msgTypes[8]
17121914 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
17131915 ms.StoreMessageInfo(mi)
17141916 }
@@ -1720,7 +1922,7 @@ func (x *CIActionResolution) String() string {
17201922 func (*CIActionResolution) ProtoMessage() {}
17211923
17221924 func (x *CIActionResolution) ProtoReflect() protoreflect.Message {
1723- mi := &file_ermite_storage_ci_proto_msgTypes[7]
1925+ mi := &file_ermite_storage_ci_proto_msgTypes[8]
17241926 if x != nil {
17251927 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
17261928 if ms.LoadMessageInfo() == nil {
@@ -1733,7 +1935,7 @@ func (x *CIActionResolution) ProtoReflect() protoreflect.Message {
17331935
17341936 // Deprecated: Use CIActionResolution.ProtoReflect.Descriptor instead.
17351937 func (*CIActionResolution) Descriptor() ([]byte, []int) {
1736- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{7}
1938+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{8}
17371939 }
17381940
17391941 func (x *CIActionResolution) GetKey() string {
@@ -1781,7 +1983,7 @@ type CIAnnotation struct {
17811983
17821984 func (x *CIAnnotation) Reset() {
17831985 *x = CIAnnotation{}
1784- mi := &file_ermite_storage_ci_proto_msgTypes[8]
1986+ mi := &file_ermite_storage_ci_proto_msgTypes[9]
17851987 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
17861988 ms.StoreMessageInfo(mi)
17871989 }
@@ -1793,7 +1995,7 @@ func (x *CIAnnotation) String() string {
17931995 func (*CIAnnotation) ProtoMessage() {}
17941996
17951997 func (x *CIAnnotation) ProtoReflect() protoreflect.Message {
1796- mi := &file_ermite_storage_ci_proto_msgTypes[8]
1998+ mi := &file_ermite_storage_ci_proto_msgTypes[9]
17971999 if x != nil {
17982000 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
17992001 if ms.LoadMessageInfo() == nil {
@@ -1806,7 +2008,7 @@ func (x *CIAnnotation) ProtoReflect() protoreflect.Message {
18062008
18072009 // Deprecated: Use CIAnnotation.ProtoReflect.Descriptor instead.
18082010 func (*CIAnnotation) Descriptor() ([]byte, []int) {
1809- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{8}
2011+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{9}
18102012 }
18112013
18122014 func (x *CIAnnotation) GetLevel() CIAnnotationLevel {
@@ -1878,7 +2080,7 @@ type CIStepAnnotations struct {
18782080
18792081 func (x *CIStepAnnotations) Reset() {
18802082 *x = CIStepAnnotations{}
1881- mi := &file_ermite_storage_ci_proto_msgTypes[9]
2083+ mi := &file_ermite_storage_ci_proto_msgTypes[10]
18822084 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
18832085 ms.StoreMessageInfo(mi)
18842086 }
@@ -1890,7 +2092,7 @@ func (x *CIStepAnnotations) String() string {
18902092 func (*CIStepAnnotations) ProtoMessage() {}
18912093
18922094 func (x *CIStepAnnotations) ProtoReflect() protoreflect.Message {
1893- mi := &file_ermite_storage_ci_proto_msgTypes[9]
2095+ mi := &file_ermite_storage_ci_proto_msgTypes[10]
18942096 if x != nil {
18952097 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
18962098 if ms.LoadMessageInfo() == nil {
@@ -1903,7 +2105,7 @@ func (x *CIStepAnnotations) ProtoReflect() protoreflect.Message {
19032105
19042106 // Deprecated: Use CIStepAnnotations.ProtoReflect.Descriptor instead.
19052107 func (*CIStepAnnotations) Descriptor() ([]byte, []int) {
1906- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{9}
2108+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{10}
19072109 }
19082110
19092111 func (x *CIStepAnnotations) GetAnnotations() []*CIAnnotation {
@@ -1933,7 +2135,7 @@ type CIConcurrency struct {
19332135
19342136 func (x *CIConcurrency) Reset() {
19352137 *x = CIConcurrency{}
1936- mi := &file_ermite_storage_ci_proto_msgTypes[10]
2138+ mi := &file_ermite_storage_ci_proto_msgTypes[11]
19372139 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
19382140 ms.StoreMessageInfo(mi)
19392141 }
@@ -1945,7 +2147,7 @@ func (x *CIConcurrency) String() string {
19452147 func (*CIConcurrency) ProtoMessage() {}
19462148
19472149 func (x *CIConcurrency) ProtoReflect() protoreflect.Message {
1948- mi := &file_ermite_storage_ci_proto_msgTypes[10]
2150+ mi := &file_ermite_storage_ci_proto_msgTypes[11]
19492151 if x != nil {
19502152 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
19512153 if ms.LoadMessageInfo() == nil {
@@ -1958,7 +2160,7 @@ func (x *CIConcurrency) ProtoReflect() protoreflect.Message {
19582160
19592161 // Deprecated: Use CIConcurrency.ProtoReflect.Descriptor instead.
19602162 func (*CIConcurrency) Descriptor() ([]byte, []int) {
1961- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{10}
2163+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{11}
19622164 }
19632165
19642166 func (x *CIConcurrency) GetGroup() string {
@@ -2019,7 +2221,7 @@ type CIConcurrencyEntry struct {
20192221
20202222 func (x *CIConcurrencyEntry) Reset() {
20212223 *x = CIConcurrencyEntry{}
2022- mi := &file_ermite_storage_ci_proto_msgTypes[11]
2224+ mi := &file_ermite_storage_ci_proto_msgTypes[12]
20232225 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
20242226 ms.StoreMessageInfo(mi)
20252227 }
@@ -2031,7 +2233,7 @@ func (x *CIConcurrencyEntry) String() string {
20312233 func (*CIConcurrencyEntry) ProtoMessage() {}
20322234
20332235 func (x *CIConcurrencyEntry) ProtoReflect() protoreflect.Message {
2034- mi := &file_ermite_storage_ci_proto_msgTypes[11]
2236+ mi := &file_ermite_storage_ci_proto_msgTypes[12]
20352237 if x != nil {
20362238 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
20372239 if ms.LoadMessageInfo() == nil {
@@ -2044,7 +2246,7 @@ func (x *CIConcurrencyEntry) ProtoReflect() protoreflect.Message {
20442246
20452247 // Deprecated: Use CIConcurrencyEntry.ProtoReflect.Descriptor instead.
20462248 func (*CIConcurrencyEntry) Descriptor() ([]byte, []int) {
2047- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{11}
2249+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{12}
20482250 }
20492251
20502252 func (x *CIConcurrencyEntry) GetRunId() int64 {
@@ -2095,7 +2297,7 @@ type CIConcurrencyRecord struct {
20952297
20962298 func (x *CIConcurrencyRecord) Reset() {
20972299 *x = CIConcurrencyRecord{}
2098- mi := &file_ermite_storage_ci_proto_msgTypes[12]
2300+ mi := &file_ermite_storage_ci_proto_msgTypes[13]
20992301 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
21002302 ms.StoreMessageInfo(mi)
21012303 }
@@ -2107,7 +2309,7 @@ func (x *CIConcurrencyRecord) String() string {
21072309 func (*CIConcurrencyRecord) ProtoMessage() {}
21082310
21092311 func (x *CIConcurrencyRecord) ProtoReflect() protoreflect.Message {
2110- mi := &file_ermite_storage_ci_proto_msgTypes[12]
2312+ mi := &file_ermite_storage_ci_proto_msgTypes[13]
21112313 if x != nil {
21122314 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
21132315 if ms.LoadMessageInfo() == nil {
@@ -2120,7 +2322,7 @@ func (x *CIConcurrencyRecord) ProtoReflect() protoreflect.Message {
21202322
21212323 // Deprecated: Use CIConcurrencyRecord.ProtoReflect.Descriptor instead.
21222324 func (*CIConcurrencyRecord) Descriptor() ([]byte, []int) {
2123- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{12}
2325+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{13}
21242326 }
21252327
21262328 func (x *CIConcurrencyRecord) GetRunning() *CIConcurrencyEntry {
@@ -2156,7 +2358,7 @@ type CIConcurrencyMark struct {
21562358
21572359 func (x *CIConcurrencyMark) Reset() {
21582360 *x = CIConcurrencyMark{}
2159- mi := &file_ermite_storage_ci_proto_msgTypes[13]
2361+ mi := &file_ermite_storage_ci_proto_msgTypes[14]
21602362 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
21612363 ms.StoreMessageInfo(mi)
21622364 }
@@ -2168,7 +2370,7 @@ func (x *CIConcurrencyMark) String() string {
21682370 func (*CIConcurrencyMark) ProtoMessage() {}
21692371
21702372 func (x *CIConcurrencyMark) ProtoReflect() protoreflect.Message {
2171- mi := &file_ermite_storage_ci_proto_msgTypes[13]
2373+ mi := &file_ermite_storage_ci_proto_msgTypes[14]
21722374 if x != nil {
21732375 ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x))
21742376 if ms.LoadMessageInfo() == nil {
@@ -2181,7 +2383,7 @@ func (x *CIConcurrencyMark) ProtoReflect() protoreflect.Message {
21812383
21822384 // Deprecated: Use CIConcurrencyMark.ProtoReflect.Descriptor instead.
21832385 func (*CIConcurrencyMark) Descriptor() ([]byte, []int) {
2184- return file_ermite_storage_ci_proto_rawDescGZIP(), []int{13}
2386+ return file_ermite_storage_ci_proto_rawDescGZIP(), []int{14}
21852387 }
21862388
21872389 func (x *CIConcurrencyMark) GetEntry() *CIConcurrencyEntry {
@@ -2260,7 +2462,7 @@ const file_ermite_storage_ci_proto_rawDesc = "" +
22602462 "\tlog_start\x18\a \x01(\x03R\blogStart\x12\x1b\n" +
22612463 "\thas_lines\x18\t \x01(\bR\bhasLines\x126\n" +
22622464 "\aoutcome\x18\v \x01(\x0e2\x1c.ermite.storage.CIConclusionR\aoutcomeJ\x04\b\b\x10\tJ\x04\b\n" +
2263- "\x10\v\"\xd3\x0e\n" +
2465+ "\x10\v\"\xae\x0f\n" +
22642466 "\x05CIJob\x12\x0e\n" +
22652467 "\x02id\x18\x01 \x01(\tR\x02id\x12\x15\n" +
22662468 "\x06run_id\x18\x02 \x01(\x03R\x05runId\x12\x18\n" +
@@ -2314,13 +2516,39 @@ const file_ermite_storage_ci_proto_rawDesc = "" +
23142516 "\x11continue_on_error\x18( \x01(\bR\x0fcontinueOnError\x12 \n" +
23152517 "\venvironment\x18) \x01(\tR\venvironment\x12H\n" +
23162518 "\vpermissions\x18* \x03(\v2&.ermite.storage.CIJob.PermissionsEntryR\vpermissions\x12?\n" +
2317- "\vconcurrency\x18+ \x01(\v2\x1d.ermite.storage.CIConcurrencyR\vconcurrency\x1a:\n" +
2519+ "\vconcurrency\x18+ \x01(\v2\x1d.ermite.storage.CIConcurrencyR\vconcurrency\x12\x14\n" +
2520+ "\x05scope\x18, \x01(\tR\x05scope\x12\x17\n" +
2521+ "\acall_id\x18- \x01(\tR\x06callId\x12*\n" +
2522+ "\x04call\x18. \x01(\v2\x16.ermite.storage.CICallR\x04call\x1a:\n" +
23182523 "\fOutputsEntry\x12\x10\n" +
23192524 "\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
23202525 "\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\x1a>\n" +
23212526 "\x10PermissionsEntry\x12\x10\n" +
23222527 "\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
2323- "\x05value\x18\x02 \x01(\tR\x05value:\x028\x01J\x04\b$\x10%\"\xe5\x03\n" +
2528+ "\x05value\x18\x02 \x01(\tR\x05value:\x028\x01J\x04\b$\x10%\"\xec\x03\n" +
2529+ "\x06CICall\x12\x12\n" +
2530+ "\x04uses\x18\x01 \x01(\tR\x04uses\x12\x12\n" +
2531+ "\x04path\x18\x02 \x01(\tR\x04path\x12\x1e\n" +
2532+ "\n" +
2533+ "repository\x18\x03 \x01(\tR\n" +
2534+ "repository\x12\x10\n" +
2535+ "\x03ref\x18\x04 \x01(\tR\x03ref\x12\x10\n" +
2536+ "\x03sha\x18\x05 \x01(\tR\x03sha\x12\x12\n" +
2537+ "\x04yaml\x18\x06 \x01(\tR\x04yaml\x12\x1f\n" +
2538+ "\vinputs_json\x18\a \x01(\tR\n" +
2539+ "inputsJson\x12=\n" +
2540+ "\asecrets\x18\b \x03(\v2#.ermite.storage.CICall.SecretsEntryR\asecrets\x12\x18\n" +
2541+ "\ainherit\x18\t \x01(\bR\ainherit\x12\x14\n" +
2542+ "\x05depth\x18\n" +
2543+ " \x01(\x05R\x05depth\x12\x1a\n" +
2544+ "\bexpanded\x18\v \x01(\bR\bexpanded\x12?\n" +
2545+ "\vconcurrency\x18\f \x01(\v2\x1d.ermite.storage.CIConcurrencyR\vconcurrency\x12\x1a\n" +
2546+ "\badmitted\x18\r \x01(\bR\badmitted\x12\x1d\n" +
2547+ "\n" +
2548+ "secret_ctx\x18\x0e \x01(\tR\tsecretCtx\x1a:\n" +
2549+ "\fSecretsEntry\x12\x10\n" +
2550+ "\x03key\x18\x01 \x01(\tR\x03key\x12\x14\n" +
2551+ "\x05value\x18\x02 \x01(\tR\x05value:\x028\x01\"\xe5\x03\n" +
23242552 "\aCICheck\x12\x0e\n" +
23252553 "\x02id\x18\x01 \x01(\tR\x02id\x12\x12\n" +
23262554 "\x04repo\x18\x02 \x01(\tR\x04repo\x12\x10\n" +
@@ -2470,7 +2698,7 @@ func file_ermite_storage_ci_proto_rawDescGZIP() []byte {
24702698 }
24712699
24722700 var file_ermite_storage_ci_proto_enumTypes = make([]protoimpl.EnumInfo, 8)
2473-var file_ermite_storage_ci_proto_msgTypes = make([]protoimpl.MessageInfo, 17)
2701+var file_ermite_storage_ci_proto_msgTypes = make([]protoimpl.MessageInfo, 19)
24742702 var file_ermite_storage_ci_proto_goTypes = []any{
24752703 (CIStatus)(0), // 0: ermite.storage.CIStatus
24762704 (CIConclusion)(0), // 1: ermite.storage.CIConclusion
@@ -2484,72 +2712,77 @@ var file_ermite_storage_ci_proto_goTypes = []any{
24842712 (*CIRunSource)(nil), // 9: ermite.storage.CIRunSource
24852713 (*CIStep)(nil), // 10: ermite.storage.CIStep
24862714 (*CIJob)(nil), // 11: ermite.storage.CIJob
2487- (*CICheck)(nil), // 12: ermite.storage.CICheck
2488- (*CIRunner)(nil), // 13: ermite.storage.CIRunner
2489- (*CIRepoToken)(nil), // 14: ermite.storage.CIRepoToken
2490- (*CIActionResolution)(nil), // 15: ermite.storage.CIActionResolution
2491- (*CIAnnotation)(nil), // 16: ermite.storage.CIAnnotation
2492- (*CIStepAnnotations)(nil), // 17: ermite.storage.CIStepAnnotations
2493- (*CIConcurrency)(nil), // 18: ermite.storage.CIConcurrency
2494- (*CIConcurrencyEntry)(nil), // 19: ermite.storage.CIConcurrencyEntry
2495- (*CIConcurrencyRecord)(nil), // 20: ermite.storage.CIConcurrencyRecord
2496- (*CIConcurrencyMark)(nil), // 21: ermite.storage.CIConcurrencyMark
2497- nil, // 22: ermite.storage.CIJob.OutputsEntry
2498- nil, // 23: ermite.storage.CIJob.PermissionsEntry
2499- nil, // 24: ermite.storage.CIRepoToken.PermissionsEntry
2500- (*timestamppb.Timestamp)(nil), // 25: google.protobuf.Timestamp
2715+ (*CICall)(nil), // 12: ermite.storage.CICall
2716+ (*CICheck)(nil), // 13: ermite.storage.CICheck
2717+ (*CIRunner)(nil), // 14: ermite.storage.CIRunner
2718+ (*CIRepoToken)(nil), // 15: ermite.storage.CIRepoToken
2719+ (*CIActionResolution)(nil), // 16: ermite.storage.CIActionResolution
2720+ (*CIAnnotation)(nil), // 17: ermite.storage.CIAnnotation
2721+ (*CIStepAnnotations)(nil), // 18: ermite.storage.CIStepAnnotations
2722+ (*CIConcurrency)(nil), // 19: ermite.storage.CIConcurrency
2723+ (*CIConcurrencyEntry)(nil), // 20: ermite.storage.CIConcurrencyEntry
2724+ (*CIConcurrencyRecord)(nil), // 21: ermite.storage.CIConcurrencyRecord
2725+ (*CIConcurrencyMark)(nil), // 22: ermite.storage.CIConcurrencyMark
2726+ nil, // 23: ermite.storage.CIJob.OutputsEntry
2727+ nil, // 24: ermite.storage.CIJob.PermissionsEntry
2728+ nil, // 25: ermite.storage.CICall.SecretsEntry
2729+ nil, // 26: ermite.storage.CIRepoToken.PermissionsEntry
2730+ (*timestamppb.Timestamp)(nil), // 27: google.protobuf.Timestamp
25012731 }
25022732 var file_ermite_storage_ci_proto_depIdxs = []int32{
25032733 0, // 0: ermite.storage.CIRun.status:type_name -> ermite.storage.CIStatus
25042734 1, // 1: ermite.storage.CIRun.conclusion:type_name -> ermite.storage.CIConclusion
2505- 25, // 2: ermite.storage.CIRun.created_at:type_name -> google.protobuf.Timestamp
2506- 25, // 3: ermite.storage.CIRun.started_at:type_name -> google.protobuf.Timestamp
2507- 25, // 4: ermite.storage.CIRun.completed_at:type_name -> google.protobuf.Timestamp
2508- 25, // 5: ermite.storage.CIRun.cancel_requested_at:type_name -> google.protobuf.Timestamp
2509- 18, // 6: ermite.storage.CIRun.concurrency:type_name -> ermite.storage.CIConcurrency
2735+ 27, // 2: ermite.storage.CIRun.created_at:type_name -> google.protobuf.Timestamp
2736+ 27, // 3: ermite.storage.CIRun.started_at:type_name -> google.protobuf.Timestamp
2737+ 27, // 4: ermite.storage.CIRun.completed_at:type_name -> google.protobuf.Timestamp
2738+ 27, // 5: ermite.storage.CIRun.cancel_requested_at:type_name -> google.protobuf.Timestamp
2739+ 19, // 6: ermite.storage.CIRun.concurrency:type_name -> ermite.storage.CIConcurrency
25102740 2, // 7: ermite.storage.CIStep.state:type_name -> ermite.storage.CIStepState
25112741 1, // 8: ermite.storage.CIStep.conclusion:type_name -> ermite.storage.CIConclusion
2512- 25, // 9: ermite.storage.CIStep.started_at:type_name -> google.protobuf.Timestamp
2513- 25, // 10: ermite.storage.CIStep.completed_at:type_name -> google.protobuf.Timestamp
2742+ 27, // 9: ermite.storage.CIStep.started_at:type_name -> google.protobuf.Timestamp
2743+ 27, // 10: ermite.storage.CIStep.completed_at:type_name -> google.protobuf.Timestamp
25142744 1, // 11: ermite.storage.CIStep.outcome:type_name -> ermite.storage.CIConclusion
25152745 3, // 12: ermite.storage.CIJob.phase:type_name -> ermite.storage.CIJobPhase
25162746 1, // 13: ermite.storage.CIJob.conclusion:type_name -> ermite.storage.CIConclusion
25172747 4, // 14: ermite.storage.CIJob.isolation:type_name -> ermite.storage.CIIsolation
25182748 10, // 15: ermite.storage.CIJob.steps:type_name -> ermite.storage.CIStep
2519- 25, // 16: ermite.storage.CIJob.created_at:type_name -> google.protobuf.Timestamp
2520- 25, // 17: ermite.storage.CIJob.started_at:type_name -> google.protobuf.Timestamp
2521- 25, // 18: ermite.storage.CIJob.completed_at:type_name -> google.protobuf.Timestamp
2522- 22, // 19: ermite.storage.CIJob.outputs:type_name -> ermite.storage.CIJob.OutputsEntry
2523- 25, // 20: ermite.storage.CIJob.ready_at:type_name -> google.protobuf.Timestamp
2524- 25, // 21: ermite.storage.CIJob.assigned_at:type_name -> google.protobuf.Timestamp
2525- 25, // 22: ermite.storage.CIJob.acked_at:type_name -> google.protobuf.Timestamp
2526- 25, // 23: ermite.storage.CIJob.token_valid_until:type_name -> google.protobuf.Timestamp
2749+ 27, // 16: ermite.storage.CIJob.created_at:type_name -> google.protobuf.Timestamp
2750+ 27, // 17: ermite.storage.CIJob.started_at:type_name -> google.protobuf.Timestamp
2751+ 27, // 18: ermite.storage.CIJob.completed_at:type_name -> google.protobuf.Timestamp
2752+ 23, // 19: ermite.storage.CIJob.outputs:type_name -> ermite.storage.CIJob.OutputsEntry
2753+ 27, // 20: ermite.storage.CIJob.ready_at:type_name -> google.protobuf.Timestamp
2754+ 27, // 21: ermite.storage.CIJob.assigned_at:type_name -> google.protobuf.Timestamp
2755+ 27, // 22: ermite.storage.CIJob.acked_at:type_name -> google.protobuf.Timestamp
2756+ 27, // 23: ermite.storage.CIJob.token_valid_until:type_name -> google.protobuf.Timestamp
25272757 5, // 24: ermite.storage.CIJob.cancel_reason:type_name -> ermite.storage.CICancelReason
2528- 25, // 25: ermite.storage.CIJob.cancel_requested_at:type_name -> google.protobuf.Timestamp
2529- 23, // 26: ermite.storage.CIJob.permissions:type_name -> ermite.storage.CIJob.PermissionsEntry
2530- 18, // 27: ermite.storage.CIJob.concurrency:type_name -> ermite.storage.CIConcurrency
2531- 0, // 28: ermite.storage.CICheck.status:type_name -> ermite.storage.CIStatus
2532- 1, // 29: ermite.storage.CICheck.conclusion:type_name -> ermite.storage.CIConclusion
2533- 25, // 30: ermite.storage.CICheck.created_at:type_name -> google.protobuf.Timestamp
2534- 25, // 31: ermite.storage.CICheck.started_at:type_name -> google.protobuf.Timestamp
2535- 25, // 32: ermite.storage.CICheck.completed_at:type_name -> google.protobuf.Timestamp
2536- 4, // 33: ermite.storage.CIRunner.isolation:type_name -> ermite.storage.CIIsolation
2537- 25, // 34: ermite.storage.CIRunner.registered_at:type_name -> google.protobuf.Timestamp
2538- 25, // 35: ermite.storage.CIRepoToken.expires_at:type_name -> google.protobuf.Timestamp
2539- 25, // 36: ermite.storage.CIRepoToken.created_at:type_name -> google.protobuf.Timestamp
2540- 24, // 37: ermite.storage.CIRepoToken.permissions:type_name -> ermite.storage.CIRepoToken.PermissionsEntry
2541- 6, // 38: ermite.storage.CIAnnotation.level:type_name -> ermite.storage.CIAnnotationLevel
2542- 16, // 39: ermite.storage.CIStepAnnotations.annotations:type_name -> ermite.storage.CIAnnotation
2543- 7, // 40: ermite.storage.CIConcurrency.state:type_name -> ermite.storage.CIConcurrencyState
2544- 19, // 41: ermite.storage.CIConcurrencyRecord.running:type_name -> ermite.storage.CIConcurrencyEntry
2545- 19, // 42: ermite.storage.CIConcurrencyRecord.pending:type_name -> ermite.storage.CIConcurrencyEntry
2546- 19, // 43: ermite.storage.CIConcurrencyMark.entry:type_name -> ermite.storage.CIConcurrencyEntry
2547- 7, // 44: ermite.storage.CIConcurrencyMark.state:type_name -> ermite.storage.CIConcurrencyState
2548- 45, // [45:45] is the sub-list for method output_type
2549- 45, // [45:45] is the sub-list for method input_type
2550- 45, // [45:45] is the sub-list for extension type_name
2551- 45, // [45:45] is the sub-list for extension extendee
2552- 0, // [0:45] is the sub-list for field type_name
2758+ 27, // 25: ermite.storage.CIJob.cancel_requested_at:type_name -> google.protobuf.Timestamp
2759+ 24, // 26: ermite.storage.CIJob.permissions:type_name -> ermite.storage.CIJob.PermissionsEntry
2760+ 19, // 27: ermite.storage.CIJob.concurrency:type_name -> ermite.storage.CIConcurrency
2761+ 12, // 28: ermite.storage.CIJob.call:type_name -> ermite.storage.CICall
2762+ 25, // 29: ermite.storage.CICall.secrets:type_name -> ermite.storage.CICall.SecretsEntry
2763+ 19, // 30: ermite.storage.CICall.concurrency:type_name -> ermite.storage.CIConcurrency
2764+ 0, // 31: ermite.storage.CICheck.status:type_name -> ermite.storage.CIStatus
2765+ 1, // 32: ermite.storage.CICheck.conclusion:type_name -> ermite.storage.CIConclusion
2766+ 27, // 33: ermite.storage.CICheck.created_at:type_name -> google.protobuf.Timestamp
2767+ 27, // 34: ermite.storage.CICheck.started_at:type_name -> google.protobuf.Timestamp
2768+ 27, // 35: ermite.storage.CICheck.completed_at:type_name -> google.protobuf.Timestamp
2769+ 4, // 36: ermite.storage.CIRunner.isolation:type_name -> ermite.storage.CIIsolation
2770+ 27, // 37: ermite.storage.CIRunner.registered_at:type_name -> google.protobuf.Timestamp
2771+ 27, // 38: ermite.storage.CIRepoToken.expires_at:type_name -> google.protobuf.Timestamp
2772+ 27, // 39: ermite.storage.CIRepoToken.created_at:type_name -> google.protobuf.Timestamp
2773+ 26, // 40: ermite.storage.CIRepoToken.permissions:type_name -> ermite.storage.CIRepoToken.PermissionsEntry
2774+ 6, // 41: ermite.storage.CIAnnotation.level:type_name -> ermite.storage.CIAnnotationLevel
2775+ 17, // 42: ermite.storage.CIStepAnnotations.annotations:type_name -> ermite.storage.CIAnnotation
2776+ 7, // 43: ermite.storage.CIConcurrency.state:type_name -> ermite.storage.CIConcurrencyState
2777+ 20, // 44: ermite.storage.CIConcurrencyRecord.running:type_name -> ermite.storage.CIConcurrencyEntry
2778+ 20, // 45: ermite.storage.CIConcurrencyRecord.pending:type_name -> ermite.storage.CIConcurrencyEntry
2779+ 20, // 46: ermite.storage.CIConcurrencyMark.entry:type_name -> ermite.storage.CIConcurrencyEntry
2780+ 7, // 47: ermite.storage.CIConcurrencyMark.state:type_name -> ermite.storage.CIConcurrencyState
2781+ 48, // [48:48] is the sub-list for method output_type
2782+ 48, // [48:48] is the sub-list for method input_type
2783+ 48, // [48:48] is the sub-list for extension type_name
2784+ 48, // [48:48] is the sub-list for extension extendee
2785+ 0, // [0:48] is the sub-list for field type_name
25532786 }
25542787
25552788 func init() { file_ermite_storage_ci_proto_init() }
@@ -2563,7 +2796,7 @@ func file_ermite_storage_ci_proto_init() {
25632796 GoPackagePath: reflect.TypeOf(x{}).PkgPath(),
25642797 RawDescriptor: unsafe.Slice(unsafe.StringData(file_ermite_storage_ci_proto_rawDesc), len(file_ermite_storage_ci_proto_rawDesc)),
25652798 NumEnums: 8,
2566- NumMessages: 17,
2799+ NumMessages: 19,
25672800 NumExtensions: 0,
25682801 NumServices: 0,
25692802 },
A internal/adapters/githubraw/githubraw.go +101 -0
@@ -0,0 +1,101 @@
1+// SPDX-License-Identifier: AGPL-3.0-or-later
2+
3+// Package githubraw downloads a reusable workflow file from
4+// raw.githubusercontent.com at a commit (ports.WorkflowFetcher). It sends no
5+// credentials, bounds the time and the size of the download, and follows no
6+// redirect to another host.
7+package githubraw
8+
9+import (
10+ "context"
11+ "fmt"
12+ "io"
13+ "net/http"
14+ "net/url"
15+ "strings"
16+ "time"
17+
18+ "bnema.dev/ermite/internal/domain"
19+ "bnema.dev/ermite/internal/ports"
20+)
21+
22+// DefaultBaseURL is where the files are.
23+const DefaultBaseURL = "https://raw.githubusercontent.com"
24+
25+// DefaultTimeout bounds one download.
26+const DefaultTimeout = 20 * time.Second
27+
28+// Config configures a Fetcher; tests replace BaseURL and Client.
29+type Config struct {
30+ BaseURL string
31+ Timeout time.Duration
32+ Client *http.Client
33+}
34+
35+// Fetcher implements ports.WorkflowFetcher over HTTPS.
36+type Fetcher struct {
37+ base string
38+ client *http.Client
39+}
40+
41+var _ ports.WorkflowFetcher = (*Fetcher)(nil)
42+
43+// New builds a Fetcher.
44+func New(cfg Config) *Fetcher {
45+ if cfg.BaseURL == "" {
46+ cfg.BaseURL = DefaultBaseURL
47+ }
48+ if cfg.Timeout <= 0 {
49+ cfg.Timeout = DefaultTimeout
50+ }
51+ c := cfg.Client
52+ if c == nil {
53+ c = &http.Client{Transport: http.DefaultTransport}
54+ }
55+ cc := *c
56+ cc.Timeout = cfg.Timeout
57+ cc.CheckRedirect = func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }
58+ return &Fetcher{base: strings.TrimRight(cfg.BaseURL, "/"), client: &cc}
59+}
60+
61+// FetchWorkflow downloads path of owner/repo at sha.
62+func (f *Fetcher) FetchWorkflow(ctx context.Context, owner, repo, sha, path string, maxBytes int64) ([]byte, error) {
63+ segs := []string{owner, repo, sha}
64+ segs = append(segs, strings.Split(path, "/")...)
65+ for _, s := range segs {
66+ if s == "" || s == "." || s == ".." {
67+ return nil, fmt.Errorf("%w: bad workflow location", domain.ErrInvalid)
68+ }
69+ }
70+ esc := make([]string, len(segs))
71+ for i, s := range segs {
72+ esc[i] = url.PathEscape(s)
73+ }
74+ req, err := http.NewRequestWithContext(ctx, http.MethodGet, f.base+"/"+strings.Join(esc, "/"), nil)
75+ if err != nil {
76+ return nil, err
77+ }
78+ res, err := f.client.Do(req)
79+ if err != nil {
80+ if ctx.Err() != nil {
81+ return nil, ctx.Err()
82+ }
83+ return nil, fmt.Errorf("%w: download the reusable workflow: %w", domain.ErrUnavailable, err)
84+ }
85+ defer func() { _ = res.Body.Close() }()
86+ switch res.StatusCode {
87+ case http.StatusOK:
88+ case http.StatusNotFound, http.StatusUnauthorized, http.StatusForbidden:
89+ return nil, domain.ErrNotFound
90+ default:
91+ return nil, fmt.Errorf("%w: the download answered %d", domain.ErrUnavailable, res.StatusCode)
92+ }
93+ data, err := io.ReadAll(io.LimitReader(res.Body, maxBytes+1))
94+ if err != nil {
95+ return nil, fmt.Errorf("%w: download the reusable workflow: %w", domain.ErrUnavailable, err)
96+ }
97+ if int64(len(data)) > maxBytes {
98+ return nil, fmt.Errorf("%w: the reusable workflow is larger than %d bytes", domain.ErrResourceExhausted, maxBytes)
99+ }
100+ return data, nil
101+}
A internal/adapters/githubraw/githubraw_test.go +57 -0
@@ -0,0 +1,57 @@
1+// SPDX-License-Identifier: AGPL-3.0-or-later
2+
3+package githubraw_test
4+
5+import (
6+ "context"
7+ "net/http"
8+ "net/http/httptest"
9+ "strings"
10+ "testing"
11+
12+ "github.com/stretchr/testify/assert"
13+ "github.com/stretchr/testify/require"
14+
15+ "bnema.dev/ermite/internal/adapters/githubraw"
16+ "bnema.dev/ermite/internal/domain"
17+)
18+
19+// The server is local: the test never reaches github.com.
20+func TestFetchWorkflow(t *testing.T) {
21+ t.Parallel()
22+ var got string
23+ srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
24+ got = r.URL.Path
25+ switch {
26+ case strings.HasSuffix(r.URL.Path, "/ok.yml"):
27+ _, _ = w.Write([]byte("on: workflow_call\n"))
28+ case strings.HasSuffix(r.URL.Path, "/big.yml"):
29+ _, _ = w.Write([]byte(strings.Repeat("x", 100)))
30+ case strings.HasSuffix(r.URL.Path, "/moved.yml"):
31+ http.Redirect(w, r, "http://127.0.0.1:1/x", http.StatusFound)
32+ case strings.HasSuffix(r.URL.Path, "/boom.yml"):
33+ w.WriteHeader(http.StatusBadGateway)
34+ default:
35+ http.NotFound(w, r)
36+ }
37+ }))
38+ t.Cleanup(srv.Close)
39+ f := githubraw.New(githubraw.Config{BaseURL: srv.URL})
40+ ctx := context.Background()
41+
42+ data, err := f.FetchWorkflow(ctx, "o", "r", "abc", ".github/workflows/ok.yml", 1024)
43+ require.NoError(t, err)
44+ assert.Equal(t, "on: workflow_call\n", string(data))
45+ assert.Equal(t, "/o/r/abc/.github/workflows/ok.yml", got)
46+
47+ _, err = f.FetchWorkflow(ctx, "o", "r", "abc", ".github/workflows/none.yml", 1024)
48+ assert.ErrorIs(t, err, domain.ErrNotFound)
49+ _, err = f.FetchWorkflow(ctx, "o", "r", "abc", ".github/workflows/big.yml", 10)
50+ assert.ErrorIs(t, err, domain.ErrResourceExhausted)
51+ _, err = f.FetchWorkflow(ctx, "o", "r", "abc", ".github/workflows/boom.yml", 10)
52+ assert.ErrorIs(t, err, domain.ErrUnavailable)
53+ _, err = f.FetchWorkflow(ctx, "o", "r", "abc", ".github/workflows/moved.yml", 10)
54+ assert.ErrorIs(t, err, domain.ErrUnavailable, "a redirect is not followed")
55+ _, err = f.FetchWorkflow(ctx, "o", "..", "abc", ".github/workflows/ok.yml", 10)
56+ assert.ErrorIs(t, err, domain.ErrInvalid)
57+}
M internal/adapters/kvstore/ci.go +24 -1
@@ -337,7 +337,7 @@ func encodeJob(j domain.CIJob) ([]byte, error) {
337337 AckedAt: encodeTime(j.AckedAt), AssignCancels: int32(j.AssignCancels), //nolint:gosec // at most 3
338338 TokenValidUntil: encodeTime(j.TokenValidUntil), CancelReason: encodeCancelReason(j.CancelReason), CancelMessage: j.CancelMessage,
339339 CancelRequestedAt: encodeTime(j.CancelRequestedAt), Planned: j.Planned, Environment: j.Environment, Permissions: j.Permissions,
340- Concurrency: encodeConcurrency(j.Concurrency),
340+ Concurrency: encodeConcurrency(j.Concurrency), Scope: j.Scope, CallId: j.CallID, Call: encodeCall(j.Call),
341341 }
342342 if j.HasToken() {
343343 pb.TokenHash = j.TokenHash[:]
@@ -368,6 +368,7 @@ func decodeJob(item kv.Item) (domain.CIJob, error) {
368368 AssignedAt: decodeTime(pb.GetAssignedAt()), AckedAt: decodeTime(pb.GetAckedAt()), AssignCancels: int(pb.GetAssignCancels()),
369369 CancelMessage: pb.GetCancelMessage(), CancelRequestedAt: decodeTime(pb.GetCancelRequestedAt()),
370370 Environment: pb.GetEnvironment(), Permissions: pb.GetPermissions(), Concurrency: decodeConcurrency(pb.GetConcurrency()), Version: item.Version,
371+ Scope: pb.GetScope(), CallID: pb.GetCallId(), Call: decodeCall(pb.GetCall()),
371372 }
372373 j.CancelReason = decodeCancelReason(pb.GetCancelReason())
373374 copy(j.TokenHash[:], pb.GetTokenHash())
@@ -1062,3 +1063,25 @@ func (c *CI) RemoveRunner(ctx context.Context, id string) (domain.CIRunner, erro
10621063 }
10631064 return domain.CIRunner{}, fmt.Errorf("remove runner %q: %w", id, domain.ErrConflict)
10641065 }
1066+
1067+func encodeCall(c *domain.CICall) *storagepb.CICall {
1068+ if c == nil {
1069+ return nil
1070+ }
1071+ return &storagepb.CICall{
1072+ Uses: c.Uses, Path: c.Path, Repository: c.Repository, Ref: c.Ref, Sha: c.Sha, Yaml: c.YAML, InputsJson: c.InputsJSON,
1073+ Secrets: c.Secrets, Inherit: c.Inherit, Depth: int32(c.Depth), Expanded: c.Expanded, Concurrency: encodeConcurrency(c.Concurrency), Admitted: c.Admitted, SecretCtx: c.SecretCtx, //nolint:gosec // at most 4
1074+ }
1075+}
1076+
1077+//hexcheck:ignore suspicious-business-logic-in-adapter maps the stored call to the domain call field by field: no rule is decided here
1078+func decodeCall(pb *storagepb.CICall) *domain.CICall {
1079+ if pb == nil {
1080+ return nil
1081+ }
1082+ return &domain.CICall{
1083+ Uses: pb.GetUses(), Path: pb.GetPath(), Repository: pb.GetRepository(), Ref: pb.GetRef(), Sha: pb.GetSha(), YAML: pb.GetYaml(),
1084+ InputsJSON: pb.GetInputsJson(), Secrets: pb.GetSecrets(), Inherit: pb.GetInherit(), Depth: int(pb.GetDepth()), Expanded: pb.GetExpanded(), Concurrency: decodeConcurrency(pb.GetConcurrency()),
1085+ Admitted: pb.GetAdmitted(), SecretCtx: pb.GetSecretCtx(),
1086+ }
1087+}
M internal/adapters/kvstore/store.go +5 -2
@@ -20,8 +20,11 @@ import (
2020 // SchemaVersion is the storage schema this build reads and writes. Version 2
2121 // adds branch prefixes to tokens: a build that does not know them would treat
2222 // a branch-limited token as unrestricted, so it must refuse the data
23-// (ErrSchemaTooNew) instead.
24-const SchemaVersion = 2
23+// (ErrSchemaTooNew) instead. Version 3 adds the records of CI-3 that an older
24+// build would misread: the call and scoped jobs of reusable workflows (it would
25+// run the jobs of a called workflow as jobs of the caller's file, and plan the
26+// call itself).
27+const SchemaVersion = 3
2528
2629 const schemaKey = "meta/schema"
2730
M internal/adapters/kvstore/store_test.go +28 -4
@@ -210,7 +210,7 @@ func TestSchema(t *testing.T) {
210210 assert.Equal(t, uint32(kvstore.SchemaVersion), readSchema(t, dir))
211211 })
212212
213- t.Run("version 1 is upgraded to 2, its data kept", func(t *testing.T) {
213+ t.Run("version 1 is upgraded to the current one, its data kept", func(t *testing.T) {
214214 dir := t.TempDir()
215215 putRaw(t, dir, "meta/schema", mustProto(t, &storagepb.SchemaVersion{Version: 1}))
216216 putRaw(t, dir, "repo/old", mustProto(t, &storagepb.Repo{Name: "old", Visibility: storagepb.Visibility_VISIBILITY_PUBLIC}))
@@ -220,15 +220,39 @@ func TestSchema(t *testing.T) {
220220 require.NoError(t, err)
221221 assert.Equal(t, "old", got.Name.String())
222222 require.NoError(t, s.Close(ctx))
223- assert.Equal(t, uint32(2), readSchema(t, dir))
223+ assert.Equal(t, uint32(kvstore.SchemaVersion), readSchema(t, dir))
224+ })
225+
226+ t.Run("version 2 is upgraded to 3, its data kept", func(t *testing.T) {
227+ dir := t.TempDir()
228+ putRaw(t, dir, "meta/schema", mustProto(t, &storagepb.SchemaVersion{Version: 2}))
229+ putRaw(t, dir, "repo/old", mustProto(t, &storagepb.Repo{Name: "old", Visibility: storagepb.Visibility_VISIBILITY_PUBLIC}))
230+ s, err := kvstore.Open(ctx, dir)
231+ require.NoError(t, err)
232+ _, err = s.Get(ctx, name(t, "old"))
233+ require.NoError(t, err)
234+ require.NoError(t, s.Close(ctx))
235+ assert.Equal(t, uint32(3), readSchema(t, dir))
236+ })
237+
238+ // Downgrade guard: once a build with reusable workflows has opened the data
239+ // (and may have stored calls), a build that knows schema 2 only must refuse
240+ // it; it does so for any version above its own (ErrSchemaTooNew), which is
241+ // the version 3 this build writes.
242+ t.Run("data written by this build is above schema 2", func(t *testing.T) {
243+ dir := t.TempDir()
244+ s, err := kvstore.Open(ctx, dir)
245+ require.NoError(t, err)
246+ require.NoError(t, s.Close(ctx))
247+ assert.Greater(t, readSchema(t, dir), uint32(2), "a build that supports 2 refuses it")
224248 })
225249
226250 t.Run("newer stops startup and releases the lock", func(t *testing.T) {
227251 dir := t.TempDir()
228- putRaw(t, dir, "meta/schema", mustProto(t, &storagepb.SchemaVersion{Version: 3}))
252+ putRaw(t, dir, "meta/schema", mustProto(t, &storagepb.SchemaVersion{Version: kvstore.SchemaVersion + 1}))
229253 _, err := kvstore.Open(ctx, dir)
230254 require.ErrorIs(t, err, kvstore.ErrSchemaTooNew)
231- assert.Equal(t, uint32(3), readSchema(t, dir), "the lock was released and the value kept")
255+ assert.Equal(t, uint32(kvstore.SchemaVersion+1), readSchema(t, dir), "the lock was released and the value kept")
232256 })
233257
234258 t.Run("garbage and zero are errors", func(t *testing.T) {
M internal/app/ci_flow_test.go +3 -3
@@ -316,8 +316,8 @@ jobs:
316316 assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_FAILURE, c.runs()[0].GetConclusion())
317317 }
318318
319-// A job with a feature of a later phase fails clearly, never silently.
320-func TestCIUnsupportedFeatureFailsTheJob(t *testing.T) {
319+// A job that calls a reusable workflow which is not there fails clearly, never silently.
320+func TestCICallOfAMissingWorkflowFailsTheJob(t *testing.T) {
321321 c := newCIHarness(t, nil)
322322 c.registerRunner("alpha", "ubuntu-latest")
323323 c.push("main", map[string]string{".github/workflows/u.yml": `on: push
@@ -328,7 +328,7 @@ jobs:
328328 c.waitFor("the run fails", func() bool { return len(c.runs()) == 1 && c.runs()[0].GetStatus() == pb.CiStatus_CI_STATUS_COMPLETED })
329329 j := c.getRun(c.runs()[0].GetId()).GetJobs()[0]
330330 assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_FAILURE, j.GetConclusion())
331- assert.Equal(t, "not supported yet: reusable workflows (uses at the job level)", j.GetError())
331+ assert.Contains(t, j.GetError(), "Unable to find reusable workflow '.github/workflows/callee.yml'")
332332 }
333333
334334 // A job for a label no runner has is "waiting for a runner" and fails after the
M internal/app/ci_parity_test.go +1 -23
@@ -308,7 +308,7 @@ jobs:
308308 }
309309
310310 // CI-2: planning runs what CI-1 refused (uses, container, services, a step
311-// timeout) and CI-3 environments, and still refuses job-level uses.
311+// timeout).
312312 func TestCIPlanningAcceptsCI2Features(t *testing.T) {
313313 c := newCIHarness(t, nil)
314314 runner := c.registerRunner("alpha", "ubuntu-latest")
@@ -327,28 +327,6 @@ jobs:
327327 `}, "")
328328 job := runner.await()
329329 assert.Equal(t, "j", job.GetJobKey())
330-
331- for _, tc := range []struct{ name, job, want string }{
332- {"reusable", "uses: ./.github/workflows/x.yml", "not supported yet: reusable workflows (uses at the job level)"},
333- } {
334- t.Run(tc.name, func(t *testing.T) {
335- body := "on: push\njobs:\n r:\n " + tc.job + "\n"
336- c.push("main", map[string]string{".github/workflows/" + tc.name + ".yml": body}, "")
337- var got string
338- c.waitFor(tc.name+" fails", func() bool {
339- for _, r := range c.runs() {
340- if r.GetWorkflowPath() == ".github/workflows/"+tc.name+".yml" && r.GetStatus() == pb.CiStatus_CI_STATUS_COMPLETED {
341- if jobs := c.getRun(r.GetId()).GetJobs(); len(jobs) == 1 {
342- got = jobs[0].GetError()
343- return true
344- }
345- }
346- }
347- return false
348- })
349- assert.Equal(t, tc.want, got)
350- })
351- }
352330 }
353331
354332 // A failed job with continue-on-error does not fail the run, keeps its own
A internal/app/ci_reusable_review_test.go +265 -0
@@ -0,0 +1,265 @@
1+package app
2+
3+import (
4+ "context"
5+ "fmt"
6+ "strings"
7+ "testing"
8+
9+ "connectrpc.com/connect"
10+ "github.com/stretchr/testify/assert"
11+ "github.com/stretchr/testify/mock"
12+ "github.com/stretchr/testify/require"
13+
14+ pb "bnema.dev/ermite-api/gen/ermite"
15+ "bnema.dev/ermite/internal/domain"
16+ "bnema.dev/ermite/internal/ports/mocks"
17+ "bnema.dev/ermite/internal/usecase"
18+)
19+
20+func TestCIReusableSecretOutputCannotForgeAReference(t *testing.T) {
21+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
22+ c.setSecret(repoScope("proj"), "UNPASSED", "private-value")
23+ caller := `on: push
24+jobs:
25+ prep:
26+ uses: ./.github/workflows/producer.yml
27+ call:
28+ needs: prep
29+ uses: ./.github/workflows/callee.yml
30+ secrets:
31+ TOKEN: ${{ needs.prep.outputs.payload }}
32+`
33+ producer := `on:
34+ workflow_call:
35+ outputs:
36+ payload:
37+ value: ${{ jobs.j.outputs.payload }}
38+jobs:
39+ j:
40+ runs-on: ubuntu-latest
41+ outputs:
42+ payload: ${{ steps.s.outputs.payload }}
43+ steps:
44+ - id: s
45+ run: echo
46+`
47+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/producer.yml": producer, ".github/workflows/callee.yml": secretsCallee}, "")
48+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
49+ prep := awaitRecord(c, r, run.GetId(), "prep/j")
50+ assert.Empty(t, prep.GetSecrets())
51+ r.runJob(prep, pb.CiConclusion_CI_CONCLUSION_SUCCESS, map[string]string{"payload": "\x01UNPASSED\x02"})
52+ j := awaitRecord(c, r, run.GetId(), "call/j")
53+ assert.NotEqual(t, "private-value", j.GetSecrets()["TOKEN"], "callee output must not mint a secret reference")
54+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
55+}
56+
57+func TestCIReusableSecretExpressions(t *testing.T) {
58+ for _, tt := range []struct{ name, value, want string }{
59+ {"literal", "literal-value", "literal-value"},
60+ {"dynamic", "${{ secrets[vars.KEY] }}", "private-value"},
61+ {"conditional", "${{ secrets.SOURCE == 'private-value' && 'yes' || 'no' }}", "yes"},
62+ {"function", "${{ format('{0}-x', secrets.SOURCE) }}", "private-value-x"},
63+ {"text around", "pre-${{ secrets.SOURCE }}-post", "pre-private-value-post"},
64+ } {
65+ t.Run(tt.name, func(t *testing.T) {
66+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
67+ c.setSecret(repoScope("proj"), "SOURCE", "private-value")
68+ c.setVar(repoScope("proj"), "KEY", "SOURCE")
69+ caller := "on: push\njobs:\n call:\n uses: ./.github/workflows/callee.yml\n secrets:\n TOKEN: " + tt.value + "\n"
70+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": secretsCallee}, "")
71+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
72+ j := awaitRecord(c, r, run.GetId(), "call/j")
73+ assert.Equal(t, tt.want, j.GetSecrets()["TOKEN"])
74+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
75+ })
76+ }
77+}
78+
79+func TestCIReusableCallMaxParallel(t *testing.T) {
80+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
81+ caller := `on: push
82+jobs:
83+ c:
84+ strategy:
85+ max-parallel: 1
86+ matrix:
87+ v: [a,b]
88+ uses: ./.github/workflows/callee.yml
89+`
90+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": ghCallee}, "")
91+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
92+ j := awaitRecord(c, r, run.GetId(), "c/test")
93+ second, err := r.fetch("")
94+ require.NoError(t, err)
95+ assert.Nil(t, second, "max-parallel: 1 must not assign the second call while the first is running")
96+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
97+ next := awaitRecord(c, r, run.GetId(), "c/test")
98+ assert.NotEqual(t, j.GetJobId(), next.GetJobId(), "the second call starts once the first has ended")
99+ r.runJob(next, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
100+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(run.GetId()).GetConclusion())
101+}
102+
103+func TestCIReusableOneResolutionPerAttempt(t *testing.T) {
104+ remote := newFakeRemote(t)
105+ remote.set("acme/shared", ptrRef("refs/heads/main", shaMain))
106+ wf := &fakeWorkflows{m: mocks.NewWorkflowFetcher(t)}
107+ wf.m.EXPECT().FetchWorkflow(mock.Anything, "acme", "shared", mock.Anything, mock.Anything, mock.Anything).RunAndReturn(func(_ context.Context, _, _, sha, path string, _ int64) ([]byte, error) {
108+ remote.set("acme/shared", ptrRef("refs/heads/main", shaOther))
109+ return []byte(ghCallee), nil
110+ }).Maybe()
111+ c, r := newReusableHarness(t, remote, wf)
112+ c.push("main", map[string]string{".github/workflows/ci.yml": "on: push\njobs:\n a:\n uses: acme/shared/.github/workflows/a.yml@main\n b:\n uses: acme/shared/.github/workflows/b.yml@main\n"}, "")
113+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
114+ for range 2 {
115+ j := r.await()
116+ assert.Contains(t, j.GetContexts(), `"workflow_sha":"`+shaMain+`"`, "the first resolution must pin all references of the attempt")
117+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
118+ }
119+ assert.Equal(t, 1, remote.count())
120+ c.waitRunDone(run.GetId())
121+}
122+
123+func TestCIReusableFailedRerunKeepsTheCallee(t *testing.T) {
124+ remote, wf := newFakeRemote(t), newFakeWorkflows(t)
125+ remote.set("acme/shared", ptrRef("refs/heads/main", shaMain))
126+ wf.set("acme/shared@"+shaMain+":.github/workflows/a.yml", ghCallee)
127+ wf.set("acme/shared@"+shaOther+":.github/workflows/a.yml", ghCallee)
128+ c, r := newReusableHarness(t, remote, wf)
129+ c.push("main", map[string]string{".github/workflows/ci.yml": "on: push\njobs:\n a:\n uses: acme/shared/.github/workflows/a.yml@main\n"}, "")
130+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
131+ j := r.await()
132+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_FAILURE, nil)
133+ c.waitRunDone(run.GetId())
134+ remote.set("acme/shared", ptrRef("refs/heads/main", shaOther))
135+ _, err := c.root.run.RerunRun(c.ctx, connect.NewRequest(&pb.RerunRunRequest{Repo: "proj", RunId: run.GetId(), FailedOnly: true}))
136+ require.NoError(t, err)
137+ again := r.await()
138+ assert.Contains(t, again.GetContexts(), `"workflow_sha":"`+shaMain+`"`, "failed-job reruns must reuse the first attempt's callee commit")
139+ r.runJob(again, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
140+}
141+
142+func TestCIReusableJobRecordBudget(t *testing.T) {
143+ c, _ := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
144+ vals := func(n int) string {
145+ v := make([]string, n)
146+ for i := range v {
147+ v[i] = fmt.Sprint(i)
148+ }
149+ return "[" + strings.Join(v, ",") + "]"
150+ }
151+ caller := "on: push\njobs:\n c:\n strategy:\n matrix:\n v: " + vals(33) + "\n uses: ./.github/workflows/callee.yml\n"
152+ callee := "on: workflow_call\njobs:\n j:\n strategy:\n matrix:\n v: " + vals(32) + "\n runs-on: ubuntu-latest\n steps:\n - run: echo\n"
153+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": callee}, "")
154+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
155+ const budgetText = "Workflows may not contain more than 1000 jobs across all referenced files"
156+ c.waitFor("the budget refusal", func() bool {
157+ for _, j := range c.getRun(run.GetId()).GetJobs() {
158+ if strings.Contains(j.GetError(), budgetText) {
159+ return true
160+ }
161+ }
162+ return false
163+ })
164+ assert.LessOrEqual(t, len(c.getRun(run.GetId()).GetJobs()), usecase.MaxCIChecks, "expanded records obey the job-count limit")
165+}
166+
167+// The ordinary matrices of a workflow share the same budget.
168+func TestCIJobRecordBudgetOfMatrices(t *testing.T) {
169+ c, _ := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
170+ v := make([]string, 200)
171+ for i := range v {
172+ v[i] = fmt.Sprint(i)
173+ }
174+ wf := "on: push\njobs:\n"
175+ for _, id := range []string{"a", "b", "c", "d", "e", "f"} {
176+ wf += " " + id + ":\n strategy:\n matrix:\n v: [" + strings.Join(v, ",") + "]\n runs-on: ubuntu-latest\n steps:\n - run: echo\n"
177+ }
178+ c.push("main", map[string]string{".github/workflows/ci.yml": wf}, "")
179+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
180+ c.waitFor("a job fails on the budget", func() bool {
181+ for _, j := range c.getRun(run.GetId()).GetJobs() {
182+ if strings.Contains(j.GetError(), "Workflows may not contain more than 1000 jobs across all referenced files") {
183+ return true
184+ }
185+ }
186+ return false
187+ })
188+ assert.LessOrEqual(t, len(c.getRun(run.GetId()).GetJobs()), usecase.MaxCIChecks)
189+}
190+
191+func TestCIReusableCachedGitHubResolutionDoesNotBypassErmite(t *testing.T) {
192+ remote, wf := newFakeRemote(t), newFakeWorkflows(t)
193+ remote.set("owner/shared", ptrRef("refs/heads/main", shaMain))
194+ wf.set("owner/shared@"+shaMain+":.github/workflows/a.yml", ghCallee)
195+ c, r := newReusableHarness(t, remote, wf)
196+ token, _ := c.branchToken([]domain.Scope{domain.ScopeRepoRead, domain.ScopeRepoWrite}, nil, "proj")
197+ caller := "on: push\njobs:\n a:\n uses: owner/shared/.github/workflows/a.yml@main\n b:\n needs: a\n uses: owner/shared/.github/workflows/a.yml@main\n"
198+ c.pushAs(token, "main", map[string]string{".github/workflows/ci.yml": caller})
199+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
200+ first := awaitRecord(c, r, run.GetId(), "a/test")
201+ c.seed("shared", domain.VisibilityPrivate)
202+ r.runJob(first, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
203+ second, err := r.fetch("")
204+ require.NoError(t, err)
205+ assert.True(t, second == nil, "must not use the cached GitHub source after an unreadable ermite name collision")
206+ if second != nil {
207+ r.runJob(second, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
208+ }
209+}
210+
211+// S2: a literal in "secrets:" arrives although no stored secret resolves; a run
212+// that is not trusted (a branch-limited token pushed it) gets nothing.
213+func TestCIReusableLiteralSecretNeedsATrustedRun(t *testing.T) {
214+ for _, tt := range []struct {
215+ name string
216+ limited bool
217+ want string
218+ }{{"owner", false, "literal-value"}, {"limited token", true, ""}} {
219+ t.Run(tt.name, func(t *testing.T) {
220+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
221+ caller := "on: push\njobs:\n call:\n uses: ./.github/workflows/callee.yml\n secrets:\n TOKEN: literal-value\n"
222+ files := map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": secretsCallee}
223+ if tt.limited {
224+ tok, _ := c.branchToken([]domain.Scope{domain.ScopeRepoRead, domain.ScopeRepoWrite}, nil)
225+ c.pushAs(tok, "main", files)
226+ } else {
227+ c.push("main", files, "")
228+ }
229+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
230+ j := awaitRecord(c, r, run.GetId(), "call/j")
231+ assert.Equal(t, tt.want, j.GetSecrets()["TOKEN"])
232+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
233+ })
234+ }
235+}
236+
237+// T3: the name of a nested job carries every caller.
238+func TestCIReusableNestedNames(t *testing.T) {
239+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
240+ leaf := "on:\n workflow_call:\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo\n"
241+ mid := "on:\n workflow_call:\njobs:\n inner:\n uses: ./.github/workflows/leaf.yml\n"
242+ c.push("main", map[string]string{".github/workflows/leaf.yml": leaf, ".github/workflows/mid.yml": mid, ".github/workflows/ci.yml": "on: push\njobs:\n top:\n uses: ./.github/workflows/mid.yml\n"}, "")
243+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
244+ j := awaitRecord(c, r, run.GetId(), "top/inner/j")
245+ names := map[string]string{}
246+ for _, x := range c.getRun(run.GetId()).GetJobs() {
247+ names[x.GetKey()] = x.GetName()
248+ }
249+ assert.Equal(t, map[string]string{"top": "top", "top/inner": "top / inner", "top/inner/j": "top / inner / j"}, names)
250+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
251+}
252+
253+// S1/S3: secrets pass through two levels of calls, each mapping evaluated at
254+// its own level.
255+func TestCIReusableSecretsThroughTwoLevels(t *testing.T) {
256+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
257+ c.setSecret(repoScope("proj"), "TOP", "top-value")
258+ leaf := "on:\n workflow_call:\n secrets:\n LEAF:\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo ${{ secrets.LEAF }}\n"
259+ mid := "on:\n workflow_call:\n secrets:\n MID:\njobs:\n inner:\n uses: ./.github/workflows/leaf.yml\n secrets:\n LEAF: ${{ secrets.MID }}-leaf\n"
260+ c.push("main", map[string]string{".github/workflows/leaf.yml": leaf, ".github/workflows/mid.yml": mid, ".github/workflows/ci.yml": "on: push\njobs:\n top:\n uses: ./.github/workflows/mid.yml\n secrets:\n MID: ${{ secrets.TOP }}\n"}, "")
261+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
262+ j := awaitRecord(c, r, run.GetId(), "top/inner/j")
263+ assert.Equal(t, map[string]string{"LEAF": "top-value-leaf"}, j.GetSecrets())
264+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
265+}
A internal/app/ci_reusable_secrets_test.go +259 -0
@@ -0,0 +1,259 @@
1+// SPDX-License-Identifier: AGPL-3.0-or-later
2+
3+package app
4+
5+import (
6+ "fmt"
7+ "strings"
8+ "testing"
9+
10+ "connectrpc.com/connect"
11+ "github.com/stretchr/testify/assert"
12+ "github.com/stretchr/testify/require"
13+
14+ pb "bnema.dev/ermite-api/gen/ermite"
15+ "bnema.dev/ermite/internal/domain"
16+)
17+
18+const producerWorkflow = `on:
19+ workflow_call:
20+ outputs:
21+ payload:
22+ value: ${{ jobs.j.outputs.payload }}
23+jobs:
24+ j:
25+ runs-on: ubuntu-latest
26+ outputs:
27+ payload: ${{ steps.s.outputs.payload }}
28+ steps:
29+ - id: s
30+ run: echo
31+`
32+
33+// secretCall runs a push whose caller maps TOKEN with value and returns the job
34+// of the callee that was assigned.
35+func secretCall(t *testing.T, setup func(c *ciHarness), value string) (*ciHarness, *fakeRunner, *pb.Job) {
36+ t.Helper()
37+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
38+ if setup != nil {
39+ setup(c)
40+ }
41+ caller := "on: push\njobs:\n call:\n uses: ./.github/workflows/callee.yml\n secrets:\n TOKEN: " + value + "\n"
42+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": secretsCallee}, "")
43+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
44+ return c, r, awaitRecord(c, r, run.GetId(), "call/j")
45+}
46+
47+// GO-009: a string that an expression produces is data. It is not read again as
48+// an expression, so it cannot name a secret.
49+func TestCIReusableEscapedExpressionStaysALiteral(t *testing.T) {
50+ c, r, j := secretCall(t, func(c *ciHarness) {
51+ c.setSecret(repoScope("proj"), "UNPASSED", "private-value")
52+ }, "${{ '${{ secrets.UNPASSED }}' }}")
53+ assert.Equal(t, "${{ secrets.UNPASSED }}", j.GetSecrets()["TOKEN"])
54+ assert.NotContains(t, c.logs.String(), "private-value")
55+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
56+}
57+
58+// GO-009: nor does a name taken from a context select a secret.
59+func TestCIReusableContextDataCannotSelectASecret(t *testing.T) {
60+ for _, tt := range []struct{ name, token, payload, want string }{
61+ {"a text that looks like an expression", "${{ needs.prep.outputs.payload }}", "${{ secrets.UNPASSED }}", "${{ secrets.UNPASSED }}"},
62+ {"a former marker", "${{ needs.prep.outputs.payload }}", "\x01UNPASSED\x02", "\x01UNPASSED\x02"},
63+ {"an escaped expression over json", "${{ '${{ secrets[fromJSON(needs.prep.outputs.payload).name] }}' }}", `{"name":"UNPASSED"}`, "${{ secrets[fromJSON(needs.prep.outputs.payload).name] }}"},
64+ } {
65+ t.Run(tt.name, func(t *testing.T) {
66+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
67+ c.setSecret(repoScope("proj"), "UNPASSED", "private-value")
68+ caller := "on: push\njobs:\n prep:\n uses: ./.github/workflows/producer.yml\n call:\n needs: prep\n uses: ./.github/workflows/callee.yml\n secrets:\n TOKEN: " + tt.token + "\n"
69+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/producer.yml": producerWorkflow, ".github/workflows/callee.yml": secretsCallee}, "")
70+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
71+ prep := awaitRecord(c, r, run.GetId(), "prep/j")
72+ assert.Empty(t, prep.GetSecrets())
73+ r.runJob(prep, pb.CiConclusion_CI_CONCLUSION_SUCCESS, map[string]string{"payload": tt.payload})
74+ j := awaitRecord(c, r, run.GetId(), "call/j")
75+ assert.Equal(t, tt.want, j.GetSecrets()["TOKEN"])
76+ assert.NotEqual(t, "private-value", j.GetSecrets()["TOKEN"])
77+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
78+ })
79+ }
80+}
81+
82+// GO-009: an expression that selects a secret by data of the caller (an input of
83+// the workflow, here) does use the secret it names.
84+func TestCIReusableMappingUsesPlainContexts(t *testing.T) {
85+ _, r, j := secretCall(t, func(c *ciHarness) {
86+ c.setSecret(repoScope("proj"), "SOURCE", "private-value")
87+ c.setVar(repoScope("proj"), "KEY", "SOURCE")
88+ }, "${{ secrets[vars.KEY] }}")
89+ assert.Equal(t, "private-value", j.GetSecrets()["TOKEN"])
90+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
91+}
92+
93+// GO-010: an expression error never reaches the log: format() puts its
94+// arguments, here a secret, in its message.
95+func TestCIReusableSecretEvaluationErrorIsNotLogged(t *testing.T) {
96+ secret := "private-value{bad-format}"
97+ c, r, j := secretCall(t, func(c *ciHarness) {
98+ c.setSecret(repoScope("proj"), "SOURCE", secret)
99+ }, "${{ format(secrets.SOURCE) }}")
100+ assert.Empty(t, j.GetSecrets()["TOKEN"])
101+ logs := c.logs.String()
102+ assert.NotContains(t, logs, secret)
103+ assert.NotContains(t, logs, "private-value")
104+ assert.Contains(t, logs, "could not evaluate", "the mapping is named, with a fixed reason")
105+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
106+}
107+
108+// GO-011: planning does not evaluate the mapping, so a value that needs a real
109+// secret (here a JSON one) is not refused for lack of it.
110+func TestCIReusableMappingNeedsNoSecretAtPlanning(t *testing.T) {
111+ c, r, j := secretCall(t, func(c *ciHarness) {
112+ c.setSecret(repoScope("proj"), "JSON", `{"token":"private-value"}`)
113+ }, "${{ fromJSON(secrets.JSON).token }}")
114+ assert.Equal(t, "private-value", j.GetSecrets()["TOKEN"])
115+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
116+ _ = c
117+}
118+
119+// GO-012: the mapping of a call reads the vars of the calling scope, not those
120+// of the environment of the job that is assigned.
121+func TestCIReusableMappingUsesTheCallersVars(t *testing.T) {
122+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
123+ c.setSecret(repoScope("proj"), "PUBLIC", "public-value")
124+ c.setSecret(repoScope("proj"), "PRIVATE", "private-value")
125+ c.setVar(repoScope("proj"), "KEY", "PUBLIC")
126+ _, err := c.root.env.CreateEnvironment(c.ctx, connect.NewRequest(&pb.CreateEnvironmentRequest{Repo: "proj", Name: "prod"}))
127+ require.NoError(t, err)
128+ c.setVar(envScope("proj", "prod"), "KEY", "PRIVATE")
129+ caller := "on: push\njobs:\n call:\n uses: ./.github/workflows/callee.yml\n secrets:\n TOKEN: ${{ secrets[vars.KEY] }}\n"
130+ callee := strings.Replace(secretsCallee, " runs-on: ubuntu-latest", " runs-on: ubuntu-latest\n environment: prod", 1)
131+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": callee}, "")
132+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
133+ j := awaitRecord(c, r, run.GetId(), "call/j")
134+ assert.Equal(t, "public-value", j.GetSecrets()["TOKEN"])
135+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
136+}
137+
138+// An input handed down as an escaped expression is data in a callee that
139+// inherits secrets, and an untrusted run gets nothing.
140+func TestCIReusableInheritScopeTrustAndInputData(t *testing.T) {
141+ for _, limited := range []bool{false, true} {
142+ t.Run(map[bool]string{false: "trusted", true: "untrusted"}[limited], func(t *testing.T) {
143+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
144+ c.setSecret(repoScope("proj"), "TOP", "top-value")
145+ c.setSecret(repoScope("proj"), "UNPASSED", "private-value")
146+ leaf := "on:\n workflow_call:\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo ${{ toJSON(secrets) }}\n"
147+ mid := "on:\n workflow_call:\n inputs:\n text:\n type: string\n secrets:\n MID:\njobs:\n inner:\n uses: ./.github/workflows/leaf.yml\n secrets: inherit\n"
148+ caller := "on: push\njobs:\n top:\n uses: ./.github/workflows/mid.yml\n with:\n text: '${{ ''${{ secrets.UNPASSED }}'' }}'\n secrets:\n MID: ${{ secrets.TOP }}\n"
149+ files := map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/mid.yml": mid, ".github/workflows/leaf.yml": leaf}
150+ if limited {
151+ tok, _ := c.branchToken([]domain.Scope{domain.ScopeRepoRead, domain.ScopeRepoWrite}, nil)
152+ c.pushAs(tok, "main", files)
153+ } else {
154+ c.push("main", files, "")
155+ }
156+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
157+ j := awaitRecord(c, r, run.GetId(), "top/inner/j")
158+ if limited {
159+ assert.Empty(t, j.GetSecrets())
160+ } else {
161+ assert.Equal(t, map[string]string{"MID": "top-value"}, j.GetSecrets())
162+ }
163+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
164+ })
165+ }
166+}
167+
168+// A secret cannot be handed to the callee as an input.
169+func TestCIReusableSecretsAreRefusedInWith(t *testing.T) {
170+ c, _ := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
171+ c.setSecret(repoScope("proj"), "SOURCE", "private-value")
172+ caller := "on: push\njobs:\n call:\n uses: ./.github/workflows/callee.yml\n with:\n go-version: ${{ secrets.SOURCE }}\n"
173+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": ghCallee}, "")
174+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
175+ c.waitRunDone(run.GetId())
176+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_FAILURE, c.getRun(run.GetId()).GetRun().GetConclusion())
177+ assert.NotContains(t, c.logs.String(), "private-value")
178+}
179+
180+// A re-run of everything resolves the reference of a called workflow afresh.
181+func TestCIReusableRerunAllRefreshesTheCallee(t *testing.T) {
182+ remote, wf := newFakeRemote(t), newFakeWorkflows(t)
183+ remote.set("acme/shared", ptrRef("refs/heads/main", shaMain))
184+ wf.set("acme/shared@"+shaMain+":.github/workflows/a.yml", ghCallee)
185+ wf.set("acme/shared@"+shaOther+":.github/workflows/a.yml", ghCallee)
186+ c, r := newReusableHarness(t, remote, wf)
187+ c.push("main", map[string]string{".github/workflows/ci.yml": "on: push\njobs:\n a:\n uses: acme/shared/.github/workflows/a.yml@main\n"}, "")
188+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
189+ first := r.await()
190+ assert.Contains(t, first.GetContexts(), `"workflow_sha":"`+shaMain+`"`)
191+ r.runJob(first, pb.CiConclusion_CI_CONCLUSION_FAILURE, nil)
192+ c.waitRunDone(run.GetId())
193+
194+ remote.set("acme/shared", ptrRef("refs/heads/main", shaOther))
195+ _, err := c.root.run.RerunRun(c.ctx, connect.NewRequest(&pb.RerunRunRequest{Repo: "proj", RunId: run.GetId()}))
196+ require.NoError(t, err)
197+ again := r.await()
198+ assert.Contains(t, again.GetContexts(), `"workflow_sha":"`+shaOther+`"`, "the reference is resolved again")
199+ r.runJob(again, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
200+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(run.GetId()).GetConclusion())
201+}
202+
203+// A matrix of calls with max-parallel and both concurrency groups progresses
204+// one call at a time to the end, and the groups are free for the next run.
205+func TestCIReusableMaxParallelWithBothGroups(t *testing.T) {
206+ c, r := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
207+ caller := "on: push\njobs:\n c:\n strategy:\n max-parallel: 1\n matrix:\n v: [a, b, c]\n concurrency: caller-group\n uses: ./.github/workflows/callee.yml\n"
208+ callee := strings.Replace(ghCallee, "jobs:\n", "concurrency: callee-group\njobs:\n", 1)
209+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": callee}, "")
210+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
211+ for range 3 {
212+ j := awaitRecord(c, r, run.GetId(), "c/test")
213+ c.forge.CI.ReconcileConcurrency(c.ctx)
214+ second, err := r.fetch("")
215+ require.NoError(t, err)
216+ assert.Nil(t, second, "one call at a time")
217+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
218+ }
219+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(run.GetId()).GetConclusion())
220+ c.push("main", map[string]string{"x": "second"}, "")
221+ next := c.awaitRunOf(".github/workflows/ci.yml", run.GetId())
222+ for range 3 {
223+ j := awaitRecord(c, r, next.GetId(), "c/test")
224+ r.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
225+ }
226+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(next.GetId()).GetConclusion())
227+}
228+
229+// The job-record budget holds again after a re-run of the failed jobs.
230+func TestCIReusableBudgetHoldsOnRerun(t *testing.T) {
231+ c, _ := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
232+ vals := func(n int) string {
233+ v := make([]string, n)
234+ for i := range v {
235+ v[i] = fmt.Sprint(i)
236+ }
237+ return "[" + strings.Join(v, ",") + "]"
238+ }
239+ caller := "on: push\njobs:\n c:\n strategy:\n matrix:\n v: " + vals(33) + "\n uses: ./.github/workflows/callee.yml\n"
240+ callee := "on: workflow_call\njobs:\n j:\n strategy:\n matrix:\n v: " + vals(32) + "\n runs-on: ubuntu-latest\n steps:\n - run: echo\n"
241+ c.push("main", map[string]string{".github/workflows/ci.yml": caller, ".github/workflows/callee.yml": callee}, "")
242+ run := c.awaitRunOf(".github/workflows/ci.yml", 0)
243+ refused := func() bool {
244+ for _, j := range c.getRun(run.GetId()).GetJobs() {
245+ if strings.Contains(j.GetError(), "Workflows may not contain more than 1000 jobs across all referenced files") {
246+ return true
247+ }
248+ }
249+ return false
250+ }
251+ c.waitFor("the budget refusal", refused)
252+ assert.LessOrEqual(t, len(c.getRun(run.GetId()).GetJobs()), 1000)
253+ _, _ = c.root.run.CancelRun(c.ctx, connect.NewRequest(&pb.CancelRunRequest{Repo: "proj", RunId: run.GetId()}))
254+ c.waitRunDone(run.GetId())
255+ _, err := c.root.run.RerunRun(c.ctx, connect.NewRequest(&pb.RerunRunRequest{Repo: "proj", RunId: run.GetId(), FailedOnly: true}))
256+ require.NoError(t, err)
257+ c.waitFor("the refusal again", refused)
258+ assert.LessOrEqual(t, len(c.getRun(run.GetId()).GetJobs()), 1000)
259+}
A internal/app/ci_reusable_test.go +697 -0
@@ -0,0 +1,697 @@
1+// SPDX-License-Identifier: AGPL-3.0-or-later
2+
3+package app
4+
5+import (
6+ "context"
7+ "strings"
8+ "sync"
9+ "testing"
10+ "time"
11+
12+ "connectrpc.com/connect"
13+ "github.com/stretchr/testify/assert"
14+ "github.com/stretchr/testify/mock"
15+ "github.com/stretchr/testify/require"
16+
17+ pb "bnema.dev/ermite-api/gen/ermite"
18+ "bnema.dev/ermite/internal/domain"
19+ "bnema.dev/ermite/internal/ports"
20+ "bnema.dev/ermite/internal/ports/mocks"
21+ "bnema.dev/ermite/internal/usecase"
22+)
23+
24+// fakeWorkflows stands in for github.com's file download: no test reaches the
25+// network. It answers from a table of "owner/repo@sha:path" to file content.
26+type fakeWorkflows struct {
27+ m *mocks.WorkflowFetcher
28+ mu sync.Mutex
29+ files map[string]string
30+ calls []string
31+}
32+
33+func newFakeWorkflows(t *testing.T) *fakeWorkflows {
34+ t.Helper()
35+ f := &fakeWorkflows{m: mocks.NewWorkflowFetcher(t), files: map[string]string{}}
36+ f.m.EXPECT().FetchWorkflow(mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything, mock.Anything).RunAndReturn(
37+ func(_ context.Context, owner, repo, sha, path string, _ int64) ([]byte, error) {
38+ f.mu.Lock()
39+ defer f.mu.Unlock()
40+ key := owner + "/" + repo + "@" + sha + ":" + path
41+ f.calls = append(f.calls, key)
42+ if s, ok := f.files[key]; ok {
43+ return []byte(s), nil
44+ }
45+ return nil, domain.ErrNotFound
46+ }).Maybe()
47+ return f
48+}
49+
50+func (f *fakeWorkflows) set(key, content string) {
51+ f.mu.Lock()
52+ defer f.mu.Unlock()
53+ f.files[key] = content
54+}
55+
56+func newReusableHarness(t *testing.T, remote *fakeRemote, wf *fakeWorkflows) (*ciHarness, *fakeRunner) {
57+ t.Helper()
58+ h := newHarnessWith(t, harnessOpts{env: map[string]string{"ERMITE_SECRETS_KEY": secretsKeyA}}, func(o *options) {
59+ o.remoteLister, o.workflowFetcher = remote.m, wf.m
60+ o.ci = func(cfg *usecase.CIConfig) { cfg.Tick, cfg.PollTimeout = 20*time.Millisecond, 100*time.Millisecond }
61+ })
62+ c := &ciHarness{harness: h, root: h.api(adminToken), repo: "proj"}
63+ _, err := h.forge.Repos.Create(h.ctx, domain.Root(), ports.CreateRepoParams{Name: h.name("proj"), Visibility: domain.VisibilityPrivate})
64+ require.NoError(t, err)
65+ c.wt = h.workTree("main")
66+ return c, c.registerRunner("alpha", "ubuntu-latest")
67+}
68+
69+func jobByKey(c *ciHarness, runID int64, key string) *pb.RunJob {
70+ for _, j := range c.getRun(runID).GetJobs() {
71+ if j.GetKey() == key {
72+ return j
73+ }
74+ }
75+ return nil
76+}
77+
78+// awaitRecord fetches jobs until the one of the record with the key comes. The
79+// runner is given the id of the job in its own workflow as the job key, so the
80+// record is told apart by its job id.
81+func awaitRecord(c *ciHarness, r *fakeRunner, runID int64, key string) *pb.Job {
82+ c.t.Helper()
83+ for range 20 {
84+ j := r.await()
85+ for _, rec := range c.getRun(runID).GetJobs() {
86+ if rec.GetKey() == key && rec.GetId() == j.GetJobId() {
87+ return j
88+ }
89+ }
90+ }
91+ c.t.Fatalf("no job %s of run %d", key, runID)
92+ return nil
93+}
94+
95+func (c *ciHarness) waitRunDone(runID int64) *pb.Run {
96+ c.t.Helper()
97+ c.waitFor("the run completes", func() bool {
98+ return c.getRun(runID).GetRun().GetStatus() == pb.CiStatus_CI_STATUS_COMPLETED
99+ })
100+ return c.getRun(runID).GetRun()
101+}
102+
103+const localCaller = `name: caller
104+on: push
105+jobs:
106+ prep:
107+ runs-on: ubuntu-latest
108+ outputs:
109+ who: ${{ steps.s.outputs.who }}
110+ steps:
111+ - id: s
112+ run: echo who=world >> $GITHUB_OUTPUT
113+ call:
114+ needs: prep
115+ uses: ./.github/workflows/callee.yml
116+ with:
117+ name: ${{ needs.prep.outputs.who }}
118+ count: 3
119+ secrets: inherit
120+ after:
121+ needs: call
122+ runs-on: ubuntu-latest
123+ steps:
124+ - run: echo ${{ needs.call.outputs.greeting }} ${{ needs.call.outputs.total }}
125+`
126+
127+const localCallee = `name: callee
128+on:
129+ workflow_call:
130+ inputs:
131+ name:
132+ type: string
133+ required: true
134+ count:
135+ type: number
136+ default: 1
137+ loud:
138+ type: boolean
139+ default: false
140+ outputs:
141+ greeting:
142+ value: ${{ jobs.hello.outputs.msg }}
143+ total:
144+ value: ${{ inputs.count }}
145+jobs:
146+ hello:
147+ runs-on: ubuntu-latest
148+ outputs:
149+ msg: ${{ steps.m.outputs.msg }}
150+ steps:
151+ - id: m
152+ run: echo "msg=hello ${{ inputs.name }} ${{ secrets.TOKEN }} ${{ inputs.loud }}" >> $GITHUB_OUTPUT
153+ second:
154+ needs: hello
155+ runs-on: ubuntu-latest
156+ steps:
157+ - run: echo ${{ needs.hello.outputs.msg }}
158+`
159+
160+// AC6: a local callee with inputs, secrets: inherit and outputs consumed by a
161+// later job of the caller; the callee's jobs are keyed and named after the call.
162+func TestCIReusableLocalCalleeWithInputsInheritAndOutputs(t *testing.T) {
163+ c, runner := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
164+ c.setSecret(repoScope("proj"), "TOKEN", "s3cr3t")
165+ c.setSecret(repoScope("proj"), "OTHER", "not-sent")
166+ c.push("main", map[string]string{".github/workflows/caller.yml": localCaller, ".github/workflows/callee.yml": localCallee}, "")
167+
168+ var run *pb.Run
169+ c.waitFor("the run exists", func() bool {
170+ for _, r := range c.runs() {
171+ if r.GetWorkflowPath() == ".github/workflows/caller.yml" {
172+ run = r
173+ return true
174+ }
175+ }
176+ return false
177+ })
178+ prep := awaitRecord(c, runner, run.GetId(), "prep")
179+ runner.runJob(prep, pb.CiConclusion_CI_CONCLUSION_SUCCESS, map[string]string{"who": "world"})
180+
181+ hello := awaitRecord(c, runner, run.GetId(), "call/hello")
182+ // The runner is given the called file and the callee's id of the job.
183+ assert.Equal(t, ".github/workflows/callee.yml", hello.GetWorkflowPath())
184+ assert.Contains(t, hello.GetWorkflowYaml(), "name: callee")
185+ assert.Equal(t, "s3cr3t", hello.GetSecrets()["TOKEN"], "inherit passes the caller's secrets")
186+ assert.NotContains(t, hello.GetSecrets(), "OTHER", "only the secrets the callee reads")
187+ assert.Contains(t, hello.GetContexts(), `"inputs":{"name":"world","count":3,"loud":false}`)
188+ assert.Contains(t, hello.GetContexts(), `"workflow_file_path":".github/workflows/callee.yml"`)
189+ assert.Equal(t, "call/hello", jobByKey(c, run.GetId(), "call/hello").GetKey())
190+ assert.Equal(t, "call / hello", jobByKey(c, run.GetId(), "call/hello").GetName())
191+ assert.Equal(t, []string{"call/hello"}, jobByKey(c, run.GetId(), "call/second").GetNeeds())
192+ runner.runJob(hello, pb.CiConclusion_CI_CONCLUSION_SUCCESS, map[string]string{"msg": "hello world s3cr3t false"})
193+
194+ second := awaitRecord(c, runner, run.GetId(), "call/second")
195+ assert.Contains(t, second.GetContexts(), `"msg":"hello world s3cr3t false"`)
196+ runner.runJob(second, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
197+
198+ after := awaitRecord(c, runner, run.GetId(), "after")
199+ assert.Contains(t, after.GetContexts(), `"greeting":"hello world s3cr3t false"`, "outputs mapped from the callee's jobs")
200+ assert.Contains(t, after.GetContexts(), `"total":"3"`)
201+ runner.runJob(after, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
202+
203+ done := c.waitRunDone(run.GetId())
204+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, done.GetConclusion())
205+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, jobByKey(c, run.GetId(), "call").GetConclusion())
206+}
207+
208+const secretsCallee = `on:
209+ workflow_call:
210+ secrets:
211+ TOKEN:
212+ required: true
213+ OPTIONAL:
214+jobs:
215+ j:
216+ runs-on: ubuntu-latest
217+ steps:
218+ - run: echo ${{ secrets.TOKEN }} ${{ secrets.OPTIONAL }}
219+`
220+
221+// D2: secrets are passed only when listed or inherited, renamed by the mapping.
222+func TestCIReusableSecretsAreNotPassedUnlessExplicitOrInherited(t *testing.T) {
223+ c, runner := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
224+ c.setSecret(repoScope("proj"), "MY_TOKEN", "mapped-value")
225+ c.setSecret(repoScope("proj"), "OPTIONAL", "caller-optional")
226+ files := map[string]string{".github/workflows/callee.yml": secretsCallee}
227+ caller := func(secrets string) string {
228+ return "on: push\njobs:\n c:\n uses: ./.github/workflows/callee.yml\n" + secrets
229+ }
230+
231+ // Explicit: TOKEN from MY_TOKEN; OPTIONAL is not passed although the caller has one.
232+ files[".github/workflows/explicit.yml"] = caller(" secrets:\n TOKEN: ${{ secrets.MY_TOKEN }}\n")
233+ // Nothing: the callee requires TOKEN, so the call is refused.
234+ files[".github/workflows/none.yml"] = caller("")
235+ c.push("main", files, "")
236+ var explicit, none *pb.Run
237+ c.waitFor("two runs", func() bool {
238+ for _, r := range c.runs() {
239+ switch r.GetWorkflowPath() {
240+ case ".github/workflows/explicit.yml":
241+ explicit = r
242+ case ".github/workflows/none.yml":
243+ none = r
244+ }
245+ }
246+ return explicit != nil && none != nil
247+ })
248+ j := awaitRecord(c, runner, explicit.GetId(), "c/j")
249+ assert.Equal(t, map[string]string{"TOKEN": "mapped-value"}, j.GetSecrets())
250+ runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
251+
252+ done := c.waitRunDone(none.GetId())
253+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_FAILURE, done.GetConclusion())
254+ assert.Contains(t, jobByKey(c, none.GetId(), "c").GetError(), "Secret TOKEN is required, but not provided while calling.")
255+}
256+
257+// The callee must declare workflow_call; with: is checked against its inputs.
258+func TestCIReusableCallErrors(t *testing.T) {
259+ c, _ := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
260+ callee := "on:\n workflow_call:\n inputs:\n n:\n type: number\n required: true\n flag:\n type: boolean\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo\n"
261+ caller := func(with string) string {
262+ return "on: push\njobs:\n c:\n uses: ./.github/workflows/callee.yml\n" + with
263+ }
264+ files := map[string]string{
265+ ".github/workflows/callee.yml": callee,
266+ ".github/workflows/nocall.yml": "on: push\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo\n",
267+ ".github/workflows/missing.yml": "on: push\njobs:\n c:\n uses: ./.github/workflows/nocall.yml\n",
268+ ".github/workflows/noreq.yml": caller(" with:\n flag: true\n"),
269+ ".github/workflows/badtype.yml": caller(" with:\n n: abc\n"),
270+ ".github/workflows/unknown.yml": caller(" with:\n n: 1\n zzz: 2\n"),
271+ ".github/workflows/gone.yml": "on: push\njobs:\n c:\n uses: ./.github/workflows/nothere.yml\n",
272+ ".github/workflows/ver.yml": "on: push\njobs:\n c:\n uses: ./.github/workflows/callee.yml@main\n",
273+ }
274+ c.push("main", files, "")
275+ want := map[string]string{
276+ ".github/workflows/missing.yml": "workflow_call key is not defined in the referenced workflow.",
277+ ".github/workflows/noreq.yml": "Input n is required, but not provided while calling.",
278+ ".github/workflows/badtype.yml": "Unexpected value 'abc'",
279+ ".github/workflows/unknown.yml": "Invalid input, zzz is not defined in the referenced workflow.",
280+ ".github/workflows/gone.yml": "Unable to find reusable workflow",
281+ ".github/workflows/ver.yml": "cannot specify version when calling local workflows",
282+ }
283+ runs := map[string]*pb.Run{}
284+ c.waitFor("the runs exist", func() bool {
285+ for _, r := range c.runs() {
286+ runs[r.GetWorkflowPath()] = r
287+ }
288+ for p := range want {
289+ if runs[p] == nil {
290+ return false
291+ }
292+ }
293+ return true
294+ })
295+ for path, text := range want {
296+ c.waitRunDone(runs[path].GetId())
297+ j := jobByKey(c, runs[path].GetId(), "c")
298+ require.NotNil(t, j, path)
299+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_FAILURE, j.GetConclusion(), path)
300+ assert.Contains(t, j.GetError(), text, path)
301+ }
302+ assert.Contains(t, jobByKey(c, runs[".github/workflows/noreq.yml"].GetId(), "c").GetError(), "In .github/workflows/noreq.yml (Line: 4, Col: 11): Error from called workflow")
303+}
304+
305+// AC6: depth 5 fails with the documented error; depth 4 runs.
306+func TestCIReusableDepth(t *testing.T) {
307+ c, runner := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
308+ files := map[string]string{}
309+ for i := 1; i <= 5; i++ {
310+ body := "on:\n workflow_call:\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo leaf\n"
311+ if i < 5 {
312+ body = "on:\n workflow_call:\njobs:\n next:\n uses: ./.github/workflows/l" + string(rune('0'+i+1)) + ".yml\n"
313+ }
314+ files[".github/workflows/l"+string(rune('0'+i))+".yml"] = body
315+ }
316+ files[".github/workflows/top4.yml"] = "on: push\njobs:\n a:\n uses: ./.github/workflows/l2.yml\n"
317+ files[".github/workflows/top5.yml"] = "on: push\njobs:\n a:\n uses: ./.github/workflows/l1.yml\n"
318+ delete(files, ".github/workflows/l5.yml")
319+ files[".github/workflows/l5.yml"] = "on:\n workflow_call:\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo leaf\n"
320+ c.push("main", files, "")
321+ var top4, top5 *pb.Run
322+ c.waitFor("both runs", func() bool {
323+ for _, r := range c.runs() {
324+ switch r.GetWorkflowPath() {
325+ case ".github/workflows/top4.yml":
326+ top4 = r
327+ case ".github/workflows/top5.yml":
328+ top5 = r
329+ }
330+ }
331+ return top4 != nil && top5 != nil
332+ })
333+ // top4: a(1) -> l2(2) -> l3(3) -> l4(4) -> l5 (leaf): the leaf runs at depth 4.
334+ leaf := awaitRecord(c, runner, top4.GetId(), "a/next/next/next/j")
335+ runner.runJob(leaf, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
336+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(top4.GetId()).GetConclusion())
337+
338+ // top5: one more level: l1 -> ... -> l5 holds a call at depth 5.
339+ c.waitRunDone(top5.GetId())
340+ var failed *pb.RunJob
341+ for _, j := range c.getRun(top5.GetId()).GetJobs() {
342+ if j.GetConclusion() == pb.CiConclusion_CI_CONCLUSION_FAILURE && strings.HasSuffix(j.GetKey(), "/next") {
343+ failed = j
344+ }
345+ }
346+ require.NotNil(t, failed, "the call that is too deep fails")
347+ assert.Contains(t, failed.GetError(), "Nested reusable workflow depth exceeded 4.")
348+}
349+
350+const ghCallee = `on:
351+ workflow_call:
352+ inputs:
353+ go-version:
354+ type: string
355+ default: stable
356+jobs:
357+ test:
358+ runs-on: ubuntu-latest
359+ steps:
360+ - run: echo ${{ inputs.go-version }}
361+`
362+
363+// AC6: a github.com callee, resolved with the CI-2 ref resolution and fetched
364+// at the resolved commit through the fetcher port (a fake here).
365+func TestCIReusableFromGitHub(t *testing.T) {
366+ remote, wf := newFakeRemote(t), newFakeWorkflows(t)
367+ remote.set("acme/gh-actions", ptrRef("refs/heads/main", shaMain))
368+ wf.set("acme/gh-actions@"+shaMain+":.github/workflows/go-ci.yml", ghCallee)
369+ c, runner := newReusableHarness(t, remote, wf)
370+ c.push("main", map[string]string{".github/workflows/ci.yml": "on: push\njobs:\n ci:\n uses: acme/gh-actions/.github/workflows/go-ci.yml@main\n with:\n go-version: '1.27'\n"}, "")
371+ var r *pb.Run
372+ c.waitFor("the run", func() bool {
373+ if rs := c.runs(); len(rs) > 0 {
374+ r = rs[0]
375+ }
376+ return r != nil
377+ })
378+ j := awaitRecord(c, runner, r.GetId(), "ci/test")
379+ assert.Contains(t, j.GetContexts(), `"inputs":{"go-version":"1.27"}`)
380+ assert.Contains(t, j.GetContexts(), `"workflow_ref":"acme/gh-actions/.github/workflows/go-ci.yml@main"`)
381+ assert.Contains(t, j.GetContexts(), `"workflow_sha":"`+shaMain+`"`)
382+ runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
383+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(r.GetId()).GetConclusion())
384+ assert.True(t, remote.asked("acme/gh-actions"))
385+ wf.mu.Lock()
386+ assert.Equal(t, []string{"acme/gh-actions@" + shaMain + ":.github/workflows/go-ci.yml"}, wf.calls)
387+ wf.mu.Unlock()
388+}
389+
390+// AC6: an owner repository of the server wins over github.com.
391+func TestCIReusableFromAnErmiteRepository(t *testing.T) {
392+ remote, wf := newFakeRemote(t), newFakeWorkflows(t)
393+ remote.set("owner/shared", ptrRef("refs/heads/main", shaMain)) // shadowed by ermite
394+ c, runner := newReusableHarness(t, remote, wf)
395+ c.seed("shared", domain.VisibilityPrivate)
396+ wt := c.workTree("main")
397+ c.writeFile(wt, ".github/workflows/lib.yml", []byte(ghCallee))
398+ c.mustGit(wt, "push", "-q", c.urlAuth(adminToken, "/shared.git"), "main")
399+
400+ c.push("main", map[string]string{".github/workflows/ci.yml": "on: push\njobs:\n lib:\n uses: owner/shared/.github/workflows/lib.yml@main\n"}, "")
401+ var r *pb.Run
402+ c.waitFor("the run", func() bool {
403+ if rs := c.runs(); len(rs) > 0 {
404+ r = rs[0]
405+ }
406+ return r != nil
407+ })
408+ j := awaitRecord(c, runner, r.GetId(), "lib/test")
409+ assert.Contains(t, j.GetContexts(), `"inputs":{"go-version":"stable"}`, "the default")
410+ runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
411+ c.waitRunDone(r.GetId())
412+ assert.False(t, remote.asked("owner/shared"))
413+ wf.mu.Lock()
414+ assert.Empty(t, wf.calls, "github.com is not asked for an ermite repository")
415+ wf.mu.Unlock()
416+}
417+
418+// A matrix on the calling job expands the callee once per leg.
419+func TestCIReusableMatrixCaller(t *testing.T) {
420+ c, runner := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
421+ callee := "on:\n workflow_call:\n inputs:\n v:\n type: string\n required: true\njobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo ${{ inputs.v }}\n"
422+ caller := "on: push\njobs:\n c:\n strategy:\n matrix:\n v: [a, b]\n uses: ./.github/workflows/callee.yml\n with:\n v: ${{ matrix.v }}\n"
423+ c.push("main", map[string]string{".github/workflows/callee.yml": callee, ".github/workflows/ci.yml": caller}, "")
424+ var r *pb.Run
425+ c.waitFor("the run", func() bool {
426+ for _, x := range c.runs() {
427+ if x.GetWorkflowPath() == ".github/workflows/ci.yml" {
428+ r = x
429+ }
430+ }
431+ return r != nil
432+ })
433+ got := map[string]bool{}
434+ for range 2 {
435+ j := awaitRecord(c, runner, r.GetId(), "c/j")
436+ switch {
437+ case strings.Contains(j.GetContexts(), `"inputs":{"v":"a"}`):
438+ got["a"] = true
439+ case strings.Contains(j.GetContexts(), `"inputs":{"v":"b"}`):
440+ got["b"] = true
441+ }
442+ runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
443+ }
444+ assert.Equal(t, map[string]bool{"a": true, "b": true}, got)
445+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(r.GetId()).GetConclusion())
446+ names := map[string]bool{}
447+ for _, j := range c.getRun(r.GetId()).GetJobs() {
448+ names[j.GetName()] = true
449+ }
450+ assert.True(t, names["c (a) / j"] && names["c (b) / j"], "%v", names)
451+}
452+
453+// D3: the callee can only lower the caller's permissions; a request above is
454+// refused with the runner's text, and a lower one is kept.
455+func TestCIReusablePermissionsOnlyLowered(t *testing.T) {
456+ c, runner := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
457+ callee := func(perm string) string {
458+ return "on:\n workflow_call:\n" + perm + "jobs:\n j:\n runs-on: ubuntu-latest\n steps:\n - run: echo\n"
459+ }
460+ caller := func(perm, callee string) string {
461+ return "on: push\n" + perm + "jobs:\n c:\n uses: ./.github/workflows/" + callee + ".yml\n"
462+ }
463+ c.push("main", map[string]string{
464+ ".github/workflows/lower.yml": callee("permissions:\n contents: read\n"),
465+ ".github/workflows/higher.yml": callee("permissions:\n contents: write\n"),
466+ ".github/workflows/plain.yml": callee(""),
467+ ".github/workflows/a.yml": caller("permissions:\n contents: write\n pull-requests: write\n", "lower"),
468+ ".github/workflows/b.yml": caller("permissions:\n contents: read\n", "higher"),
469+ ".github/workflows/p.yml": caller("permissions:\n contents: write\n", "plain"),
470+ }, "")
471+ runs := map[string]*pb.Run{}
472+ c.waitFor("three runs", func() bool {
473+ for _, r := range c.runs() {
474+ runs[r.GetWorkflowPath()] = r
475+ }
476+ return runs[".github/workflows/a.yml"] != nil && runs[".github/workflows/b.yml"] != nil && runs[".github/workflows/p.yml"] != nil
477+ })
478+ lower := awaitRecord(c, runner, runs[".github/workflows/a.yml"].GetId(), "c/j")
479+ assert.Equal(t, "read", lower.GetPermissions().GetLevels()["contents"], "the callee lowers")
480+ assert.NotContains(t, lower.GetPermissions().GetLevels(), "pull-requests")
481+ runner.runJob(lower, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
482+
483+ plain := awaitRecord(c, runner, runs[".github/workflows/p.yml"].GetId(), "c/j")
484+ assert.Equal(t, "write", plain.GetPermissions().GetLevels()["contents"], "a callee without permissions keeps the caller's")
485+ runner.runJob(plain, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
486+
487+ c.waitRunDone(runs[".github/workflows/b.yml"].GetId())
488+ var errText string
489+ for _, j := range c.getRun(runs[".github/workflows/b.yml"].GetId()).GetJobs() {
490+ if j.GetKey() == "c/j" {
491+ errText = j.GetError()
492+ }
493+ }
494+ assert.Contains(t, errText, "is requesting 'contents: write', but is only allowed 'contents: read'.")
495+}
496+
497+// D4: the job-level concurrency of a callee's job applies.
498+func TestCIReusableCalleeJobConcurrency(t *testing.T) {
499+ c, runner := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
500+ callee := "on:\n workflow_call:\njobs:\n j:\n runs-on: ubuntu-latest\n concurrency:\n group: callee-job-group\n steps:\n - run: echo\n"
501+ caller := "on: push\njobs:\n c:\n uses: ./.github/workflows/callee.yml\n"
502+ c.push("main", map[string]string{".github/workflows/callee.yml": callee, ".github/workflows/ci.yml": caller}, "")
503+ r1 := c.awaitRunOf(".github/workflows/ci.yml", 0)
504+ j1 := awaitRecord(c, runner, r1.GetId(), "c/j")
505+ assert.Equal(t, "callee-job-group", jobByKey(c, r1.GetId(), "c/j").GetConcurrencyGroup())
506+ c.push("main", map[string]string{"x.txt": "1"}, "second")
507+ r2 := c.awaitRunOf(".github/workflows/ci.yml", r1.GetId())
508+ c.waitFor("the second run's job is pending on the group", func() bool {
509+ j := jobByKey(c, r2.GetId(), "c/j")
510+ return j != nil && j.GetStatus() == pb.CiStatus_CI_STATUS_PENDING
511+ })
512+ runner.runJob(j1, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
513+ j2 := awaitRecord(c, runner, r2.GetId(), "c/j")
514+ runner.runJob(j2, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
515+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(r2.GetId()).GetConclusion())
516+}
517+
518+// awaitRunOf waits for the newest run of a workflow file whose id is above after.
519+func (c *ciHarness) awaitRunOf(path string, after int64) *pb.Run {
520+ c.t.Helper()
521+ var r *pb.Run
522+ c.waitFor("a run of "+path, func() bool {
523+ for _, x := range c.runs() {
524+ if x.GetWorkflowPath() == path && x.GetId() > after {
525+ r = x
526+ return true
527+ }
528+ }
529+ return false
530+ })
531+ return r
532+}
533+
534+const concCallee = `on:
535+ workflow_call:
536+ inputs:
537+ env:
538+ type: string
539+ default: prod
540+%s
541+jobs:
542+ j:
543+ runs-on: ubuntu-latest
544+ steps:
545+ - run: echo
546+`
547+
548+func concCaller(top string) string {
549+ return "on: push\njobs:\n c:\n" + top + " uses: ./.github/workflows/callee.yml\n with:\n env: ${{ github.ref_name }}\n"
550+}
551+
552+// D4: the concurrency of the calling job, and the workflow-level concurrency of
553+// the called workflow (evaluated with the callee's inputs), hold the whole call:
554+// its jobs are not assignable until the call has the group, and the group is
555+// freed when the call ends. Both groups are tested in turn.
556+func TestCIReusableCallConcurrency(t *testing.T) {
557+ for _, tt := range []struct{ name, caller, callee, group string }{
558+ {"calling job", " concurrency:\n group: call-group\n", "", "call-group"},
559+ {"called workflow", "", "concurrency: deploy-${{ inputs.env }}\n", "deploy-main"},
560+ } {
561+ t.Run(tt.name, func(t *testing.T) {
562+ c, runner := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
563+ callee := strings.Replace(concCallee, "%s\n", tt.callee, 1)
564+ files := map[string]string{".github/workflows/callee.yml": callee, ".github/workflows/ci.yml": concCaller(tt.caller)}
565+ c.push("main", files, "")
566+ r1 := c.awaitRunOf(".github/workflows/ci.yml", 0)
567+ j1 := awaitRecord(c, runner, r1.GetId(), "c/j")
568+
569+ c.push("main", map[string]string{"x.txt": "1"}, "second")
570+ r2 := c.awaitRunOf(".github/workflows/ci.yml", r1.GetId())
571+ c.waitFor("the second call waits for the group", func() bool {
572+ call := jobByKey(c, r2.GetId(), "c")
573+ return call != nil && call.GetStatus() == pb.CiStatus_CI_STATUS_PENDING
574+ })
575+ // Its jobs exist but are not assignable; they show the group.
576+ j, err := runner.fetch("")
577+ require.NoError(t, err)
578+ assert.Nil(t, j, "no job of the second call is given while the first holds the group")
579+
580+ runner.runJob(j1, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
581+ j2 := awaitRecord(c, runner, r2.GetId(), "c/j")
582+ runner.runJob(j2, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
583+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(r2.GetId()).GetConclusion())
584+
585+ // Freed with the call: a third call takes it at once.
586+ c.push("main", map[string]string{"y.txt": "1"}, "third")
587+ r3 := c.awaitRunOf(".github/workflows/ci.yml", r2.GetId())
588+ j3 := awaitRecord(c, runner, r3.GetId(), "c/j")
589+ runner.runJob(j3, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
590+ c.waitRunDone(r3.GetId())
591+ })
592+ }
593+}
594+
595+// D4: cancel-in-progress on a call cancels the running call: its running job is
596+// asked to stop, the call ends cancelled with GitHub's annotation, and the
597+// waiting call takes the group.
598+func TestCIReusableCallCancelInProgress(t *testing.T) {
599+ for _, tt := range []struct{ name, caller, callee string }{
600+ {"calling job", " concurrency:\n group: Cip\n cancel-in-progress: true\n", ""},
601+ {"called workflow", "", "concurrency:\n group: Cip\n cancel-in-progress: true\n"},
602+ } {
603+ t.Run(tt.name, func(t *testing.T) {
604+ c, runner := newReusableHarness(t, newFakeRemote(t), newFakeWorkflows(t))
605+ callee := strings.Replace(concCallee, "%s\n", tt.callee, 1)
606+ c.push("main", map[string]string{".github/workflows/callee.yml": callee, ".github/workflows/ci.yml": concCaller(tt.caller)}, "")
607+ r1 := c.awaitRunOf(".github/workflows/ci.yml", 0)
608+ j1 := awaitRecord(c, runner, r1.GetId(), "c/j")
609+ watch, err := runner.jobAPI(j1).runner.WatchJob(c.ctx, connect.NewRequest(&pb.WatchJobRequest{JobId: j1.GetJobId()}))
610+ require.NoError(t, err)
611+
612+ c.push("main", map[string]string{"x.txt": "1"}, "second")
613+ r2 := c.awaitRunOf(".github/workflows/ci.yml", r1.GetId())
614+ var cancel *pb.CancelRequest
615+ for watch.Receive() {
616+ if m := watch.Msg().GetCancel(); m != nil {
617+ cancel = m
618+ break
619+ }
620+ }
621+ require.NotNil(t, cancel, "the running job of the call is asked to stop")
622+ assert.Contains(t, cancel.GetMessage(), "higher priority waiting request for 'Cip'")
623+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_CANCELLED, c.cancelledCall(r1), "the call ends cancelled")
624+
625+ _, err = runner.jobAPI(j1).runner.CompleteJob(c.ctx, connect.NewRequest(&pb.CompleteJobRequest{
626+ JobId: j1.GetJobId(), Conclusion: pb.CiConclusion_CI_CONCLUSION_CANCELLED,
627+ }))
628+ require.NoError(t, err)
629+ j2 := awaitRecord(c, runner, r2.GetId(), "c/j")
630+ runner.runJob(j2, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
631+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_SUCCESS, c.waitRunDone(r2.GetId()).GetConclusion())
632+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_CANCELLED, c.waitRunDone(r1.GetId()).GetConclusion())
633+ })
634+ }
635+}
636+
637+// cancelledCall waits for the call record "c" of a run to end and returns its conclusion.
638+func (c *ciHarness) cancelledCall(r *pb.Run) pb.CiConclusion {
639+ c.t.Helper()
640+ c.waitFor("the call ends", func() bool {
641+ j := jobByKey(c, r.GetId(), "c")
642+ return j != nil && j.GetStatus() == pb.CiStatus_CI_STATUS_COMPLETED
643+ })
644+ return jobByKey(c, r.GetId(), "c").GetConclusion()
645+}
646+
647+// A private repository of the server that the actor cannot read is not a
648+// source of reusable workflows: the call fails as for a repository that does
649+// not exist, and a reader gets it.
650+func TestCIReusableFromAPrivateErmiteRepository(t *testing.T) {
651+ for _, tt := range []struct {
652+ name string
653+ token func(c *ciHarness) (string, string)
654+ reads bool
655+ }{
656+ {"an owner token", nil, true},
657+ {"a token that lists the caller only", func(c *ciHarness) (string, string) {
658+ return c.branchToken([]domain.Scope{domain.ScopeRepoRead, domain.ScopeRepoWrite}, nil, "proj")
659+ }, false},
660+ {"a token that lists both", func(c *ciHarness) (string, string) {
661+ return c.branchToken([]domain.Scope{domain.ScopeRepoRead, domain.ScopeRepoWrite}, nil, "proj", "shared")
662+ }, true},
663+ } {
664+ t.Run(tt.name, func(t *testing.T) {
665+ remote := newFakeRemote(t)
666+ remote.set("owner/shared", ptrRef("refs/heads/main", shaMain))
667+ wf := newFakeWorkflows(t)
668+ c, runner := newReusableHarness(t, remote, wf)
669+ c.seed("shared", domain.VisibilityPrivate)
670+ wt := c.workTree("main")
671+ c.writeFile(wt, ".github/workflows/lib.yml", []byte(ghCallee))
672+ c.mustGit(wt, "push", "-q", c.urlAuth(adminToken, "/shared.git"), "main")
673+ secret := adminToken
674+ if tt.token != nil {
675+ secret, _ = tt.token(c)
676+ }
677+ c.pushAs(secret, "main", map[string]string{".github/workflows/ci.yml": "on: push\njobs:\n lib:\n uses: owner/shared/.github/workflows/lib.yml@main\n"})
678+ r := c.awaitRunOf(".github/workflows/ci.yml", 0)
679+ if tt.reads {
680+ j := awaitRecord(c, runner, r.GetId(), "lib/test")
681+ runner.runJob(j, pb.CiConclusion_CI_CONCLUSION_SUCCESS, nil)
682+ c.waitRunDone(r.GetId())
683+ return
684+ }
685+ c.waitRunDone(r.GetId())
686+ j := jobByKey(c, r.GetId(), "lib")
687+ require.NotNil(t, j)
688+ assert.Equal(t, pb.CiConclusion_CI_CONCLUSION_FAILURE, j.GetConclusion())
689+ assert.Equal(t, "Unable to resolve action `owner/shared@main`, repository not found", j.GetError(),
690+ "the text of a repository that is not there, nothing about the private one")
691+ assert.False(t, remote.asked("owner/shared"), "never falls through to github.com")
692+ wf.mu.Lock()
693+ assert.Empty(t, wf.calls)
694+ wf.mu.Unlock()
695+ })
696+ }
697+}
M internal/app/forge.go +16 -2
@@ -22,6 +22,7 @@ import (
2222 "bnema.dev/ermite/internal/adapters/gitcli"
2323 "bnema.dev/ermite/internal/adapters/githttp"
2424 "bnema.dev/ermite/internal/adapters/githubapi"
25+ "bnema.dev/ermite/internal/adapters/githubraw"
2526 "bnema.dev/ermite/internal/adapters/gitremote"
2627 "bnema.dev/ermite/internal/adapters/httpserver"
2728 "bnema.dev/ermite/internal/adapters/kvstore"
@@ -60,6 +61,9 @@ type options struct {
6061 // remote replaces the lister of github.com refs (tests never reach the
6162 // network).
6263 remoteLister ports.GitRemote
64+ // workflowFetcher replaces the downloader of github.com reusable workflows
65+ // (tests never reach the network).
66+ workflowFetcher ports.WorkflowFetcher
6367 // ci overrides the settings of the CI engine (tests shorten its
6468 // durations).
6569 ci func(*usecase.CIConfig)
@@ -73,6 +77,15 @@ func (o options) remote(def ports.GitRemote) ports.GitRemote {
7377 return def
7478 }
7579
80+// workflows returns the downloader of github.com reusable workflows: the
81+// test's, else def.
82+func (o options) workflows(def ports.WorkflowFetcher) ports.WorkflowFetcher {
83+ if o.workflowFetcher != nil {
84+ return o.workflowFetcher
85+ }
86+ return def
87+}
88+
7689 // forge is the wired server: every adapter and use case, ready to serve.
7790 type forge struct {
7891 cfg config.Config
@@ -241,8 +254,9 @@ func newForge(ctx context.Context, cfg config.Config, opts options) (_ *forge, e
241254 Bin: cfg.Git, Path: path, Home: filepath.Join(cfg.DataDir, "run", "git-home"),
242255 Proxy: gitremote.ProxyEnv(os.Getenv),
243256 })),
244- Tokens: store.Tokens(),
245- Settings: f.Settings,
257+ Tokens: store.Tokens(),
258+ Settings: f.Settings,
259+ Workflows: opts.workflows(githubraw.New(githubraw.Config{})),
246260 }
247261 if opts.ci != nil {
248262 opts.ci(&ciCfg)
M internal/domain/ci.go +46 -0
@@ -288,6 +288,15 @@ type CIJob struct {
288288 // the job is planned, else the one of its run (FromRun).
289289 Concurrency CIConcurrency
290290
291+ // Scope is the id of the record of the job that called the reusable workflow
292+ // this job belongs to, "" for a job of the run's own workflow. CallID is the
293+ // id of the job in that called workflow (Key is "<caller key>/<call id>").
294+ Scope string
295+ CallID string
296+ // Call is set on the record of a job that calls a reusable workflow: it never
297+ // runs, it ends when the jobs of its scope have.
298+ Call *CICall
299+
291300 // TokenHash is the hash of the job token of the current assignment; zero
292301 // when there is none.
293302 TokenHash TokenHash
@@ -305,6 +314,37 @@ type CIJob struct {
305314 Version uint64
306315 }
307316
317+// CICall is what a job calling a reusable workflow resolved (CI-3, D7).
318+type CICall struct {
319+ Uses string
320+ // Path is the file in its repository, Repository "owner/repo" for github.com
321+ // or the name on this server, Ref what the workflow wrote ("" for a local
322+ // call), Sha the commit the call resolved to.
323+ Path, Repository, Ref, Sha string
324+ YAML string
325+ // InputsJSON is the inputs context of the called workflow.
326+ InputsJSON string
327+ // Secrets holds the names (upper case) of the secrets of the called workflow
328+ // that the calling workflow's "secrets:" gives, with empty values: the
329+ // expressions are read from the pinned calling workflow when a job is
330+ // assigned. Inherit: "secrets: inherit".
331+ Secrets map[string]string
332+ Inherit bool
333+ // Depth is 1 for a call from the run's own workflow.
334+ Depth int
335+ // Expanded: the jobs of the called workflow exist.
336+ Expanded bool
337+ // Concurrency is the workflow-level group of the called workflow. The call
338+ // holds it, with the group of the calling job (CIJob.Concurrency), until it
339+ // ends; its jobs wait for both.
340+ Concurrency CIConcurrency
341+ // Admitted: the call may start, the max-parallel of the calling job's
342+ // strategy allows it.
343+ Admitted bool
344+ // SecretCtx is the JSON of the other contexts the Secrets expressions read.
345+ SecretCtx string
346+}
347+
308348 // Status is the status owners see: a job is queued until its runner
309349 // acknowledged it.
310350 func (j CIJob) Status() CIStatus {
@@ -317,6 +357,12 @@ func (j CIJob) Status() CIStatus {
317357 if j.Concurrency.State == CIConcurrencyPending {
318358 return CIPending
319359 }
360+ if j.Call != nil && j.Call.Concurrency.State == CIConcurrencyPending {
361+ return CIPending
362+ }
363+ if j.Call != nil && j.Call.Expanded && j.Call.Admitted {
364+ return CIInProgress
365+ }
320366 }
321367 return CIQueued
322368 }
M internal/domain/ci_permissions.go +36 -0
@@ -2,6 +2,8 @@
22
33 package domain
44
5+import "slices"
6+
57 // The permissions of a job repository token (CI-3, D3) are the keys and levels
68 // of the "permissions:" of a GitHub workflow. They are computed once, when the
79 // job is planned, and stored as a map from key to level; a key that is absent
@@ -172,3 +174,37 @@ func RepoTokenScopes(perms map[string]string) []Scope {
172174 }
173175 return scopes
174176 }
177+
178+// permissionRank orders the levels: none, read, write.
179+func permissionRank(level string) int {
180+ switch level {
181+ case PermissionRead:
182+ return 1
183+ case PermissionWrite:
184+ return 2
185+ }
186+ return 0
187+}
188+
189+// PermissionViolations lists the keys that requested asks for above what max
190+// allows (an absent key is none): a reusable workflow can only keep or lower the
191+// permissions of its caller. The two slices hold "key: level" in the same
192+// order, sorted by key, as GitHub's error message shows them.
193+func PermissionViolations(requested, max map[string]string) (asked, allowed []string) {
194+ keys := make([]string, 0, len(requested))
195+ for k := range requested {
196+ keys = append(keys, k)
197+ }
198+ slices.Sort(keys)
199+ for _, k := range keys {
200+ if permissionRank(requested[k]) > permissionRank(max[k]) {
201+ lv := max[k]
202+ if lv == "" {
203+ lv = PermissionNone
204+ }
205+ asked = append(asked, k+": "+requested[k])
206+ allowed = append(allowed, k+": "+lv)
207+ }
208+ }
209+ return asked, allowed
210+}
M internal/domain/ci_permissions_test.go +11 -0
@@ -136,3 +136,14 @@ func TestRepoTokenPrincipalFollowsItsPermissions(t *testing.T) {
136136 assert.False(t, domain.Anonymous().ViaJobToken())
137137 assert.False(t, domain.TokenPrincipal(domain.Token{ID: "0123456789abcdef"}).ViaJobToken())
138138 }
139+
140+func TestPermissionViolations(t *testing.T) {
141+ t.Parallel()
142+ asked, allowed := domain.PermissionViolations(
143+ map[string]string{"contents": "write", "issues": "read", "packages": "read", "pages": "none"},
144+ map[string]string{"contents": "read", "packages": "write"})
145+ assert.Equal(t, []string{"contents: write", "issues: read"}, asked)
146+ assert.Equal(t, []string{"contents: read", "issues: none"}, allowed)
147+ asked, _ = domain.PermissionViolations(map[string]string{"contents": "read"}, map[string]string{"contents": "write"})
148+ assert.Empty(t, asked, "lowering is allowed")
149+}
M internal/ports/ci.go +9 -0
@@ -438,6 +438,15 @@ type GitRemote interface {
438438 ListRefs(ctx context.Context, owner, repo string) ([]RemoteRef, error)
439439 }
440440
441+// WorkflowFetcher downloads one reusable workflow file from github.com at a
442+// commit that was already resolved (CI-3, D7), with no credentials. It returns
443+// domain.ErrNotFound for a file or repository that does not exist or needs
444+// credentials, domain.ErrResourceExhausted for a file larger than maxBytes, and
445+// an error matching domain.ErrUnavailable when it cannot tell.
446+type WorkflowFetcher interface {
447+ FetchWorkflow(ctx context.Context, owner, repo, sha, path string, maxBytes int64) ([]byte, error)
448+}
449+
441450 // JobSignal is one message of WatchJob.
442451 type JobSignal struct {
443452 Keepalive bool
A internal/ports/mocks/WorkflowFetcher.go +139 -0
@@ -0,0 +1,139 @@
1+// Code generated by mockery; DO NOT EDIT.
2+// github.com/vektra/mockery
3+// template: testify
4+
5+package mocks
6+
7+import (
8+ "context"
9+
10+ mock "github.com/stretchr/testify/mock"
11+)
12+
13+// NewWorkflowFetcher creates a new instance of WorkflowFetcher. It also registers a testing interface on the mock and a cleanup function to assert the mocks expectations.
14+// The first argument is typically a *testing.T value.
15+func NewWorkflowFetcher(t interface {
16+ mock.TestingT
17+ Cleanup(func())
18+}) *WorkflowFetcher {
19+ if helper, ok := t.(interface{ Helper() }); ok {
20+ helper.Helper()
21+ }
22+
23+ mock := &WorkflowFetcher{}
24+ mock.Mock.Test(t)
25+
26+ t.Cleanup(func() {
27+ if helper, ok := t.(interface{ Helper() }); ok {
28+ helper.Helper()
29+ }
30+ mock.AssertExpectations(t)
31+ })
32+
33+ return mock
34+}
35+
36+// WorkflowFetcher is an autogenerated mock type for the WorkflowFetcher type
37+type WorkflowFetcher struct {
38+ mock.Mock
39+}
40+
41+type WorkflowFetcher_Expecter struct {
42+ mock *mock.Mock
43+}
44+
45+func (_m *WorkflowFetcher) EXPECT() *WorkflowFetcher_Expecter {
46+ return &WorkflowFetcher_Expecter{mock: &_m.Mock}
47+}
48+
49+// FetchWorkflow provides a mock function for the type WorkflowFetcher
50+func (_mock *WorkflowFetcher) FetchWorkflow(ctx context.Context, owner string, repo string, sha string, path string, maxBytes int64) ([]byte, error) {
51+ ret := _mock.Called(ctx, owner, repo, sha, path, maxBytes)
52+
53+ if len(ret) == 0 {
54+ panic("no return value specified for FetchWorkflow")
55+ }
56+
57+ var r0 []byte
58+ var r1 error
59+ if returnFunc, ok := ret.Get(0).(func(context.Context, string, string, string, string, int64) ([]byte, error)); ok {
60+ return returnFunc(ctx, owner, repo, sha, path, maxBytes)
61+ }
62+ if returnFunc, ok := ret.Get(0).(func(context.Context, string, string, string, string, int64) []byte); ok {
63+ r0 = returnFunc(ctx, owner, repo, sha, path, maxBytes)
64+ } else {
65+ if ret.Get(0) != nil {
66+ r0 = ret.Get(0).([]byte)
67+ }
68+ }
69+ if returnFunc, ok := ret.Get(1).(func(context.Context, string, string, string, string, int64) error); ok {
70+ r1 = returnFunc(ctx, owner, repo, sha, path, maxBytes)
71+ } else {
72+ r1 = ret.Error(1)
73+ }
74+ return r0, r1
75+}
76+
77+// WorkflowFetcher_FetchWorkflow_Call is a *mock.Call that shadows Run/Return methods with type explicit version for method 'FetchWorkflow'
78+type WorkflowFetcher_FetchWorkflow_Call struct {
79+ *mock.Call
80+}
81+
82+// FetchWorkflow is a helper method to define mock.On call
83+// - ctx context.Context
84+// - owner string
85+// - repo string
86+// - sha string
87+// - path string
88+// - maxBytes int64
89+func (_e *WorkflowFetcher_Expecter) FetchWorkflow(ctx any, owner any, repo any, sha any, path any, maxBytes any) *WorkflowFetcher_FetchWorkflow_Call {
90+ return &WorkflowFetcher_FetchWorkflow_Call{Call: _e.mock.On("FetchWorkflow", ctx, owner, repo, sha, path, maxBytes)}
91+}
92+
93+func (_c *WorkflowFetcher_FetchWorkflow_Call) Run(run func(ctx context.Context, owner string, repo string, sha string, path string, maxBytes int64)) *WorkflowFetcher_FetchWorkflow_Call {
94+ _c.Call.Run(func(args mock.Arguments) {
95+ var arg0 context.Context
96+ if args[0] != nil {
97+ arg0 = args[0].(context.Context)
98+ }
99+ var arg1 string
100+ if args[1] != nil {
101+ arg1 = args[1].(string)
102+ }
103+ var arg2 string
104+ if args[2] != nil {
105+ arg2 = args[2].(string)
106+ }
107+ var arg3 string
108+ if args[3] != nil {
109+ arg3 = args[3].(string)
110+ }
111+ var arg4 string
112+ if args[4] != nil {
113+ arg4 = args[4].(string)
114+ }
115+ var arg5 int64
116+ if args[5] != nil {
117+ arg5 = args[5].(int64)
118+ }
119+ run(
120+ arg0,
121+ arg1,
122+ arg2,
123+ arg3,
124+ arg4,
125+ arg5,
126+ )
127+ })
128+ return _c
129+}
130+
131+func (_c *WorkflowFetcher_FetchWorkflow_Call) Return(bytes []byte, err error) *WorkflowFetcher_FetchWorkflow_Call {
132+ _c.Call.Return(bytes, err)
133+ return _c
134+}
135+
136+func (_c *WorkflowFetcher_FetchWorkflow_Call) RunAndReturn(run func(ctx context.Context, owner string, repo string, sha string, path string, maxBytes int64) ([]byte, error)) *WorkflowFetcher_FetchWorkflow_Call {
137+ _c.Call.Return(run)
138+ return _c
139+}
M internal/usecase/ci.go +3 -0
@@ -69,6 +69,9 @@ type CIConfig struct {
6969 // Remote lists the refs of github.com repositories for ResolveActions; nil
7070 // answers every github.com action with unavailable.
7171 Remote ports.GitRemote
72+ // Workflows downloads reusable workflow files from github.com at a resolved
73+ // commit; nil fails such a call as unavailable.
74+ Workflows ports.WorkflowFetcher
7275 // Archive serves the tarballs of the REST subset.
7376 Archive ports.GitArchive
7477 // Tokens finds the stored token of the pusher of a run, to ask what it may
M internal/usecase/ci_concurrency.go +25 -7
@@ -313,7 +313,7 @@ func (c *CI) cancelLater(group string, e domain.CIConcurrencyEntry) {
313313 // where it stands: its own record does not say want yet (a promotion whose
314314 // planning failed or was lost, or a crash between the write of the group and the
315315 // write of the entry). An entry that is gone or over is the pruning's business.
316-func (c *CI) entryNeeds(ctx context.Context, repo domain.RepoName, e domain.CIConcurrencyEntry, want domain.CIConcurrencyState) bool {
316+func (c *CI) entryNeeds(ctx context.Context, repo domain.RepoName, group string, e domain.CIConcurrencyEntry, want domain.CIConcurrencyState) bool {
317317 if e.JobID == "" {
318318 run, err := c.cfg.Store.GetRun(ctx, repo, e.RunID)
319319 return err == nil && run.Status != domain.CICompleted && run.Attempt == e.Attempt && run.Concurrency.State != want
@@ -325,7 +325,16 @@ func (c *CI) entryNeeds(ctx context.Context, repo domain.RepoName, e domain.CICo
325325 run, err := c.cfg.Store.GetRun(ctx, repo, e.RunID)
326326 return err == nil && run.Status != domain.CICompleted && run.Attempt == e.Attempt
327327 }
328- return err == nil && !j.Ended() && j.Phase == domain.CIPhaseWaiting && j.Concurrency.State != want
328+ if err != nil || j.Ended() || j.Phase != domain.CIPhaseWaiting {
329+ return false
330+ }
331+ // A call holds two groups, its own and its callee's: the state to compare
332+ // is the one of the group asked about.
333+ if j.Call != nil && domain.ConcurrencyKey(j.Call.Concurrency.Group) == domain.ConcurrencyKey(group) &&
334+ domain.ConcurrencyKey(j.Concurrency.Group) != domain.ConcurrencyKey(group) {
335+ return j.Call.Concurrency.State != want
336+ }
337+ return j.Concurrency.State != want
329338 }
330339
331340 // ReconcileConcurrency repairs the groups. It runs at startup and every minute.
@@ -376,11 +385,11 @@ func (c *CI) ReconcileConcurrency(ctx context.Context) {
376385 if r.Canceling {
377386 c.cancelLater(st.Marks[r.Key()].Group, *r)
378387 }
379- if c.entryNeeds(ctx, g.Repo, *r, domain.CIConcurrencyHolding) {
388+ if c.entryNeeds(ctx, g.Repo, g.Group, *r, domain.CIConcurrencyHolding) {
380389 c.wake(g.Repo, g.Group, *r)
381390 }
382391 }
383- if p := st.Record.Pending; p != nil && c.entryNeeds(ctx, g.Repo, *p, domain.CIConcurrencyPending) {
392+ if p := st.Record.Pending; p != nil && c.entryNeeds(ctx, g.Repo, g.Group, *p, domain.CIConcurrencyPending) {
384393 c.wake(g.Repo, g.Group, *p)
385394 }
386395 for _, m := range st.Marks {
@@ -618,12 +627,21 @@ func (rs *runState) releaseEnded(ctx context.Context, changed []int, runDone boo
618627 c := rs.c
619628 for _, i := range changed {
620629 j := rs.jobs[i]
621- if !j.Ended() || j.Concurrency.Group == "" || j.Concurrency.FromRun {
630+ if !j.Ended() {
622631 continue
623632 }
624633 e := domain.CIConcurrencyEntry{RunID: j.RunID, Attempt: j.RunAttempt, JobID: j.ID}
625- if err := c.releaseEntry(ctx, rs.run.Repo, j.Concurrency.Group, e); err != nil && !errors.Is(err, domain.ErrNotFound) {
626- ctxLog(ctx).Error().Err(err).Str("job_id", j.ID).Msg("ci: free a concurrency group")
634+ groups := []string{}
635+ if j.Concurrency.Group != "" && !j.Concurrency.FromRun {
636+ groups = append(groups, j.Concurrency.Group)
637+ }
638+ if j.Call != nil && j.Call.Concurrency.Group != "" {
639+ groups = append(groups, j.Call.Concurrency.Group) // a call ends holding both
640+ }
641+ for _, g := range groups {
642+ if err := c.releaseEntry(ctx, rs.run.Repo, g, e); err != nil && !errors.Is(err, domain.ErrNotFound) {
643+ ctxLog(ctx).Error().Err(err).Str("job_id", j.ID).Msg("ci: free a concurrency group")
644+ }
627645 }
628646 }
629647 if runDone && rs.run.Concurrency.Group != "" {
M internal/usecase/ci_plan.go +51 -31
@@ -32,10 +32,12 @@ type wfCache struct {
3232 // never a value), per run then job key: the workflow file of a run does not
3333 // change, so the walk of its YAML is done once per job.
3434 uses map[int64]map[string]secretUse
35+ // called keeps the parsed reusable workflows, by hash of path and content.
36+ called map[string]*workflow.Workflow
3537 }
3638
3739 func newWfCache(max int) *wfCache {
38- return &wfCache{max: max, m: map[int64]*workflow.Workflow{}, uses: map[int64]map[string]secretUse{}}
40+ return &wfCache{max: max, m: map[int64]*workflow.Workflow{}, uses: map[int64]map[string]secretUse{}, called: map[string]*workflow.Workflow{}}
3941 }
4042
4143 // secretUse returns the secrets the job refers to, walking the YAML once per
@@ -80,19 +82,6 @@ func (w *wfCache) get(runID int64, path, yaml string) (*workflow.Workflow, error
8082 return wf, nil
8183 }
8284
83-// unsupported names the feature of a job that ermite does not run yet, or "".
84-// Such a job fails with "not supported yet: <feature>" instead of running
85-// wrong. CI-2 runs steps with uses:, job container:, services: and step
86-// timeout-minutes (ADR 0025), CI-3 runs environment:; what stays is the rest of
87-// CI-3.
88-func unsupported(j *workflow.Job) string {
89- switch {
90- case j.IsReusable():
91- return "reusable workflows (uses at the job level)"
92- }
93- return ""
94-}
95-
9685 // instances returns the indexes of the records of the job key, in order.
9786 func (rs *runState) instances(key string) []int {
9887 var out []int
@@ -153,12 +142,12 @@ func result(insts []domain.CIJob) string {
153142 // ancestor failed (documented, not recorded). An ancestor beyond the direct
154143 // needs only changes success() and failure(): cancelled() is not recorded for
155144 // them, and a skipped ancestor behind a need that did run is not either.
156-func (rs *runState) needsOf(wf *workflow.Workflow, j *workflow.Job) (needs *expr.Dictionary, st workflow.StatusFuncs, ready bool) {
145+func (rs *runState) needsOf(wf *workflow.Workflow, j *workflow.Job, scope string) (needs *expr.Dictionary, st workflow.StatusFuncs, ready bool) {
157146 needs = expr.NewDictionary()
158147 st = workflow.StatusFuncs{Success: true, Cancelled: rs.run.CancelRequested}
159148 for _, n := range wf.Ancestors(j.ID) {
160149 var insts []domain.CIJob
161- for _, i := range rs.instances(n) {
150+ for _, i := range rs.instancesIn(scope, n) {
162151 if rs.jobs[i].Ended() {
163152 insts = append(insts, rs.jobs[i])
164153 }
@@ -172,7 +161,7 @@ func (rs *runState) needsOf(wf *workflow.Workflow, j *workflow.Job) (needs *expr
172161 }
173162 for _, n := range j.Needs {
174163 var insts []domain.CIJob
175- for _, i := range rs.instances(n) {
164+ for _, i := range rs.instancesIn(scope, n) {
176165 if !rs.jobs[i].Ended() {
177166 return nil, st, false
178167 }
@@ -275,7 +264,7 @@ func (rs *runState) end(j *domain.CIJob, conclusion domain.CIConclusion, msg str
275264 if j.FailFast && !j.ContinueOnError && conclusion.IsFailure() {
276265 for i := range rs.jobs {
277266 s := &rs.jobs[i]
278- if s.Key != j.Key || s.ID == j.ID || s.Ended() {
267+ if s.Key != j.Key || s.Scope != j.Scope || s.ID == j.ID || s.Ended() {
279268 continue
280269 }
281270 if s.Active() {
@@ -344,13 +333,21 @@ func (rs *runState) advance(ctx context.Context) error {
344333 if rec.Planned || rec.Ended() || rec.Phase != domain.CIPhaseWaiting {
345334 continue
346335 }
347- needs, st, ready := rs.needsOf(wf, wj)
336+ needs, st, ready := rs.needsOf(wf, wj, "")
348337 if !ready {
349338 continue
350339 }
351340 rs.planJob(ctx, wf, wj, insts[0], needs, st)
352341 changed = true
353342 }
343+ // The jobs of called workflows, and the calls that have ended.
344+ ch, err := rs.advanceCalls(ctx)
345+ if err != nil {
346+ return err
347+ }
348+ if ch {
349+ changed = true
350+ }
354351 }
355352 return rs.promote(ctx)
356353 }
@@ -363,15 +360,15 @@ func (rs *runState) promote(ctx context.Context) error {
363360 counts := map[string]int{}
364361 for _, j := range rs.jobs {
365362 if j.Phase == domain.CIPhaseReady || j.Active() {
366- counts[j.Key]++
363+ counts[j.Scope+"\x00"+j.Key]++
367364 }
368365 }
369366 for i := range rs.jobs {
370367 j := &rs.jobs[i]
371- if !j.Planned || j.Phase != domain.CIPhaseWaiting {
372- continue
368+ if !j.Planned || j.Phase != domain.CIPhaseWaiting || j.Call != nil {
369+ continue // a call never runs: it ends with the jobs of its scope
373370 }
374- if j.MaxParallel > 0 && counts[j.Key] >= j.MaxParallel {
371+ if j.MaxParallel > 0 && counts[j.Scope+"\x00"+j.Key] >= j.MaxParallel {
375372 continue
376373 }
377374 if g := j.Concurrency; g.Group != "" && !g.FromRun {
@@ -388,7 +385,7 @@ func (rs *runState) promote(ctx context.Context) error {
388385 }
389386 }
390387 j.Phase, j.ReadyAt = domain.CIPhaseReady, now
391- counts[j.Key]++
388+ counts[j.Scope+"\x00"+j.Key]++
392389 rs.touch(j)
393390 }
394391 return nil
@@ -415,7 +412,7 @@ func (rs *runState) planJob(ctx context.Context, wf *workflow.Workflow, wj *work
415412 return
416413 }
417414 env := template.Env{Contexts: map[string]expr.Value{
418- "github": gh, "needs": needs, "inputs": expr.NewDictionary(), "vars": vars,
415+ "github": gh, "needs": needs, "inputs": rs.inputsOf(rec), "vars": vars,
419416 }}
420417 run, err := wj.EvaluateIf(ctx, env, st)
421418 switch {
@@ -424,11 +421,20 @@ func (rs *runState) planJob(ctx context.Context, wf *workflow.Workflow, wj *work
424421 return
425422 case !run:
426423 rs.jobs[idx].Name = rs.skippedName(ctx, wj, env)
424+ if call := rs.scopeOf(rec); call != nil {
425+ rs.jobs[idx].Name = call.Name + " / " + rs.jobs[idx].Name
426+ }
427427 rs.end(&rs.jobs[idx], domain.CISkipped, "")
428428 return
429429 }
430- if feature := unsupported(wj); feature != "" {
431- fail("not supported yet: " + feature)
430+ if wj.IsReusable() {
431+ if err := rs.planCall(ctx, wf, wj, idx, env); err != nil {
432+ if text, ok := isCallFailure(err); ok {
433+ fail(text)
434+ } else {
435+ fail("reusable workflow " + wj.Uses.Value + ": " + err.Error())
436+ }
437+ }
432438 return
433439 }
434440 plan, err := wj.EvaluateStrategy(ctx, env)
@@ -437,6 +443,10 @@ func (rs *runState) planJob(ctx context.Context, wf *workflow.Workflow, wj *work
437443 return
438444 }
439445
446+ if err := rs.checkJobBudget(len(plan.Combos) - 1); err != nil {
447+ fail(err.Error())
448+ return
449+ }
440450 insts := make([]domain.CIJob, 0, len(plan.Combos))
441451 for i := range plan.Combos {
442452 menv, err := plan.MatrixEnv(env, i)
@@ -484,6 +494,10 @@ func (rs *runState) instance(ctx context.Context, wf *workflow.Workflow, wj *wor
484494 return j, err
485495 }
486496 name = strings.TrimSpace(name) // GitHub shows "Display name of", not "Display name of "
497+ call := rs.scopeOf(base)
498+ if call != nil {
499+ name = call.Name + " / " + name // "<caller job name> / <callee job name>"
500+ }
487501 labels, _, err := wj.EvaluateRunsOn(ctx, env)
488502 if err != nil {
489503 return j, err
@@ -517,12 +531,18 @@ func (rs *runState) instance(ctx context.Context, wf *workflow.Workflow, wj *wor
517531 if vars == nil {
518532 vars = expr.NewDictionary()
519533 }
520- contexts, err := contextsJSON(gh, needs, plan.Context(i), matrixCtx, vars, rs.c.jobKeys(ctx, rs.run, base.Index, i))
534+ contexts, err := contextsJSON(gh, needs, plan.Context(i), matrixCtx, vars, rs.inputsOf(base), rs.jobKeysOf(ctx, base, i))
521535 if err != nil {
522536 return j, err
523537 }
524538 j.Environment = environment
525- j.Permissions = domain.ResolvePermissions(permissionSpec(wf.Permissions), permissionSpec(wj.Permissions))
539+ if call != nil {
540+ if j.Permissions, err = rs.scopedPermissions(wf, wj, call); err != nil {
541+ return j, err
542+ }
543+ } else {
544+ j.Permissions = domain.ResolvePermissions(permissionSpec(wf.Permissions), permissionSpec(wj.Permissions))
545+ }
526546 j.Name, j.Labels, j.TimeoutMinutes = name, labels, timeout
527547 j.MatrixJSON = compactJSON(expr.ToJSON(matrix))
528548 j.ContextsJSON, j.ComboTotal = contexts, len(plan.Combos)
@@ -565,11 +585,11 @@ func compactJSON(s string) string {
565585 // workflow_ref, workflow_sha, workflow_repository, workflow_file_path); the
566586 // runner adds status. The runner copies the keys of this "job" into its job
567587 // context (executor.State.initRest), whatever they are.
568-func contextsJSON(github, needs, strategy *expr.Dictionary, matrix expr.Value, vars *expr.Dictionary, job ordered) (string, error) {
588+func contextsJSON(github, needs, strategy *expr.Dictionary, matrix expr.Value, vars, inputs *expr.Dictionary, job ordered) (string, error) {
569589 out := map[string]json.RawMessage{}
570590 for k, v := range map[string]expr.Value{
571591 "github": github, "needs": needs, "strategy": strategy, "matrix": matrix,
572- "inputs": expr.NewDictionary(), "vars": vars,
592+ "inputs": inputs, "vars": vars,
573593 } {
574594 out[k] = json.RawMessage(compactJSON(expr.ToJSON(v)))
575595 }
A internal/usecase/ci_reusable.go +1298 -0
@@ -0,0 +1,1298 @@
1+// SPDX-License-Identifier: AGPL-3.0-or-later
2+
3+package usecase
4+
5+import (
6+ "context"
7+ "crypto/sha256"
8+ "encoding/hex"
9+ "encoding/json"
10+ "errors"
11+ "fmt"
12+ "io"
13+ "slices"
14+ "strconv"
15+ "strings"
16+
17+ "go.yaml.in/yaml/v3"
18+
19+ "bnema.dev/ermite-actions/expr"
20+ "bnema.dev/ermite-actions/template"
21+ "bnema.dev/ermite-actions/workflow"
22+ "bnema.dev/ermite/internal/domain"
23+)
24+
25+// Reusable workflows (CI-3, D7). ermite owns the job graph, so a job with
26+// "uses:" is expanded at planning, not by the runner:
27+//
28+// - the job becomes a call record (one per matrix leg) that never runs; it ends
29+// when the jobs of its scope have, with their aggregate conclusion, and its
30+// outputs are the callee's "on.workflow_call.outputs";
31+// - each job of the called workflow becomes a record of that scope, keyed
32+// "<caller key>/<callee id>", named "<caller name> / <callee name>", planned
33+// when the callee's own "needs" have ended (the caller's "needs" ended
34+// before the call was expanded, so they hold for every callee root);
35+// - the called file is kept in the call record, so every job of the call runs
36+// from the file of the commit that was resolved, and the runner is given that
37+// file and the callee job id.
38+//
39+// Texts are the ones of actions/runner (ReusableWorkflowsLoader.cs,
40+// WorkflowTemplateConverter.cs, PermissionsHelper.cs) and languageservices
41+// (validate.ts) where they exist.
42+
43+// maxReusableWorkflowDepth is the nesting limit of reusable workflows
44+// (languageservices workflow-parser/src/model/convert.ts:49, 4; the runner's
45+// check is ReusableWorkflowsLoader.LoadRecursive).
46+const maxReusableWorkflowDepth = 4
47+
48+// maxJobIndex is the highest job position a check id can hold (checkIndexBits).
49+const maxJobIndex = 1<<checkIndexBits - 1
50+
51+// callRef is a parsed "uses:" of a job.
52+type callRef struct {
53+ local bool
54+ owner, repo string
55+ path string // .github/workflows/x.yml
56+ ref string
57+}
58+
59+// parseCallRef reads the "uses:" of a job calling a workflow. The texts are
60+// languageservices' (validate.ts validateWorkflowUsesFormat).
61+func parseCallRef(uses string) (callRef, string) {
62+ bad := func(reason string) (callRef, string) {
63+ return callRef{}, "Invalid workflow reference '" + uses + "': " + reason
64+ }
65+ if strings.HasPrefix(uses, "./.github/workflows/") {
66+ switch {
67+ case strings.Contains(uses, "@"):
68+ return bad("cannot specify version when calling local workflows")
69+ case !strings.HasSuffix(uses, ".yml") && !strings.HasSuffix(uses, ".yaml"):
70+ return bad("workflow file should have either a '.yml' or '.yaml' file extension")
71+ case len(strings.Split(uses, "/")) != 4:
72+ return bad("workflows must be defined at the top level of the .github/workflows/ directory")
73+ case strings.HasSuffix(uses, "/.yml") || strings.HasSuffix(uses, "/.yaml"):
74+ return bad("invalid workflow file name")
75+ }
76+ return callRef{local: true, path: strings.TrimPrefix(uses, "./")}, ""
77+ }
78+ if strings.HasPrefix(uses, "./") {
79+ return bad("local workflow references must be rooted in '.github/workflows'")
80+ }
81+ parts := strings.Split(uses, "@")
82+ switch {
83+ case len(parts) == 1 || (len(parts) == 2 && parts[1] == ""):
84+ return bad("no version specified")
85+ case len(parts) > 2:
86+ return bad("too many '@' in workflow reference")
87+ }
88+ i := strings.Index(parts[0], ".github/workflows/")
89+ if i < 0 {
90+ return bad("references to workflows must be rooted in '.github/workflows'")
91+ }
92+ nwo := strings.FieldsFunc(parts[0][:i], func(r rune) bool { return r == '/' })
93+ if len(nwo) != 2 {
94+ return bad("references to workflows must be prefixed with format 'owner/repository/' or './' for local workflows")
95+ }
96+ if !validActionName(nwo[0]) || !validActionName(nwo[1]) {
97+ return bad("owner name must be a valid repository owner name")
98+ }
99+ path := parts[0][i:]
100+ for _, s := range strings.Split(path, "/") {
101+ if s == "" || s == "." || s == ".." {
102+ return bad("references to workflows must be rooted in '.github/workflows'")
103+ }
104+ }
105+ return callRef{owner: nwo[0], repo: nwo[1], path: path, ref: parts[1]}, ""
106+}
107+
108+// callFailure is a call that cannot be expanded; text is the job's error.
109+type callFailure struct{ text string }
110+
111+func (e callFailure) Error() string { return e.text }
112+
113+func isCallFailure(err error) (string, bool) {
114+ var cf callFailure
115+ if errors.As(err, &cf) {
116+ return cf.text, true
117+ }
118+ return "", false
119+}
120+
121+// calledFile is a workflow file read for a call.
122+type calledFile struct {
123+ repository, path, ref, sha string
124+ yaml []byte
125+}
126+
127+// loadCallee finds the file a "uses:" names. home is the call whose file holds
128+// the "uses:" (nil for the run's own workflow): a local reference is read in
129+// the repository and at the commit of that file.
130+func (c *CI) loadCallee(ctx context.Context, rs *runState, home *domain.CICall, rec domain.CIJob, ref callRef) (calledFile, error) {
131+ var f calledFile
132+ if ref.local {
133+ f.path = ref.path
134+ if home == nil {
135+ f.repository, f.sha = rs.run.Repo.String(), rs.run.Sha
136+ } else {
137+ f.repository, f.sha = home.Repository, home.Sha
138+ }
139+ return f, c.readCallee(ctx, &f)
140+ }
141+ f.path, f.ref = ref.path, ref.ref
142+ ar := actionRef{uses: ref.owner + "/" + ref.repo + "@" + ref.ref, owner: ref.owner, repo: ref.repo, ref: ref.ref}
143+ by := c.onBehalfOf(*rs.src)
144+ // One resolution per reference and attempt: what this planning resolved
145+ // already (not stored yet), else what the attempt stored, else a fresh one.
146+ var res domain.CIActionResolution
147+ found := false
148+ for _, r := range rs.resolutions {
149+ if r.RunID == rec.RunID && r.RunAttempt == rec.RunAttempt && r.Key == ar.key() {
150+ res, found = r, true
151+ break
152+ }
153+ }
154+ if !found {
155+ stored, err := c.cfg.Store.GetActionResolution(ctx, rec.RunID, rec.RunAttempt, ar.key())
156+ switch {
157+ case err == nil:
158+ res = stored
159+ case errors.Is(err, domain.ErrNotFound):
160+ var msg string
161+ res, msg, err = c.resolveAction(ctx, rec, ar, newRemoteRefs(c.cfg.Remote, c.remoteSlots), by)
162+ if err != nil {
163+ return f, err
164+ }
165+ if msg != "" {
166+ return f, callFailure{msg}
167+ }
168+ rs.resolutions = append(rs.resolutions, res)
169+ default:
170+ return f, err
171+ }
172+ }
173+ // The source guard of ResolveActions (answer), whatever the resolution came
174+ // from: a repository of this server that the actor cannot read is never
175+ // served from anywhere else, and a stored ermite resolution of a repository
176+ // that is gone has nothing to serve.
177+ repo, onErmite, err := c.ermiteRepoOf(ctx, ar)
178+ if err != nil {
179+ return f, err
180+ }
181+ if onErmite {
182+ if ok, err := c.pusherCanRead(ctx, by, rs.run.Repo, repo); err != nil {
183+ return f, err
184+ } else if !ok {
185+ return f, callFailure{errUnresolvedRepo(ar)}
186+ }
187+ }
188+ f.sha = res.ResolvedSHA
189+ switch res.Source {
190+ case domain.CIActionErmite:
191+ if !onErmite {
192+ return f, callFailure{errUnresolvedRepo(ar)}
193+ }
194+ f.repository = repo.Name.String()
195+ default:
196+ f.repository = ref.owner + "/" + ref.repo
197+ }
198+ return f, c.readCallee(ctx, &f)
199+}
200+
201+// readCallee fills f.yaml from the repository of the server or github.com.
202+func (c *CI) readCallee(ctx context.Context, f *calledFile) error {
203+ notFound := callFailure{"Unable to find reusable workflow '" + f.path + "' in '" + f.repository + "' at '" + f.sha + "'"}
204+ if owner, repo, ok := strings.Cut(f.repository, "/"); ok {
205+ if c.cfg.Workflows == nil {
206+ return fmt.Errorf("%w: this server cannot reach github.com", domain.ErrUnavailable)
207+ }
208+ data, err := c.cfg.Workflows.FetchWorkflow(ctx, owner, repo, f.sha, f.path, maxCIWorkflowBytes)
209+ switch {
210+ case errors.Is(err, domain.ErrNotFound):
211+ return notFound
212+ case errors.Is(err, domain.ErrResourceExhausted):
213+ return callFailure{f.path + ": the workflow file is larger than 1 MiB"}
214+ case err != nil:
215+ return err
216+ }
217+ f.yaml = data
218+ return nil
219+ }
220+ name, err := domain.ParseRepoName(f.repository)
221+ if err != nil {
222+ return notFound
223+ }
224+ id, err := domain.ParseObjectID(f.sha)
225+ if err != nil {
226+ return notFound
227+ }
228+ tp, err := domain.ParseTreePath(f.path)
229+ if err != nil {
230+ return notFound
231+ }
232+ _, rc, err := c.cfg.Reader.OpenBlob(ctx, name, domain.RevisionOf(id), tp)
233+ if errors.Is(err, domain.ErrNotFound) {
234+ return notFound
235+ }
236+ if err != nil {
237+ return err
238+ }
239+ defer func() { _ = rc.Close() }()
240+ data, err := io.ReadAll(io.LimitReader(rc, maxCIWorkflowBytes+1))
241+ if err != nil {
242+ return err
243+ }
244+ if len(data) > maxCIWorkflowBytes {
245+ return callFailure{f.path + ": the workflow file is larger than 1 MiB"}
246+ }
247+ f.yaml = data
248+ return nil
249+}
250+
251+// callerPrefix is the text GitHub puts before an error of the called workflow
252+// (TemplateStrings: CalledWorkflowNotValidWithErrors, LineColumn).
253+func callerPrefix(wj *workflow.Job) string {
254+ file, pos := "", template.Pos{}
255+ if wj.Uses != nil {
256+ file, pos = wj.Uses.File(), wj.Uses.Pos()
257+ }
258+ return fmt.Sprintf("In %s (Line: %d, Col: %d): Error from called workflow", file, pos.Line, pos.Col)
259+}
260+
261+// calleeSpec is what ermite reads of "on.workflow_call" that the workflow model
262+// does not keep: the outputs and the secrets (ermite-actions parses the inputs).
263+type calleeSpec struct {
264+ outputs map[string]string
265+ secrets map[string]bool // name (upper case) to required
266+}
267+
268+// readCalleeSpec reads on.workflow_call.outputs and .secrets from the YAML.
269+func readCalleeSpec(src []byte) calleeSpec {
270+ spec := calleeSpec{outputs: map[string]string{}, secrets: map[string]bool{}}
271+ var doc yaml.Node
272+ if yaml.Unmarshal(src, &doc) != nil || len(doc.Content) == 0 {
273+ return spec
274+ }
275+ root := derefYAML(doc.Content[0])
276+ for i := 0; root.Kind == yaml.MappingNode && i+1 < len(root.Content); i += 2 {
277+ if root.Content[i].Value != "on" {
278+ continue
279+ }
280+ on := derefYAML(root.Content[i+1])
281+ for j := 0; on.Kind == yaml.MappingNode && j+1 < len(on.Content); j += 2 {
282+ if on.Content[j].Value != "workflow_call" {
283+ continue
284+ }
285+ wc := derefYAML(on.Content[j+1])
286+ for k := 0; wc.Kind == yaml.MappingNode && k+1 < len(wc.Content); k += 2 {
287+ body := derefYAML(wc.Content[k+1])
288+ for m := 0; body.Kind == yaml.MappingNode && m+1 < len(body.Content); m += 2 {
289+ name, def := body.Content[m].Value, derefYAML(body.Content[m+1])
290+ switch wc.Content[k].Value {
291+ case "outputs":
292+ for q := 0; def.Kind == yaml.MappingNode && q+1 < len(def.Content); q += 2 {
293+ if def.Content[q].Value == "value" {
294+ spec.outputs[name] = derefYAML(def.Content[q+1]).Value
295+ }
296+ }
297+ case "secrets":
298+ req := false
299+ for q := 0; def.Kind == yaml.MappingNode && q+1 < len(def.Content); q += 2 {
300+ if def.Content[q].Value == "required" {
301+ req = derefYAML(def.Content[q+1]).Value == "true"
302+ }
303+ }
304+ spec.secrets[strings.ToUpper(name)] = req
305+ }
306+ }
307+ }
308+ }
309+ }
310+ return spec
311+}
312+
313+func derefYAML(n *yaml.Node) *yaml.Node {
314+ for range 16 {
315+ if n.Kind != yaml.AliasNode || n.Alias == nil {
316+ return n
317+ }
318+ n = n.Alias
319+ }
320+ return n
321+}
322+
323+// callInputs checks the "with:" of a call against the inputs of the called
324+// workflow and returns the inputs context: the value given, else the default,
325+// else the zero of the type. Texts: WorkflowTemplateConverter.cs
326+// (ConvertToWorkflowJobInputs, ValidateUndefinedParameters).
327+func callInputs(ctx context.Context, defs []workflow.Input, given *expr.Dictionary, defEnv template.Env) (*expr.Dictionary, error) {
328+ out := expr.NewDictionary()
329+ byName := map[string]bool{}
330+ for _, in := range defs {
331+ byName[strings.ToLower(in.Name)] = true
332+ var v expr.Value
333+ have := false
334+ if given != nil {
335+ for _, p := range given.Pairs() {
336+ if strings.EqualFold(p.Key, in.Name) {
337+ cv, err := coerceInput(in, p.Value)
338+ if err != nil {
339+ return nil, err
340+ }
341+ v, have = cv, true
342+ }
343+ }
344+ }
345+ if !have && in.HasDefault {
346+ dv, err := evalInterpolated(ctx, in.Default, defEnv)
347+ if err != nil {
348+ return nil, err
349+ }
350+ cv, err := coerceInput(in, dv)
351+ if err != nil {
352+ return nil, err
353+ }
354+ v, have = cv, true
355+ }
356+ if !have && in.Required {
357+ return nil, callFailure{"Input " + in.Name + " is required, but not provided while calling."}
358+ }
359+ if !have {
360+ switch in.Type {
361+ case "boolean":
362+ v = expr.Boolean(false)
363+ case "number":
364+ v = expr.Number(0)
365+ default:
366+ v = expr.String("")
367+ }
368+ }
369+ out.Add(in.Name, v)
370+ }
371+ if given != nil {
372+ for _, p := range given.Pairs() {
373+ if !byName[strings.ToLower(p.Key)] {
374+ return nil, callFailure{"Invalid input, " + p.Key + " is not defined in the referenced workflow."}
375+ }
376+ }
377+ }
378+ return out, nil
379+}
380+
381+// coerceInput converts a value to the type of an input, as the template engine
382+// does for a boolean, number or string (GetDefaultValueByType,
383+// ConvertToInputValueDefinedType): a mismatch is "Unexpected value".
384+func coerceInput(in workflow.Input, v expr.Value) (expr.Value, error) {
385+ unexpected := func() (expr.Value, error) {
386+ return nil, callFailure{"Unexpected value '" + expr.CoerceString(v) + "' for input '" + in.Name + "' of type '" + in.Type + "'"}
387+ }
388+ switch in.Type {
389+ case "boolean":
390+ switch x := v.(type) {
391+ case expr.Boolean:
392+ return x, nil
393+ case expr.String:
394+ switch strings.ToLower(string(x)) {
395+ case "true":
396+ return expr.Boolean(true), nil
397+ case "false":
398+ return expr.Boolean(false), nil
399+ }
400+ }
401+ return unexpected()
402+ case "number":
403+ switch x := v.(type) {
404+ case expr.Number:
405+ return x, nil
406+ case expr.String:
407+ if f, err := strconv.ParseFloat(strings.TrimSpace(string(x)), 64); err == nil {
408+ return expr.Number(f), nil
409+ }
410+ }
411+ return unexpected()
412+ }
413+ switch v.(type) {
414+ case expr.String, expr.Number, expr.Boolean:
415+ if s, ok := v.(expr.String); ok {
416+ return s, nil
417+ }
418+ return expr.String(expr.CoerceString(v)), nil
419+ }
420+ return unexpected()
421+}
422+
423+// evalInterpolated evaluates a string with ${{ }} expressions (a default of an
424+// input, the value of an output). A string that is one expression keeps the
425+// type of its value.
426+func evalInterpolated(ctx context.Context, s string, env template.Env) (expr.Value, error) {
427+ if !strings.Contains(s, "${{") {
428+ return expr.String(s), nil
429+ }
430+ ctxNames := make([]string, 0, len(env.Contexts))
431+ for k := range env.Contexts {
432+ ctxNames = append(ctxNames, k)
433+ }
434+ eval := func(src string) (expr.Value, error) {
435+ node, err := expr.Parse(src, ctxNames, exprFuncs)
436+ if err != nil {
437+ return nil, err
438+ }
439+ return expr.Evaluate(ctx, node, expr.EvalContext{Contexts: env.Contexts})
440+ }
441+ exprs := expressionsIn(s)
442+ if t := strings.TrimSpace(s); len(exprs) == 1 && strings.HasPrefix(t, "${{") && strings.HasSuffix(t, "}}") && strings.Count(t, "${{") == 1 {
443+ return eval(exprs[0])
444+ }
445+ var b strings.Builder
446+ rest := s
447+ for _, src := range exprs {
448+ i := strings.Index(rest, "${{")
449+ b.WriteString(rest[:i])
450+ v, err := eval(src)
451+ if err != nil {
452+ return nil, err
453+ }
454+ b.WriteString(expr.CoerceString(v))
455+ rest = rest[i+3+len(src)+2:]
456+ }
457+ b.WriteString(rest)
458+ return expr.String(b.String()), nil
459+}
460+
461+// calleeOf parses the workflow a call record kept.
462+func (rs *runState) calleeOf(call *domain.CIJob) (*workflow.Workflow, error) {
463+ if call == nil || call.Call == nil {
464+ return nil, fmt.Errorf("the call of this job is gone")
465+ }
466+ return rs.c.flows.parseCalled(call.Call.Path, call.Call.YAML)
467+}
468+
469+// scopeOf finds the call record of a scoped job, nil for a job of the run's own
470+// workflow.
471+func (rs *runState) scopeOf(j domain.CIJob) *domain.CIJob {
472+ if j.Scope == "" {
473+ return nil
474+ }
475+ return rs.find(j.Scope)
476+}
477+
478+// inputsOf is the "inputs" context of a job: its call's inputs, empty at the
479+// top.
480+func (rs *runState) inputsOf(j domain.CIJob) *expr.Dictionary {
481+ if sc := rs.scopeOf(j); sc != nil && sc.Call != nil {
482+ if v, err := expr.FromJSON([]byte(sc.Call.InputsJSON)); err == nil {
483+ if d, ok := v.(*expr.Dictionary); ok {
484+ return d
485+ }
486+ }
487+ }
488+ return expr.NewDictionary()
489+}
490+
491+// instancesIn returns the indexes of the records of job id in a scope.
492+func (rs *runState) instancesIn(scope, id string) []int {
493+ var out []int
494+ for i, j := range rs.jobs {
495+ if j.Scope == scope && ((scope == "" && j.Key == id) || (scope != "" && j.CallID == id)) {
496+ out = append(out, i)
497+ }
498+ }
499+ return out
500+}
501+
502+// rootKey is the key of the job of the run's own workflow a record descends
503+// from.
504+func rootKey(key string) string {
505+ root, _, _ := strings.Cut(key, "/")
506+ return root
507+}
508+
509+// nextIndex is a position for a job added to the run.
510+func (rs *runState) nextIndex() int {
511+ n := 0
512+ for _, j := range rs.jobs {
513+ n = max(n, j.Index+1)
514+ }
515+ return n
516+}
517+
518+// planCall expands a job that calls a reusable workflow. rec (at idx) becomes
519+// the first call record, one per matrix leg; the jobs of the called workflow are
520+// added as waiting records of the scope of each leg.
521+func (rs *runState) planCall(ctx context.Context, wf *workflow.Workflow, wj *workflow.Job, idx int, env template.Env) error {
522+ rec := rs.jobs[idx]
523+ parent := rs.scopeOf(rec)
524+ var home *domain.CICall
525+ depth := 1
526+ if parent != nil {
527+ home, depth = parent.Call, parent.Call.Depth+1
528+ }
529+ if depth > maxReusableWorkflowDepth {
530+ return callFailure{fmt.Sprintf("Nested reusable workflow depth exceeded %d.", maxReusableWorkflowDepth)}
531+ }
532+ ref, text := parseCallRef(wj.Uses.Value)
533+ if text != "" {
534+ return callFailure{text}
535+ }
536+ file, err := rs.c.loadCallee(ctx, rs, home, rec, ref)
537+ if err != nil {
538+ return err
539+ }
540+ prefix := callerPrefix(wj)
541+ cw, errs := workflow.Parse(file.path, file.yaml)
542+ if len(errs) > 0 {
543+ return callFailure{prefix + " " + errs[0].Error()}
544+ }
545+ wc := cw.On.Event("workflow_call")
546+ if wc == nil {
547+ return callFailure{prefix + " workflow_call key is not defined in the referenced workflow."}
548+ }
549+ spec := readCalleeSpec(file.yaml)
550+
551+ eff, err := rs.callPermissions(wf, wj, rec, parent)
552+ if err != nil {
553+ return err
554+ }
555+ plan, err := wj.EvaluateStrategy(ctx, env)
556+ if err != nil {
557+ return err
558+ }
559+ vars, _ := env.Contexts["vars"].(*expr.Dictionary)
560+ if vars == nil {
561+ vars = expr.NewDictionary()
562+ }
563+ gh := env.Contexts["github"]
564+
565+ // The records of the run are bounded before anything changes: the calls, and
566+ // the jobs of the called workflow for each (a job that is a matrix adds its
567+ // legs when it is planned, checked there).
568+ if err := rs.checkJobBudget(len(plan.Combos) - 1 + len(plan.Combos)*len(cw.Jobs)); err != nil {
569+ return err
570+ }
571+ legs := make([]domain.CIJob, 0, len(plan.Combos))
572+ for i := range plan.Combos {
573+ menv, err := plan.MatrixEnv(env, i)
574+ if err != nil {
575+ return err
576+ }
577+ var given *expr.Dictionary
578+ if wj.With != nil {
579+ v, err := template.Evaluate(ctx, wj.With, menv)
580+ if err != nil {
581+ return err
582+ }
583+ given, _ = v.(*expr.Dictionary)
584+ }
585+ inEnv := template.Env{Contexts: map[string]expr.Value{"github": gh, "inputs": expr.NewDictionary(), "vars": vars}}
586+ inputs, err := callInputs(ctx, wc.Inputs, given, inEnv)
587+ if err != nil {
588+ if text, ok := isCallFailure(err); ok {
589+ return callFailure{prefix + " " + text}
590+ }
591+ return err
592+ }
593+ // The group of the calling job (job-level, evaluated like the group of
594+ // any job) and the workflow-level group of the called workflow
595+ // (evaluated with the caller's github context and the call's inputs).
596+ // The call record is the entry of both.
597+ var jobConc, wfConc domain.CIConcurrency
598+ if wj.Concurrency != nil {
599+ if jobConc, err = evalConcurrency(ctx, wj.Concurrency, menv); err != nil {
600+ return err
601+ }
602+ jobConc.Evaluated = true
603+ }
604+ if cw.Concurrency != nil {
605+ wenv := template.Env{Contexts: map[string]expr.Value{"github": gh, "inputs": inputs, "vars": vars}}
606+ if wfConc, err = evalConcurrency(ctx, cw.Concurrency, wenv); err != nil {
607+ return callFailure{prefix + " " + err.Error()}
608+ }
609+ wfConc.Evaluated = true
610+ }
611+ call := domain.CICall{
612+ Uses: wj.Uses.Value, Path: file.path, Repository: file.repository, Ref: file.ref, Sha: file.sha, YAML: string(file.yaml),
613+ InputsJSON: compactJSON(expr.ToJSON(inputs)), Depth: depth, Expanded: true, Concurrency: wfConc,
614+ }
615+ if err := rs.callSecrets(wj, menv, spec, &call); err != nil {
616+ if text, ok := isCallFailure(err); ok {
617+ return callFailure{prefix + " " + text}
618+ }
619+ return err
620+ }
621+ name, err := wj.DisplayName(ctx, menv, plan, i)
622+ if err != nil {
623+ return err
624+ }
625+ leg := rec
626+ leg.Combo = i
627+ leg.ID = domain.NewJobID(rs.run.ID, rec.Index, i)
628+ if i > 0 {
629+ leg.Version = 0
630+ }
631+ leg.Name = strings.TrimSpace(name)
632+ if parent != nil {
633+ leg.Name = parent.Name + " / " + leg.Name // "<caller> / <nested caller>"
634+ }
635+ leg.Call, leg.Planned, leg.Phase = &call, true, domain.CIPhaseWaiting
636+ leg.Permissions = eff
637+ leg.Concurrency = jobConc
638+ leg.MatrixJSON = compactJSON(expr.ToJSON(plan.Combos[i].Values))
639+ leg.ContextsJSON, leg.ComboTotal = "{}", len(plan.Combos)
640+ leg.FailFast, leg.MaxParallel = plan.FailFast, plan.MaxParallel
641+ leg.Labels, leg.TimeoutMinutes = nil, 0
642+ legs = append(legs, leg)
643+ }
644+
645+ // The new positions follow the highest one in use.
646+ next := rs.nextIndex()
647+ if next+len(legs)*len(cw.Jobs) > maxJobIndex+1 {
648+ return callFailure{"The workflow has more jobs than the server tracks (" + strconv.Itoa(maxJobIndex+1) + ")."}
649+ }
650+ rs.jobs[idx] = legs[0]
651+ rs.touch(&rs.jobs[idx])
652+ for _, l := range legs[1:] {
653+ rs.jobs = append(rs.jobs, l)
654+ rs.dirty[l.ID] = true
655+ }
656+ next = rs.nextIndex()
657+ now := rs.c.now()
658+ for _, l := range legs {
659+ for _, cj := range cw.Jobs {
660+ needs := make([]string, len(cj.Needs))
661+ for i, n := range cj.Needs {
662+ needs[i] = l.Key + "/" + n
663+ }
664+ id := domain.NewJobID(rs.run.ID, next, 0)
665+ rs.jobs = append(rs.jobs, domain.CIJob{
666+ ID: id, RunID: rs.run.ID, Attempt: rs.attempt, RunAttempt: rs.attempt, Repo: rs.run.Repo, Key: l.Key + "/" + cj.ID,
667+ Name: cj.ID, Phase: domain.CIPhaseWaiting, Needs: needs, Index: next, CreatedAt: now, Scope: l.ID, CallID: cj.ID,
668+ })
669+ rs.dirty[id] = true
670+ next++
671+ }
672+ }
673+ rs.sortJobs()
674+ return nil
675+}
676+
677+// callPermissions is the permissions a call grants to the jobs of its scope: the
678+// "permissions:" of the calling job, else of its workflow, else those the
679+// containing call allows. A reusable workflow can only keep or lower them: a
680+// request above the allowed is the error of PermissionsHelper.cs.
681+func (rs *runState) callPermissions(wf *workflow.Workflow, wj *workflow.Job, rec domain.CIJob, parent *domain.CIJob) (map[string]string, error) {
682+ got := domain.ResolvePermissions(permissionSpec(wf.Permissions), permissionSpec(wj.Permissions))
683+ if parent == nil {
684+ return got, nil
685+ }
686+ allowed := domain.TokenPermissions(parent.Permissions, true)
687+ if got == nil {
688+ return allowed, nil
689+ }
690+ if asked, ok := domain.PermissionViolations(got, allowed); len(asked) > 0 {
691+ return nil, callFailure{fmt.Sprintf("Error calling workflow '%s'. The nested job '%s' is requesting '%s', but is only allowed '%s'.",
692+ parent.Call.Uses, wj.ID, strings.Join(asked, ", "), strings.Join(ok, ", "))}
693+ }
694+ return got, nil
695+}
696+
697+// scopedPermissions are the permissions of a job of a called workflow: its own
698+// "permissions:", else its workflow's, else what the call allows; asking for
699+// more than the call allows is an error.
700+func (rs *runState) scopedPermissions(wf *workflow.Workflow, wj *workflow.Job, call *domain.CIJob) (map[string]string, error) {
701+ allowed := domain.TokenPermissions(call.Permissions, true)
702+ got := domain.ResolvePermissions(permissionSpec(wf.Permissions), permissionSpec(wj.Permissions))
703+ if got == nil {
704+ return call.Permissions, nil
705+ }
706+ if asked, ok := domain.PermissionViolations(got, allowed); len(asked) > 0 {
707+ what := "The nested job '" + wj.ID + "'"
708+ if wj.Permissions == nil {
709+ what = "The workflow"
710+ }
711+ return nil, callFailure{fmt.Sprintf("Error calling workflow '%s'. %s is requesting '%s', but is only allowed '%s'.",
712+ call.Call.Uses, what, strings.Join(asked, ", "), strings.Join(ok, ", "))}
713+ }
714+ return got, nil
715+}
716+
717+// callSecrets reads the "secrets:" of a call: "inherit", or a mapping. Planning
718+// validates names only (the expressions were checked against the contexts the
719+// schema allows when the file was parsed) and keeps no expression and no value:
720+// the mapping is read from the pinned workflow file, as a template token, when
721+// a job is assigned (callMapping, mapSecrets), and evaluated once there. Only
722+// the plain contexts the expressions read (needs, matrix, strategy, inputs,
723+// vars: none is secret) are kept, in call.SecretCtx, as they are now.
724+func (rs *runState) callSecrets(wj *workflow.Job, env template.Env, spec calleeSpec, call *domain.CICall) error {
725+ if wj.InheritSecrets {
726+ call.Inherit = true
727+ return nil
728+ }
729+ if wj.Secrets == nil {
730+ return rs.checkRequiredSecrets(spec, map[string]bool{})
731+ }
732+ call.Secrets = map[string]string{}
733+ given := map[string]bool{}
734+ for _, p := range wj.Secrets.Pairs {
735+ name := strings.ToUpper(p.Key.String())
736+ if _, ok := spec.secrets[name]; !ok {
737+ return callFailure{"Invalid secret, " + p.Key.String() + " is not defined in the referenced workflow."}
738+ }
739+ given[name] = true
740+ call.Secrets[name] = "" // the name only
741+ }
742+ plain := map[string]json.RawMessage{}
743+ for _, k := range []string{"needs", "matrix", "strategy", "inputs", "vars"} {
744+ if v, ok := env.Contexts[k]; ok && v != nil {
745+ plain[k] = json.RawMessage(compactJSON(expr.ToJSON(v)))
746+ }
747+ }
748+ b, err := json.Marshal(plain)
749+ if err != nil {
750+ return err
751+ }
752+ call.SecretCtx = string(b)
753+ return rs.checkRequiredSecrets(spec, given)
754+}
755+
756+// callMapping returns the "secrets:" mapping of a call as the template tokens of
757+// the workflow file that holds the calling job: the run's own workflow, or the
758+// called workflow of the call that contains it. The tokens come from the file
759+// itself and are never rebuilt from text; names are upper-cased.
760+func (rs *runState) callMapping(cr *domain.CIJob) (map[string]template.Token, error) {
761+ path, yamlText, id := rs.run.WorkflowPath, rs.src.WorkflowYAML, cr.Key
762+ if p := rs.scopeOf(*cr); p != nil && p.Call != nil {
763+ path, yamlText, id = p.Call.Path, p.Call.YAML, cr.CallID
764+ }
765+ wf, err := rs.c.flows.parseCalled(path, yamlText)
766+ if err != nil {
767+ return nil, err
768+ }
769+ wj := wf.Job(id)
770+ if wj == nil {
771+ return nil, fmt.Errorf("the calling job is gone")
772+ }
773+ out := map[string]template.Token{}
774+ if wj.Secrets != nil {
775+ for _, p := range wj.Secrets.Pairs {
776+ out[strings.ToUpper(p.Key.String())] = p.Value
777+ }
778+ }
779+ return out, nil
780+}
781+
782+// secretsReadBy returns the secrets a template token refers to: the expressions
783+// of the token are parsed, not the text they produce.
784+func secretsReadBy(t template.Token) secretUse {
785+ w := &secretWalk{names: map[string]bool{}}
786+ template.Walk(t, func(x template.Token) bool {
787+ if be, ok := x.(*template.BasicExpression); ok {
788+ w.expression(be.Expression)
789+ }
790+ return true
791+ })
792+ use := secretUse{All: w.all || w.tooDeep}
793+ for n := range w.names {
794+ use.Names = append(use.Names, n)
795+ }
796+ slices.Sort(use.Names)
797+ return use
798+}
799+
800+func (rs *runState) checkRequiredSecrets(spec calleeSpec, given map[string]bool) error {
801+ for _, name := range sortedKeys(spec.secrets) {
802+ if spec.secrets[name] && !given[name] {
803+ return callFailure{"Secret " + name + " is required, but not provided while calling."}
804+ }
805+ }
806+ return nil
807+}
808+
809+// chainOf returns the call records a job runs under, innermost first.
810+func (rs *runState) chainOf(j domain.CIJob) []*domain.CIJob {
811+ var out []*domain.CIJob
812+ for sc := rs.scopeOf(j); sc != nil && len(out) <= maxReusableWorkflowDepth; sc = rs.scopeOf(*sc) {
813+ out = append(out, sc)
814+ }
815+ return out
816+}
817+
818+// scopedSecretUse is the secrets of the run's own workflow that a job of a
819+// called workflow reads, through the "secrets:" of each call that leads to it.
820+// inner is the use inside the called workflow of the job.
821+func (rs *runState) scopedSecretUse(j domain.CIJob) (top, inner secretUse) {
822+ chain := rs.chainOf(j)
823+ inner = secretsUsedBy(chain[0].Call.YAML, j.CallID)
824+ cur := inner
825+ for _, c := range chain {
826+ cur = rs.translateUse(cur, c)
827+ }
828+ return cur, inner
829+}
830+
831+// translateUse turns the secrets a called workflow reads into the secrets its
832+// caller must hold: with "inherit" the same ones, else the secrets that the
833+// expressions of the call's mapping, for the ones it reads, refer to (all of
834+// them when an expression is not a plain name). A mapping that cannot be read
835+// asks for all of them: the safe side for the job, which then gets none it
836+// cannot evaluate.
837+func (rs *runState) translateUse(use secretUse, cr *domain.CIJob) secretUse {
838+ if cr.Call.Inherit {
839+ return use
840+ }
841+ mapping, err := rs.callMapping(cr)
842+ if err != nil {
843+ return secretUse{All: true}
844+ }
845+ var out secretUse
846+ add := func(t template.Token) {
847+ u := secretsReadBy(t)
848+ out.All = out.All || u.All
849+ for _, n := range u.Names {
850+ if !slices.Contains(out.Names, n) {
851+ out.Names = append(out.Names, n)
852+ }
853+ }
854+ }
855+ for name, t := range mapping {
856+ if use.All || slices.Contains(use.Names, name) {
857+ add(t)
858+ }
859+ }
860+ slices.Sort(out.Names)
861+ return out
862+}
863+
864+// mapSecrets gives a job of a called workflow the secrets it reads, named as the
865+// called workflow names them. top holds the secrets of the run's own workflow
866+// that it may use (empty for a run that withholds them: the caller decides who
867+// gets any). From the outermost call, each mapping value is the template token
868+// of the workflow file, evaluated once with the secrets of its calling scope and
869+// the plain contexts the call kept; the result is a final string, never parsed
870+// again, and is the secrets of the scope below. A secret not passed (not in
871+// "secrets:" and no "inherit") is not there. A value that does not evaluate is
872+// empty, and nothing of the error is logged: it may hold the secret.
873+func (rs *runState) mapSecrets(ctx context.Context, j domain.CIJob, top map[string]string, inner secretUse) map[string]string {
874+ chain := rs.chainOf(j)
875+ vals := top
876+ gh, ghErr := rs.githubValue("", rs.run.Attempt)
877+ for i := len(chain) - 1; i >= 0; i-- {
878+ cr := chain[i]
879+ if cr.Call.Inherit {
880+ continue
881+ }
882+ mapping, merr := rs.callMapping(cr)
883+ sd := expr.NewDictionary()
884+ for _, k := range sortedKeys(vals) {
885+ sd.Add(k, expr.String(vals[k]))
886+ }
887+ env := template.Env{Contexts: map[string]expr.Value{"secrets": sd}}
888+ if ghErr == nil {
889+ env.Contexts["github"] = gh
890+ }
891+ var plain map[string]json.RawMessage
892+ _ = json.Unmarshal([]byte(cr.Call.SecretCtx), &plain)
893+ for k, raw := range plain {
894+ if v, err := expr.FromJSON(raw); err == nil {
895+ env.Contexts[k] = v
896+ }
897+ }
898+ next := map[string]string{}
899+ for name := range cr.Call.Secrets {
900+ t, ok := mapping[name]
901+ if merr != nil || !ok {
902+ next[name] = ""
903+ continue
904+ }
905+ v, err := template.EvaluateString(ctx, t, env)
906+ if err != nil {
907+ // Never the error: a format() error holds its arguments.
908+ ctxLog(ctx).Warn().Str("secret", name).Msg("ci: a secret passed to a called workflow could not evaluate: it is empty")
909+ v = ""
910+ }
911+ next[name] = v
912+ }
913+ vals = next
914+ }
915+ out := map[string]string{}
916+ for n, v := range vals {
917+ if inner.All || slices.Contains(inner.Names, n) {
918+ out[n] = v
919+ }
920+ }
921+ return out
922+}
923+
924+// callHeld reports whether a call holds the concurrency groups it asked for: the
925+// group of the calling job and the workflow-level group of the called workflow.
926+// The jobs of a call are not planned before.
927+func callHeld(call *domain.CIJob) bool {
928+ if call.Call == nil || !call.Call.Admitted {
929+ return false
930+ }
931+ if call.Concurrency.Group != "" && !call.Concurrency.FromRun && call.Concurrency.State != domain.CIConcurrencyHolding {
932+ return false
933+ }
934+ return call.Call == nil || call.Call.Concurrency.Group == "" || call.Call.Concurrency.State == domain.CIConcurrencyHolding
935+}
936+
937+// admitCalls lets the calls of a matrix start while the max-parallel of the
938+// calling job's strategy allows it, in the order of the legs, as promote does
939+// for the jobs that run. A call that is admitted asks for its groups and has its
940+// jobs planned; one that is not waits, and starts nothing.
941+func (rs *runState) admitCalls() bool {
942+ running := map[string]int{}
943+ for _, j := range rs.jobs {
944+ if j.Call != nil && j.Call.Admitted && !j.Ended() {
945+ running[j.Scope+"\x00"+j.Key]++
946+ }
947+ }
948+ changed := false
949+ for i := range rs.jobs {
950+ j := &rs.jobs[i]
951+ if j.Call == nil || j.Call.Admitted || j.Ended() || j.Phase != domain.CIPhaseWaiting {
952+ continue
953+ }
954+ k := j.Scope + "\x00" + j.Key
955+ if j.MaxParallel > 0 && running[k] >= j.MaxParallel {
956+ continue
957+ }
958+ call := *j.Call
959+ call.Admitted = true
960+ j.Call, changed = &call, true
961+ running[k]++
962+ rs.touch(j)
963+ }
964+ return changed
965+}
966+
967+// checkJobBudget refuses an expansion that would give the run more job records
968+// than a commit may have checks (MaxCIChecks: each record has one check, and the
969+// merge rule cannot read more). extra is the number of records the expansion
970+// adds (the record of the job being expanded becomes the first of its legs).
971+// GitHub's text is the runner's (JobCountValidator.cs); its limit applies to the
972+// jobs across all referenced files, and the matrix of one job is capped at 256
973+// by ermite-actions (ErrTooManyConfigurations).
974+func (rs *runState) checkJobBudget(extra int) error {
975+ if len(rs.jobs)+extra > MaxCIChecks {
976+ return callFailure{fmt.Sprintf("Workflows may not contain more than %d jobs across all referenced files", MaxCIChecks)}
977+ }
978+ return nil
979+}
980+
981+// holdCalls asks for the turn of the calls that wait for a group: the group of
982+// the calling job first, then the one of the called workflow, so that a call
983+// waits in one group at a time. The call record is the entry, and holds both
984+// until it ends (releaseEnded).
985+func (rs *runState) holdCalls(ctx context.Context) (bool, error) {
986+ changed := false
987+ for i := range rs.jobs {
988+ j := &rs.jobs[i]
989+ if j.Call == nil || !j.Call.Expanded || !j.Call.Admitted || j.Ended() || j.Phase != domain.CIPhaseWaiting {
990+ continue
991+ }
992+ entry := domain.CIConcurrencyEntry{RunID: j.RunID, Attempt: j.RunAttempt, JobID: j.ID}
993+ if g := j.Concurrency; g.Group != "" && !g.FromRun && g.State != domain.CIConcurrencyHolding {
994+ e := entry
995+ e.Cancel = g.Cancel
996+ st, err := rs.acquire(ctx, g.Group, e)
997+ if err != nil {
998+ return changed, err
999+ }
1000+ if st != g.State {
1001+ j.Concurrency.State, changed = st, true
1002+ rs.touch(j)
1003+ }
1004+ if st != domain.CIConcurrencyHolding {
1005+ continue
1006+ }
1007+ }
1008+ if g := j.Call.Concurrency; g.Group != "" && g.State != domain.CIConcurrencyHolding {
1009+ e := entry
1010+ e.Cancel = g.Cancel
1011+ st, err := rs.acquire(ctx, g.Group, e)
1012+ if err != nil {
1013+ return changed, err
1014+ }
1015+ if st != g.State {
1016+ call := *j.Call
1017+ call.Concurrency.State = st
1018+ j.Call, changed = &call, true
1019+ rs.touch(j)
1020+ }
1021+ }
1022+ }
1023+ return changed, nil
1024+}
1025+
1026+// advanceCalls plans the jobs of called workflows whose needs ended and ends
1027+// the call records whose jobs have. It returns at the first change, since a
1028+// change reorders the records.
1029+func (rs *runState) advanceCalls(ctx context.Context) (bool, error) {
1030+ if rs.admitCalls() {
1031+ return true, nil
1032+ }
1033+ if changed, err := rs.holdCalls(ctx); err != nil || changed {
1034+ return changed, err
1035+ }
1036+ for i := range rs.jobs {
1037+ rec := rs.jobs[i]
1038+ if rec.Scope == "" || rec.Planned || rec.Ended() || rec.Phase != domain.CIPhaseWaiting {
1039+ continue
1040+ }
1041+ call := rs.scopeOf(rec)
1042+ if call == nil {
1043+ rs.end(&rs.jobs[i], domain.CIFailure, "the call of this job is gone")
1044+ return true, nil
1045+ }
1046+ if call.Ended() {
1047+ rs.end(&rs.jobs[i], domain.CICancelled, "")
1048+ return true, nil
1049+ }
1050+ if call.Call == nil || !call.Call.Admitted {
1051+ continue // the max-parallel of the calling job holds the call back
1052+ }
1053+ if !callHeld(call) {
1054+ // Waits for the group of its call: shown as pending, as the jobs
1055+ // of a run behind a workflow-level group.
1056+ if rs.run.Concurrency.Group == "" && rec.Concurrency.State != domain.CIConcurrencyPending {
1057+ g := call.Concurrency
1058+ if g.Group == "" || g.State == domain.CIConcurrencyHolding {
1059+ g = call.Call.Concurrency
1060+ }
1061+ rs.jobs[i].Concurrency = domain.CIConcurrency{Group: g.Group, State: domain.CIConcurrencyPending, FromRun: true, Evaluated: true}
1062+ rs.touch(&rs.jobs[i])
1063+ return true, nil
1064+ }
1065+ continue
1066+ }
1067+ if rec.Concurrency.FromRun && rs.run.Concurrency.Group == "" {
1068+ rs.jobs[i].Concurrency = domain.CIConcurrency{}
1069+ rs.touch(&rs.jobs[i])
1070+ }
1071+ cw, err := rs.calleeOf(call)
1072+ if err != nil {
1073+ rs.end(&rs.jobs[i], domain.CIFailure, err.Error())
1074+ return true, nil
1075+ }
1076+ cj := cw.Job(rec.CallID)
1077+ if cj == nil {
1078+ rs.end(&rs.jobs[i], domain.CIFailure, "the called workflow has no job '"+rec.CallID+"'")
1079+ return true, nil
1080+ }
1081+ if len(rs.instancesIn(rec.Scope, rec.CallID)) != 1 {
1082+ continue
1083+ }
1084+ needs, st, ready := rs.needsOf(cw, cj, rec.Scope)
1085+ if !ready {
1086+ continue
1087+ }
1088+ rs.planJob(ctx, cw, cj, i, needs, st)
1089+ return true, nil
1090+ }
1091+ for i := range rs.jobs {
1092+ call := &rs.jobs[i]
1093+ if call.Call == nil || !call.Call.Expanded {
1094+ continue
1095+ }
1096+ var scoped []int
1097+ allEnded := true
1098+ for k, j := range rs.jobs {
1099+ if j.Scope == call.ID {
1100+ scoped = append(scoped, k)
1101+ allEnded = allEnded && j.Ended()
1102+ }
1103+ }
1104+ if call.Ended() {
1105+ changed := false
1106+ for _, k := range scoped {
1107+ s := &rs.jobs[k]
1108+ switch {
1109+ case s.Ended():
1110+ case s.Active():
1111+ rs.requestCancel(s, domain.CICancelUser, endedCallMessage(*call))
1112+ default:
1113+ rs.end(s, domain.CICancelled, "")
1114+ changed = true
1115+ }
1116+ }
1117+ if changed {
1118+ return true, nil
1119+ }
1120+ continue
1121+ }
1122+ if allEnded {
1123+ rs.finishCall(ctx, i, scoped)
1124+ return true, nil
1125+ }
1126+ }
1127+ return false, nil
1128+}
1129+
1130+// finishCall ends a call record with the conclusion of its jobs and the outputs
1131+// of the called workflow.
1132+func (rs *runState) finishCall(ctx context.Context, idx int, scoped []int) {
1133+ call := &rs.jobs[idx]
1134+ var top []domain.CIJob
1135+ for _, k := range scoped {
1136+ top = append(top, rs.jobs[k])
1137+ }
1138+ conc, msg := domain.CISuccess, ""
1139+ switch result(top) {
1140+ case "failure":
1141+ conc = domain.CIFailure
1142+ msg = "A job of the called workflow failed"
1143+ for _, j := range top {
1144+ if j.Conclusion.IsFailure() && !j.ContinueOnError {
1145+ msg = "The job '" + j.Name + "' of the called workflow failed"
1146+ break
1147+ }
1148+ }
1149+ case "cancelled":
1150+ conc = domain.CICancelled
1151+ case "skipped":
1152+ conc = domain.CISkipped
1153+ }
1154+ jobs := expr.NewDictionary()
1155+ for _, id := range distinctCallIDs(top) {
1156+ var insts []domain.CIJob
1157+ for _, j := range top {
1158+ if j.CallID == id {
1159+ insts = append(insts, j)
1160+ }
1161+ }
1162+ outs := expr.NewDictionary()
1163+ for _, in := range insts {
1164+ for _, k := range sortedKeys(in.Outputs) {
1165+ if v := in.Outputs[k]; v != "" || !hasKey(outs, k) {
1166+ setKey(outs, k, expr.String(v))
1167+ }
1168+ }
1169+ }
1170+ d := expr.NewDictionary()
1171+ d.Add("result", expr.String(result(insts)))
1172+ d.Add("outputs", outs)
1173+ jobs.Add(id, d)
1174+ }
1175+ outputs := map[string]string{}
1176+ spec := readCalleeSpec([]byte(call.Call.YAML))
1177+ if len(spec.outputs) > 0 {
1178+ gh, err := rs.githubValue(call.Key, rs.run.Attempt)
1179+ vars, verr := rs.c.varsContext(ctx, rs.run.Repo)
1180+ if err == nil && verr == nil {
1181+ env := template.Env{Contexts: map[string]expr.Value{"github": gh, "inputs": expr.NewDictionary(), "vars": vars, "jobs": jobs}}
1182+ // The inputs of the call itself.
1183+ if v, err := expr.FromJSON([]byte(call.Call.InputsJSON)); err == nil {
1184+ env.Contexts["inputs"] = v
1185+ }
1186+ for _, name := range sortedKeys(spec.outputs) {
1187+ v, err := evalInterpolated(ctx, spec.outputs[name], env)
1188+ if err != nil {
1189+ ctxLog(ctx).Warn().Str("output", name).Msg("ci: an output of a called workflow could not evaluate")
1190+ continue
1191+ }
1192+ outputs[name] = expr.CoerceString(v)
1193+ }
1194+ }
1195+ }
1196+ call.Outputs = outputs
1197+ rs.end(call, conc, msg)
1198+}
1199+
1200+func distinctCallIDs(js []domain.CIJob) []string {
1201+ var out []string
1202+ for _, j := range js {
1203+ if !slices.Contains(out, j.CallID) {
1204+ out = append(out, j.CallID)
1205+ }
1206+ }
1207+ return out
1208+}
1209+
1210+// parseCalled parses a called workflow, once per distinct file.
1211+func (w *wfCache) parseCalled(path, yamlText string) (*workflow.Workflow, error) {
1212+ sum := sha256.Sum256([]byte(path + "\x00" + yamlText))
1213+ key := hex.EncodeToString(sum[:])
1214+ w.mu.Lock()
1215+ if wf, ok := w.called[key]; ok {
1216+ w.mu.Unlock()
1217+ return wf, nil
1218+ }
1219+ w.mu.Unlock()
1220+ wf, errs := workflow.Parse(path, []byte(yamlText))
1221+ if len(errs) > 0 {
1222+ return nil, fmt.Errorf("the called workflow no longer parses: %s", errs[0].Error())
1223+ }
1224+ w.mu.Lock()
1225+ defer w.mu.Unlock()
1226+ if len(w.called) >= w.max {
1227+ clear(w.called)
1228+ }
1229+ w.called[key] = wf
1230+ return wf, nil
1231+}
1232+
1233+// jobKeysOf are the keys of the "job" context the server knows. A job of a
1234+// called workflow names the called workflow in them (workflow_ref,
1235+// workflow_sha, workflow_repository, workflow_file_path); github.workflow_ref
1236+// and github.workflow_sha stay the caller's (the github context is the
1237+// caller's).
1238+func (rs *runState) jobKeysOf(ctx context.Context, base domain.CIJob, combo int) ordered {
1239+ keys := rs.c.jobKeys(ctx, rs.run, base.Index, combo)
1240+ call := rs.scopeOf(base)
1241+ if call == nil || call.Call == nil {
1242+ return keys
1243+ }
1244+ c := call.Call
1245+ full := c.Repository
1246+ if !strings.Contains(full, "/") {
1247+ full = rs.c.cfg.Owner + "/" + full
1248+ }
1249+ ref := c.Ref
1250+ if ref == "" {
1251+ ref = rs.run.Ref
1252+ }
1253+ return ordered{
1254+ keys[0], {"workflow_ref", full + "/" + c.Path + "@" + ref}, {"workflow_sha", c.Sha},
1255+ {"workflow_repository", full}, {"workflow_file_path", c.Path},
1256+ }
1257+}
1258+
1259+// endedCallMessage is what the jobs of a call that ended are told: the reason
1260+// the call was cancelled for (a newer call of its concurrency group), else that
1261+// the calling job ended.
1262+func endedCallMessage(call domain.CIJob) string {
1263+ if call.Concurrency.Preempted || call.Error != "" {
1264+ return call.Error
1265+ }
1266+ return "Canceled because the calling job ended"
1267+}
1268+
1269+// callResolutions copies the resolutions of the reusable workflows that the
1270+// attempt of rs called to the attempt next.
1271+func (c *CI) callResolutions(ctx context.Context, rs *runState, next int32) ([]domain.CIActionResolution, error) {
1272+ var out []domain.CIActionResolution
1273+ seen := map[string]bool{}
1274+ for _, j := range rs.jobs {
1275+ if j.Call == nil {
1276+ continue
1277+ }
1278+ ref, text := parseCallRef(j.Call.Uses)
1279+ if text != "" || ref.local {
1280+ continue
1281+ }
1282+ key := actionRef{owner: ref.owner, repo: ref.repo, ref: ref.ref}.key()
1283+ if seen[key] {
1284+ continue
1285+ }
1286+ seen[key] = true
1287+ res, err := c.cfg.Store.GetActionResolution(ctx, rs.run.ID, rs.attempt, key)
1288+ if errors.Is(err, domain.ErrNotFound) {
1289+ continue
1290+ }
1291+ if err != nil {
1292+ return nil, err
1293+ }
1294+ res.RunAttempt = next
1295+ out = append(out, res)
1296+ }
1297+ return out, nil
1298+}
M internal/usecase/ci_runner.go +7 -1
@@ -242,9 +242,15 @@ func (c *CI) runnerJob(rs *runState, j domain.CIJob, token, ghToken string, sett
242242 if settings.Vars != nil {
243243 j.ContextsJSON = withVars(j.ContextsJSON, settings.Vars)
244244 }
245+ path, yaml := rs.run.WorkflowPath, rs.src.WorkflowYAML
246+ if call := rs.scopeOf(j); call != nil && call.Call != nil {
247+ // A job of a called workflow runs from the file of the call, under its
248+ // id in that file.
249+ path, yaml, j.Key = call.Call.Path, call.Call.YAML, j.CallID
250+ }
245251 return &ports.RunnerJob{
246252 Job: j, Token: token, GithubToken: ghToken, Repo: rs.run.Repo, Sha: rs.run.Sha, Ref: rs.run.Ref,
247- WorkflowPath: rs.run.WorkflowPath, WorkflowYAML: rs.src.WorkflowYAML,
253+ WorkflowPath: path, WorkflowYAML: yaml,
248254 Environment: j.Environment, Vars: settings.Vars, Secrets: settings.Secrets, Trusted: rs.run.Trusted,
249255 Permissions: domain.TokenPermissions(j.Permissions, rs.run.Trusted),
250256 }
M internal/usecase/ci_runs.go +17 -6
@@ -378,15 +378,15 @@ func (c *CI) RerunRun(ctx context.Context, p domain.Principal, name domain.RepoN
378378 w := ports.CIWrite{}
379379 var fresh []domain.CIJob
380380 for _, j := range rs.jobs {
381- if !redo[j.Key] {
381+ if !redo[rootKey(j.Key)] {
382382 kept := j // kept with its own attempt, steps and log
383383 kept.RunAttempt, kept.Version = next, 0
384384 w.Jobs = append(w.Jobs, kept)
385385 fresh = append(fresh, kept)
386386 continue
387387 }
388- if j.Combo > 0 {
389- continue // the matrix is expanded again when the job is planned
388+ if j.Combo > 0 || j.Scope != "" {
389+ continue // the matrix, and the jobs of a call, are expanded again when the job is planned
390390 }
391391 n := domain.CIJob{
392392 ID: domain.NewJobID(runID, j.Index, 0), RunID: runID, Attempt: next, RunAttempt: next, Repo: rs.run.Repo,
@@ -395,6 +395,17 @@ func (c *CI) RerunRun(ctx context.Context, p domain.Principal, name domain.RepoN
395395 w.Jobs = append(w.Jobs, n)
396396 fresh = append(fresh, n)
397397 }
398+ if failedOnly {
399+ // The callee of a reusable workflow stays at the commit the run first
400+ // resolved, as the jobs that are kept ran from it; a re-run of
401+ // everything resolves afresh (as for the actions of CI-2, which are
402+ // resolved again in every attempt).
403+ res, err := c.callResolutions(ctx, rs, next)
404+ if err != nil {
405+ return err
406+ }
407+ w.Resolutions = append(w.Resolutions, res...)
408+ }
398409 run := rs.run
399410 run.Attempt, run.Status, run.Conclusion = next, domain.CIQueued, domain.CINoConclusion
400411 run.StartedAt, run.CompletedAt, run.CreatedAt = time.Time{}, time.Time{}, now
@@ -443,14 +454,14 @@ func rerunSet(rs *runState, failedOnly bool) (map[string]bool, error) {
443454 redo := map[string]bool{}
444455 if !failedOnly {
445456 for _, j := range rs.jobs {
446- redo[j.Key] = true
457+ redo[rootKey(j.Key)] = true
447458 }
448459 return redo, nil
449460 }
450461 for _, j := range rs.jobs {
451462 switch j.Conclusion {
452463 case domain.CIFailure, domain.CICancelled, domain.CITimedOut, domain.CIActionRequired:
453- redo[j.Key] = true
464+ redo[rootKey(j.Key)] = true
454465 }
455466 }
456467 if len(redo) == 0 {
@@ -459,7 +470,7 @@ func rerunSet(rs *runState, failedOnly bool) (map[string]bool, error) {
459470 for changed := true; changed; {
460471 changed = false
461472 for _, j := range rs.jobs {
462- if redo[j.Key] {
473+ if redo[j.Key] || j.Scope != "" {
463474 continue
464475 }
465476 if slices.ContainsFunc(j.Needs, func(n string) bool { return redo[n] }) {
M internal/usecase/ci_secrets.go +15 -1
@@ -294,14 +294,28 @@ func (c *CI) settingsOf(ctx context.Context, rs *runState, j domain.CIJob) (port
294294 // that is not trusted does not add environments to the repository.
295295 CreateEnvironment: j.Environment != "" && rs.run.Trusted,
296296 }
297+ var inner secretUse
298+ scoped := rs.scopeOf(j) != nil
297299 if rs.run.Trusted {
298- use := c.flows.secretUse(rs.run.ID, rs.src.WorkflowYAML, j.Key)
300+ var use secretUse
301+ if scoped {
302+ // A job of a called workflow gets the secrets its caller passes
303+ // it, under the names the called workflow gives them.
304+ use, inner = rs.scopedSecretUse(j)
305+ } else {
306+ use = c.flows.secretUse(rs.run.ID, rs.src.WorkflowYAML, j.Key)
307+ }
299308 req.WantSecrets, req.AllSecrets, req.SecretNames = !use.none(), use.All, use.Names
300309 }
301310 got, err := c.cfg.Settings.Resolve(ctx, req)
302311 if err != nil {
303312 return ports.JobSettings{}, &settingsError{err}
304313 }
314+ if scoped && rs.run.Trusted {
315+ // Also when no stored secret resolved: a mapping can give a literal
316+ // value. A run that is not trusted gets nothing.
317+ got.Secrets = rs.mapSecrets(ctx, j, got.Secrets, inner)
318+ }
305319 return got, nil
306320 }
307321
M proto/ermite/storage/ci.proto +50 -0
@@ -213,6 +213,56 @@ message CIJob {
213213 // The job's concurrency group (its own, else the one of its run) and where
214214 // it is in it.
215215 CIConcurrency concurrency = 43;
216+ // Reusable workflows (D7). scope is the id of the record of the job that
217+ // called the workflow this job belongs to, empty for a job of the run's own
218+ // workflow; call_id is the id of the job in that called workflow (key is
219+ // "<caller key>/<call id>"). call is set on the record that stands for a job
220+ // calling a reusable workflow: it never runs, it ends when the jobs of its
221+ // scope have.
222+ string scope = 44;
223+ string call_id = 45;
224+ CICall call = 46;
225+}
226+
227+// CICall is the call a reusable-workflow job makes, set when the calling job is
228+// planned. The called workflow's file is kept here, so that the jobs of the call
229+// are planned and run from the file of the commit that was resolved.
230+message CICall {
231+ string uses = 1;
232+ // path of the file in its repository, repository as "owner/repo" (github.com)
233+ // or the name of the repository on this server, ref as the calling workflow
234+ // wrote it ("" for a local call), sha the commit it resolved to.
235+ string path = 2;
236+ string repository = 3;
237+ string ref = 4;
238+ string sha = 5;
239+ string yaml = 6;
240+ // inputs_json is the inputs context of the called workflow: the values given
241+ // and the defaults, typed.
242+ string inputs_json = 7;
243+ // secrets holds the names of the secrets of the called workflow that the
244+ // calling workflow's "secrets:" gives (upper case), with an empty value: the
245+ // expressions are not stored. They are read as template tokens from the pinned
246+ // workflow file of the caller when a job is assigned, and evaluated once there.
247+ map<string, string> secrets = 8;
248+ bool inherit = 9;
249+ int32 depth = 10;
250+ // expanded: the jobs of the called workflow exist.
251+ bool expanded = 11;
252+ // concurrency is the workflow-level group of the called workflow, evaluated
253+ // with the caller's github context and the call's inputs when the call is
254+ // planned. The call record is its entry, like the group of the calling job
255+ // (CIJob.concurrency): the jobs of the call run once the call holds both.
256+ CIConcurrency concurrency = 12;
257+ // admitted: the call may start (the max-parallel of the calling job's
258+ // strategy allows it); it asks for its groups and plans its jobs only then.
259+ bool admitted = 13;
260+ // secret_ctx is the JSON of the contexts the "secrets:" expressions read apart
261+ // from the secrets, as they were when the call was planned: needs, matrix,
262+ // strategy, inputs and vars of the calling scope (none is secret). Plain data,
263+ // never interpreted as a reference; the vars are the caller's, not those of
264+ // the environment of the job that is assigned.
265+ string secret_ctx = 14;
216266 }
217267
218268 // CICheck is stored under "check/<repo>/<sha>/<id>": the id of its job, or

Keyboard shortcuts

Single-key shortcuts are on. ctrl k and esc work whatever this says.

ctrl k :
go to a section, branch, tag, file or repository
?
this list
j k
move the row cursor down / up
enter
open the row under the cursor; on Files, a directory opens or closes in place
↑ ↓ ← → home end
in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
l h
on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. Elsewhere: into the side panel (on a merge request, its files) / back to the content; there j k move between its controls
g g G
first / last row
/
filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
b
switch branch or tag
g s g f g c g b g m g i g r g a
tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
g h
home
w
wide mode: use the whole width of the screen, or not (remembered)
y
copy a permalink pinned to the commit
d u x
diff: split / unified / collapse or expand all
e E
diff: the whole file in view / every file, with the lines the hunks hide
n
merge request: next note in the diff
esc
close a dialog, leave a field

Branches 2

Tags 44

↑↓ moveenter openesc close