1# Porting notes for package workflow
2
3`Parse` is a port of `model/convert` and `model/converter/*` of
4`@actions/workflow-parser` 0.3.61 on top of package `template`. The reference's
5errors are replayed in `TestConvertAgainstReference` (153 files in
6`testdata/convert`, answers in `testdata/convert.json`, generated by
7`gen/convert.mjs` from the unmodified npm package).
8
9The parts below are **not** in that package. They are written from GitHub's
10documentation and from what is known of the official runner, and each needs a
11recording of github.com (`ermite-conformance`) or a look at the C# runner
12before being trusted. Every one carries a `TODO(verify with recording)` in the
13code.
14
15## Differences from the reference
16
17- **`branches` with `branches-ignore`** (and `tags`, `paths`): the reference
18  accepts them; GitHub refuses the file. `Parse` reports
19  `Event 'push' cannot use both 'branches' and 'branches-ignore'. Use only one
20  of them.` (our wording, to replace with GitHub's). Applied to every event.
21- **A filter list made only of `!` patterns** (`branches: ['!main']`) is an
22  error (`Event 'push' filter 'branches' has only patterns that start with
23  '!'...`), and so is a pattern that does not compile. Both are ours; GitHub's
24  texts are unknown. Applied to every event and every filter.
25- **Matrix exclude** with a key that is not a vector of the matrix is
26  `Matrix exclude key '<k>' does not match any key within the matrix`
27  (GitHub's text as remembered, not recorded).
28- **Limits that GitHub does not state**: the cross product of the vectors may
29  not exceed `MaxMatrixProduct` (65 536) before exclusions, include and exclude
30  have at most `MaxMatrixEntries` (256) entries each, and the context is
31  checked every 1 024 combinations. They keep a hostile matrix from running
32  for long; a legitimate one (at most 256 jobs) is far below them. The
33  product limit also refuses a matrix GitHub would accept by excluding most
34  of a huge product.
35- **Reusable workflows** are parsed (`uses`, `with`, `secrets`, `inherit`) but
36  not followed, and the called workflow's inputs and secrets are not checked.
37- Unknown event types are reported by the schema (`Unexpected value 'x'`).
38- `permissions` are modelled but not validated beyond the schema.
39- `Job.If`, `Step.If`: the status-function wrapping is the reference's
40  (`success() && (cond)` when the condition calls none of `success`,
41  `failure`, `cancelled`, `always`, looking through nested calls).
42
43## Recorded (github.com, ermite-conformance corpus)
44
45Opt-in check against the corpus itself: `ERMITE_CORPUS=<ermite-conformance
46checkout> go test ./workflow -run Corpus` (`corpus_test.go`) replays the
47scenario files and compares the runs SkipCI, MatchPush and Parse predict with
48the recordings. The cases copied into the unit tests stay.
49
50- Status functions (`success()`, `failure()`, `always()`, `cancelled()`) in
51  `env` or `run` expressions make the workflow invalid (recorded:
52  expr-status-in-env, expr-status-in-run, each a failed run with no job). The
53  text `Unrecognized function: 'success'. Located at position 1 within
54  expression: success()` is the one observed on the github.com run page, not in
55  the recordings; `TestRecordedInvalidWorkflows`.
56- Invalid workflow messages (the recordings hold the failed run only, not the
57  text; the messages are those of the reference parser, which refuses the same
58  files): `The workflow must contain at least one job with
59  no dependencies.` (no root job: needs on an unknown job or a cycle with no
60  other root), `Job 'x' depends on unknown job 'y'.`, `Job 'x' depends on job
61  'y' which creates a cycle in the dependency graph.`, `Required property is
62  missing: runs-on`, `Unexpected value 'unknown-key'` (invalid-needs,
63  invalid-schema).
64
65## To verify (recordings)
66
67- [ ] **Glob syntax** (`glob.go`): `*`, `**`, `?`, `+`, `[]`, `\`, `!`. The
68      cheat sheet's examples pass; `**/` matching nothing, `?` and `+` after a
69      `]`, `*` or an escaped character, `[!x]`, an unclosed `[` (literal here)
70      and `/`-prefixed patterns are assumed.
71- [x] **`MatchPush`** (recorded: push-tags-limit, filter-tags,
72      filter-branches-ignore, filter-tag-branches-only, filter-paths-tag,
73      filter-paths*, push-delete-*): deletes never run; more than 3 tags run
74      nothing for tags (not for the branch); branches-only filter and a tag
75      push (no run) and the reverse; with both filters each ref uses its own;
76      `paths` only for branches; the first **3000** files, also for
77      `paths-ignore` (not 300). Still assumed: a push whose diff is empty
78      never matches a `paths` filter and always matches a `paths-ignore` one;
79      unknown ref kinds never match; truncated list passes.
80- [ ] **`PushInfo.Created`** is not used: a new branch's file list is the
81      caller's job (diff against the default branch merge-base).
82- [x] **`SkipCI`** (recorded: skip-directives, skip-directives-placement,
83      skip-checks-trailer, skip-directive-tag): markers are case-sensitive,
84      anywhere; `skip-checks: true` needs two blank lines before it and
85      nothing after it, in exact case. The tag case is the caller's: the
86      message of the commit the tag points at. Assumed, not recorded: only
87      the head commit counts (the scenarios push one commit at a time); a
88      line of spaces or tabs is not a blank line; more than two blank lines
89      count like two.
90- [ ] **`ExpandMatrix`**: the unknown exclude key error (once per offending
91      entry) and the over-256 message (`configurationLimitError`, which matches
92      `ErrTooManyConfigurations` with `errors.Is`) are the
93      texts observed on the github.com run page for matrix-edge-values and
94      matrix-over-256, not in the recordings (a failed run with no job). To
95      verify: order of the product (first key slowest), include
96      semantics (added to every matching combination, never overwriting
97      original values, later includes overwrite earlier added values, a
98      non-matching include is a new combination, match by vector keys only),
99      exclude (partial match, applied before include), the 256 limit counting
100      combinations after include, key case-insensitivity, value comparison
101      (same kind; objects by containment), the error when all combinations are
102      excluded (GitHub may fail the run differently), empty vector error.
103- [x] A **null or empty (`''`) matrix value** is not shown in the job name
104      (recorded: matrix-edge-values).
105- [ ] **Job display names** (`MatrixCombo.DisplayName`, `Job.DisplayName`):
106      recorded in matrix-job-name and matrix-edge-values: `name (v1, v2)`
107      listing the values in key order, numbers and booleans as
108      `CoerceString` shows them, a name that does not read `matrix` followed
109      by the values. Assumed: the keys added by include come after the vector
110      keys; arrays and objects are left out; a name that reads the `matrix`
111      context (found in the expression's syntax tree, not by searching the
112      text) is used as is.
113- [ ] **`strategy` context** (`Strategy.Context`): `fail-fast`, `job-index`,
114      `job-total`, `max-parallel` (the job count when unlimited).
115- [ ] `fail-fast` and `max-parallel` accept strings (`"false"`, `"2"`) after
116      evaluation; `max-parallel` must be a whole number ≥ 1.
117- [ ] The 1 000-commit and 3 000-file limits themselves are the caller's.
118
119## New in the second pass
120
121- `Job.EvaluateRunsOn`, `Job.EvaluateIf`, `Step.EvaluateIf` and
122  `Workflow.JobLevels` are conveniences for the planner and are not in the
123  reference. `EvaluateRunsOn` applies the reference's runs-on checks to the
124  evaluated value (labels must be strings; group names as in the converter).
125  `JobLevels` is a layering of `needs` by depth, file order inside a level.
126- The reference versions and the regeneration of its answers are in
127  `../template/PORTING.md`.