Merge pull request 'feat(expr): GitHub Actions expressions' (#1) from feat/expr into main
101 files · +6557 -0
this diff holds invisible characters (direction controls, zero-width characters). They are shown as <U+XXXX> markers, never applied.
A expr/PORTING.md +206 -0
| @@ -0,0 +1,206 @@ | |||
| 1 | + | # Porting notes for package expr | |
| 2 | + | ||
| 3 | + | The package is a port of `@actions/expressions` 0.3.61 (MIT, GitHub, | |
| 4 | + | actions/languageservices). The JavaScript is the specification wherever this | |
| 5 | + | package could be checked against it: `testdata/oracle.json.gz` holds what the | |
| 6 | + | reference answers for 33 575 expressions (tokens with positions, error | |
| 7 | + | messages, result kinds, string coercions, `toJSON` output), and | |
| 8 | + | `TestOracle` replays it. `gen/oracle.mjs` regenerates it, `gen/golden.mjs` the | |
| 9 | + | `testdata/tojson` goldens, and `gen/casing_dump.mjs` + `gen/casing.go` the | |
| 10 | + | casing tables (`go generate ./expr`; Node 25 / V8 14.1, Unicode 16). The | |
| 11 | + | tables are relative to the Go toolchain's own Unicode tables | |
| 12 | + | (`unicode.Version`); `TestCasingTablesMatchGoUnicodeVersion` fails with the | |
| 13 | + | regeneration command when the toolchain changes. The oracle records the | |
| 14 | + | npm integrity (sha512) of the reference tarball and the Node/V8/Unicode | |
| 15 | + | versions in its `reference` field, and `TestOracle` logs them. | |
| 16 | + | ||
| 17 | + | The C# runner (actions/runner, `DistributedTask/Expressions2`) is the | |
| 18 | + | implementation that runs on github.com. It was not available while this was | |
| 19 | + | written. Nothing below has been checked against it. | |
| 20 | + | ||
| 21 | + | ## Limits added by this port (not in the JavaScript) | |
| 22 | + | ||
| 23 | + | The reference has no bound on what an evaluation may build. These are | |
| 24 | + | additions, each reported as an `*EvalError` (or `*Error` for the parser) and | |
| 25 | + | none changes the result of an evaluation that stays within them. | |
| 26 | + | ||
| 27 | + | - **Output budget** (`MaxResultBytes`, 1 MiB by default, `EvalContext. | |
| 28 | + | MaxResultBytes` to override, negative to disable). One budget per | |
| 29 | + | `Evaluate`, charged by `format` (every piece appended), `join` (items and | |
| 30 | + | separators), `toJSON` (the encoder stops as soon as the output would exceed | |
| 31 | + | what is left) and by every element of the arrays built by `.*` / `[*]` | |
| 32 | + | (16 bytes each). `format('{0}{0}', format('{0}{0}', ...))` chains and | |
| 33 | + | `a.*.*.*` fan-outs over shared arrays are cut off instead of exhausting | |
| 34 | + | memory. Strings that come out of `fromJSON` are not charged: their size is | |
| 35 | + | bounded by the input the caller provided. The cost per filtered element is a | |
| 36 | + | guess; revisit it if real workflows hit the limit. | |
| 37 | + | - **Cancellation**: `Evaluate(ctx, ...)` checks `ctx` every 64 nodes and in the | |
| 38 | + | loops of filters, `join` and `contains`, and passes `ctx` to extension | |
| 39 | + | functions. | |
| 40 | + | - **Evaluation depth** of 1000 for hand-built trees (parsed trees stay far | |
| 41 | + | below it), **JSON depth** `MaxJSONDepth` = 1000 for `FromJSON` and for the | |
| 42 | + | encoder (cyclic values built with `Array.Append` yield an error from | |
| 43 | + | `EncodeJSON`, `"null"` from `ToJSON`, an `*EvalError` from `toJSON()`). | |
| 44 | + | The JS recursion limit is engine-dependent, around 5 000 to 10 000. | |
| 45 | + | - **Built-in arity is re-checked at evaluation**, so a hand-built AST gets an | |
| 46 | + | `*EvalError`, not a panic. | |
| 47 | + | - **Input length** is checked before the input is converted, without | |
| 48 | + | allocating. | |
| 49 | + | ||
| 50 | + | Other limits that exist in the JS and are ported: expression length 21000 | |
| 51 | + | UTF-16 units and parser depth 50 (see below). | |
| 52 | + | ||
| 53 | + | ## Performance notes (deferred) | |
| 54 | + | ||
| 55 | + | - GO-008: `Evaluate` rebuilds the case-folded context and function maps on | |
| 56 | + | every call, and `foldKey` lower-cases on each lookup. Fine for per-step | |
| 57 | + | evaluation; when a caller evaluates thousands of expressions against the | |
| 58 | + | same contexts, add a prepared `Environment` (fold once, reuse) and cache | |
| 59 | + | folded names in the AST. | |
| 60 | + | ||
| 61 | + | ## Where the JavaScript is believed to differ from the C# runner | |
| 62 | + | ||
| 63 | + | Each item is implemented as the JavaScript does. Check the runner's tests | |
| 64 | + | (`src/Test/L0/Expressions*`, `Sdk/Expressions2/...`) and decide for each one | |
| 65 | + | whether to keep the JS behaviour or follow the runner. | |
| 66 | + | ||
| 67 | + | - [ ] **String escapes: only the first `''` is unescaped.** The lexer does | |
| 68 | + | `value.replace("''", "'")` with a string pattern, which replaces one | |
| 69 | + | occurrence: `'a''b''c'` is `a'b''c`. The C# lexer should unescape every | |
| 70 | + | pair (`StringBuilder`/`Replace`). Probable JS bug. (`lexer.go`, | |
| 71 | + | `consumeString`; tests: `TestEvalLiteralsAndOperators`.) | |
| 72 | + | - [ ] **`case()` availability.** The JS gates it behind the experimental flag | |
| 73 | + | `allowCaseFunction`, but only in completion; the parser and the | |
| 74 | + | evaluator always accept it. Here it is always on. Confirm whether the | |
| 75 | + | hosted service accepts `case` and with which arity rules (the JS: at | |
| 76 | + | least 3 arguments, an odd number, predicates must be booleans). | |
| 77 | + | - [ ] **`format()`**: the reference uses `parseInt` on the digits, so | |
| 78 | + | `{99999999999999999999}` is an Infinity-like index and reports "more | |
| 79 | + | arguments than were supplied". The `{0:x}` form silently ignores the | |
| 80 | + | specifier; C# may throw. The error texts are the JS ones. Check the C# | |
| 81 | + | `Format` node for the exact messages and for `{N}` with `N` > 255 or | |
| 82 | + | leading zeros. | |
| 83 | + | - [ ] **`contains()` lower-cases (`toLowerCase`), `startsWith`/`endsWith` | |
| 84 | + | upper-case (`toUpperSpecial`).** The C# uses `OrdinalIgnoreCase` | |
| 85 | + | comparisons, which differ for `ß`, `İ`, `ı`, the Kelvin sign and final | |
| 86 | + | sigma. `contains('ß', 'ss')` is false in JS; `startsWith('ß', 'ss')` is | |
| 87 | + | true. Check what the runner does. | |
| 88 | + | - [ ] **`==` on strings uses `toUpperSpecial`** (upper-case with the dotless | |
| 89 | + | `ı` left alone, full Unicode mapping so `ß == 'SS'`), and `<`/`>` compare | |
| 90 | + | the upper-cased UTF-16 code units. C# `OrdinalIgnoreCase` upper-cases | |
| 91 | + | per UTF-16 code unit with simple mappings only: `ß == 'SS'` should be | |
| 92 | + | false there. | |
| 93 | + | - [ ] **Number to string (`CoerceString`)**: `(+x.toFixed(15)).toString()`. | |
| 94 | + | C# formats a `double` with `"G15"`-like rules. They agree for ordinary | |
| 95 | + | values; check very small and very large numbers (`1e-7` here is `1e-7`, | |
| 96 | + | `1e21` is `1e+21`), `-0`, and 16-17 significant digit values. | |
| 97 | + | - [ ] **String to number**: `Number()` semantics (hex, octal, binary, | |
| 98 | + | `Infinity`, whitespace trimmed, empty is 0). C# `double.TryParse` with | |
| 99 | + | `NumberStyles` differs (no octal/binary; check `0x` and `Infinity`). | |
| 100 | + | The lexer applies the same conversion to number literals, so `0x10`, | |
| 101 | + | `0b11`, `1e3`, `.5`, `5.` and `+1` are all literals here. | |
| 102 | + | - [ ] **Number literals after `-` / `+`**: `-1` is one literal token; `a -1` | |
| 103 | + | is therefore a parse error, as in the JS. | |
| 104 | + | - [ ] **Object key order from `fromJSON`**: array-index keys first, ascending | |
| 105 | + | (JavaScript `Object.keys`), then insertion order. `Json.NET` keeps | |
| 106 | + | document order. The same ordering is applied by `toJSON` (the JS | |
| 107 | + | replacer builds a plain object). Check against the runner for | |
| 108 | + | `toJSON(fromJSON('{"b":1,"1":2}'))`. | |
| 109 | + | - [ ] **Duplicate JSON keys**: last value wins at the first position (JS); | |
| 110 | + | keys that differ only by case collapse to the first. | |
| 111 | + | - [ ] **`fromJSON` errors**: the JS message is | |
| 112 | + | `Error parsing JSON when evaluating fromJson` with the V8 `SyntaxError` | |
| 113 | + | as cause; here the cause is a Go error with similar but not identical | |
| 114 | + | wording (only the outer message is stable). `empty input` for blank | |
| 115 | + | input. C# messages differ; check. | |
| 116 | + | - [ ] **`toJSON`**: `JSON.stringify(v, replacer, " ")`: two-space indent, | |
| 117 | + | `[]` and `{}` for empties, `NaN`/`Infinity` as `null`, `-0` as `0`, | |
| 118 | + | no escaping of `<`, `>`, `&`, U+2028/U+2029; lone surrogates as | |
| 119 | + | `\udXXX`. Json.NET differs on some of these (`Infinity`, `-0`, escapes). | |
| 120 | + | - [ ] **Index access**: `a[b]` where `b` is a number uses `floor(b)` and | |
| 121 | + | rejects NaN, negatives and infinities; where `b` is a string it is a key | |
| 122 | + | or a decimal index (`'1'`, `' 1 '`, `'1.9'` all index 1 of an array). | |
| 123 | + | Booleans and null index as `1`/`0` on arrays and as `"true"`/`""` keys on | |
| 124 | + | dictionaries. | |
| 125 | + | - [ ] **Filtered arrays** (`a.*`): a dictionary filtered by `*` yields its | |
| 126 | + | values; `.*.b` skips elements without `b`; `.*.*` flattens one level; | |
| 127 | + | arrays produced by `*` are fresh, so `a.* == a.*` is false. | |
| 128 | + | - [ ] **Comparison**: `null` is only ordered against numbers/booleans (it | |
| 129 | + | becomes 0); arrays and dictionaries are never ordered and `==` is | |
| 130 | + | identity; `NaN` is unordered and unequal to itself; `true > false`. | |
| 131 | + | - [ ] **`&&`/`||` return an operand**, not a boolean (`'' || 'x'` is `'x'`). | |
| 132 | + | - [ ] **Limits**: max expression length 21000 (UTF-16 units; the JS throws a | |
| 133 | + | bare `Error("ErrorExceededMaxLength")`), max parse depth 50. The depth | |
| 134 | + | counting (including `index()` and the flat logical chains) is ported | |
| 135 | + | literally; C# `MaxDepth` is 50 with its own counting. Cross-check with | |
| 136 | + | deep-nesting tests. There is no node-count or evaluation-memory limit | |
| 137 | + | here; the C# runner has `MaxMemory` (1 MB) and a result-size limit. | |
| 138 | + | - [ ] **Trailing comma in calls**: `contains('a', 'b',)` parses (the JS loop | |
| 139 | + | only requires a comma when the next token is not `)`). `f(,)` does not. | |
| 140 | + | - [ ] **Whitespace**: only space, `\t`, `\r`, `\n` separate tokens. U+00A0 and | |
| 141 | + | other Unicode spaces end a word but are not skipped, so `1 \u00a0 2` | |
| 142 | + | reports an unexpected symbol. `\v`, `\f` likewise. | |
| 143 | + | - [ ] **Empty expression**: the JS parser returns `undefined` for no tokens. | |
| 144 | + | `Parse` returns a nil Node and no error; `Evaluate(nil)` errors. | |
| 145 | + | - [ ] **Unknown contexts at evaluation time** are null (the JS `Dictionary.get` | |
| 146 | + | returns `undefined` and `visitContextAccess` then crashes later); here | |
| 147 | + | declared-but-absent contexts are null, never a crash. | |
| 148 | + | - [ ] **Error positions**: the JS EOF token's start column is negative after a | |
| 149 | + | trailing newline (stale `start`); ported as is | |
| 150 | + | (`TestEOFTokenPosition`). | |
| 151 | + | - [ ] **Function names and context names are case-insensitive**; the JS | |
| 152 | + | lower-cases with `toLowerCase`, as does this port (`foldKey`). Extension | |
| 153 | + | functions override built-ins at evaluation (`functions?.get(...) || | |
| 154 | + | wellKnownFunctions[...]`) but built-ins win at validation time (arity). | |
| 155 | + | ||
| 156 | + | ## JavaScript behaviours deliberately NOT ported | |
| 157 | + | ||
| 158 | + | These are accidents of JavaScript objects used as maps; porting them would be | |
| 159 | + | a bug. | |
| 160 | + | ||
| 161 | + | - `Dictionary.add` tests `key.toLowerCase() in this.indexMap` on a plain | |
| 162 | + | object, so keys that collide with `Object.prototype` properties | |
| 163 | + | (`constructor`, `__proto__`, `valueOf`, ...) are silently dropped or hidden. | |
| 164 | + | Here every key is stored. | |
| 165 | + | - `wellKnownFunctions[name]` is a plain-object lookup, so `constructor(1)`, | |
| 166 | + | `toString(1)` and `hasOwnProperty(1)` pass validation or crash with a | |
| 167 | + | `TypeError` instead of "Unrecognized function". Here they are "Unrecognized | |
| 168 | + | function". | |
| 169 | + | - A `TypeError` in the JS (`Cannot read properties of undefined`) when | |
| 170 | + | evaluating e.g. `github.true` with an empty context: the reference does not | |
| 171 | + | handle a missing root context. Here it is null. | |
| 172 | + | - Recursion limits: JS recursion depth for `fromJSON` input is engine | |
| 173 | + | dependent (about 5 000 to 10 000 levels). `FromJSON` accepts `MaxJSONDepth` | |
| 174 | + | (1000) levels. | |
| 175 | + | - Lone surrogates: JS strings are UTF-16 and can hold them; Go strings are | |
| 176 | + | UTF-8. They are kept as WTF-8 (`0xED 0xA0-0xBF xx`) so that `fromJSON` / | |
| 177 | + | `toJSON` round-trip, `==`/`<`/`contains`/`startsWith`/`endsWith` compare | |
| 178 | + | them like code units, case mapping copies them, and `format`/`join` rejoin a | |
| 179 | + | high surrogate followed by a low one into the character they form. This is | |
| 180 | + | covered by oracle cases. Two surrogate strings concatenated by other means | |
| 181 | + | (an extension function) are not rejoined. | |
| 182 | + | ||
| 183 | + | ## Other things to check once the runner is available | |
| 184 | + | ||
| 185 | + | - [ ] Run the runner's expression test vectors (`ExpressionParserL0`, | |
| 186 | + | `ExpressionTests`, `ExpressionValuesL0`, `ExpressionTestUtil`) through a | |
| 187 | + | table adapter; list every disagreement here before deciding. | |
| 188 | + | - [ ] Decide whether `EvalContext` needs a hook for the functions the runner | |
| 189 | + | implements natively with special semantics: `success()`, `failure()`, | |
| 190 | + | `always()`, `cancelled()` (take no arguments; read job status) and | |
| 191 | + | `hashFiles()` (variadic glob patterns, string result). The API supports | |
| 192 | + | them through `FuncInfo` + `Func` without changes; arity is declared by | |
| 193 | + | the caller. | |
| 194 | + | - [ ] Context names the runner allows per location (`github`, `env`, `vars`, | |
| 195 | + | `job`, `jobs`, `steps`, `runner`, `secrets`, `strategy`, `matrix`, | |
| 196 | + | `needs`, `inputs`) and which functions each location may call | |
| 197 | + | (`success`... only in `if`/`jobs.*.if`...). This belongs to the template | |
| 198 | + | layer, not to package expr. | |
| 199 | + | - [ ] `format()` argument limit is 255 in the JS (`MAX_ARGUMENTS`); the C# | |
| 200 | + | limit is `ExpressionConstants.MaxArguments` (check). | |
| 201 | + | - [ ] Case-insensitive comparison and `contains`: if C# `OrdinalIgnoreCase` | |
| 202 | + | is the truth, replace `toUpperSpecial`/`jsToLower` by per-UTF-16-unit | |
| 203 | + | simple upper-casing and regenerate the oracle tests accordingly. | |
| 204 | + | - [ ] Secret masking and the `${{ }}` splitting of a larger string are not | |
| 205 | + | part of this package (they belong to the template layer): `Parse` | |
| 206 | + | takes the text between the braces only. | |
A expr/ast.go +73 -0
| @@ -0,0 +1,73 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | // Node is a node of a parsed expression. The concrete types are [*Literal], | |
| 6 | + | // [*Unary], [*FunctionCall], [*Binary], [*Logical], [*Grouping], | |
| 7 | + | // [*ContextAccess], [*IndexAccess] and [*Star]. | |
| 8 | + | type Node interface { | |
| 9 | + | isNode() | |
| 10 | + | } | |
| 11 | + | ||
| 12 | + | // Literal is a null, boolean, number or string constant. | |
| 13 | + | type Literal struct { | |
| 14 | + | Value Value | |
| 15 | + | Token Token | |
| 16 | + | } | |
| 17 | + | ||
| 18 | + | // Unary is a negation: "!x". | |
| 19 | + | type Unary struct { | |
| 20 | + | Operator Token | |
| 21 | + | Expr Node | |
| 22 | + | } | |
| 23 | + | ||
| 24 | + | // FunctionCall is a call such as "contains(a, b)". Name is the identifier | |
| 25 | + | // token as written; function names are case-insensitive. | |
| 26 | + | type FunctionCall struct { | |
| 27 | + | Name Token | |
| 28 | + | Args []Node | |
| 29 | + | } | |
| 30 | + | ||
| 31 | + | // Binary is a comparison: ==, !=, <, <=, > or >=. | |
| 32 | + | type Binary struct { | |
| 33 | + | Left Node | |
| 34 | + | Operator Token | |
| 35 | + | Right Node | |
| 36 | + | } | |
| 37 | + | ||
| 38 | + | // Logical is a chain of && or || operands. The chain is flat: "a || b || c" | |
| 39 | + | // is one Logical with three Args. Operator is the first operator token. | |
| 40 | + | type Logical struct { | |
| 41 | + | Operator Token | |
| 42 | + | Args []Node | |
| 43 | + | } | |
| 44 | + | ||
| 45 | + | // Grouping is a parenthesized expression. | |
| 46 | + | type Grouping struct { | |
| 47 | + | Group Node | |
| 48 | + | } | |
| 49 | + | ||
| 50 | + | // ContextAccess is the name of a context, such as "github". | |
| 51 | + | type ContextAccess struct { | |
| 52 | + | Name Token | |
| 53 | + | } | |
| 54 | + | ||
| 55 | + | // IndexAccess is "expr[index]" or "expr.name" (Index is then a string | |
| 56 | + | // [Literal]) or "expr.*" (Index is a [Star]). | |
| 57 | + | type IndexAccess struct { | |
| 58 | + | Expr Node | |
| 59 | + | Index Node | |
| 60 | + | } | |
| 61 | + | ||
| 62 | + | // Star is the "*" of "expr.*" and "expr[*]". | |
| 63 | + | type Star struct{} | |
| 64 | + | ||
| 65 | + | func (*Literal) isNode() {} | |
| 66 | + | func (*Unary) isNode() {} | |
| 67 | + | func (*FunctionCall) isNode() {} | |
| 68 | + | func (*Binary) isNode() {} | |
| 69 | + | func (*Logical) isNode() {} | |
| 70 | + | func (*Grouping) isNode() {} | |
| 71 | + | func (*ContextAccess) isNode() {} | |
| 72 | + | func (*IndexAccess) isNode() {} | |
| 73 | + | func (*Star) isNode() {} | |
A expr/casing.go +327 -0
| @@ -0,0 +1,327 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "strings" | |
| 7 | + | "unicode" | |
| 8 | + | "unicode/utf16" | |
| 9 | + | "unicode/utf8" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | // The tables in casing_tables.go are generated; see go:generate in doc.go. | |
| 13 | + | ||
| 14 | + | // This file reproduces the JavaScript string functions the reference | |
| 15 | + | // implementation relies on: toUpperCase and toLowerCase (full Unicode case | |
| 16 | + | // mapping, including multi-character results such as "ß" to "SS" and the | |
| 17 | + | // Final_Sigma rule), trim, and the \s character class. | |
| 18 | + | ||
| 19 | + | // isJSSpace reports whether r is in JavaScript's WhiteSpace or | |
| 20 | + | // LineTerminator sets. It is both the \s regular expression class and the | |
| 21 | + | // set removed by String.prototype.trim and by Number(). | |
| 22 | + | func isJSSpace(r rune) bool { | |
| 23 | + | switch r { | |
| 24 | + | case '\t', '\n', '\v', '\f', '\r', ' ', 0xA0, 0x1680, 0x2028, 0x2029, 0x202F, 0x205F, 0x3000, 0xFEFF: | |
| 25 | + | return true | |
| 26 | + | } | |
| 27 | + | return r >= 0x2000 && r <= 0x200A | |
| 28 | + | } | |
| 29 | + | ||
| 30 | + | // jsTrim is String.prototype.trim. | |
| 31 | + | func jsTrim(s string) string { | |
| 32 | + | return strings.TrimFunc(s, isJSSpace) | |
| 33 | + | } | |
| 34 | + | ||
| 35 | + | func isASCII(s string) bool { | |
| 36 | + | for i := 0; i < len(s); i++ { | |
| 37 | + | if s[i] >= utf8.RuneSelf { | |
| 38 | + | return false | |
| 39 | + | } | |
| 40 | + | } | |
| 41 | + | return true | |
| 42 | + | } | |
| 43 | + | ||
| 44 | + | // Strings are Go strings, so text that JavaScript would hold as a lone | |
| 45 | + | // UTF-16 surrogate (from fromJSON('"\ud800"')) is kept in WTF-8: the | |
| 46 | + | // three-byte form 0xED 0xA0-0xBF 0x80-0xBF that UTF-8 forbids. The helpers | |
| 47 | + | // below make every case and comparison function treat such a sequence as one | |
| 48 | + | // code unit and copy it unchanged, as JavaScript does, instead of letting Go | |
| 49 | + | // turn each byte into U+FFFD. | |
| 50 | + | ||
| 51 | + | // wtf8Surrogate reports whether s starts with a WTF-8 lone surrogate and | |
| 52 | + | // returns its code unit. | |
| 53 | + | func wtf8Surrogate(s string) (rune, bool) { | |
| 54 | + | if len(s) >= 3 && s[0] == 0xED && s[1] >= 0xA0 && s[1] <= 0xBF && s[2]&0xC0 == 0x80 { | |
| 55 | + | return rune(s[0]&0x0F)<<12 | rune(s[1]&0x3F)<<6 | rune(s[2]&0x3F), true | |
| 56 | + | } | |
| 57 | + | return 0, false | |
| 58 | + | } | |
| 59 | + | ||
| 60 | + | // decodeRune is utf8.DecodeRuneInString that also decodes a WTF-8 lone | |
| 61 | + | // surrogate (width 3) to its code unit value. | |
| 62 | + | func decodeRune(s string) (rune, int) { | |
| 63 | + | r, w := utf8.DecodeRuneInString(s) | |
| 64 | + | if r == utf8.RuneError && w == 1 { | |
| 65 | + | if sur, ok := wtf8Surrogate(s); ok { | |
| 66 | + | return sur, 3 | |
| 67 | + | } | |
| 68 | + | } | |
| 69 | + | return r, w | |
| 70 | + | } | |
| 71 | + | ||
| 72 | + | // writeRune writes r, using the WTF-8 form for a surrogate code unit. | |
| 73 | + | func writeRune(b *strings.Builder, r rune) { | |
| 74 | + | if r >= 0xD800 && r < 0xE000 { | |
| 75 | + | b.WriteByte(0xE0 | byte(r>>12)) | |
| 76 | + | b.WriteByte(0x80 | byte(r>>6)&0x3F) | |
| 77 | + | b.WriteByte(0x80 | byte(r)&0x3F) | |
| 78 | + | return | |
| 79 | + | } | |
| 80 | + | b.WriteRune(r) | |
| 81 | + | } | |
| 82 | + | ||
| 83 | + | // normalizeWTF8 joins a lone high surrogate immediately followed by a lone low | |
| 84 | + | // surrogate into the supplementary character they form, as concatenating two | |
| 85 | + | // JavaScript strings does. Strings built by concatenation (format, join) go | |
| 86 | + | // through it. | |
| 87 | + | func normalizeWTF8(s string) string { | |
| 88 | + | if !strings.Contains(s, "\xed") { | |
| 89 | + | return s | |
| 90 | + | } | |
| 91 | + | var b strings.Builder | |
| 92 | + | b.Grow(len(s)) | |
| 93 | + | for i := 0; i < len(s); { | |
| 94 | + | r, w := decodeRune(s[i:]) | |
| 95 | + | if r >= 0xD800 && r < 0xDC00 { | |
| 96 | + | if lo, lw := decodeRune(s[i+w:]); lo >= 0xDC00 && lo < 0xE000 && lw == 3 && i+w < len(s) { | |
| 97 | + | b.WriteRune(0x10000 + (r-0xD800)<<10 + (lo - 0xDC00)) | |
| 98 | + | i += w + lw | |
| 99 | + | continue | |
| 100 | + | } | |
| 101 | + | } | |
| 102 | + | b.WriteString(s[i : i+w]) | |
| 103 | + | i += w | |
| 104 | + | } | |
| 105 | + | return b.String() | |
| 106 | + | } | |
| 107 | + | ||
| 108 | + | func writeUpper(b *strings.Builder, r rune, keepDotlessI bool) { | |
| 109 | + | if keepDotlessI && r == 'ı' { | |
| 110 | + | b.WriteRune(r) | |
| 111 | + | return | |
| 112 | + | } | |
| 113 | + | if s, ok := jsUpperDelta[r]; ok { | |
| 114 | + | b.WriteString(s) | |
| 115 | + | return | |
| 116 | + | } | |
| 117 | + | writeRune(b, unicode.ToUpper(r)) | |
| 118 | + | } | |
| 119 | + | ||
| 120 | + | func upperWith(s string, keepDotlessI bool) string { | |
| 121 | + | if isASCII(s) { | |
| 122 | + | return strings.ToUpper(s) | |
| 123 | + | } | |
| 124 | + | var b strings.Builder | |
| 125 | + | b.Grow(len(s)) | |
| 126 | + | for i := 0; i < len(s); { | |
| 127 | + | r, w := decodeRune(s[i:]) | |
| 128 | + | i += w | |
| 129 | + | writeUpper(&b, r, keepDotlessI) | |
| 130 | + | } | |
| 131 | + | return b.String() | |
| 132 | + | } | |
| 133 | + | ||
| 134 | + | // jsToUpper is String.prototype.toUpperCase. | |
| 135 | + | func jsToUpper(s string) string { return upperWith(s, false) } | |
| 136 | + | ||
| 137 | + | // toUpperSpecial upper-cases s like jsToUpper but leaves the dotless i | |
| 138 | + | // (U+0131) alone, as the reference implementation does for its | |
| 139 | + | // case-insensitive comparisons and for startsWith and endsWith. | |
| 140 | + | func toUpperSpecial(s string) string { return upperWith(s, true) } | |
| 141 | + | ||
| 142 | + | // jsToLower is String.prototype.toLowerCase, including the Final_Sigma | |
| 143 | + | // context rule (a capital sigma at the end of a word becomes ς). | |
| 144 | + | func jsToLower(s string) string { | |
| 145 | + | if isASCII(s) { | |
| 146 | + | return strings.ToLower(s) | |
| 147 | + | } | |
| 148 | + | runes := make([]rune, 0, len(s)) | |
| 149 | + | for i := 0; i < len(s); { | |
| 150 | + | r, w := decodeRune(s[i:]) | |
| 151 | + | runes = append(runes, r) | |
| 152 | + | i += w | |
| 153 | + | } | |
| 154 | + | var b strings.Builder | |
| 155 | + | b.Grow(len(s)) | |
| 156 | + | for i, r := range runes { | |
| 157 | + | switch r { | |
| 158 | + | case 0x3A3: | |
| 159 | + | if finalSigma(runes, i) { | |
| 160 | + | b.WriteRune('ς') | |
| 161 | + | } else { | |
| 162 | + | b.WriteRune('σ') | |
| 163 | + | } | |
| 164 | + | default: | |
| 165 | + | if m, ok := jsLowerDelta[r]; ok { | |
| 166 | + | b.WriteString(m) | |
| 167 | + | } else { | |
| 168 | + | writeRune(&b, unicode.ToLower(r)) | |
| 169 | + | } | |
| 170 | + | } | |
| 171 | + | } | |
| 172 | + | return b.String() | |
| 173 | + | } | |
| 174 | + | ||
| 175 | + | // finalSigma implements the Final_Sigma condition of Unicode SpecialCasing: | |
| 176 | + | // the sigma is preceded by a cased letter, ignoring case-ignorable | |
| 177 | + | // characters, and is not followed by one. | |
| 178 | + | func finalSigma(runes []rune, i int) bool { | |
| 179 | + | before := false | |
| 180 | + | for j := i - 1; j >= 0; j-- { | |
| 181 | + | if isCaseIgnorable(runes[j]) { | |
| 182 | + | continue | |
| 183 | + | } | |
| 184 | + | before = isCased(runes[j]) | |
| 185 | + | break | |
| 186 | + | } | |
| 187 | + | if !before { | |
| 188 | + | return false | |
| 189 | + | } | |
| 190 | + | for j := i + 1; j < len(runes); j++ { | |
| 191 | + | if isCaseIgnorable(runes[j]) { | |
| 192 | + | continue | |
| 193 | + | } | |
| 194 | + | return !isCased(runes[j]) | |
| 195 | + | } | |
| 196 | + | return true | |
| 197 | + | } | |
| 198 | + | ||
| 199 | + | func isCased(r rune) bool { | |
| 200 | + | return unicode.IsUpper(r) || unicode.IsLower(r) || unicode.IsTitle(r) || | |
| 201 | + | unicode.Is(unicode.Other_Lowercase, r) || unicode.Is(unicode.Other_Uppercase, r) | |
| 202 | + | } | |
| 203 | + | ||
| 204 | + | func isCaseIgnorable(r rune) bool { | |
| 205 | + | switch r { | |
| 206 | + | case '\'', '.', ':', '^', '`', 0xA8, 0xAD, 0xAF, 0xB4, 0xB7, 0xB8, 0x2018, 0x2019, 0x2024, 0x2027, 0xFE13, 0xFE52, 0xFE55, 0xFF07, 0xFF0E, 0xFF1A: | |
| 207 | + | return true | |
| 208 | + | } | |
| 209 | + | return unicode.In(r, unicode.Mn, unicode.Me, unicode.Cf, unicode.Lm, unicode.Sk) | |
| 210 | + | } | |
| 211 | + | ||
| 212 | + | // utf16Units encodes s as the UTF-16 code units a JavaScript string holds, | |
| 213 | + | // WTF-8 lone surrogates included. | |
| 214 | + | func utf16Units(s string) []uint16 { | |
| 215 | + | out := make([]uint16, 0, len(s)) | |
| 216 | + | for i := 0; i < len(s); { | |
| 217 | + | r, w := decodeRune(s[i:]) | |
| 218 | + | i += w | |
| 219 | + | if r >= 0xD800 && r < 0xE000 { | |
| 220 | + | out = append(out, uint16(r)) | |
| 221 | + | continue | |
| 222 | + | } | |
| 223 | + | if r >= 0x10000 { | |
| 224 | + | hi, lo := utf16.EncodeRune(r) | |
| 225 | + | out = append(out, uint16(hi), uint16(lo)) | |
| 226 | + | continue | |
| 227 | + | } | |
| 228 | + | out = append(out, uint16(r)) | |
| 229 | + | } | |
| 230 | + | return out | |
| 231 | + | } | |
| 232 | + | ||
| 233 | + | // compareUTF16 compares two strings by UTF-16 code unit, which is how | |
| 234 | + | // JavaScript orders strings with < and >. It differs from byte or rune order | |
| 235 | + | // only for supplementary-plane characters against U+E000..U+FFFF. | |
| 236 | + | func compareUTF16(a, b string) int { | |
| 237 | + | if isASCII(a) && isASCII(b) { | |
| 238 | + | return strings.Compare(a, b) | |
| 239 | + | } | |
| 240 | + | ua, ub := utf16Units(a), utf16Units(b) | |
| 241 | + | for i := 0; i < len(ua) && i < len(ub); i++ { | |
| 242 | + | if ua[i] != ub[i] { | |
| 243 | + | if ua[i] < ub[i] { | |
| 244 | + | return -1 | |
| 245 | + | } | |
| 246 | + | return 1 | |
| 247 | + | } | |
| 248 | + | } | |
| 249 | + | switch { | |
| 250 | + | case len(ua) > len(ub): | |
| 251 | + | return 1 | |
| 252 | + | case len(ua) < len(ub): | |
| 253 | + | return -1 | |
| 254 | + | } | |
| 255 | + | return 0 | |
| 256 | + | } | |
| 257 | + | ||
| 258 | + | // exceedsUTF16Len reports whether s is longer than max UTF-16 code units. It | |
| 259 | + | // stops counting at max and allocates nothing. Invalid UTF-8 bytes count as | |
| 260 | + | // one unit each (U+FFFD), like the conversion Lex applies. | |
| 261 | + | func exceedsUTF16Len(s string, max int) bool { | |
| 262 | + | n := 0 | |
| 263 | + | for _, r := range s { | |
| 264 | + | n++ | |
| 265 | + | if r >= 0x10000 { | |
| 266 | + | n++ | |
| 267 | + | } | |
| 268 | + | if n > max { | |
| 269 | + | return true | |
| 270 | + | } | |
| 271 | + | } | |
| 272 | + | return false | |
| 273 | + | } | |
| 274 | + | ||
| 275 | + | // hasWTF8 reports whether s holds a lone surrogate in WTF-8 form. | |
| 276 | + | func hasWTF8(s string) bool { | |
| 277 | + | for i := 0; i < len(s); i++ { | |
| 278 | + | if s[i] == 0xED { | |
| 279 | + | if _, ok := wtf8Surrogate(s[i:]); ok { | |
| 280 | + | return true | |
| 281 | + | } | |
| 282 | + | } | |
| 283 | + | } | |
| 284 | + | return false | |
| 285 | + | } | |
| 286 | + | ||
| 287 | + | // jsIncludes, jsHasPrefix and jsHasSuffix are String.prototype.includes, | |
| 288 | + | // startsWith and endsWith. They work on bytes, which is equivalent to UTF-16 | |
| 289 | + | // code units except around lone surrogates (a lone high surrogate is found | |
| 290 | + | // inside a supplementary character), where they switch to code units. | |
| 291 | + | func jsIncludes(s, sub string) bool { | |
| 292 | + | if !hasWTF8(s) && !hasWTF8(sub) { | |
| 293 | + | return strings.Contains(s, sub) | |
| 294 | + | } | |
| 295 | + | us, usub := utf16Units(s), utf16Units(sub) | |
| 296 | + | for i := 0; i+len(usub) <= len(us); i++ { | |
| 297 | + | if unitsEqual(us[i:i+len(usub)], usub) { | |
| 298 | + | return true | |
| 299 | + | } | |
| 300 | + | } | |
| 301 | + | return false | |
| 302 | + | } | |
| 303 | + | ||
| 304 | + | func jsHasPrefix(s, prefix string) bool { | |
| 305 | + | if !hasWTF8(s) && !hasWTF8(prefix) { | |
| 306 | + | return strings.HasPrefix(s, prefix) | |
| 307 | + | } | |
| 308 | + | us, up := utf16Units(s), utf16Units(prefix) | |
| 309 | + | return len(up) <= len(us) && unitsEqual(us[:len(up)], up) | |
| 310 | + | } | |
| 311 | + | ||
| 312 | + | func jsHasSuffix(s, suffix string) bool { | |
| 313 | + | if !hasWTF8(s) && !hasWTF8(suffix) { | |
| 314 | + | return strings.HasSuffix(s, suffix) | |
| 315 | + | } | |
| 316 | + | us, up := utf16Units(s), utf16Units(suffix) | |
| 317 | + | return len(up) <= len(us) && unitsEqual(us[len(us)-len(up):], up) | |
| 318 | + | } | |
| 319 | + | ||
| 320 | + | func unitsEqual(a, b []uint16) bool { | |
| 321 | + | for i := range a { | |
| 322 | + | if a[i] != b[i] { | |
| 323 | + | return false | |
| 324 | + | } | |
| 325 | + | } | |
| 326 | + | return len(a) == len(b) | |
| 327 | + | } | |
A expr/casing_tables.go +176 -0
| @@ -0,0 +1,176 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | // Code generated by gen/casing.go from gen/casing_dump.mjs; DO NOT EDIT. | |
| 4 | + | ||
| 5 | + | package expr | |
| 6 | + | ||
| 7 | + | // casingGoUnicodeVersion is the Unicode version of the Go toolchain that generated | |
| 8 | + | // the tables below: they are relative to its unicode.ToUpper and unicode.ToLower. | |
| 9 | + | const casingGoUnicodeVersion = "17.0.0" | |
| 10 | + | ||
| 11 | + | // jsUpperDelta maps the code points whose JavaScript String.prototype.toUpperCase result differs from the Go simple mapping. | |
| 12 | + | var jsUpperDelta = map[rune]string{ | |
| 13 | + | 0x00DF: "SS", // "ß" | |
| 14 | + | 0x0149: "ʼN", // "ʼn" | |
| 15 | + | 0x01F0: "J̌", // "ǰ" | |
| 16 | + | 0x0390: "Ϊ́", // "ΐ" | |
| 17 | + | 0x03B0: "Ϋ́", // "ΰ" | |
| 18 | + | 0x0587: "ԵՒ", // "և" | |
| 19 | + | 0x1E96: "H̱", // "ẖ" | |
| 20 | + | 0x1E97: "T̈", // "ẗ" | |
| 21 | + | 0x1E98: "W̊", // "ẘ" | |
| 22 | + | 0x1E99: "Y̊", // "ẙ" | |
| 23 | + | 0x1E9A: "Aʾ", // "ẚ" | |
| 24 | + | 0x1F50: "Υ̓", // "ὐ" | |
| 25 | + | 0x1F52: "Υ̓̀", // "ὒ" | |
| 26 | + | 0x1F54: "Υ̓́", // "ὔ" | |
| 27 | + | 0x1F56: "Υ̓͂", // "ὖ" | |
| 28 | + | 0x1F80: "ἈΙ", // "ᾀ" | |
| 29 | + | 0x1F81: "ἉΙ", // "ᾁ" | |
| 30 | + | 0x1F82: "ἊΙ", // "ᾂ" | |
| 31 | + | 0x1F83: "ἋΙ", // "ᾃ" | |
| 32 | + | 0x1F84: "ἌΙ", // "ᾄ" | |
| 33 | + | 0x1F85: "ἍΙ", // "ᾅ" | |
| 34 | + | 0x1F86: "ἎΙ", // "ᾆ" | |
| 35 | + | 0x1F87: "ἏΙ", // "ᾇ" | |
| 36 | + | 0x1F88: "ἈΙ", // "ᾈ" | |
| 37 | + | 0x1F89: "ἉΙ", // "ᾉ" | |
| 38 | + | 0x1F8A: "ἊΙ", // "ᾊ" | |
| 39 | + | 0x1F8B: "ἋΙ", // "ᾋ" | |
| 40 | + | 0x1F8C: "ἌΙ", // "ᾌ" | |
| 41 | + | 0x1F8D: "ἍΙ", // "ᾍ" | |
| 42 | + | 0x1F8E: "ἎΙ", // "ᾎ" | |
| 43 | + | 0x1F8F: "ἏΙ", // "ᾏ" | |
| 44 | + | 0x1F90: "ἨΙ", // "ᾐ" | |
| 45 | + | 0x1F91: "ἩΙ", // "ᾑ" | |
| 46 | + | 0x1F92: "ἪΙ", // "ᾒ" | |
| 47 | + | 0x1F93: "ἫΙ", // "ᾓ" | |
| 48 | + | 0x1F94: "ἬΙ", // "ᾔ" | |
| 49 | + | 0x1F95: "ἭΙ", // "ᾕ" | |
| 50 | + | 0x1F96: "ἮΙ", // "ᾖ" | |
| 51 | + | 0x1F97: "ἯΙ", // "ᾗ" | |
| 52 | + | 0x1F98: "ἨΙ", // "ᾘ" | |
| 53 | + | 0x1F99: "ἩΙ", // "ᾙ" | |
| 54 | + | 0x1F9A: "ἪΙ", // "ᾚ" | |
| 55 | + | 0x1F9B: "ἫΙ", // "ᾛ" | |
| 56 | + | 0x1F9C: "ἬΙ", // "ᾜ" | |
| 57 | + | 0x1F9D: "ἭΙ", // "ᾝ" | |
| 58 | + | 0x1F9E: "ἮΙ", // "ᾞ" | |
| 59 | + | 0x1F9F: "ἯΙ", // "ᾟ" | |
| 60 | + | 0x1FA0: "ὨΙ", // "ᾠ" | |
| 61 | + | 0x1FA1: "ὩΙ", // "ᾡ" | |
| 62 | + | 0x1FA2: "ὪΙ", // "ᾢ" | |
| 63 | + | 0x1FA3: "ὫΙ", // "ᾣ" | |
| 64 | + | 0x1FA4: "ὬΙ", // "ᾤ" | |
| 65 | + | 0x1FA5: "ὭΙ", // "ᾥ" | |
| 66 | + | 0x1FA6: "ὮΙ", // "ᾦ" | |
| 67 | + | 0x1FA7: "ὯΙ", // "ᾧ" | |
| 68 | + | 0x1FA8: "ὨΙ", // "ᾨ" | |
| 69 | + | 0x1FA9: "ὩΙ", // "ᾩ" | |
| 70 | + | 0x1FAA: "ὪΙ", // "ᾪ" | |
| 71 | + | 0x1FAB: "ὫΙ", // "ᾫ" | |
| 72 | + | 0x1FAC: "ὬΙ", // "ᾬ" | |
| 73 | + | 0x1FAD: "ὭΙ", // "ᾭ" | |
| 74 | + | 0x1FAE: "ὮΙ", // "ᾮ" | |
| 75 | + | 0x1FAF: "ὯΙ", // "ᾯ" | |
| 76 | + | 0x1FB2: "ᾺΙ", // "ᾲ" | |
| 77 | + | 0x1FB3: "ΑΙ", // "ᾳ" | |
| 78 | + | 0x1FB4: "ΆΙ", // "ᾴ" | |
| 79 | + | 0x1FB6: "Α͂", // "ᾶ" | |
| 80 | + | 0x1FB7: "Α͂Ι", // "ᾷ" | |
| 81 | + | 0x1FBC: "ΑΙ", // "ᾼ" | |
| 82 | + | 0x1FC2: "ῊΙ", // "ῂ" | |
| 83 | + | 0x1FC3: "ΗΙ", // "ῃ" | |
| 84 | + | 0x1FC4: "ΉΙ", // "ῄ" | |
| 85 | + | 0x1FC6: "Η͂", // "ῆ" | |
| 86 | + | 0x1FC7: "Η͂Ι", // "ῇ" | |
| 87 | + | 0x1FCC: "ΗΙ", // "ῌ" | |
| 88 | + | 0x1FD2: "Ϊ̀", // "ῒ" | |
| 89 | + | 0x1FD3: "Ϊ́", // "ΐ" | |
| 90 | + | 0x1FD6: "Ι͂", // "ῖ" | |
| 91 | + | 0x1FD7: "Ϊ͂", // "ῗ" | |
| 92 | + | 0x1FE2: "Ϋ̀", // "ῢ" | |
| 93 | + | 0x1FE3: "Ϋ́", // "ΰ" | |
| 94 | + | 0x1FE4: "Ρ̓", // "ῤ" | |
| 95 | + | 0x1FE6: "Υ͂", // "ῦ" | |
| 96 | + | 0x1FE7: "Ϋ͂", // "ῧ" | |
| 97 | + | 0x1FF2: "ῺΙ", // "ῲ" | |
| 98 | + | 0x1FF3: "ΩΙ", // "ῳ" | |
| 99 | + | 0x1FF4: "ΏΙ", // "ῴ" | |
| 100 | + | 0x1FF6: "Ω͂", // "ῶ" | |
| 101 | + | 0x1FF7: "Ω͂Ι", // "ῷ" | |
| 102 | + | 0x1FFC: "ΩΙ", // "ῼ" | |
| 103 | + | 0xA7CF: "", // "" | |
| 104 | + | 0xA7D3: "ꟓ", // "ꟓ" | |
| 105 | + | 0xA7D5: "ꟕ", // "ꟕ" | |
| 106 | + | 0xFB00: "FF", // "ff" | |
| 107 | + | 0xFB01: "FI", // "fi" | |
| 108 | + | 0xFB02: "FL", // "fl" | |
| 109 | + | 0xFB03: "FFI", // "ffi" | |
| 110 | + | 0xFB04: "FFL", // "ffl" | |
| 111 | + | 0xFB05: "ST", // "ſt" | |
| 112 | + | 0xFB06: "ST", // "st" | |
| 113 | + | 0xFB13: "ՄՆ", // "ﬓ" | |
| 114 | + | 0xFB14: "ՄԵ", // "ﬔ" | |
| 115 | + | 0xFB15: "ՄԻ", // "ﬕ" | |
| 116 | + | 0xFB16: "ՎՆ", // "ﬖ" | |
| 117 | + | 0xFB17: "ՄԽ", // "ﬗ" | |
| 118 | + | 0x16EBB: "", // "" | |
| 119 | + | 0x16EBC: "", // "" | |
| 120 | + | 0x16EBD: "", // "" | |
| 121 | + | 0x16EBE: "", // "" | |
| 122 | + | 0x16EBF: "", // "" | |
| 123 | + | 0x16EC0: "", // "" | |
| 124 | + | 0x16EC1: "", // "" | |
| 125 | + | 0x16EC2: "", // "" | |
| 126 | + | 0x16EC3: "", // "" | |
| 127 | + | 0x16EC4: "", // "" | |
| 128 | + | 0x16EC5: "", // "" | |
| 129 | + | 0x16EC6: "", // "" | |
| 130 | + | 0x16EC7: "", // "" | |
| 131 | + | 0x16EC8: "", // "" | |
| 132 | + | 0x16EC9: "", // "" | |
| 133 | + | 0x16ECA: "", // "" | |
| 134 | + | 0x16ECB: "", // "" | |
| 135 | + | 0x16ECC: "", // "" | |
| 136 | + | 0x16ECD: "", // "" | |
| 137 | + | 0x16ECE: "", // "" | |
| 138 | + | 0x16ECF: "", // "" | |
| 139 | + | 0x16ED0: "", // "" | |
| 140 | + | 0x16ED1: "", // "" | |
| 141 | + | 0x16ED2: "", // "" | |
| 142 | + | 0x16ED3: "", // "" | |
| 143 | + | } | |
| 144 | + | ||
| 145 | + | // jsLowerDelta maps the code points whose JavaScript String.prototype.toLowerCase (without the Final_Sigma context) result differs from the Go simple mapping. | |
| 146 | + | var jsLowerDelta = map[rune]string{ | |
| 147 | + | 0x0130: "i̇", // "İ" | |
| 148 | + | 0xA7CE: "", // "" | |
| 149 | + | 0xA7D2: "", // "" | |
| 150 | + | 0xA7D4: "", // "" | |
| 151 | + | 0x16EA0: "", // "" | |
| 152 | + | 0x16EA1: "", // "" | |
| 153 | + | 0x16EA2: "", // "" | |
| 154 | + | 0x16EA3: "", // "" | |
| 155 | + | 0x16EA4: "", // "" | |
| 156 | + | 0x16EA5: "", // "" | |
| 157 | + | 0x16EA6: "", // "" | |
| 158 | + | 0x16EA7: "", // "" | |
| 159 | + | 0x16EA8: "", // "" | |
| 160 | + | 0x16EA9: "", // "" | |
| 161 | + | 0x16EAA: "", // "" | |
| 162 | + | 0x16EAB: "", // "" | |
| 163 | + | 0x16EAC: "", // "" | |
| 164 | + | 0x16EAD: "", // "" | |
| 165 | + | 0x16EAE: "", // "" | |
| 166 | + | 0x16EAF: "", // "" | |
| 167 | + | 0x16EB0: "", // "" | |
| 168 | + | 0x16EB1: "", // "" | |
| 169 | + | 0x16EB2: "", // "" | |
| 170 | + | 0x16EB3: "", // "" | |
| 171 | + | 0x16EB4: "", // "" | |
| 172 | + | 0x16EB5: "", // "" | |
| 173 | + | 0x16EB6: "", // "" | |
| 174 | + | 0x16EB7: "", // "" | |
| 175 | + | 0x16EB8: "", // "" | |
| 176 | + | } | |
A expr/data.go +301 -0
| @@ -0,0 +1,301 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import "math" | |
| 6 | + | ||
| 7 | + | // Kind is the type of a [Value]. | |
| 8 | + | type Kind int | |
| 9 | + | ||
| 10 | + | // The kinds of value an expression can produce. | |
| 11 | + | const ( | |
| 12 | + | KindNull Kind = iota | |
| 13 | + | KindBoolean | |
| 14 | + | KindNumber | |
| 15 | + | KindString | |
| 16 | + | KindArray | |
| 17 | + | KindDictionary | |
| 18 | + | ) | |
| 19 | + | ||
| 20 | + | // String returns the name GitHub uses for the kind when a non-primitive | |
| 21 | + | // value is converted to a string: "Array" and "Object" for the two compound | |
| 22 | + | // kinds. | |
| 23 | + | func (k Kind) String() string { | |
| 24 | + | switch k { | |
| 25 | + | case KindNull: | |
| 26 | + | return "Null" | |
| 27 | + | case KindBoolean: | |
| 28 | + | return "Boolean" | |
| 29 | + | case KindNumber: | |
| 30 | + | return "Number" | |
| 31 | + | case KindString: | |
| 32 | + | return "String" | |
| 33 | + | case KindArray: | |
| 34 | + | return "Array" | |
| 35 | + | case KindDictionary: | |
| 36 | + | return "Object" | |
| 37 | + | } | |
| 38 | + | return "unknown" | |
| 39 | + | } | |
| 40 | + | ||
| 41 | + | // Value is the result of an expression or an element of a context: one of | |
| 42 | + | // [Null], [Boolean], [Number], [String], [*Array] and [*Dictionary]. The set | |
| 43 | + | // is closed. A nil Value, or a nil *Array or *Dictionary, behaves as [Null]. | |
| 44 | + | type Value interface { | |
| 45 | + | // Kind reports the type of the value. | |
| 46 | + | Kind() Kind | |
| 47 | + | isValue() | |
| 48 | + | } | |
| 49 | + | ||
| 50 | + | // Null is the null value. | |
| 51 | + | type Null struct{} | |
| 52 | + | ||
| 53 | + | // Boolean is a true or false value. | |
| 54 | + | type Boolean bool | |
| 55 | + | ||
| 56 | + | // Number is a double-precision floating-point number. NaN and the infinities | |
| 57 | + | // are legal values. | |
| 58 | + | type Number float64 | |
| 59 | + | ||
| 60 | + | // String is a text value. | |
| 61 | + | type String string | |
| 62 | + | ||
| 63 | + | func (Null) Kind() Kind { return KindNull } | |
| 64 | + | func (Boolean) Kind() Kind { return KindBoolean } | |
| 65 | + | func (Number) Kind() Kind { return KindNumber } | |
| 66 | + | func (String) Kind() Kind { return KindString } | |
| 67 | + | func (*Array) Kind() Kind { return KindArray } | |
| 68 | + | func (*Dictionary) Kind() Kind { return KindDictionary } | |
| 69 | + | ||
| 70 | + | func (Null) isValue() {} | |
| 71 | + | func (Boolean) isValue() {} | |
| 72 | + | func (Number) isValue() {} | |
| 73 | + | func (String) isValue() {} | |
| 74 | + | func (*Array) isValue() {} | |
| 75 | + | func (*Dictionary) isValue() {} | |
| 76 | + | ||
| 77 | + | // Array is an ordered list of values. Two arrays are equal in an expression | |
| 78 | + | // only if they are the same *Array. | |
| 79 | + | type Array struct { | |
| 80 | + | items []Value | |
| 81 | + | // filtered is set on arrays produced by the * filter ("a.*", "a[*]"), | |
| 82 | + | // which flatten one level when indexed again ("a.*.b"). | |
| 83 | + | filtered bool | |
| 84 | + | } | |
| 85 | + | ||
| 86 | + | // NewArray returns an array holding items. | |
| 87 | + | func NewArray(items ...Value) *Array { | |
| 88 | + | return &Array{items: append([]Value(nil), items...)} | |
| 89 | + | } | |
| 90 | + | ||
| 91 | + | // Len returns the number of elements. | |
| 92 | + | func (a *Array) Len() int { | |
| 93 | + | if a == nil { | |
| 94 | + | return 0 | |
| 95 | + | } | |
| 96 | + | return len(a.items) | |
| 97 | + | } | |
| 98 | + | ||
| 99 | + | // At returns the element at index i, or [Null] when i is out of range. | |
| 100 | + | func (a *Array) At(i int) Value { | |
| 101 | + | if a == nil || i < 0 || i >= len(a.items) { | |
| 102 | + | return Null{} | |
| 103 | + | } | |
| 104 | + | return normalize(a.items[i]) | |
| 105 | + | } | |
| 106 | + | ||
| 107 | + | // Items returns a copy of the elements. | |
| 108 | + | func (a *Array) Items() []Value { | |
| 109 | + | if a == nil { | |
| 110 | + | return nil | |
| 111 | + | } | |
| 112 | + | out := make([]Value, len(a.items)) | |
| 113 | + | for i, v := range a.items { | |
| 114 | + | out[i] = normalize(v) | |
| 115 | + | } | |
| 116 | + | return out | |
| 117 | + | } | |
| 118 | + | ||
| 119 | + | // Append adds v at the end of the array. | |
| 120 | + | func (a *Array) Append(v Value) { a.items = append(a.items, v) } | |
| 121 | + | ||
| 122 | + | // Filtered reports whether the array was produced by a * filter. | |
| 123 | + | func (a *Array) Filtered() bool { return a != nil && a.filtered } | |
| 124 | + | ||
| 125 | + | // Pair is a key and its value in a [Dictionary]. | |
| 126 | + | type Pair struct { | |
| 127 | + | Key string | |
| 128 | + | Value Value | |
| 129 | + | } | |
| 130 | + | ||
| 131 | + | // Dictionary is an ordered map from string keys to values. Keys are looked up | |
| 132 | + | // case-insensitively and keep the spelling and position of their first | |
| 133 | + | // insertion. Two dictionaries are equal in an expression only if they are | |
| 134 | + | // the same *Dictionary. | |
| 135 | + | type Dictionary struct { | |
| 136 | + | keys []string | |
| 137 | + | vals []Value | |
| 138 | + | index map[string]int | |
| 139 | + | } | |
| 140 | + | ||
| 141 | + | // NewDictionary returns an empty dictionary. | |
| 142 | + | func NewDictionary() *Dictionary { | |
| 143 | + | return &Dictionary{index: map[string]int{}} | |
| 144 | + | } | |
| 145 | + | ||
| 146 | + | // Add inserts key with value v and reports whether it did. When a key that | |
| 147 | + | // differs only by case is already present it does nothing and returns false: | |
| 148 | + | // the first insertion wins. | |
| 149 | + | func (d *Dictionary) Add(key string, v Value) bool { | |
| 150 | + | if d.index == nil { | |
| 151 | + | d.index = map[string]int{} | |
| 152 | + | } | |
| 153 | + | folded := foldKey(key) | |
| 154 | + | if _, ok := d.index[folded]; ok { | |
| 155 | + | return false | |
| 156 | + | } | |
| 157 | + | d.index[folded] = len(d.vals) | |
| 158 | + | d.keys = append(d.keys, key) | |
| 159 | + | d.vals = append(d.vals, v) | |
| 160 | + | return true | |
| 161 | + | } | |
| 162 | + | ||
| 163 | + | // Get returns the value stored under key, compared case-insensitively. | |
| 164 | + | func (d *Dictionary) Get(key string) (Value, bool) { | |
| 165 | + | if d == nil { | |
| 166 | + | return nil, false | |
| 167 | + | } | |
| 168 | + | i, ok := d.index[foldKey(key)] | |
| 169 | + | if !ok { | |
| 170 | + | return nil, false | |
| 171 | + | } | |
| 172 | + | return normalize(d.vals[i]), true | |
| 173 | + | } | |
| 174 | + | ||
| 175 | + | // Len returns the number of entries. | |
| 176 | + | func (d *Dictionary) Len() int { | |
| 177 | + | if d == nil { | |
| 178 | + | return 0 | |
| 179 | + | } | |
| 180 | + | return len(d.vals) | |
| 181 | + | } | |
| 182 | + | ||
| 183 | + | // Keys returns the keys in insertion order. | |
| 184 | + | func (d *Dictionary) Keys() []string { | |
| 185 | + | if d == nil { | |
| 186 | + | return nil | |
| 187 | + | } | |
| 188 | + | return append([]string(nil), d.keys...) | |
| 189 | + | } | |
| 190 | + | ||
| 191 | + | // Values returns the values in insertion order. | |
| 192 | + | func (d *Dictionary) Values() []Value { | |
| 193 | + | if d == nil { | |
| 194 | + | return nil | |
| 195 | + | } | |
| 196 | + | out := make([]Value, len(d.vals)) | |
| 197 | + | for i, v := range d.vals { | |
| 198 | + | out[i] = normalize(v) | |
| 199 | + | } | |
| 200 | + | return out | |
| 201 | + | } | |
| 202 | + | ||
| 203 | + | // Pairs returns the entries in insertion order. | |
| 204 | + | func (d *Dictionary) Pairs() []Pair { | |
| 205 | + | if d == nil { | |
| 206 | + | return nil | |
| 207 | + | } | |
| 208 | + | out := make([]Pair, len(d.keys)) | |
| 209 | + | for i, k := range d.keys { | |
| 210 | + | out[i] = Pair{Key: k, Value: normalize(d.vals[i])} | |
| 211 | + | } | |
| 212 | + | return out | |
| 213 | + | } | |
| 214 | + | ||
| 215 | + | // foldKey is the case-insensitive form of a dictionary key. GitHub lower-cases | |
| 216 | + | // keys with JavaScript's toLowerCase. | |
| 217 | + | func foldKey(key string) string { return jsToLower(key) } | |
| 218 | + | ||
| 219 | + | // normalize maps nil values to Null so that callers can switch on Kind. | |
| 220 | + | func normalize(v Value) Value { | |
| 221 | + | switch x := v.(type) { | |
| 222 | + | case nil: | |
| 223 | + | return Null{} | |
| 224 | + | case *Array: | |
| 225 | + | if x == nil { | |
| 226 | + | return Null{} | |
| 227 | + | } | |
| 228 | + | case *Dictionary: | |
| 229 | + | if x == nil { | |
| 230 | + | return Null{} | |
| 231 | + | } | |
| 232 | + | } | |
| 233 | + | return v | |
| 234 | + | } | |
| 235 | + | ||
| 236 | + | // isPrimitive reports whether v is a Null, Boolean, Number or String. | |
| 237 | + | func isPrimitive(v Value) bool { | |
| 238 | + | switch normalize(v).Kind() { | |
| 239 | + | case KindNull, KindBoolean, KindNumber, KindString: | |
| 240 | + | return true | |
| 241 | + | } | |
| 242 | + | return false | |
| 243 | + | } | |
| 244 | + | ||
| 245 | + | // toNumber converts v to a number the way GitHub does: null is 0, booleans | |
| 246 | + | // are 1 and 0, strings are parsed like JavaScript's Number(), and arrays and | |
| 247 | + | // dictionaries are NaN. | |
| 248 | + | func toNumber(v Value) float64 { | |
| 249 | + | switch x := normalize(v).(type) { | |
| 250 | + | case Null: | |
| 251 | + | return 0 | |
| 252 | + | case Boolean: | |
| 253 | + | if x { | |
| 254 | + | return 1 | |
| 255 | + | } | |
| 256 | + | return 0 | |
| 257 | + | case Number: | |
| 258 | + | return float64(x) | |
| 259 | + | case String: | |
| 260 | + | return jsToNumber(string(x)) | |
| 261 | + | } | |
| 262 | + | return nan | |
| 263 | + | } | |
| 264 | + | ||
| 265 | + | // CoerceString converts v to the string GitHub shows for it: null is "", booleans | |
| 266 | + | // are "true" and "false", numbers are formatted with at most 15 decimals | |
| 267 | + | // ("0.1" for 0.1+0.2) and arrays and dictionaries are "Array" and "Object". | |
| 268 | + | func CoerceString(v Value) string { | |
| 269 | + | switch x := normalize(v).(type) { | |
| 270 | + | case Null: | |
| 271 | + | return "" | |
| 272 | + | case Boolean: | |
| 273 | + | if x { | |
| 274 | + | return "true" | |
| 275 | + | } | |
| 276 | + | return "false" | |
| 277 | + | case Number: | |
| 278 | + | return numberToCoercedString(float64(x)) | |
| 279 | + | case String: | |
| 280 | + | return string(x) | |
| 281 | + | default: | |
| 282 | + | return x.Kind().String() | |
| 283 | + | } | |
| 284 | + | } | |
| 285 | + | ||
| 286 | + | // Truthy reports whether v is true when used as a condition. Null, false, 0, | |
| 287 | + | // NaN and the empty string are falsy; everything else, including every array | |
| 288 | + | // and dictionary, is truthy. | |
| 289 | + | func Truthy(v Value) bool { | |
| 290 | + | switch x := normalize(v).(type) { | |
| 291 | + | case Null: | |
| 292 | + | return false | |
| 293 | + | case Boolean: | |
| 294 | + | return bool(x) | |
| 295 | + | case Number: | |
| 296 | + | return x != 0 && !math.IsNaN(float64(x)) | |
| 297 | + | case String: | |
| 298 | + | return len(x) != 0 | |
| 299 | + | } | |
| 300 | + | return true | |
| 301 | + | } | |
A expr/doc.go +35 -0
| @@ -0,0 +1,35 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | // Regenerate the case-mapping tables (needs node; see gen/casing_dump.mjs). | |
| 4 | + | //go:generate sh -c "node gen/casing_dump.mjs | go run gen/casing.go -o casing_tables.go" | |
| 5 | + | ||
| 6 | + | // Package expr implements GitHub Actions expressions: the language found | |
| 7 | + | // between the braces of ${{ }} in workflow files. | |
| 8 | + | // | |
| 9 | + | // The behaviour is a faithful port of GitHub's own implementation published | |
| 10 | + | // as the npm package @actions/expressions (MIT, actions/languageservices): | |
| 11 | + | // same tokens, precedence, error messages, coercions and built-in functions. | |
| 12 | + | // Where that implementation is believed to differ from the hosted service | |
| 13 | + | // (the C# runner in actions/runner), the Go code follows the JavaScript and | |
| 14 | + | // says so in a comment; PORTING.md lists what remains to be checked. | |
| 15 | + | // | |
| 16 | + | // The entry points are [Parse], which turns the text of one expression into a | |
| 17 | + | // [Node], and [Evaluate], which computes its [Value] against the named | |
| 18 | + | // contexts and functions of an [EvalContext]: | |
| 19 | + | // | |
| 20 | + | // node, err := expr.Parse("github.ref == 'refs/heads/main'", []string{"github"}, nil) | |
| 21 | + | // if err != nil { /* *expr.Error */ } | |
| 22 | + | // v, err := expr.Evaluate(ctx, node, expr.EvalContext{Contexts: map[string]expr.Value{"github": github}}) | |
| 23 | + | // if err != nil { /* *expr.EvalError */ } | |
| 24 | + | // if expr.Truthy(v) { /* ... */ } | |
| 25 | + | // | |
| 26 | + | // Values are the JSON-like [Null], [Boolean], [Number], [String], [*Array] | |
| 27 | + | // and [*Dictionary]. Dictionaries keep insertion order and look keys up | |
| 28 | + | // case-insensitively. [FromJSON] and [ToJSON] convert to and from JSON text | |
| 29 | + | // exactly as the fromJSON and toJSON functions do. | |
| 30 | + | // | |
| 31 | + | // Evaluation is bounded: [Evaluate] honours its context.Context and stops | |
| 32 | + | // when the strings and filtered arrays it builds exceed | |
| 33 | + | // [EvalContext.MaxResultBytes] (default [MaxResultBytes]); the parser limits | |
| 34 | + | // length and nesting ([MaxExpressionLength], [MaxParserDepth]). | |
| 35 | + | package expr | |
A expr/errors.go +107 -0
| @@ -0,0 +1,107 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "fmt" | |
| 7 | + | ) | |
| 8 | + | ||
| 9 | + | // Limits of the reference implementation. | |
| 10 | + | const ( | |
| 11 | + | // MaxParserDepth is the deepest nesting of groupings, function arguments, | |
| 12 | + | // index accesses, negations and logical operators the parser accepts. | |
| 13 | + | MaxParserDepth = 50 | |
| 14 | + | // MaxExpressionLength is the longest expression accepted, counted in | |
| 15 | + | // UTF-16 code units like the JavaScript string length it mirrors. | |
| 16 | + | MaxExpressionLength = 21000 | |
| 17 | + | ) | |
| 18 | + | ||
| 19 | + | // ErrorType classifies an [Error]. | |
| 20 | + | type ErrorType int | |
| 21 | + | ||
| 22 | + | // The error types of the reference implementation, in its order. | |
| 23 | + | const ( | |
| 24 | + | ErrorUnexpectedSymbol ErrorType = iota | |
| 25 | + | // ErrorUnrecognizedNamedValue is declared by the reference but never | |
| 26 | + | // produced; ErrorUnrecognizedContext carries the same text. | |
| 27 | + | ErrorUnrecognizedNamedValue | |
| 28 | + | ErrorUnexpectedEndOfExpression | |
| 29 | + | ErrorExceededMaxDepth | |
| 30 | + | ErrorExceededMaxLength | |
| 31 | + | ErrorTooFewParameters | |
| 32 | + | ErrorTooManyParameters | |
| 33 | + | ErrorEvenParameters | |
| 34 | + | ErrorUnrecognizedContext | |
| 35 | + | ErrorUnrecognizedFunction | |
| 36 | + | ) | |
| 37 | + | ||
| 38 | + | func (t ErrorType) description() string { | |
| 39 | + | switch t { | |
| 40 | + | case ErrorUnexpectedEndOfExpression: | |
| 41 | + | return "Unexpected end of expression" | |
| 42 | + | case ErrorUnexpectedSymbol: | |
| 43 | + | return "Unexpected symbol" | |
| 44 | + | case ErrorUnrecognizedNamedValue: | |
| 45 | + | return "Unrecognized named-value" | |
| 46 | + | case ErrorExceededMaxDepth: | |
| 47 | + | return fmt.Sprintf("Exceeded max expression depth %d", MaxParserDepth) | |
| 48 | + | case ErrorExceededMaxLength: | |
| 49 | + | return fmt.Sprintf("Exceeded max expression length %d", MaxExpressionLength) | |
| 50 | + | case ErrorTooFewParameters: | |
| 51 | + | return "Too few parameters supplied" | |
| 52 | + | case ErrorTooManyParameters: | |
| 53 | + | return "Too many parameters supplied" | |
| 54 | + | case ErrorEvenParameters: | |
| 55 | + | return "Even number of parameters supplied, requires an odd number of parameters" | |
| 56 | + | case ErrorUnrecognizedContext: | |
| 57 | + | return "Unrecognized named-value" | |
| 58 | + | case ErrorUnrecognizedFunction: | |
| 59 | + | return "Unrecognized function" | |
| 60 | + | } | |
| 61 | + | return "Unknown error" | |
| 62 | + | } | |
| 63 | + | ||
| 64 | + | // Position is a zero-based line and column in an expression. Columns count | |
| 65 | + | // UTF-16 code units, as in the reference implementation. | |
| 66 | + | type Position struct { | |
| 67 | + | Line int | |
| 68 | + | Column int | |
| 69 | + | } | |
| 70 | + | ||
| 71 | + | // Error is a lexing or parsing error. Its message is exactly the one the | |
| 72 | + | // reference implementation produces. | |
| 73 | + | type Error struct { | |
| 74 | + | Type ErrorType | |
| 75 | + | // Message is the full text, e.g. "Unexpected symbol: ')'". | |
| 76 | + | Message string | |
| 77 | + | // Pos is where the offending token starts. | |
| 78 | + | Pos Position | |
| 79 | + | // Token is the text of the offending token. | |
| 80 | + | Token string | |
| 81 | + | } | |
| 82 | + | ||
| 83 | + | func (e *Error) Error() string { return e.Message } | |
| 84 | + | ||
| 85 | + | func newTokenError(t ErrorType, tok Token) *Error { | |
| 86 | + | return &Error{ | |
| 87 | + | Type: t, | |
| 88 | + | Message: t.description() + ": '" + tokenString(tok) + "'", | |
| 89 | + | Pos: tok.Range.Start, | |
| 90 | + | Token: tokenString(tok), | |
| 91 | + | } | |
| 92 | + | } | |
| 93 | + | ||
| 94 | + | // EvalError is an error raised while evaluating an expression: a failing | |
| 95 | + | // function, such as fromJSON on invalid input, or an invalid format string. | |
| 96 | + | type EvalError struct { | |
| 97 | + | Message string | |
| 98 | + | // Err is the underlying error, if any. | |
| 99 | + | Err error | |
| 100 | + | } | |
| 101 | + | ||
| 102 | + | func (e *EvalError) Error() string { return e.Message } | |
| 103 | + | func (e *EvalError) Unwrap() error { return e.Err } | |
| 104 | + | ||
| 105 | + | func evalErrorf(format string, a ...any) *EvalError { | |
| 106 | + | return &EvalError{Message: fmt.Sprintf(format, a...)} | |
| 107 | + | } | |
A expr/eval.go +507 -0
| @@ -0,0 +1,507 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "errors" | |
| 8 | + | "fmt" | |
| 9 | + | "math" | |
| 10 | + | "sort" | |
| 11 | + | ) | |
| 12 | + | ||
| 13 | + | // MaxResultBytes is the default budget of an evaluation: the total size, in | |
| 14 | + | // bytes, of the strings that format, join and toJSON build plus a nominal | |
| 15 | + | // cost for every element of the arrays that the * filter builds. The | |
| 16 | + | // reference implementation has no such limit; without one a few kilobytes of | |
| 17 | + | // expression can allocate gigabytes (format('{0}{0}{0}...', format(...))), | |
| 18 | + | // and arrays shared in a context can multiply under .* filters. | |
| 19 | + | const MaxResultBytes = 1 << 20 | |
| 20 | + | ||
| 21 | + | // filterElementCost is what one element of a filtered array costs against the | |
| 22 | + | // budget: the size of an interface value. | |
| 23 | + | const filterElementCost = 16 | |
| 24 | + | ||
| 25 | + | // maxEvalDepth bounds the recursion of the evaluator. The parser never | |
| 26 | + | // produces trees anywhere near it; it protects against hand-built ASTs. | |
| 27 | + | const maxEvalDepth = 1000 | |
| 28 | + | ||
| 29 | + | // Func implements a function callable from expressions. It receives the | |
| 30 | + | // evaluation's context, which it should honour if it blocks or loops, and the | |
| 31 | + | // evaluated arguments in order, and returns the result or an error. | |
| 32 | + | // [Evaluate] wraps the error in an [*EvalError] that unwraps to it. Arity is | |
| 33 | + | // not checked at call time: declare it in a [FuncInfo] given to [Parse]. | |
| 34 | + | type Func func(ctx context.Context, args []Value) (Value, error) | |
| 35 | + | ||
| 36 | + | // EvalContext is the environment an expression is evaluated in. | |
| 37 | + | type EvalContext struct { | |
| 38 | + | // Contexts maps context names ("github", "env", "matrix"...) to their | |
| 39 | + | // values. Names are matched case-insensitively; when several keys differ | |
| 40 | + | // only by case, the first in sorted order wins. A name that is absent | |
| 41 | + | // evaluates to null. | |
| 42 | + | Contexts map[string]Value | |
| 43 | + | // Funcs implements the extra functions declared to [Parse], such as | |
| 44 | + | // hashFiles, success, always, cancelled and failure. Names are matched | |
| 45 | + | // case-insensitively. As in the reference implementation these take | |
| 46 | + | // precedence over the built-in functions. | |
| 47 | + | Funcs map[string]Func | |
| 48 | + | // MaxResultBytes is the output budget of the evaluation, see | |
| 49 | + | // [MaxResultBytes]. Zero means the default; a negative value removes the | |
| 50 | + | // limit. The budget is shared by the whole evaluation, not per call. | |
| 51 | + | MaxResultBytes int | |
| 52 | + | } | |
| 53 | + | ||
| 54 | + | // Evaluate computes the value of a parsed expression. | |
| 55 | + | // | |
| 56 | + | // Evaluation stops with an error that wraps ctx.Err() when ctx is done, and | |
| 57 | + | // with an [*EvalError] when the output budget is exceeded. Values of the | |
| 58 | + | // context are never modified, and results may share them with the context: | |
| 59 | + | // treat the result as read-only. The error is an [*EvalError]. | |
| 60 | + | func Evaluate(ctx context.Context, node Node, ec EvalContext) (Value, error) { | |
| 61 | + | if ctx == nil { | |
| 62 | + | ctx = context.Background() | |
| 63 | + | } | |
| 64 | + | if node == nil { | |
| 65 | + | return nil, &EvalError{Message: "cannot evaluate an empty expression"} | |
| 66 | + | } | |
| 67 | + | if err := ctx.Err(); err != nil { | |
| 68 | + | return nil, cancelError(err) | |
| 69 | + | } | |
| 70 | + | ev := &evaluator{ | |
| 71 | + | ctx: ctx, | |
| 72 | + | contexts: make(map[string]Value, len(ec.Contexts)), | |
| 73 | + | funcs: make(map[string]Func, len(ec.Funcs)), | |
| 74 | + | limit: ec.MaxResultBytes, | |
| 75 | + | } | |
| 76 | + | if ev.limit == 0 { | |
| 77 | + | ev.limit = MaxResultBytes | |
| 78 | + | } | |
| 79 | + | names := make([]string, 0, len(ec.Contexts)) | |
| 80 | + | for k := range ec.Contexts { | |
| 81 | + | names = append(names, k) | |
| 82 | + | } | |
| 83 | + | sort.Strings(names) | |
| 84 | + | for _, k := range names { | |
| 85 | + | if _, dup := ev.contexts[foldKey(k)]; !dup { | |
| 86 | + | ev.contexts[foldKey(k)] = ec.Contexts[k] | |
| 87 | + | } | |
| 88 | + | } | |
| 89 | + | names = names[:0] | |
| 90 | + | for k := range ec.Funcs { | |
| 91 | + | names = append(names, k) | |
| 92 | + | } | |
| 93 | + | sort.Strings(names) | |
| 94 | + | for _, k := range names { | |
| 95 | + | if _, dup := ev.funcs[foldKey(k)]; !dup { | |
| 96 | + | ev.funcs[foldKey(k)] = ec.Funcs[k] | |
| 97 | + | } | |
| 98 | + | } | |
| 99 | + | v, err := ev.eval(node) | |
| 100 | + | if err != nil { | |
| 101 | + | return nil, err | |
| 102 | + | } | |
| 103 | + | return normalize(v), nil | |
| 104 | + | } | |
| 105 | + | ||
| 106 | + | func cancelError(err error) *EvalError { | |
| 107 | + | return &EvalError{Message: "evaluation canceled: " + err.Error(), Err: err} | |
| 108 | + | } | |
| 109 | + | ||
| 110 | + | type evaluator struct { | |
| 111 | + | ctx context.Context | |
| 112 | + | contexts map[string]Value | |
| 113 | + | funcs map[string]Func | |
| 114 | + | limit int // output budget; negative: unlimited | |
| 115 | + | used int | |
| 116 | + | depth int | |
| 117 | + | ticks int | |
| 118 | + | } | |
| 119 | + | ||
| 120 | + | // tick is called in loops and on every node: it checks the context once in a | |
| 121 | + | // while. | |
| 122 | + | func (e *evaluator) tick() error { | |
| 123 | + | e.ticks++ | |
| 124 | + | if e.ticks&63 == 0 { | |
| 125 | + | if err := e.ctx.Err(); err != nil { | |
| 126 | + | return cancelError(err) | |
| 127 | + | } | |
| 128 | + | } | |
| 129 | + | return nil | |
| 130 | + | } | |
| 131 | + | ||
| 132 | + | // charge spends n bytes of the output budget. | |
| 133 | + | func (e *evaluator) charge(n int) error { | |
| 134 | + | if e.limit < 0 { | |
| 135 | + | return nil | |
| 136 | + | } | |
| 137 | + | if n > e.limit-e.used { // written to be overflow-proof | |
| 138 | + | return e.budgetError() | |
| 139 | + | } | |
| 140 | + | e.used += n | |
| 141 | + | return nil | |
| 142 | + | } | |
| 143 | + | ||
| 144 | + | func (e *evaluator) budgetError() *EvalError { | |
| 145 | + | return &EvalError{Message: fmt.Sprintf("expression output exceeds the limit of %d bytes", e.limit)} | |
| 146 | + | } | |
| 147 | + | ||
| 148 | + | func (e *evaluator) eval(n Node) (Value, error) { | |
| 149 | + | if err := e.tick(); err != nil { | |
| 150 | + | return nil, err | |
| 151 | + | } | |
| 152 | + | e.depth++ | |
| 153 | + | defer func() { e.depth-- }() | |
| 154 | + | if e.depth > maxEvalDepth { | |
| 155 | + | return nil, &EvalError{Message: fmt.Sprintf("expression tree deeper than %d levels", maxEvalDepth)} | |
| 156 | + | } | |
| 157 | + | switch n := n.(type) { | |
| 158 | + | case *Literal: | |
| 159 | + | return normalize(n.Value), nil | |
| 160 | + | case *Unary: | |
| 161 | + | r, err := e.eval(n.Expr) | |
| 162 | + | if err != nil { | |
| 163 | + | return nil, err | |
| 164 | + | } | |
| 165 | + | return Boolean(!Truthy(r)), nil | |
| 166 | + | case *Binary: | |
| 167 | + | return e.binary(n) | |
| 168 | + | case *Logical: | |
| 169 | + | return e.logical(n) | |
| 170 | + | case *Grouping: | |
| 171 | + | return e.eval(n.Group) | |
| 172 | + | case *ContextAccess: | |
| 173 | + | if v, ok := e.contexts[foldKey(n.Name.Lexeme)]; ok { | |
| 174 | + | return normalize(v), nil | |
| 175 | + | } | |
| 176 | + | return Null{}, nil | |
| 177 | + | case *IndexAccess: | |
| 178 | + | return e.indexAccess(n) | |
| 179 | + | case *FunctionCall: | |
| 180 | + | return e.call(n) | |
| 181 | + | } | |
| 182 | + | // *Star is only valid as an index and never evaluated on its own; the | |
| 183 | + | // reference throws "Method not implemented." for it. | |
| 184 | + | return nil, &EvalError{Message: "Method not implemented."} | |
| 185 | + | } | |
| 186 | + | ||
| 187 | + | func (e *evaluator) binary(b *Binary) (Value, error) { | |
| 188 | + | // Both operands are always evaluated: comparisons do not short-circuit. | |
| 189 | + | left, err := e.eval(b.Left) | |
| 190 | + | if err != nil { | |
| 191 | + | return nil, err | |
| 192 | + | } | |
| 193 | + | right, err := e.eval(b.Right) | |
| 194 | + | if err != nil { | |
| 195 | + | return nil, err | |
| 196 | + | } | |
| 197 | + | switch b.Operator.Type { | |
| 198 | + | case TokenEqualEqual: | |
| 199 | + | return Boolean(equals(left, right)), nil | |
| 200 | + | case TokenBangEqual: | |
| 201 | + | return Boolean(!equals(left, right)), nil | |
| 202 | + | case TokenGreater: | |
| 203 | + | return Boolean(greaterThan(left, right)), nil | |
| 204 | + | case TokenGreaterEqual: | |
| 205 | + | return Boolean(equals(left, right) || greaterThan(left, right)), nil | |
| 206 | + | case TokenLess: | |
| 207 | + | return Boolean(lessThan(left, right)), nil | |
| 208 | + | case TokenLessEqual: | |
| 209 | + | return Boolean(equals(left, right) || lessThan(left, right)), nil | |
| 210 | + | } | |
| 211 | + | return nil, evalErrorf("unknown binary operator: %s", b.Operator.Lexeme) | |
| 212 | + | } | |
| 213 | + | ||
| 214 | + | // logical returns the operand that decided the chain, not a boolean: | |
| 215 | + | // "a || b" is a when a is truthy, else b; "a && b" is a when a is falsy, else b. | |
| 216 | + | func (e *evaluator) logical(l *Logical) (Value, error) { | |
| 217 | + | var result Value | |
| 218 | + | for _, arg := range l.Args { | |
| 219 | + | r, err := e.eval(arg) | |
| 220 | + | if err != nil { | |
| 221 | + | return nil, err | |
| 222 | + | } | |
| 223 | + | result = r | |
| 224 | + | if (l.Operator.Type == TokenAnd && !Truthy(r)) || (l.Operator.Type == TokenOr && Truthy(r)) { | |
| 225 | + | break | |
| 226 | + | } | |
| 227 | + | } | |
| 228 | + | if result == nil { | |
| 229 | + | return Null{}, nil // a Logical without operands, possible only in a hand-built tree | |
| 230 | + | } | |
| 231 | + | return result, nil | |
| 232 | + | } | |
| 233 | + | ||
| 234 | + | func (e *evaluator) call(c *FunctionCall) (Value, error) { | |
| 235 | + | name := foldKey(c.Name.Lexeme) | |
| 236 | + | ext, isExt := e.funcs[name] | |
| 237 | + | isExt = isExt && ext != nil | |
| 238 | + | var impl builtinFunc | |
| 239 | + | if !isExt { | |
| 240 | + | // Arity of the built-ins is checked by the parser, and again here | |
| 241 | + | // so that a hand-built tree cannot reach an implementation with the | |
| 242 | + | // wrong number of arguments. | |
| 243 | + | f, ok := builtinFuncs[name] | |
| 244 | + | if !ok { | |
| 245 | + | return nil, evalErrorf("function %s is not implemented", c.Name.Lexeme) | |
| 246 | + | } | |
| 247 | + | if t, bad := arityError(name, f, len(c.Args)); bad { | |
| 248 | + | return nil, &EvalError{Message: t.description() + ": '" + c.Name.Lexeme + "'"} | |
| 249 | + | } | |
| 250 | + | impl = builtinImpls[name] | |
| 251 | + | } | |
| 252 | + | args := make([]Value, len(c.Args)) | |
| 253 | + | for i, a := range c.Args { | |
| 254 | + | v, err := e.eval(a) | |
| 255 | + | if err != nil { | |
| 256 | + | return nil, err | |
| 257 | + | } | |
| 258 | + | args[i] = v | |
| 259 | + | } | |
| 260 | + | if isExt { | |
| 261 | + | v, err := ext(e.ctx, args) | |
| 262 | + | if err != nil { | |
| 263 | + | var ee *EvalError | |
| 264 | + | if errors.As(err, &ee) { | |
| 265 | + | return nil, err | |
| 266 | + | } | |
| 267 | + | return nil, &EvalError{Message: err.Error(), Err: err} | |
| 268 | + | } | |
| 269 | + | return v, nil | |
| 270 | + | } | |
| 271 | + | return impl(e, args) | |
| 272 | + | } | |
| 273 | + | ||
| 274 | + | // indexer is idxHelper of the reference: how an index value is read, as a | |
| 275 | + | // star, as a string key and as an array position. | |
| 276 | + | type indexer struct { | |
| 277 | + | star bool | |
| 278 | + | hasStr bool | |
| 279 | + | str string | |
| 280 | + | hasInt bool | |
| 281 | + | // intv is only meaningful when hasInt; it is a float because positions | |
| 282 | + | // beyond the int range must simply compare as out of bounds. | |
| 283 | + | intv float64 | |
| 284 | + | } | |
| 285 | + | ||
| 286 | + | func newIndexer(star bool, v Value) indexer { | |
| 287 | + | ix := indexer{star: star} | |
| 288 | + | if star || v == nil { | |
| 289 | + | return ix | |
| 290 | + | } | |
| 291 | + | v = normalize(v) | |
| 292 | + | if isPrimitive(v) { | |
| 293 | + | ix.hasStr = true | |
| 294 | + | ix.str = CoerceString(v) | |
| 295 | + | } | |
| 296 | + | f := toNumber(v) | |
| 297 | + | if !math.IsNaN(f) && !math.IsInf(f, 0) && f >= 0 { | |
| 298 | + | ix.hasInt = true | |
| 299 | + | ix.intv = math.Floor(f) | |
| 300 | + | } | |
| 301 | + | return ix | |
| 302 | + | } | |
| 303 | + | ||
| 304 | + | // pos returns the position as an int when it is within [0, n). | |
| 305 | + | func (ix indexer) pos(n int) (int, bool) { | |
| 306 | + | if !ix.hasInt || ix.intv >= float64(n) { | |
| 307 | + | return 0, false | |
| 308 | + | } | |
| 309 | + | return int(ix.intv), true | |
| 310 | + | } | |
| 311 | + | ||
| 312 | + | func (e *evaluator) indexAccess(ia *IndexAccess) (Value, error) { | |
| 313 | + | var ix indexer | |
| 314 | + | if _, isStar := ia.Index.(*Star); isStar { | |
| 315 | + | ix = newIndexer(true, nil) | |
| 316 | + | } else { | |
| 317 | + | v, err := e.eval(ia.Index) | |
| 318 | + | if err != nil { | |
| 319 | + | return nil, &EvalError{Message: "could not evaluate index for index access: " + err.Error(), Err: err} | |
| 320 | + | } | |
| 321 | + | ix = newIndexer(false, v) | |
| 322 | + | } | |
| 323 | + | obj, err := e.eval(ia.Expr) | |
| 324 | + | if err != nil { | |
| 325 | + | return nil, err | |
| 326 | + | } | |
| 327 | + | switch o := normalize(obj).(type) { | |
| 328 | + | case *Array: | |
| 329 | + | if o.filtered { | |
| 330 | + | return e.filteredArrayAccess(o, ix) | |
| 331 | + | } | |
| 332 | + | return e.arrayAccess(o, ix) | |
| 333 | + | case *Dictionary: | |
| 334 | + | return e.objectAccess(o, ix) | |
| 335 | + | } | |
| 336 | + | if ix.star { | |
| 337 | + | return &Array{filtered: true}, nil | |
| 338 | + | } | |
| 339 | + | return Null{}, nil | |
| 340 | + | } | |
| 341 | + | ||
| 342 | + | // filteredArrayAccess applies an index to each element of a filtered array | |
| 343 | + | // and collects the hits: "a.*.b" is the b of every element that has one. | |
| 344 | + | func (e *evaluator) filteredArrayAccess(fa *Array, ix indexer) (Value, error) { | |
| 345 | + | result := &Array{filtered: true} | |
| 346 | + | for _, item := range fa.items { | |
| 347 | + | if err := e.tick(); err != nil { | |
| 348 | + | return nil, err | |
| 349 | + | } | |
| 350 | + | switch it := normalize(item).(type) { | |
| 351 | + | case *Dictionary: | |
| 352 | + | if ix.star { | |
| 353 | + | if err := e.charge(len(it.vals) * filterElementCost); err != nil { | |
| 354 | + | return nil, err | |
| 355 | + | } | |
| 356 | + | result.items = append(result.items, it.vals...) | |
| 357 | + | } else if ix.hasStr { | |
| 358 | + | if v, ok := it.Get(ix.str); ok { | |
| 359 | + | if err := e.charge(filterElementCost); err != nil { | |
| 360 | + | return nil, err | |
| 361 | + | } | |
| 362 | + | result.items = append(result.items, v) | |
| 363 | + | } | |
| 364 | + | } | |
| 365 | + | case *Array: | |
| 366 | + | if ix.star { | |
| 367 | + | if err := e.charge(len(it.items) * filterElementCost); err != nil { | |
| 368 | + | return nil, err | |
| 369 | + | } | |
| 370 | + | result.items = append(result.items, it.items...) | |
| 371 | + | } else if i, ok := ix.pos(len(it.items)); ok { | |
| 372 | + | if err := e.charge(filterElementCost); err != nil { | |
| 373 | + | return nil, err | |
| 374 | + | } | |
| 375 | + | result.items = append(result.items, it.items[i]) | |
| 376 | + | } | |
| 377 | + | } | |
| 378 | + | } | |
| 379 | + | return result, nil | |
| 380 | + | } | |
| 381 | + | ||
| 382 | + | func (e *evaluator) arrayAccess(a *Array, ix indexer) (Value, error) { | |
| 383 | + | if ix.star { | |
| 384 | + | if err := e.charge(len(a.items) * filterElementCost); err != nil { | |
| 385 | + | return nil, err | |
| 386 | + | } | |
| 387 | + | return &Array{items: append([]Value(nil), a.items...), filtered: true}, nil | |
| 388 | + | } | |
| 389 | + | if i, ok := ix.pos(len(a.items)); ok { | |
| 390 | + | return normalize(a.items[i]), nil | |
| 391 | + | } | |
| 392 | + | return Null{}, nil | |
| 393 | + | } | |
| 394 | + | ||
| 395 | + | func (e *evaluator) objectAccess(d *Dictionary, ix indexer) (Value, error) { | |
| 396 | + | if ix.star { | |
| 397 | + | if err := e.charge(len(d.vals) * filterElementCost); err != nil { | |
| 398 | + | return nil, err | |
| 399 | + | } | |
| 400 | + | return &Array{items: append([]Value(nil), d.vals...), filtered: true}, nil | |
| 401 | + | } | |
| 402 | + | if ix.hasStr { | |
| 403 | + | if v, ok := d.Get(ix.str); ok { | |
| 404 | + | return v, nil | |
| 405 | + | } | |
| 406 | + | } | |
| 407 | + | return Null{}, nil | |
| 408 | + | } | |
| 409 | + | ||
| 410 | + | // coerceTypes converts two values of different kinds to a common kind the | |
| 411 | + | // way the reference does, which is like the JavaScript abstract equality | |
| 412 | + | // algorithm except that arrays and dictionaries are never converted: null | |
| 413 | + | // and booleans become numbers, and a string compared with a number becomes | |
| 414 | + | // a number. | |
| 415 | + | func coerceTypes(l, r Value) (Value, Value) { | |
| 416 | + | l, r = normalize(l), normalize(r) | |
| 417 | + | if l.Kind() == r.Kind() { | |
| 418 | + | return l, r | |
| 419 | + | } | |
| 420 | + | switch l.Kind() { | |
| 421 | + | case KindNumber: | |
| 422 | + | if r.Kind() == KindString { | |
| 423 | + | return l, Number(toNumber(r)) | |
| 424 | + | } | |
| 425 | + | case KindString: | |
| 426 | + | if r.Kind() == KindNumber { | |
| 427 | + | return Number(toNumber(l)), r | |
| 428 | + | } | |
| 429 | + | case KindNull, KindBoolean: | |
| 430 | + | return coerceTypes(Number(toNumber(l)), r) | |
| 431 | + | } | |
| 432 | + | switch r.Kind() { | |
| 433 | + | case KindNull, KindBoolean: | |
| 434 | + | return coerceTypes(l, Number(toNumber(r))) | |
| 435 | + | } | |
| 436 | + | return l, r | |
| 437 | + | } | |
| 438 | + | ||
| 439 | + | // equals is the == operator: loose equality, case-insensitive for strings | |
| 440 | + | // (see toUpperSpecial), false for NaN, and by identity for arrays and | |
| 441 | + | // dictionaries. | |
| 442 | + | func equals(lhs, rhs Value) bool { | |
| 443 | + | l, r := coerceTypes(lhs, rhs) | |
| 444 | + | if l.Kind() != r.Kind() { | |
| 445 | + | return false | |
| 446 | + | } | |
| 447 | + | switch lv := l.(type) { | |
| 448 | + | case Null: | |
| 449 | + | return true | |
| 450 | + | case Number: | |
| 451 | + | return float64(lv) == float64(r.(Number)) // false when either is NaN | |
| 452 | + | case String: | |
| 453 | + | return toUpperSpecial(string(lv)) == toUpperSpecial(string(r.(String))) | |
| 454 | + | case Boolean: | |
| 455 | + | return lv == r.(Boolean) | |
| 456 | + | case *Array: | |
| 457 | + | return lv == r.(*Array) | |
| 458 | + | case *Dictionary: | |
| 459 | + | return lv == r.(*Dictionary) | |
| 460 | + | } | |
| 461 | + | return false | |
| 462 | + | } | |
| 463 | + | ||
| 464 | + | // compare implements < and >: it returns -1, 0 or 1, and ok false when the | |
| 465 | + | // values are not ordered (different kinds after coercion, NaN, null, arrays, | |
| 466 | + | // dictionaries). | |
| 467 | + | func compare(lhs, rhs Value) (c int, ok bool) { | |
| 468 | + | l, r := coerceTypes(lhs, rhs) | |
| 469 | + | if l.Kind() != r.Kind() { | |
| 470 | + | return 0, false | |
| 471 | + | } | |
| 472 | + | switch lv := l.(type) { | |
| 473 | + | case Number: | |
| 474 | + | lf, rf := float64(lv), float64(r.(Number)) | |
| 475 | + | switch { | |
| 476 | + | case math.IsNaN(lf) || math.IsNaN(rf): | |
| 477 | + | return 0, false | |
| 478 | + | case lf < rf: | |
| 479 | + | return -1, true | |
| 480 | + | case lf > rf: | |
| 481 | + | return 1, true | |
| 482 | + | } | |
| 483 | + | return 0, true | |
| 484 | + | case String: | |
| 485 | + | return compareUTF16(toUpperSpecial(string(lv)), toUpperSpecial(string(r.(String)))), true | |
| 486 | + | case Boolean: | |
| 487 | + | // false < true | |
| 488 | + | switch rb := r.(Boolean); { | |
| 489 | + | case lv == rb: | |
| 490 | + | return 0, true | |
| 491 | + | case !bool(lv): | |
| 492 | + | return -1, true | |
| 493 | + | } | |
| 494 | + | return 1, true | |
| 495 | + | } | |
| 496 | + | return 0, false | |
| 497 | + | } | |
| 498 | + | ||
| 499 | + | func greaterThan(l, r Value) bool { | |
| 500 | + | c, ok := compare(l, r) | |
| 501 | + | return ok && c > 0 | |
| 502 | + | } | |
| 503 | + | ||
| 504 | + | func lessThan(l, r Value) bool { | |
| 505 | + | c, ok := compare(l, r) | |
| 506 | + | return ok && c < 0 | |
| 507 | + | } | |
A expr/eval_test.go +952 -0
| @@ -0,0 +1,952 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "errors" | |
| 8 | + | "math" | |
| 9 | + | "strings" | |
| 10 | + | "testing" | |
| 11 | + | ) | |
| 12 | + | ||
| 13 | + | type evalCase struct { | |
| 14 | + | expr string | |
| 15 | + | want string // "Kind:string", "PARSE message" or "ERR message" | |
| 16 | + | } | |
| 17 | + | ||
| 18 | + | func runEvalCases(t *testing.T, cases []evalCase) { | |
| 19 | + | t.Helper() | |
| 20 | + | for _, c := range cases { | |
| 21 | + | t.Run(c.expr, func(t *testing.T) { | |
| 22 | + | if got := evalString(t, c.expr); got != c.want { | |
| 23 | + | t.Errorf("%s\n got: %q\nwant: %q", c.expr, got, c.want) | |
| 24 | + | } | |
| 25 | + | }) | |
| 26 | + | } | |
| 27 | + | } | |
| 28 | + | ||
| 29 | + | // Examples from the GitHub documentation of expressions. | |
| 30 | + | func TestEvalDocumentationExamples(t *testing.T) { | |
| 31 | + | runEvalCases(t, []evalCase{ | |
| 32 | + | {"contains('Hello world', 'llo')", "Boolean:true"}, | |
| 33 | + | {"contains('Hello world', 'LLO')", "Boolean:true"}, | |
| 34 | + | {"contains(github.event.issue.labels.*.name, 'bug')", "Boolean:true"}, | |
| 35 | + | {"contains(github.event.issue.labels.*.name, 'BUG')", "Boolean:true"}, | |
| 36 | + | {"contains(github.event.issue.labels.*.name, 'nope')", "Boolean:false"}, | |
| 37 | + | {"startsWith('Hello world', 'He')", "Boolean:true"}, | |
| 38 | + | {"startsWith('Hello world', 'he')", "Boolean:true"}, | |
| 39 | + | {"startsWith('Hello world', 'world')", "Boolean:false"}, | |
| 40 | + | {"endsWith('Hello world', 'ld')", "Boolean:true"}, | |
| 41 | + | {"endsWith('Hello world', 'Hello')", "Boolean:false"}, | |
| 42 | + | {"format('Hello {0} {1} {2}', 'Mona', 'the', 'Octocat')", "String:Hello Mona the Octocat"}, | |
| 43 | + | {"format('{{Hello {0} {1} {2}!}}', 'Mona', 'the', 'Octocat')", "String:{Hello Mona the Octocat!}"}, | |
| 44 | + | {"join(github.event.issue.labels.*.name, ', ')", "String:bug, Docs"}, | |
| 45 | + | {"join(github.event.issue.labels.*.name)", "String:bug,Docs"}, | |
| 46 | + | {"toJSON(fromJSON('{\"a\":1}'))", "String:{\n \"a\": 1\n}"}, | |
| 47 | + | {"fromJSON('{\"a\":[1,2]}').a[1]", "Number:2"}, | |
| 48 | + | {"fromJSON('true')", "Boolean:true"}, | |
| 49 | + | {"fromJSON('\"str\"')", "String:str"}, | |
| 50 | + | {"fromJSON('null')", "Null:"}, | |
| 51 | + | // The documented idiom for matrix from JSON and for converting a | |
| 52 | + | // string to a number: fromJSON('6') is a number. | |
| 53 | + | {"fromJSON('6') == 6", "Boolean:true"}, | |
| 54 | + | {"github.event_name == 'push' && contains(github.ref, 'main')", "Boolean:true"}, | |
| 55 | + | {"github.event_name == 'push' || false", "Boolean:true"}, | |
| 56 | + | {"!(github.event_name == 'push')", "Boolean:false"}, | |
| 57 | + | // Documented ternary idiom: cond && a || b. | |
| 58 | + | {"github.event_name == 'push' && 'yes' || 'no'", "String:yes"}, | |
| 59 | + | {"github.event_name == 'pull_request' && 'yes' || 'no'", "String:no"}, | |
| 60 | + | // A missing property is null, so the default after || applies. | |
| 61 | + | {"github.event.nope || 'default'", "String:default"}, | |
| 62 | + | {"env.MISSING || 'fallback'", "String:fallback"}, | |
| 63 | + | {"github.event.commits.*.id", "Array:Array"}, | |
| 64 | + | {"join(github.event.commits.*.id, '-')", "String:1-2-3"}, | |
| 65 | + | }) | |
| 66 | + | } | |
| 67 | + | ||
| 68 | + | func TestEvalLiteralsAndOperators(t *testing.T) { | |
| 69 | + | runEvalCases(t, []evalCase{ | |
| 70 | + | {"true", "Boolean:true"}, | |
| 71 | + | {"false", "Boolean:false"}, | |
| 72 | + | {"null", "Null:"}, | |
| 73 | + | {"1", "Number:1"}, | |
| 74 | + | {"-1", "Number:-1"}, | |
| 75 | + | {"+1", "Number:1"}, | |
| 76 | + | {"1.5", "Number:1.5"}, | |
| 77 | + | {".5", "Number:0.5"}, | |
| 78 | + | {"-.5", "Number:-0.5"}, | |
| 79 | + | {"5.", "Number:5"}, | |
| 80 | + | {"1e3", "Number:1000"}, | |
| 81 | + | {"0x10", "Number:16"}, | |
| 82 | + | {"0o17", "Number:15"}, | |
| 83 | + | {"0b101", "Number:5"}, | |
| 84 | + | {"-0x10", "PARSE Unexpected symbol: '-0x10'. Located at position 1 within expression: -0x10"}, | |
| 85 | + | {"NaN", "Number:NaN"}, | |
| 86 | + | {"Infinity", "Number:Infinity"}, | |
| 87 | + | {"-Infinity", "Number:-Infinity"}, | |
| 88 | + | {"'a'", "String:a"}, | |
| 89 | + | {"''", "String:"}, | |
| 90 | + | {"'it''s'", "String:it's"}, | |
| 91 | + | // Only the first doubled quote is unescaped: the reference's | |
| 92 | + | // value.replace("''", "'") is not global. | |
| 93 | + | {"'a''b''c'", "String:a'b''c"}, | |
| 94 | + | {"''''", "String:'"}, | |
| 95 | + | {"'a''''b'", "String:a'''b"}, | |
| 96 | + | ||
| 97 | + | {"1 == 1", "Boolean:true"}, | |
| 98 | + | {"1 == 2", "Boolean:false"}, | |
| 99 | + | {"1 != 2", "Boolean:true"}, | |
| 100 | + | {"1 < 2", "Boolean:true"}, | |
| 101 | + | {"2 <= 2", "Boolean:true"}, | |
| 102 | + | {"2 > 3", "Boolean:false"}, | |
| 103 | + | {"3 >= 3", "Boolean:true"}, | |
| 104 | + | ||
| 105 | + | // Loose equality: strings compare to numbers as numbers, null and | |
| 106 | + | // booleans become numbers. | |
| 107 | + | {"1 == '1'", "Boolean:true"}, | |
| 108 | + | {"'1' == 1", "Boolean:true"}, | |
| 109 | + | {"'1.0' == 1", "Boolean:true"}, | |
| 110 | + | {"' 1 ' == 1", "Boolean:true"}, | |
| 111 | + | {"'0x10' == 16", "Boolean:true"}, | |
| 112 | + | {"'abc' == 0", "Boolean:false"}, | |
| 113 | + | {"'' == 0", "Boolean:true"}, | |
| 114 | + | {"'' == null", "Boolean:true"}, // both become 0 | |
| 115 | + | {"null == 0", "Boolean:true"}, | |
| 116 | + | {"null == false", "Boolean:true"}, | |
| 117 | + | {"null == ''", "Boolean:true"}, | |
| 118 | + | {"null == null", "Boolean:true"}, | |
| 119 | + | {"true == 1", "Boolean:true"}, | |
| 120 | + | {"true == '1'", "Boolean:true"}, | |
| 121 | + | {"true == 'true'", "Boolean:false"}, | |
| 122 | + | {"false == 0", "Boolean:true"}, | |
| 123 | + | {"false == ''", "Boolean:true"}, | |
| 124 | + | {"true == true", "Boolean:true"}, | |
| 125 | + | {"true != false", "Boolean:true"}, | |
| 126 | + | {"matrix.nothing == null", "Boolean:true"}, | |
| 127 | + | {"env.EMPTY == null", "Boolean:true"}, | |
| 128 | + | {"env.EMPTY == false", "Boolean:true"}, | |
| 129 | + | ||
| 130 | + | // NaN is never equal, never ordered. | |
| 131 | + | {"NaN == NaN", "Boolean:false"}, | |
| 132 | + | {"NaN != NaN", "Boolean:true"}, | |
| 133 | + | {"NaN < 1", "Boolean:false"}, | |
| 134 | + | {"NaN > 1", "Boolean:false"}, | |
| 135 | + | {"NaN <= NaN", "Boolean:false"}, | |
| 136 | + | {"NaN >= NaN", "Boolean:false"}, | |
| 137 | + | {"'abc' > 1", "Boolean:false"}, | |
| 138 | + | {"'abc' < 1", "Boolean:false"}, | |
| 139 | + | {"'abc' != 1", "Boolean:true"}, | |
| 140 | + | {"Infinity == Infinity", "Boolean:true"}, | |
| 141 | + | {"-Infinity < Infinity", "Boolean:true"}, | |
| 142 | + | ||
| 143 | + | // Strings compare case-insensitively and by upper-cased value. | |
| 144 | + | {"'a' == 'A'", "Boolean:true"}, | |
| 145 | + | {"'abc' == 'ABC'", "Boolean:true"}, | |
| 146 | + | {"'a' < 'B'", "Boolean:true"}, | |
| 147 | + | {"'B' > 'a'", "Boolean:true"}, | |
| 148 | + | {"'a' < 'A'", "Boolean:false"}, | |
| 149 | + | {"'a' <= 'A'", "Boolean:true"}, | |
| 150 | + | {"'Z' < 'a'", "Boolean:false"}, | |
| 151 | + | {"'_' < 'a'", "Boolean:false"}, // '_' (0x5F) sorts after upper-case letters | |
| 152 | + | {"'ß' == 'SS'", "Boolean:true"}, | |
| 153 | + | {"'ß' == 'ss'", "Boolean:true"}, | |
| 154 | + | {"'ı' == 'I'", "Boolean:false"}, // dotless i is not upper-cased | |
| 155 | + | {"'ı' == 'i'", "Boolean:false"}, | |
| 156 | + | {"'ı' == 'ı'", "Boolean:true"}, | |
| 157 | + | {"'İ' == 'i'", "Boolean:false"}, | |
| 158 | + | {"'σ' == 'ς'", "Boolean:true"}, | |
| 159 | + | {"'dž' == 'DŽ'", "Boolean:true"}, | |
| 160 | + | ||
| 161 | + | // Booleans order false < true; null does not order (it becomes 0 | |
| 162 | + | // against a number but is only equal to itself). | |
| 163 | + | {"false < true", "Boolean:true"}, | |
| 164 | + | {"true > false", "Boolean:true"}, | |
| 165 | + | {"true < false", "Boolean:false"}, | |
| 166 | + | {"true >= true", "Boolean:true"}, | |
| 167 | + | {"null < 1", "Boolean:true"}, | |
| 168 | + | {"null < null", "Boolean:false"}, | |
| 169 | + | {"null <= null", "Boolean:true"}, | |
| 170 | + | {"null >= null", "Boolean:true"}, | |
| 171 | + | {"true > 0", "Boolean:true"}, | |
| 172 | + | ||
| 173 | + | // Arrays and dictionaries are equal only by identity and never ordered. | |
| 174 | + | {"matrix.node == matrix.node", "Boolean:true"}, | |
| 175 | + | {"matrix.node != matrix.node", "Boolean:false"}, | |
| 176 | + | {"matrix.node == fromJSON('[14,16,18]')", "Boolean:false"}, | |
| 177 | + | {"matrix.obj == matrix.obj", "Boolean:true"}, | |
| 178 | + | {"matrix.obj == fromJSON('{\"k\":\"v\"}')", "Boolean:false"}, | |
| 179 | + | {"matrix.node == 1", "Boolean:false"}, | |
| 180 | + | {"matrix.node < matrix.node", "Boolean:false"}, | |
| 181 | + | {"matrix.node <= matrix.node", "Boolean:true"}, | |
| 182 | + | {"matrix.node == null", "Boolean:false"}, | |
| 183 | + | {"matrix.obj == true", "Boolean:false"}, | |
| 184 | + | {"matrix.obj == 'Object'", "Boolean:false"}, | |
| 185 | + | // A filtered array is a new array each time. | |
| 186 | + | {"github.event.commits.* == github.event.commits.*", "Boolean:false"}, | |
| 187 | + | }) | |
| 188 | + | } | |
| 189 | + | ||
| 190 | + | func TestEvalLogicalReturnsOperands(t *testing.T) { | |
| 191 | + | runEvalCases(t, []evalCase{ | |
| 192 | + | {"1 || 2", "Number:1"}, | |
| 193 | + | {"0 || 2", "Number:2"}, | |
| 194 | + | {"0 || ''", "String:"}, | |
| 195 | + | {"0 || '' || null", "Null:"}, | |
| 196 | + | {"0 || '' || 'x'", "String:x"}, | |
| 197 | + | {"null || null", "Null:"}, | |
| 198 | + | {"1 && 2", "Number:2"}, | |
| 199 | + | {"0 && 2", "Number:0"}, | |
| 200 | + | {"1 && 0 && 3", "Number:0"}, | |
| 201 | + | {"1 && 2 && 3", "Number:3"}, | |
| 202 | + | {"'a' && 'b' || 'c'", "String:b"}, | |
| 203 | + | {"'' && 'b' || 'c'", "String:c"}, | |
| 204 | + | {"NaN || 'x'", "String:x"}, | |
| 205 | + | {"NaN && 'x'", "Number:NaN"}, | |
| 206 | + | {"'0' || 'x'", "String:0"}, // the string "0" is truthy | |
| 207 | + | {"matrix.node || 'x'", "Array:Array"}, | |
| 208 | + | {"matrix.obj && 'x'", "String:x"}, | |
| 209 | + | {"fromJSON('[]') && 'x'", "String:x"}, | |
| 210 | + | {"fromJSON('{}') || 'x'", "Object:Object"}, | |
| 211 | + | // Short-circuit: the failing call is never evaluated. | |
| 212 | + | {"true || fromJSON('')", "Boolean:true"}, | |
| 213 | + | {"false && fromJSON('')", "Boolean:false"}, | |
| 214 | + | {"false || fromJSON('')", "ERR empty input"}, | |
| 215 | + | {"true && fromJSON('')", "ERR empty input"}, | |
| 216 | + | // Comparisons evaluate both sides. | |
| 217 | + | {"false == fromJSON('')", "ERR empty input"}, | |
| 218 | + | {"fromJSON('') == false", "ERR empty input"}, | |
| 219 | + | // Precedence: && binds tighter than ||, ! tighter than comparison. | |
| 220 | + | {"true || false && false", "Boolean:true"}, | |
| 221 | + | {"(true || false) && false", "Boolean:false"}, | |
| 222 | + | {"false && true || true", "Boolean:true"}, | |
| 223 | + | {"1 == 1 == true", "Boolean:true"}, | |
| 224 | + | {"1 < 2 == true", "Boolean:true"}, | |
| 225 | + | {"true == 1 < 2", "Boolean:true"}, | |
| 226 | + | {"!1 == 2", "Boolean:false"}, | |
| 227 | + | {"!(1 == 2)", "Boolean:true"}, | |
| 228 | + | {"!!1", "Boolean:true"}, | |
| 229 | + | {"!0", "Boolean:true"}, | |
| 230 | + | {"!''", "Boolean:true"}, | |
| 231 | + | {"!'0'", "Boolean:false"}, | |
| 232 | + | {"!null", "Boolean:true"}, | |
| 233 | + | {"!NaN", "Boolean:true"}, | |
| 234 | + | {"!matrix.node", "Boolean:false"}, | |
| 235 | + | {"!matrix.obj", "Boolean:false"}, | |
| 236 | + | {"!github.event.commits.*", "Boolean:false"}, | |
| 237 | + | }) | |
| 238 | + | } | |
| 239 | + | ||
| 240 | + | func TestEvalAccess(t *testing.T) { | |
| 241 | + | runEvalCases(t, []evalCase{ | |
| 242 | + | {"github.event_name", "String:push"}, | |
| 243 | + | {"GITHUB.EVENT_NAME", "String:push"}, | |
| 244 | + | {"GitHub.Event.Number", "Number:12"}, | |
| 245 | + | {"github['event_name']", "String:push"}, | |
| 246 | + | {"github['EVENT_NAME']", "String:push"}, | |
| 247 | + | {"github[ 'event_name' ]", "String:push"}, | |
| 248 | + | {"(github).event_name", "String:push"}, | |
| 249 | + | {"(github.event).number", "Number:12"}, | |
| 250 | + | {"github.nope", "Null:"}, | |
| 251 | + | {"github.nope.deeper", "Null:"}, | |
| 252 | + | {"github.nope[0]", "Null:"}, | |
| 253 | + | {"github.nope.*", "Array:Array"}, | |
| 254 | + | {"github.event.commits[0].id", "String:1"}, | |
| 255 | + | {"github.event.commits[2].id", "String:3"}, | |
| 256 | + | {"github.event.commits[3]", "Null:"}, | |
| 257 | + | {"github.event.commits[-1]", "Null:"}, | |
| 258 | + | {"github.event.commits[1.9].id", "String:2"}, | |
| 259 | + | {"github.event.commits['1'].id", "String:2"}, | |
| 260 | + | {"github.event.commits['1.9'].id", "String:2"}, | |
| 261 | + | {"github.event.commits[' 1 '].id", "String:2"}, | |
| 262 | + | {"github.event.commits[true].id", "String:2"}, | |
| 263 | + | {"github.event.commits[false].id", "String:1"}, | |
| 264 | + | {"github.event.commits[null].id", "String:1"}, | |
| 265 | + | {"github.event.commits[0x1].id", "String:2"}, | |
| 266 | + | {"github.event.commits[NaN]", "Null:"}, | |
| 267 | + | {"github.event.commits[Infinity]", "Null:"}, | |
| 268 | + | {"github.event.commits['a']", "Null:"}, | |
| 269 | + | {"github.event.commits[github]", "Null:"}, | |
| 270 | + | {"github.event.commits[0]['author']['name']", "String:Ann"}, | |
| 271 | + | {"github.event.commits[0].author.name", "String:Ann"}, | |
| 272 | + | {"github.event.commits[2].author.name", "Null:"}, | |
| 273 | + | {"github.event.commits[1e20]", "Null:"}, | |
| 274 | + | // Index by number or boolean on a dictionary uses the string form. | |
| 275 | + | {"steps[10].outputs.r", "String:ten"}, | |
| 276 | + | {"steps['10'].outputs.r", "String:ten"}, | |
| 277 | + | // After an identifier, ".10" is not a number: the lexer reads a DOT and then "10.outputs.r" as one number-like word. | |
| 278 | + | {"steps.10.outputs.r", "PARSE Unexpected symbol: '10.outputs.r'. Located at position 7 within expression: steps.10.outputs.r"}, | |
| 279 | + | {"steps[2].outputs.r", "String:two"}, | |
| 280 | + | {"matrix.obj[0]", "Null:"}, | |
| 281 | + | {"matrix.node['0']", "Number:14"}, | |
| 282 | + | {"matrix.node[0] == 14", "Boolean:true"}, | |
| 283 | + | {"matrix.ver", "String:3.10"}, | |
| 284 | + | {"matrix.ver == 3.1", "Boolean:true"}, | |
| 285 | + | {"matrix.ver == '3.10'", "Boolean:true"}, | |
| 286 | + | // Indexing a non-container gives null (or an empty filter with *). | |
| 287 | + | {"'abc'[0]", "PARSE Unexpected symbol: '['"}, | |
| 288 | + | {"(1)[0]", "Null:"}, | |
| 289 | + | {"(true).a", "Null:"}, | |
| 290 | + | {"(null).a", "Null:"}, | |
| 291 | + | {"(1).*", "Array:Array"}, | |
| 292 | + | {"toJSON((1).*)", "String:[]"}, | |
| 293 | + | // Missing contexts at evaluation time are null. | |
| 294 | + | {"toJSON(github.event.commits[0].added)", "String:[\n \"a.txt\"\n]"}, | |
| 295 | + | }) | |
| 296 | + | } | |
| 297 | + | ||
| 298 | + | func TestEvalFilters(t *testing.T) { | |
| 299 | + | runEvalCases(t, []evalCase{ | |
| 300 | + | {"toJSON(github.event.commits.*.id)", "String:[\n \"1\",\n \"2\",\n \"3\"\n]"}, | |
| 301 | + | {"toJSON(github.event.commits[*].id)", "String:[\n \"1\",\n \"2\",\n \"3\"\n]"}, | |
| 302 | + | // Elements without the property are skipped. | |
| 303 | + | {"toJSON(github.event.commits.*.author.name)", "String:[\n \"Ann\",\n \"Bob\"\n]"}, | |
| 304 | + | // ".*" on a filtered array flattens one level of arrays and objects. | |
| 305 | + | {"toJSON(github.event.commits.*.added.*)", "String:[\n \"a.txt\",\n \"b\",\n \"c\"\n]"}, | |
| 306 | + | {"toJSON(github.event.commits.*.added[0])", "String:[\n \"a.txt\",\n \"b\"\n]"}, | |
| 307 | + | {"toJSON(github.event.commits.*.added[1])", "String:[\n \"c\"\n]"}, | |
| 308 | + | {"toJSON(github.event.nested.*.*)", "String:[\n 1,\n 2,\n 3,\n 4,\n 5,\n 6\n]"}, | |
| 309 | + | {"toJSON(github.event.nested.*[1])", "String:[\n 2,\n 5\n]"}, | |
| 310 | + | {"toJSON(github.event.nested[*][0])", "String:[\n 1,\n 3,\n 4\n]"}, | |
| 311 | + | {"toJSON(github.event.nested.*.*.*)", "String:[]"}, | |
| 312 | + | {"toJSON(matrix.obj.*)", "String:[\n \"v\"\n]"}, | |
| 313 | + | {"toJSON(matrix.node.*)", "String:[\n 14,\n 16,\n 18\n]"}, | |
| 314 | + | {"toJSON(matrix.nothing.*)", "String:[]"}, | |
| 315 | + | {"toJSON(inputs.empty.*)", "String:[]"}, | |
| 316 | + | {"toJSON(steps.*.outputs.r)", "String:[\n \"two\",\n \"ten\"\n]"}, | |
| 317 | + | {"toJSON(fromJSON('{\"a\":{\"x\":1},\"b\":{\"x\":2}}').*.x)", "String:[\n 1,\n 2\n]"}, | |
| 318 | + | {"toJSON(fromJSON('[1,2]').*.x)", "String:[]"}, | |
| 319 | + | {"toJSON(fromJSON('[{\"a\":[1,2]},{\"a\":[3]}]').*.a.*)", "String:[\n 1,\n 2,\n 3\n]"}, | |
| 320 | + | {"contains(github.event.commits.*.message, 'FIX: BUG')", "Boolean:true"}, | |
| 321 | + | {"contains(github.event.commits.*.id, 2)", "Boolean:true"}, | |
| 322 | + | {"contains(github.event.commits.*.id, 4)", "Boolean:false"}, | |
| 323 | + | {"contains(matrix.node, 16)", "Boolean:true"}, | |
| 324 | + | {"contains(matrix.node, '16')", "Boolean:true"}, | |
| 325 | + | {"contains(matrix.node, 17)", "Boolean:false"}, | |
| 326 | + | {"join(matrix.node.*, '.')", "String:14.16.18"}, | |
| 327 | + | }) | |
| 328 | + | } | |
| 329 | + | ||
| 330 | + | func TestEvalBuiltinFunctions(t *testing.T) { | |
| 331 | + | runEvalCases(t, []evalCase{ | |
| 332 | + | // contains | |
| 333 | + | {"contains('abc', 'B')", "Boolean:true"}, | |
| 334 | + | {"contains('abc', '')", "Boolean:true"}, | |
| 335 | + | {"contains('', '')", "Boolean:true"}, | |
| 336 | + | {"contains('', 'a')", "Boolean:false"}, | |
| 337 | + | {"contains(null, '')", "Boolean:true"}, | |
| 338 | + | {"contains(1234, 23)", "Boolean:true"}, | |
| 339 | + | {"contains(true, 'RU')", "Boolean:true"}, | |
| 340 | + | {"contains(github, 'a')", "Boolean:false"}, | |
| 341 | + | {"contains('a', github)", "Boolean:false"}, | |
| 342 | + | {"contains(fromJSON('[]'), null)", "Boolean:false"}, | |
| 343 | + | {"contains(fromJSON('[1,\"2\",null]'), null)", "Boolean:true"}, | |
| 344 | + | {"contains(fromJSON('[1,\"2\",null]'), '')", "Boolean:true"}, | |
| 345 | + | {"contains(fromJSON('[[1]]'), fromJSON('[1]'))", "Boolean:false"}, | |
| 346 | + | {"contains('İ', 'i')", "Boolean:true"}, // İ lower-cases to i + U+0307, which contains i | |
| 347 | + | {"contains('ß', 'ss')", "Boolean:false"}, // contains lower-cases: no ß folding | |
| 348 | + | {"contains('A\u03a3', 'a\u03c2')", "Boolean:true"}, | |
| 349 | + | {"contains('K', 'k')", "Boolean:true"}, // KELVIN SIGN lower-cases to k | |
| 350 | + | {"contains()", "PARSE Too few parameters supplied: 'contains'"}, | |
| 351 | + | {"contains(1)", "PARSE Too few parameters supplied: 'contains'"}, | |
| 352 | + | {"contains(1, 2, 3)", "PARSE Too many parameters supplied: 'contains'"}, | |
| 353 | + | ||
| 354 | + | // startsWith, endsWith | |
| 355 | + | {"startsWith('abc', 'AB')", "Boolean:true"}, | |
| 356 | + | {"startsWith('abc', '')", "Boolean:true"}, | |
| 357 | + | {"startsWith(null, '')", "Boolean:true"}, | |
| 358 | + | {"startsWith(123, 12)", "Boolean:true"}, | |
| 359 | + | {"startsWith(github, 'a')", "Boolean:false"}, | |
| 360 | + | {"startsWith('a', github)", "Boolean:false"}, | |
| 361 | + | {"startsWith('ß', 'ss')", "Boolean:true"}, // upper-cases to SS | |
| 362 | + | {"startsWith('ı', 'I')", "Boolean:false"}, | |
| 363 | + | {"endsWith('abc', 'BC')", "Boolean:true"}, | |
| 364 | + | {"endsWith('abc', 'abcd')", "Boolean:false"}, | |
| 365 | + | {"endsWith(1.50, '5')", "Boolean:true"}, | |
| 366 | + | {"endsWith(1.50, 5)", "Boolean:true"}, | |
| 367 | + | {"endsWith('abc')", "PARSE Too few parameters supplied: 'endsWith'"}, | |
| 368 | + | {"STARTSWITH('abc', 'a')", "Boolean:true"}, | |
| 369 | + | ||
| 370 | + | // join | |
| 371 | + | {"join('a')", "String:a"}, | |
| 372 | + | {"join('a', 'b')", "String:a"}, | |
| 373 | + | {"join(null)", "String:"}, | |
| 374 | + | {"join(1.50)", "String:1.5"}, | |
| 375 | + | {"join(github)", "String:"}, | |
| 376 | + | {"join(fromJSON('[]'))", "String:"}, | |
| 377 | + | {"join(fromJSON('[1,2,3]'))", "String:1,2,3"}, | |
| 378 | + | {"join(fromJSON('[1,2,3]'), '')", "String:123"}, | |
| 379 | + | {"join(fromJSON('[1,2,3]'), ', ')", "String:1, 2, 3"}, | |
| 380 | + | {"join(fromJSON('[1,2,3]'), 1)", "String:11213"}, | |
| 381 | + | {"join(fromJSON('[1,2,3]'), null)", "String:123"}, | |
| 382 | + | {"join(fromJSON('[1,2,3]'), github)", "String:1,2,3"}, // a non-primitive separator is ignored | |
| 383 | + | {"join(fromJSON('[null,true,\"a\",1.5,[1],{}]'))", "String:,true,a,1.5,Array,Object"}, | |
| 384 | + | {"join()", "PARSE Too few parameters supplied: 'join'"}, | |
| 385 | + | {"join(1, 2, 3)", "PARSE Too many parameters supplied: 'join'"}, | |
| 386 | + | ||
| 387 | + | // toJSON | |
| 388 | + | {"toJSON(null)", "String:null"}, | |
| 389 | + | {"toJSON(true)", "String:true"}, | |
| 390 | + | {"toJSON(1)", "String:1"}, | |
| 391 | + | {"toJSON(NaN)", "String:null"}, | |
| 392 | + | {"toJSON(Infinity)", "String:null"}, | |
| 393 | + | {"toJSON(-0)", "String:0"}, | |
| 394 | + | {"toJSON('x')", "String:\"x\""}, | |
| 395 | + | {"toJSON(1e21)", "String:1e+21"}, | |
| 396 | + | {"toJSON(1e-7)", "String:1e-7"}, | |
| 397 | + | {"toJSON(0.1)", "String:0.1"}, | |
| 398 | + | {"toJSON(fromJSON('[]'))", "String:[]"}, | |
| 399 | + | {"toJSON(fromJSON('{}'))", "String:{}"}, | |
| 400 | + | {"toJSON(fromJSON('[[],{}]'))", "String:[\n [],\n {}\n]"}, | |
| 401 | + | {"toJSON(1, 2)", "PARSE Too many parameters supplied: 'toJSON'"}, | |
| 402 | + | {"toJSON()", "PARSE Too few parameters supplied: 'toJSON'"}, | |
| 403 | + | {"fromJSON(toJSON(github.event.commits[0])).author.name", "String:Ann"}, | |
| 404 | + | ||
| 405 | + | // fromJSON | |
| 406 | + | {"fromJSON('')", "ERR empty input"}, | |
| 407 | + | {"fromJSON(' ')", "ERR empty input"}, | |
| 408 | + | {"fromJSON('\n')", "ERR empty input"}, | |
| 409 | + | {"fromJSON(null)", "ERR empty input"}, | |
| 410 | + | {"fromJSON('{')", "ERR Error parsing JSON when evaluating fromJson"}, | |
| 411 | + | {"fromJSON('[1,]')", "ERR Error parsing JSON when evaluating fromJson"}, | |
| 412 | + | {"fromJSON('{a:1}')", "ERR Error parsing JSON when evaluating fromJson"}, | |
| 413 | + | {"fromJSON('01')", "ERR Error parsing JSON when evaluating fromJson"}, | |
| 414 | + | {"fromJSON('NaN')", "ERR Error parsing JSON when evaluating fromJson"}, | |
| 415 | + | {"fromJSON('1 2')", "ERR Error parsing JSON when evaluating fromJson"}, | |
| 416 | + | {"fromJSON(' 1 ')", "Number:1"}, | |
| 417 | + | {"fromJSON('-1.5e2')", "Number:-150"}, | |
| 418 | + | {"fromJSON('1e999')", "Number:Infinity"}, | |
| 419 | + | {"fromJSON('\"\\u0041\"')", "String:A"}, | |
| 420 | + | {"fromJSON('\"\\ud83d\\ude00\"')", "String:😀"}, | |
| 421 | + | {"fromJSON(1)", "Number:1"}, | |
| 422 | + | {"fromJSON(true)", "Boolean:true"}, | |
| 423 | + | {"fromJSON(matrix.node)", "ERR Error parsing JSON when evaluating fromJson"}, // the string "Array" | |
| 424 | + | {"fromJSON(1, 2)", "PARSE Too many parameters supplied: 'fromJSON'"}, | |
| 425 | + | ||
| 426 | + | // format | |
| 427 | + | {"format('a')", "String:a"}, | |
| 428 | + | {"format('')", "String:"}, | |
| 429 | + | {"format(null)", "String:"}, | |
| 430 | + | {"format(1)", "String:1"}, | |
| 431 | + | {"format('{0}', 1)", "String:1"}, | |
| 432 | + | {"format('{0}{0}', 'x')", "String:xx"}, | |
| 433 | + | {"format('{0}{1}{0}', 'a', 'b')", "String:aba"}, | |
| 434 | + | {"format('{{')", "String:{"}, | |
| 435 | + | {"format('}}')", "String:}"}, | |
| 436 | + | {"format('{{}}')", "String:{}"}, | |
| 437 | + | {"format('{{0}}', 1)", "String:{0}"}, | |
| 438 | + | {"format('{{{0}}}', 1)", "String:{1}"}, | |
| 439 | + | {"format('{0}', 'a', 'b')", "String:a"}, // extra arguments are fine | |
| 440 | + | {"format('{0:x}', 1)", "String:1"}, // a format specifier is skipped | |
| 441 | + | {"format('{0 anything}', 1)", "String:1"}, | |
| 442 | + | {"format('{00}', 1)", "String:1"}, | |
| 443 | + | {"format('{01}', 1, 2)", "String:2"}, | |
| 444 | + | {"format('{0}', null)", "String:"}, | |
| 445 | + | {"format('{0}', true)", "String:true"}, | |
| 446 | + | {"format('{0}', 0.1)", "String:0.1"}, | |
| 447 | + | {"format('{0}', github)", "String:Object"}, | |
| 448 | + | {"format('{0}', matrix.node)", "String:Array"}, | |
| 449 | + | {"format('{0}-{1}', 1, 'a')", "String:1-a"}, | |
| 450 | + | {"format('x{0}y', '😀')", "String:x😀y"}, | |
| 451 | + | {"format('{1}', 1)", "ERR The following format string references more arguments than were supplied: {1}"}, | |
| 452 | + | {"format('{0}')", "ERR The following format string references more arguments than were supplied: {0}"}, | |
| 453 | + | {"format('{', 1)", "ERR The following format string is invalid: {"}, | |
| 454 | + | {"format('}', 1)", "ERR The following format string is invalid: }"}, | |
| 455 | + | {"format('{}', 1)", "ERR The following format string is invalid: {}"}, | |
| 456 | + | {"format('{ 0}', 1)", "ERR The following format string is invalid: { 0}"}, | |
| 457 | + | {"format('{-1}', 1)", "ERR The following format string is invalid: {-1}"}, | |
| 458 | + | {"format('{a}', 1)", "ERR The following format string is invalid: {a}"}, | |
| 459 | + | {"format('{0', 1)", "ERR The following format string is invalid: {0"}, | |
| 460 | + | {"format('0}', 1)", "ERR The following format string is invalid: 0}"}, | |
| 461 | + | {"format('{{0}', 1)", "ERR The following format string is invalid: {{0}"}, | |
| 462 | + | {"format('{0}}', 1)", "ERR The following format string is invalid: {0}}"}, | |
| 463 | + | {"format('{12345678901234567890123}', 1)", "ERR The following format string references more arguments than were supplied: {12345678901234567890123}"}, | |
| 464 | + | {"format()", "PARSE Too few parameters supplied: 'format'"}, | |
| 465 | + | ||
| 466 | + | // case | |
| 467 | + | {"case(true, 'a', 'b')", "String:a"}, | |
| 468 | + | {"case(false, 'a', 'b')", "String:b"}, | |
| 469 | + | {"case(false, 1, true, 2, 3)", "Number:2"}, | |
| 470 | + | {"case(false, 1, false, 2, 3)", "Number:3"}, | |
| 471 | + | {"case(true, 1, true, 2, 3)", "Number:1"}, | |
| 472 | + | {"case(github.event_name == 'push', 'a', github.event_name == 'pr', 'b', 'c')", "String:a"}, | |
| 473 | + | {"CASE(true, matrix.node, 2)", "Array:Array"}, | |
| 474 | + | {"case(1, 2, 3)", "ERR case predicate must evaluate to a boolean value"}, | |
| 475 | + | {"case('true', 2, 3)", "ERR case predicate must evaluate to a boolean value"}, | |
| 476 | + | {"case(null, 2, 3)", "ERR case predicate must evaluate to a boolean value"}, | |
| 477 | + | {"case(true, 1, 2, 3)", "PARSE Even number of parameters supplied, requires an odd number of parameters: 'case'"}, | |
| 478 | + | {"case(true, 1)", "PARSE Too few parameters supplied: 'case'"}, | |
| 479 | + | {"case()", "PARSE Too few parameters supplied: 'case'"}, | |
| 480 | + | // Predicates after the first true one are not checked... | |
| 481 | + | {"case(true, 1, 'bad', 2, 3)", "Number:1"}, | |
| 482 | + | // ...but earlier ones are. | |
| 483 | + | {"case('bad', 1, true, 2, 3)", "ERR case predicate must evaluate to a boolean value"}, | |
| 484 | + | // The default is returned as is, whatever it is. | |
| 485 | + | {"case(false, 1, null)", "Null:"}, | |
| 486 | + | }) | |
| 487 | + | } | |
| 488 | + | ||
| 489 | + | func TestEvalErrorMessages(t *testing.T) { | |
| 490 | + | runEvalCases(t, []evalCase{ | |
| 491 | + | {"", "PARSE Unexpected end of expression: 'x'"}, // replaced below: the empty expression is not an error | |
| 492 | + | }[:0]) | |
| 493 | + | cases := []evalCase{ | |
| 494 | + | {"1 +", "PARSE Unexpected symbol: '+'. Located at position 3 within expression: 1 +"}, | |
| 495 | + | {"a b", "PARSE Unrecognized named-value: 'a'"}, | |
| 496 | + | {"github github", "PARSE Unexpected symbol: 'github'"}, | |
| 497 | + | {"github.event_name ==", "PARSE Unexpected end of expression: '=='"}, | |
| 498 | + | {"== 1", "PARSE Unexpected symbol: '=='"}, | |
| 499 | + | {"(github.event_name", "PARSE Unexpected end of expression: 'event_name'"}, | |
| 500 | + | {"github.event_name)", "PARSE Unexpected symbol: ')'"}, | |
| 501 | + | {"contains('a',", "PARSE Unexpected end of expression: ','"}, | |
| 502 | + | {"contains('a'", "PARSE Unexpected symbol: 'EOF'"}, | |
| 503 | + | {"contains('a' 'b')", "PARSE Unexpected symbol: 'b'"}, | |
| 504 | + | {"foo(", "PARSE Unexpected end of expression: '('"}, | |
| 505 | + | {"foo()", "PARSE Unrecognized function: 'foo'"}, | |
| 506 | + | {"foo", "PARSE Unrecognized named-value: 'foo'"}, | |
| 507 | + | {"unknown(1)", "PARSE Unrecognized function: 'unknown'"}, | |
| 508 | + | {"success()", "PARSE Unrecognized function: 'success'"}, // not declared in these tests | |
| 509 | + | {"(", "PARSE Unexpected end of expression: '('"}, | |
| 510 | + | {")", "PARSE Unexpected symbol: ')'"}, | |
| 511 | + | {"()", "PARSE Unexpected symbol: ')'"}, | |
| 512 | + | {"!", "PARSE Unexpected end of expression: '!'"}, | |
| 513 | + | {"1 !", "PARSE Unexpected symbol: '!'"}, | |
| 514 | + | {"[1]", "PARSE Unexpected symbol: '['"}, | |
| 515 | + | {"github.[0]", "PARSE Unexpected symbol: '['"}, | |
| 516 | + | {"github.", "PARSE Unexpected symbol: 'EOF'"}, | |
| 517 | + | {"github[", "PARSE Unexpected end of expression: '['"}, | |
| 518 | + | {"github[]", "PARSE Unexpected symbol: ']'"}, | |
| 519 | + | {"github[*", "PARSE Unexpected symbol: 'EOF'"}, | |
| 520 | + | {"github.true", "Null:"}, // keywords are plain identifiers after a dot | |
| 521 | + | {"env.true", "Null:"}, | |
| 522 | + | {"1 = 1", "PARSE Unexpected symbol: '='. Located at position 3 within expression: 1 = 1"}, | |
| 523 | + | {"1 & 1", "PARSE Unexpected symbol: '&'. Located at position 3 within expression: 1 & 1"}, | |
| 524 | + | {"1 | 1", "PARSE Unexpected symbol: '|'. Located at position 3 within expression: 1 | 1"}, | |
| 525 | + | {"1 + 1", "PARSE Unexpected symbol: '+'. Located at position 3 within expression: 1 + 1"}, | |
| 526 | + | {"a-b", "PARSE Unrecognized named-value: 'a-b'"}, | |
| 527 | + | {"1a", "PARSE Unexpected symbol: '1a'. Located at position 1 within expression: 1a"}, | |
| 528 | + | {"'unterminated", "PARSE Unexpected symbol: ''unterminated'. Located at position 1 within expression: 'unterminated"}, | |
| 529 | + | {"@", "PARSE Unexpected symbol: '@'. Located at position 1 within expression: @"}, | |
| 530 | + | {"${{ 1 }}", "PARSE Unexpected symbol: '${{'. Located at position 1 within expression: ${{ 1 }}"}, | |
| 531 | + | {"1 \u00a0 2", "PARSE Unexpected symbol: '\u00a0'. Located at position 3 within expression: 1 \u00a0 2"}, | |
| 532 | + | } | |
| 533 | + | runEvalCases(t, cases) | |
| 534 | + | } | |
| 535 | + | ||
| 536 | + | func TestEmptyExpression(t *testing.T) { | |
| 537 | + | for _, s := range []string{"", " ", "\n", " \t\r\n "} { | |
| 538 | + | node, err := Parse(s, nil, nil) | |
| 539 | + | if err != nil || node != nil { | |
| 540 | + | t.Errorf("Parse(%q) = %v, %v; want nil, nil", s, node, err) | |
| 541 | + | } | |
| 542 | + | } | |
| 543 | + | if _, err := Evaluate(context.Background(), nil, EvalContext{}); err == nil { | |
| 544 | + | t.Error("Evaluate(nil) succeeded") | |
| 545 | + | } | |
| 546 | + | } | |
| 547 | + | ||
| 548 | + | func TestParseErrorPositions(t *testing.T) { | |
| 549 | + | tests := []struct { | |
| 550 | + | src string | |
| 551 | + | line, col int | |
| 552 | + | msg string | |
| 553 | + | }{ | |
| 554 | + | {"1 ==\n 2 ===\n 3", 1, 5, "Unexpected symbol: '='. Located at position 11 within expression: 1 ==\n 2 ===\n 3"}, | |
| 555 | + | {"true &&\n false ||\n nope", 2, 2, "Unrecognized named-value: 'nope'"}, | |
| 556 | + | {"\n\n bogus", 2, 2, "Unrecognized named-value: 'bogus'"}, | |
| 557 | + | {"github github", 0, 7, "Unexpected symbol: 'github'"}, | |
| 558 | + | {" ) ", 0, 2, "Unexpected symbol: ')'"}, | |
| 559 | + | {"contains('a',", 0, 12, "Unexpected end of expression: ','"}, | |
| 560 | + | {"foo()", 0, 0, "Unrecognized function: 'foo'"}, | |
| 561 | + | {"1 && foo()", 0, 5, "Unrecognized function: 'foo'"}, | |
| 562 | + | {"'a'\n 'b'\n zzz", 1, 1, "Unexpected symbol: 'b'"}, | |
| 563 | + | } | |
| 564 | + | for _, tc := range tests { | |
| 565 | + | _, err := Parse(tc.src, testContextNames, nil) | |
| 566 | + | var pe *Error | |
| 567 | + | if !errors.As(err, &pe) { | |
| 568 | + | t.Errorf("Parse(%q): error %v is not *Error", tc.src, err) | |
| 569 | + | continue | |
| 570 | + | } | |
| 571 | + | if pe.Message != tc.msg || pe.Pos.Line != tc.line || pe.Pos.Column != tc.col { | |
| 572 | + | t.Errorf("Parse(%q) = %q at %v; want %q at %d:%d", tc.src, pe.Message, pe.Pos, tc.msg, tc.line, tc.col) | |
| 573 | + | } | |
| 574 | + | } | |
| 575 | + | } | |
| 576 | + | ||
| 577 | + | func TestParseErrorTypes(t *testing.T) { | |
| 578 | + | tests := []struct { | |
| 579 | + | src string | |
| 580 | + | want ErrorType | |
| 581 | + | }{ | |
| 582 | + | {"1 ==", ErrorUnexpectedEndOfExpression}, | |
| 583 | + | {")", ErrorUnexpectedSymbol}, | |
| 584 | + | {"nope", ErrorUnrecognizedContext}, | |
| 585 | + | {"nope()", ErrorUnrecognizedFunction}, | |
| 586 | + | {"contains(1)", ErrorTooFewParameters}, | |
| 587 | + | {"contains(1,2,3)", ErrorTooManyParameters}, | |
| 588 | + | {"case(true, 1, 2, 3)", ErrorEvenParameters}, | |
| 589 | + | {strings.Repeat("(", 60) + "1" + strings.Repeat(")", 60), ErrorExceededMaxDepth}, | |
| 590 | + | {"1 = 1", ErrorUnexpectedSymbol}, | |
| 591 | + | } | |
| 592 | + | for _, tc := range tests { | |
| 593 | + | _, err := Parse(tc.src, nil, nil) | |
| 594 | + | var pe *Error | |
| 595 | + | if !errors.As(err, &pe) || pe.Type != tc.want { | |
| 596 | + | t.Errorf("Parse(%.30q): got %#v, want type %d", tc.src, err, tc.want) | |
| 597 | + | } | |
| 598 | + | } | |
| 599 | + | } | |
| 600 | + | ||
| 601 | + | func TestLimits(t *testing.T) { | |
| 602 | + | deepParens := func(n int) string { return strings.Repeat("(", n) + "1" + strings.Repeat(")", n) } | |
| 603 | + | tests := []struct { | |
| 604 | + | name string | |
| 605 | + | src string | |
| 606 | + | ok bool | |
| 607 | + | }{ | |
| 608 | + | // Each "(" costs one level, plus one for the outermost expression. | |
| 609 | + | {"49 parens", deepParens(49), true}, | |
| 610 | + | {"50 parens", deepParens(50), false}, | |
| 611 | + | {"49 nots", strings.Repeat("!", 49) + "true", true}, | |
| 612 | + | {"50 nots", strings.Repeat("!", 50) + "true", false}, | |
| 613 | + | {"49 calls", strings.Repeat("format(", 49) + "'x'" + strings.Repeat(")", 49), true}, | |
| 614 | + | {"50 calls", strings.Repeat("format(", 50) + "'x'" + strings.Repeat(")", 50), false}, | |
| 615 | + | {"long dotted path", "github" + strings.Repeat(".event", 49), true}, | |
| 616 | + | {"too long dotted path", "github" + strings.Repeat(".event", 50), false}, | |
| 617 | + | {"long flat ||", "1" + strings.Repeat(" || 1", 3000), true}, | |
| 618 | + | {"long flat ==", "1" + strings.Repeat(" == 1", 3000), true}, | |
| 619 | + | } | |
| 620 | + | for _, tc := range tests { | |
| 621 | + | _, err := Parse(tc.src, testContextNames, nil) | |
| 622 | + | if (err == nil) != tc.ok { | |
| 623 | + | t.Errorf("%s: err = %v, want ok=%v", tc.name, err, tc.ok) | |
| 624 | + | } | |
| 625 | + | var pe *Error | |
| 626 | + | if err != nil && errors.As(err, &pe) && pe.Type == ErrorExceededMaxDepth && pe.Message != "Exceeded max expression depth 50: '1'" && !strings.HasPrefix(pe.Message, "Exceeded max expression depth 50: '") { | |
| 627 | + | t.Errorf("%s: message %q", tc.name, pe.Message) | |
| 628 | + | } | |
| 629 | + | } | |
| 630 | + | ||
| 631 | + | // Length: counted in UTF-16 code units, error is the bare token. | |
| 632 | + | long := "'" + strings.Repeat("a", MaxExpressionLength-2) + "'" | |
| 633 | + | if _, err := Parse(long, nil, nil); err != nil { | |
| 634 | + | t.Errorf("expression of exactly %d characters: %v", MaxExpressionLength, err) | |
| 635 | + | } | |
| 636 | + | _, err := Parse(long+" ", nil, nil) | |
| 637 | + | if err == nil || err.Error() != "ErrorExceededMaxLength" { | |
| 638 | + | t.Errorf("expression of %d characters: err = %v", MaxExpressionLength+1, err) | |
| 639 | + | } | |
| 640 | + | // 10501 emoji are 21002 UTF-16 units but only 10501 runes. | |
| 641 | + | emoji := "'" + strings.Repeat("😀", 10500) + "'" | |
| 642 | + | if _, err := Parse(emoji, nil, nil); err == nil { | |
| 643 | + | t.Errorf("a string of 10500 emoji (21002 UTF-16 units) was accepted") | |
| 644 | + | } | |
| 645 | + | emoji = "'" + strings.Repeat("😀", 10499) + "'" | |
| 646 | + | if _, err := Parse(emoji, nil, nil); err != nil { | |
| 647 | + | t.Errorf("a string of 10499 emoji (21000 UTF-16 units) was rejected: %v", err) | |
| 648 | + | } | |
| 649 | + | } | |
| 650 | + | ||
| 651 | + | func TestDepthMessage(t *testing.T) { | |
| 652 | + | _, err := Parse(strings.Repeat("(", 60)+"1"+strings.Repeat(")", 60), nil, nil) | |
| 653 | + | want := "Exceeded max expression depth 50: '('" | |
| 654 | + | if err == nil || err.Error() != want { | |
| 655 | + | t.Errorf("got %v, want %q", err, want) | |
| 656 | + | } | |
| 657 | + | } | |
| 658 | + | ||
| 659 | + | func TestExtensionFunctions(t *testing.T) { | |
| 660 | + | funcs := []FuncInfo{ | |
| 661 | + | {Name: "hashFiles", MinArgs: 1, MaxArgs: -1}, | |
| 662 | + | {Name: "success", MinArgs: 0, MaxArgs: 0}, | |
| 663 | + | {Name: "bounded", MinArgs: 1, MaxArgs: 2}, | |
| 664 | + | {Name: "contains", MinArgs: 0, MaxArgs: 0}, // built-ins win for validation | |
| 665 | + | } | |
| 666 | + | for _, c := range []struct{ src, err string }{ | |
| 667 | + | {"hashFiles('a')", ""}, | |
| 668 | + | {"HASHFILES('a', 'b', 'c', 'd')", ""}, | |
| 669 | + | {"hashFiles()", "Too few parameters supplied: 'hashFiles'"}, | |
| 670 | + | {"success()", ""}, | |
| 671 | + | {"success(1)", "Too many parameters supplied: 'success'"}, | |
| 672 | + | {"bounded(1, 2)", ""}, | |
| 673 | + | {"bounded(1, 2, 3)", "Too many parameters supplied: 'bounded'"}, | |
| 674 | + | {"contains()", "Too few parameters supplied: 'contains'"}, | |
| 675 | + | {"nothere()", "Unrecognized function: 'nothere'"}, | |
| 676 | + | } { | |
| 677 | + | _, err := Parse(c.src, nil, funcs) | |
| 678 | + | switch { | |
| 679 | + | case c.err == "" && err != nil: | |
| 680 | + | t.Errorf("%s: %v", c.src, err) | |
| 681 | + | case c.err != "" && (err == nil || err.Error() != c.err): | |
| 682 | + | t.Errorf("%s: err = %v, want %q", c.src, err, c.err) | |
| 683 | + | } | |
| 684 | + | } | |
| 685 | + | ||
| 686 | + | calls := 0 | |
| 687 | + | ctx := EvalContext{ | |
| 688 | + | Contexts: map[string]Value{"G": String("v")}, | |
| 689 | + | Funcs: map[string]Func{ | |
| 690 | + | "HashFiles": func(_ context.Context, args []Value) (Value, error) { | |
| 691 | + | calls++ | |
| 692 | + | return String("h:" + CoerceString(args[0])), nil | |
| 693 | + | }, | |
| 694 | + | "success": func(context.Context, []Value) (Value, error) { return Boolean(true), nil }, | |
| 695 | + | "boom": func(context.Context, []Value) (Value, error) { return nil, errors.New("kaput") }, | |
| 696 | + | // An extra function replaces a built-in of the same name. | |
| 697 | + | "contains": func(context.Context, []Value) (Value, error) { return String("mine"), nil }, | |
| 698 | + | }, | |
| 699 | + | } | |
| 700 | + | node, err := Parse("hashFiles(g) == 'h:v' && success() && contains('a','b')", []string{"g"}, funcs) | |
| 701 | + | if err != nil { | |
| 702 | + | t.Fatal(err) | |
| 703 | + | } | |
| 704 | + | v, err := Evaluate(context.Background(), node, ctx) | |
| 705 | + | if err != nil || CoerceString(v) != "mine" || calls != 1 { | |
| 706 | + | t.Errorf("got %v, %v, %d calls", v, err, calls) | |
| 707 | + | } | |
| 708 | + | node, err = Parse("boom()", nil, []FuncInfo{{Name: "boom"}}) | |
| 709 | + | if err != nil { | |
| 710 | + | t.Fatal(err) | |
| 711 | + | } | |
| 712 | + | if _, err = Evaluate(context.Background(), node, ctx); err == nil || err.Error() != "kaput" { | |
| 713 | + | t.Errorf("function error not propagated: %v", err) | |
| 714 | + | } | |
| 715 | + | // A declared function without an implementation is an evaluation error. | |
| 716 | + | node, _ = Parse("success()", nil, funcs) | |
| 717 | + | if _, err = Evaluate(context.Background(), node, EvalContext{}); err == nil { | |
| 718 | + | t.Error("calling an unimplemented function succeeded") | |
| 719 | + | } | |
| 720 | + | } | |
| 721 | + | ||
| 722 | + | func TestContextNames(t *testing.T) { | |
| 723 | + | node, err := Parse("GitHub.Ref", []string{"github"}, nil) | |
| 724 | + | if err != nil { | |
| 725 | + | t.Fatal(err) | |
| 726 | + | } | |
| 727 | + | for _, key := range []string{"github", "GITHUB", "GitHub"} { | |
| 728 | + | d := NewDictionary() | |
| 729 | + | d.Add("ref", String("main")) | |
| 730 | + | v, err := Evaluate(context.Background(), node, EvalContext{Contexts: map[string]Value{key: d}}) | |
| 731 | + | if err != nil || CoerceString(v) != "main" { | |
| 732 | + | t.Errorf("context key %q: %v, %v", key, v, err) | |
| 733 | + | } | |
| 734 | + | } | |
| 735 | + | // A declared but absent context is null. | |
| 736 | + | v, err := Evaluate(context.Background(), node, EvalContext{}) | |
| 737 | + | if err != nil || v.Kind() != KindNull { | |
| 738 | + | t.Errorf("absent context: %v, %v", v, err) | |
| 739 | + | } | |
| 740 | + | // Names that were not declared to Parse are errors, even if present at evaluation. | |
| 741 | + | if _, err := Parse("secrets.X", []string{"github"}, nil); err == nil || err.Error() != "Unrecognized named-value: 'secrets'" { | |
| 742 | + | t.Errorf("undeclared context: %v", err) | |
| 743 | + | } | |
| 744 | + | // A nil Value in a context behaves as null. | |
| 745 | + | node, _ = Parse("github.a.b", []string{"github"}, nil) | |
| 746 | + | if v, err := Evaluate(context.Background(), node, EvalContext{Contexts: map[string]Value{"github": nil}}); err != nil || v.Kind() != KindNull { | |
| 747 | + | t.Errorf("nil context value: %v, %v", v, err) | |
| 748 | + | } | |
| 749 | + | var nilDict *Dictionary | |
| 750 | + | var nilArr *Array | |
| 751 | + | node, _ = Parse("(github)[0] || x", []string{"github", "x"}, nil) | |
| 752 | + | for _, bad := range []Value{nilDict, nilArr} { | |
| 753 | + | if _, err := Evaluate(context.Background(), node, EvalContext{Contexts: map[string]Value{"github": bad}}); err != nil { | |
| 754 | + | t.Errorf("typed nil %T: %v", bad, err) | |
| 755 | + | } | |
| 756 | + | } | |
| 757 | + | } | |
| 758 | + | ||
| 759 | + | func TestEvaluateDoesNotMutateContext(t *testing.T) { | |
| 760 | + | ctxs := testContexts(t) | |
| 761 | + | before := map[string]string{} | |
| 762 | + | for k, v := range ctxs { | |
| 763 | + | before[k] = ToJSON(v) | |
| 764 | + | } | |
| 765 | + | for _, src := range []string{"github.event.commits.*", "github.event.commits.*.added.*", "github.*", "matrix.node.*", "toJSON(steps.*)"} { | |
| 766 | + | node, err := Parse(src, testContextNames, nil) | |
| 767 | + | if err != nil { | |
| 768 | + | t.Fatal(err) | |
| 769 | + | } | |
| 770 | + | v, err := Evaluate(context.Background(), node, EvalContext{Contexts: ctxs}) | |
| 771 | + | if err != nil { | |
| 772 | + | t.Fatal(err) | |
| 773 | + | } | |
| 774 | + | if a, ok := v.(*Array); ok { | |
| 775 | + | a.Append(String("tampered")) | |
| 776 | + | } | |
| 777 | + | } | |
| 778 | + | for k, v := range ctxs { | |
| 779 | + | if ToJSON(v) != before[k] { | |
| 780 | + | t.Errorf("context %s was modified", k) | |
| 781 | + | } | |
| 782 | + | } | |
| 783 | + | } | |
| 784 | + | ||
| 785 | + | func TestFloatCoercion(t *testing.T) { | |
| 786 | + | tests := []struct { | |
| 787 | + | in float64 | |
| 788 | + | want string | |
| 789 | + | }{ | |
| 790 | + | {0, "0"}, | |
| 791 | + | {math.Copysign(0, -1), "0"}, | |
| 792 | + | {1, "1"}, | |
| 793 | + | {-1, "-1"}, | |
| 794 | + | {1.5, "1.5"}, | |
| 795 | + | {0.1, "0.1"}, | |
| 796 | + | {0.30000000000000004, "0.3"}, | |
| 797 | + | {1.005, "1.005"}, | |
| 798 | + | {100, "100"}, | |
| 799 | + | {1e20, "100000000000000000000"}, | |
| 800 | + | {1e21, "1e+21"}, | |
| 801 | + | {123456789012345680000, "123456789012345680000"}, | |
| 802 | + | {1e-7, "1e-7"}, // toFixed(15) keeps 0.000000100000000, which prints as 1e-7 | |
| 803 | + | {0.000001, "0.000001"}, | |
| 804 | + | {5e-16, "1e-15"}, // toFixed rounds half up on the exact binary value | |
| 805 | + | {1e-15, "1e-15"}, | |
| 806 | + | {1e-16, "0"}, | |
| 807 | + | {-1e-16, "0"}, | |
| 808 | + | {12345678901234567890, "12345678901234567000"}, | |
| 809 | + | {math.NaN(), "NaN"}, | |
| 810 | + | {math.Inf(1), "Infinity"}, | |
| 811 | + | {math.Inf(-1), "-Infinity"}, | |
| 812 | + | {2.5, "2.5"}, | |
| 813 | + | {-0.25, "-0.25"}, | |
| 814 | + | {4503599627370497.5, "4503599627370498"}, // the literal itself rounds to the nearest double | |
| 815 | + | {1.7976931348623157e308, "1.7976931348623157e+308"}, | |
| 816 | + | {5e-324, "0"}, | |
| 817 | + | } | |
| 818 | + | for _, tc := range tests { | |
| 819 | + | if got := CoerceString(Number(tc.in)); got != tc.want { | |
| 820 | + | t.Errorf("CoerceString(%v) = %q, want %q", tc.in, got, tc.want) | |
| 821 | + | } | |
| 822 | + | } | |
| 823 | + | } | |
| 824 | + | ||
| 825 | + | func TestJSNumberToString(t *testing.T) { | |
| 826 | + | one, two := 0.1, 0.2 | |
| 827 | + | sum := one + two // not a constant expression: Go would fold 0.1+0.2 exactly | |
| 828 | + | tests := []struct { | |
| 829 | + | in float64 | |
| 830 | + | want string | |
| 831 | + | }{ | |
| 832 | + | {0, "0"}, {1, "1"}, {-1.5, "-1.5"}, {100, "100"}, {1e20, "100000000000000000000"}, {1e21, "1e+21"}, | |
| 833 | + | {1.5e21, "1.5e+21"}, {1e-6, "0.000001"}, {1e-7, "1e-7"}, {1.5e-7, "1.5e-7"}, {123456.789, "123456.789"}, | |
| 834 | + | {0.000123, "0.000123"}, {5e-324, "5e-324"}, {1.7976931348623157e308, "1.7976931348623157e+308"}, | |
| 835 | + | {-1e-7, "-1e-7"}, {sum, "0.30000000000000004"}, {math.NaN(), "NaN"}, {math.Inf(-1), "-Infinity"}, | |
| 836 | + | } | |
| 837 | + | for _, tc := range tests { | |
| 838 | + | if got := jsNumberToString(tc.in); got != tc.want { | |
| 839 | + | t.Errorf("jsNumberToString(%v) = %q, want %q", tc.in, got, tc.want) | |
| 840 | + | } | |
| 841 | + | } | |
| 842 | + | } | |
| 843 | + | ||
| 844 | + | func TestJSToNumber(t *testing.T) { | |
| 845 | + | nan := math.NaN() | |
| 846 | + | tests := []struct { | |
| 847 | + | in string | |
| 848 | + | want float64 | |
| 849 | + | }{ | |
| 850 | + | {"", 0}, {" ", 0}, {"\n\t", 0}, {"1", 1}, {" 1 ", 1}, {"\u00a01\u2003", 1}, {"\ufeff1", 1}, {"+1", 1}, {"-1", -1}, | |
| 851 | + | {"1.", 1}, {".5", 0.5}, {"-.5", -0.5}, {"1e3", 1000}, {"1E-3", 0.001}, {"1e+3", 1000}, {"0x10", 16}, {"0X1f", 31}, | |
| 852 | + | {"0o17", 15}, {"0b101", 5}, {"0x", nan}, {"0xg", nan}, {"-0x10", nan}, {"+0x10", nan}, {"0b2", nan}, {"1_000", nan}, | |
| 853 | + | {"Infinity", math.Inf(1)}, {"-Infinity", math.Inf(-1)}, {"+Infinity", math.Inf(1)}, {"infinity", nan}, {"inf", nan}, | |
| 854 | + | {"NaN", nan}, {"nan", nan}, {"abc", nan}, {"1e", nan}, {"1e+", nan}, {".", nan}, {"e5", nan}, {"1 2", nan}, {"1,5", nan}, | |
| 855 | + | {"00", 0}, {"007", 7}, {"0.0", 0}, {"1e999", math.Inf(1)}, {"-1e999", math.Inf(-1)}, {"0x1fffffffffffff", 9007199254740991}, | |
| 856 | + | {"12345678901234567890", 12345678901234567890}, {"٣", nan}, {"--1", nan}, {"+-1", nan}, {"1e1.5", nan}, {"0x1.8", nan}, | |
| 857 | + | } | |
| 858 | + | for _, tc := range tests { | |
| 859 | + | got := jsToNumber(tc.in) | |
| 860 | + | if got != tc.want && !(math.IsNaN(got) && math.IsNaN(tc.want)) { | |
| 861 | + | t.Errorf("jsToNumber(%q) = %v, want %v", tc.in, got, tc.want) | |
| 862 | + | } | |
| 863 | + | } | |
| 864 | + | } | |
| 865 | + | ||
| 866 | + | func TestJSCasing(t *testing.T) { | |
| 867 | + | upper := map[string]string{"abc": "ABC", "ß": "SS", "ʼn": "ʼN", "dž": "DŽ", "ı": "I", "ffi": "FFI", "ὒ": "Υ̓̀", "ᾀ": "ἈΙ", "σ": "Σ", "ς": "Σ", "😀": "😀", "é": "É"} | |
| 868 | + | for in, want := range upper { | |
| 869 | + | if got := jsToUpper(in); got != want { | |
| 870 | + | t.Errorf("jsToUpper(%q) = %q, want %q", in, got, want) | |
| 871 | + | } | |
| 872 | + | } | |
| 873 | + | special := map[string]string{"ı": "ı", "ıi": "ıI", "I": "I", "ß": "SS", "aıb": "AıB"} | |
| 874 | + | for in, want := range special { | |
| 875 | + | if got := toUpperSpecial(in); got != want { | |
| 876 | + | t.Errorf("toUpperSpecial(%q) = %q, want %q", in, got, want) | |
| 877 | + | } | |
| 878 | + | } | |
| 879 | + | lower := map[string]string{ | |
| 880 | + | "ABC": "abc", "İ": "i\u0307", "ΑΣ": "ας", "ΑΣΑ": "ασα", "Α Σ": "α σ", "ΑΣ.": "ας.", "Σ": "σ", "ΑΣ'Α": "ασ'α", | |
| 881 | + | "K": "k", "Å": "å", "😀": "😀", "DŽ": "dž", "ΑΣΣ": "ασς", "Α.Σ": "α.ς", | |
| 882 | + | } | |
| 883 | + | for in, want := range lower { | |
| 884 | + | if got := jsToLower(in); got != want { | |
| 885 | + | t.Errorf("jsToLower(%q) = %q, want %q", in, got, want) | |
| 886 | + | } | |
| 887 | + | } | |
| 888 | + | } | |
| 889 | + | ||
| 890 | + | func TestCompareUTF16(t *testing.T) { | |
| 891 | + | // U+10000 is the surrogate pair D800 DC00, which sorts before U+E000. | |
| 892 | + | if got := compareUTF16("\U00010000", "\ue000"); got >= 0 { | |
| 893 | + | t.Errorf("compareUTF16(U+10000, U+E000) = %d, want < 0", got) | |
| 894 | + | } | |
| 895 | + | if got := compareUTF16("\uffff", "\U00010000"); got <= 0 { | |
| 896 | + | t.Errorf("compareUTF16(U+FFFF, U+10000) = %d, want > 0", got) | |
| 897 | + | } | |
| 898 | + | for _, c := range []struct { | |
| 899 | + | a, b string | |
| 900 | + | want int | |
| 901 | + | }{{"a", "a", 0}, {"a", "b", -1}, {"b", "a", 1}, {"a", "ab", -1}, {"ab", "a", 1}, {"", "", 0}, {"", "a", -1}} { | |
| 902 | + | if got := compareUTF16(c.a, c.b); got != c.want { | |
| 903 | + | t.Errorf("compareUTF16(%q, %q) = %d, want %d", c.a, c.b, got, c.want) | |
| 904 | + | } | |
| 905 | + | } | |
| 906 | + | // And through the operators. | |
| 907 | + | for _, c := range evalCasesUTF16 { | |
| 908 | + | if got := evalString(t, c.expr); got != c.want { | |
| 909 | + | t.Errorf("%s = %s, want %s", c.expr, got, c.want) | |
| 910 | + | } | |
| 911 | + | } | |
| 912 | + | } | |
| 913 | + | ||
| 914 | + | var evalCasesUTF16 = []evalCase{ | |
| 915 | + | {"'\U00010000' < '\ue000'", "Boolean:true"}, | |
| 916 | + | {"'\ue000' > '\U00010000'", "Boolean:true"}, | |
| 917 | + | {"'\U00010000' >= '\ue000'", "Boolean:false"}, | |
| 918 | + | } | |
| 919 | + | ||
| 920 | + | func TestTokenString(t *testing.T) { | |
| 921 | + | toks, err := Lex("a.b == 'x''y' && 1.5") | |
| 922 | + | if err != nil { | |
| 923 | + | t.Fatal(err) | |
| 924 | + | } | |
| 925 | + | var got []string | |
| 926 | + | for _, tk := range toks { | |
| 927 | + | got = append(got, tk.Type.String()) | |
| 928 | + | } | |
| 929 | + | want := "IDENTIFIER DOT IDENTIFIER EQUAL_EQUAL STRING AND NUMBER EOF" | |
| 930 | + | if strings.Join(got, " ") != want { | |
| 931 | + | t.Errorf("tokens = %v, want %s", got, want) | |
| 932 | + | } | |
| 933 | + | if toks[4].Value.(String) != "x'y" || toks[4].Lexeme != "'x''y'" { | |
| 934 | + | t.Errorf("string token = %#v", toks[4]) | |
| 935 | + | } | |
| 936 | + | if TokenType(99).String() != "TokenType(99)" { | |
| 937 | + | t.Error("unknown token type") | |
| 938 | + | } | |
| 939 | + | } | |
| 940 | + | ||
| 941 | + | // The EOF token of the reference is positioned from a stale token start, so | |
| 942 | + | // after a trailing newline its column is negative. Kept as is. | |
| 943 | + | func TestEOFTokenPosition(t *testing.T) { | |
| 944 | + | toks, err := Lex("a\n") | |
| 945 | + | if err != nil { | |
| 946 | + | t.Fatal(err) | |
| 947 | + | } | |
| 948 | + | eof := toks[len(toks)-1] | |
| 949 | + | if eof.Type != TokenEOF || eof.Range.Start != (Position{Line: 1, Column: -1}) || eof.Range.End != (Position{Line: 1, Column: 0}) { | |
| 950 | + | t.Errorf("EOF token = %+v", eof) | |
| 951 | + | } | |
| 952 | + | } | |
A expr/example_test.go +91 -0
| @@ -0,0 +1,91 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr_test | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "errors" | |
| 8 | + | "fmt" | |
| 9 | + | ||
| 10 | + | "bnema.dev/ermite-actions/expr" | |
| 11 | + | ) | |
| 12 | + | ||
| 13 | + | func ExampleParse() { | |
| 14 | + | node, err := expr.Parse("contains(github.ref, 'main') && !failure()", | |
| 15 | + | []string{"github"}, | |
| 16 | + | []expr.FuncInfo{{Name: "failure", MinArgs: 0, MaxArgs: 0}}) | |
| 17 | + | fmt.Println(node != nil, err) | |
| 18 | + | ||
| 19 | + | // Unknown names and wrong argument counts are rejected when parsing, | |
| 20 | + | // with the messages of GitHub's implementation. | |
| 21 | + | for _, src := range []string{"secrets.TOKEN", "contains('a')", "github.ref ==", "nope()"} { | |
| 22 | + | _, err := expr.Parse(src, []string{"github"}, nil) | |
| 23 | + | var pe *expr.Error | |
| 24 | + | if errors.As(err, &pe) { | |
| 25 | + | fmt.Printf("%s -> %s (line %d, column %d)\n", src, pe.Message, pe.Pos.Line, pe.Pos.Column) | |
| 26 | + | } | |
| 27 | + | } | |
| 28 | + | // Output: | |
| 29 | + | // true <nil> | |
| 30 | + | // secrets.TOKEN -> Unrecognized named-value: 'secrets' (line 0, column 0) | |
| 31 | + | // contains('a') -> Too few parameters supplied: 'contains' (line 0, column 0) | |
| 32 | + | // github.ref == -> Unexpected end of expression: '==' (line 0, column 11) | |
| 33 | + | // nope() -> Unrecognized function: 'nope' (line 0, column 0) | |
| 34 | + | } | |
| 35 | + | ||
| 36 | + | func ExampleEvaluate() { | |
| 37 | + | event, err := expr.FromJSON([]byte(`{"ref": "refs/heads/main", "labels": [{"name": "Bug"}, {"name": "docs"}]}`)) | |
| 38 | + | if err != nil { | |
| 39 | + | panic(err) | |
| 40 | + | } | |
| 41 | + | ec := expr.EvalContext{ | |
| 42 | + | Contexts: map[string]expr.Value{"github": event}, | |
| 43 | + | Funcs: map[string]expr.Func{ | |
| 44 | + | "always": func(context.Context, []expr.Value) (expr.Value, error) { return expr.Boolean(true), nil }, | |
| 45 | + | }, | |
| 46 | + | } | |
| 47 | + | funcs := []expr.FuncInfo{{Name: "always", MinArgs: 0, MaxArgs: 0}} | |
| 48 | + | ||
| 49 | + | for _, src := range []string{ | |
| 50 | + | "github.ref == 'refs/heads/main' && 'release' || 'dev'", | |
| 51 | + | "contains(github.labels.*.name, 'bug')", | |
| 52 | + | "join(github.labels.*.name, ', ')", | |
| 53 | + | "format('{0} has {1} labels', github.ref, 2)", | |
| 54 | + | "always() && github.missing", | |
| 55 | + | "fromJSON('{')", | |
| 56 | + | } { | |
| 57 | + | node, err := expr.Parse(src, []string{"github"}, funcs) | |
| 58 | + | if err != nil { | |
| 59 | + | panic(err) | |
| 60 | + | } | |
| 61 | + | v, err := expr.Evaluate(context.Background(), node, ec) | |
| 62 | + | if err != nil { | |
| 63 | + | fmt.Printf("%s -> error: %v\n", src, err) | |
| 64 | + | continue | |
| 65 | + | } | |
| 66 | + | fmt.Printf("%s -> %s (truthy: %t)\n", src, expr.CoerceString(v), expr.Truthy(v)) | |
| 67 | + | } | |
| 68 | + | // Output: | |
| 69 | + | // github.ref == 'refs/heads/main' && 'release' || 'dev' -> release (truthy: true) | |
| 70 | + | // contains(github.labels.*.name, 'bug') -> true (truthy: true) | |
| 71 | + | // join(github.labels.*.name, ', ') -> Bug, docs (truthy: true) | |
| 72 | + | // format('{0} has {1} labels', github.ref, 2) -> refs/heads/main has 2 labels (truthy: true) | |
| 73 | + | // always() && github.missing -> (truthy: false) | |
| 74 | + | // fromJSON('{') -> error: Error parsing JSON when evaluating fromJson | |
| 75 | + | } | |
| 76 | + | ||
| 77 | + | func ExampleToJSON() { | |
| 78 | + | d := expr.NewDictionary() | |
| 79 | + | d.Add("name", expr.String("octo")) | |
| 80 | + | d.Add("tags", expr.NewArray(expr.Number(1), expr.Null{}, expr.Number(0.5))) | |
| 81 | + | fmt.Println(expr.ToJSON(d)) | |
| 82 | + | // Output: | |
| 83 | + | // { | |
| 84 | + | // "name": "octo", | |
| 85 | + | // "tags": [ | |
| 86 | + | // 1, | |
| 87 | + | // null, | |
| 88 | + | // 0.5 | |
| 89 | + | // ] | |
| 90 | + | // } | |
| 91 | + | } | |
A expr/funcs.go +257 -0
| @@ -0,0 +1,257 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "errors" | |
| 7 | + | "math" | |
| 8 | + | "strconv" | |
| 9 | + | "strings" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | // builtinFunc implements a built-in function. It runs inside the evaluator so | |
| 13 | + | // that it can charge the output budget and watch the context. | |
| 14 | + | type builtinFunc func(e *evaluator, args []Value) (Value, error) | |
| 15 | + | ||
| 16 | + | // builtinImpls are the implementations of the built-in functions, keyed by | |
| 17 | + | // lower-cased name. Arity is validated by the parser, so each may assume its | |
| 18 | + | // documented argument count. | |
| 19 | + | var builtinImpls = map[string]builtinFunc{ | |
| 20 | + | "case": funcCase, | |
| 21 | + | "contains": funcContains, | |
| 22 | + | "endswith": funcEndsWith, | |
| 23 | + | "format": funcFormat, | |
| 24 | + | "fromjson": funcFromJSON, | |
| 25 | + | "join": funcJoin, | |
| 26 | + | "startswith": funcStartsWith, | |
| 27 | + | "tojson": funcToJSON, | |
| 28 | + | } | |
| 29 | + | ||
| 30 | + | // funcCase is case(pred1, val1, pred2, val2, ..., default). | |
| 31 | + | // | |
| 32 | + | // The JavaScript reference ships it behind an experimental feature flag | |
| 33 | + | // (allowCaseFunction) that only affects editor completion; the parser always | |
| 34 | + | // accepts it, and so does this package. | |
| 35 | + | func funcCase(e *evaluator, args []Value) (Value, error) { | |
| 36 | + | for i := 0; i < len(args)-1; i += 2 { | |
| 37 | + | pred, ok := normalize(args[i]).(Boolean) | |
| 38 | + | if !ok { | |
| 39 | + | return nil, evalErrorf("case predicate must evaluate to a boolean value") | |
| 40 | + | } | |
| 41 | + | if pred { | |
| 42 | + | return args[i+1], nil | |
| 43 | + | } | |
| 44 | + | } | |
| 45 | + | return args[len(args)-1], nil | |
| 46 | + | } | |
| 47 | + | ||
| 48 | + | // funcContains is contains(search, item): substring test, case-insensitive | |
| 49 | + | // via toLowerCase, when both are primitive; membership by == in an array; | |
| 50 | + | // false otherwise. | |
| 51 | + | func funcContains(e *evaluator, args []Value) (Value, error) { | |
| 52 | + | left, right := normalize(args[0]), normalize(args[1]) | |
| 53 | + | switch { | |
| 54 | + | case isPrimitive(left): | |
| 55 | + | if isPrimitive(right) { | |
| 56 | + | return Boolean(jsIncludes(jsToLower(CoerceString(left)), jsToLower(CoerceString(right)))), nil | |
| 57 | + | } | |
| 58 | + | case left.Kind() == KindArray: | |
| 59 | + | for _, v := range left.(*Array).items { | |
| 60 | + | if err := e.tick(); err != nil { | |
| 61 | + | return nil, err | |
| 62 | + | } | |
| 63 | + | if equals(right, v) { | |
| 64 | + | return Boolean(true), nil | |
| 65 | + | } | |
| 66 | + | } | |
| 67 | + | } | |
| 68 | + | return Boolean(false), nil | |
| 69 | + | } | |
| 70 | + | ||
| 71 | + | func funcStartsWith(e *evaluator, args []Value) (Value, error) { | |
| 72 | + | ls, rs, ok := primitivePair(args) | |
| 73 | + | return Boolean(ok && jsHasPrefix(toUpperSpecial(ls), toUpperSpecial(rs))), nil | |
| 74 | + | } | |
| 75 | + | ||
| 76 | + | func funcEndsWith(e *evaluator, args []Value) (Value, error) { | |
| 77 | + | ls, rs, ok := primitivePair(args) | |
| 78 | + | return Boolean(ok && jsHasSuffix(toUpperSpecial(ls), toUpperSpecial(rs))), nil | |
| 79 | + | } | |
| 80 | + | ||
| 81 | + | // primitivePair returns the string forms of the two arguments, and false | |
| 82 | + | // when either is an array or a dictionary. | |
| 83 | + | func primitivePair(args []Value) (string, string, bool) { | |
| 84 | + | if !isPrimitive(args[0]) || !isPrimitive(args[1]) { | |
| 85 | + | return "", "", false | |
| 86 | + | } | |
| 87 | + | return CoerceString(args[0]), CoerceString(args[1]), true | |
| 88 | + | } | |
| 89 | + | ||
| 90 | + | // funcJoin is join(array, separator). A primitive first argument is just | |
| 91 | + | // converted to a string, ignoring the separator; a dictionary gives "". A | |
| 92 | + | // separator that is not primitive means ",". | |
| 93 | + | func funcJoin(e *evaluator, args []Value) (Value, error) { | |
| 94 | + | first := normalize(args[0]) | |
| 95 | + | if isPrimitive(first) { | |
| 96 | + | return String(CoerceString(first)), nil | |
| 97 | + | } | |
| 98 | + | arr, ok := first.(*Array) | |
| 99 | + | if !ok { | |
| 100 | + | return String(""), nil | |
| 101 | + | } | |
| 102 | + | sep := "," | |
| 103 | + | if len(args) > 1 && isPrimitive(args[1]) { | |
| 104 | + | sep = CoerceString(args[1]) | |
| 105 | + | } | |
| 106 | + | var out strings.Builder | |
| 107 | + | for i, it := range arr.items { | |
| 108 | + | if err := e.tick(); err != nil { | |
| 109 | + | return nil, err | |
| 110 | + | } | |
| 111 | + | part := CoerceString(it) | |
| 112 | + | n := len(part) | |
| 113 | + | if i > 0 { | |
| 114 | + | n += len(sep) | |
| 115 | + | } | |
| 116 | + | if err := e.charge(n); err != nil { | |
| 117 | + | return nil, err | |
| 118 | + | } | |
| 119 | + | if i > 0 { | |
| 120 | + | out.WriteString(sep) | |
| 121 | + | } | |
| 122 | + | out.WriteString(part) | |
| 123 | + | } | |
| 124 | + | return String(normalizeWTF8(out.String())), nil | |
| 125 | + | } | |
| 126 | + | ||
| 127 | + | // funcToJSON is toJSON(value). The output counts against the budget and is | |
| 128 | + | // cut off as soon as it exceeds what is left. | |
| 129 | + | func funcToJSON(e *evaluator, args []Value) (Value, error) { | |
| 130 | + | max := -1 | |
| 131 | + | if e.limit >= 0 { | |
| 132 | + | max = e.limit - e.used | |
| 133 | + | } | |
| 134 | + | out, err := EncodeJSON(args[0], max) | |
| 135 | + | switch { | |
| 136 | + | case errors.Is(err, errJSONTooBig): | |
| 137 | + | return nil, e.budgetError() | |
| 138 | + | case err != nil: | |
| 139 | + | return nil, &EvalError{Message: "toJSON: " + err.Error(), Err: err} | |
| 140 | + | } | |
| 141 | + | if err := e.charge(len(out)); err != nil { | |
| 142 | + | return nil, err | |
| 143 | + | } | |
| 144 | + | return String(out), nil | |
| 145 | + | } | |
| 146 | + | ||
| 147 | + | // funcFromJSON is fromJSON(value). Blank input is "empty input"; anything | |
| 148 | + | // that is not valid JSON is "Error parsing JSON when evaluating fromJson" | |
| 149 | + | // wrapping the syntax error. | |
| 150 | + | func funcFromJSON(e *evaluator, args []Value) (Value, error) { | |
| 151 | + | in := CoerceString(args[0]) | |
| 152 | + | if jsTrim(in) == "" { | |
| 153 | + | return nil, evalErrorf("empty input") | |
| 154 | + | } | |
| 155 | + | v, err := FromJSON([]byte(in)) | |
| 156 | + | if err != nil { | |
| 157 | + | return nil, &EvalError{Message: "Error parsing JSON when evaluating fromJson", Err: err} | |
| 158 | + | } | |
| 159 | + | return v, nil | |
| 160 | + | } | |
| 161 | + | ||
| 162 | + | // funcFormat is format(fmt, args...): "{N}" inserts argument N, "{{" and "}}" | |
| 163 | + | // are literal braces. Anything between the digits of "{N" and the closing | |
| 164 | + | // brace is ignored ("{0:x}" is "{0}"). Every other use of a brace is an | |
| 165 | + | // error, as is a reference to a missing argument. | |
| 166 | + | func funcFormat(e *evaluator, args []Value) (Value, error) { | |
| 167 | + | fs := CoerceString(args[0]) | |
| 168 | + | var out strings.Builder | |
| 169 | + | index := 0 | |
| 170 | + | for index < len(fs) { | |
| 171 | + | lbrace := indexByteFrom(fs, '{', index) | |
| 172 | + | rbrace := indexByteFrom(fs, '}', index) | |
| 173 | + | switch { | |
| 174 | + | case lbrace >= 0 && (rbrace < 0 || rbrace > lbrace): | |
| 175 | + | // Escaped left brace. | |
| 176 | + | if charAt(fs, lbrace+1) == '{' { | |
| 177 | + | if err := e.charge(lbrace + 1 - index); err != nil { | |
| 178 | + | return nil, err | |
| 179 | + | } | |
| 180 | + | out.WriteString(fs[index : lbrace+1]) | |
| 181 | + | index = lbrace + 2 | |
| 182 | + | continue | |
| 183 | + | } | |
| 184 | + | // Left brace, number, optional format specifiers, right brace. | |
| 185 | + | if rbrace > lbrace+1 { | |
| 186 | + | if argIndex, ok := readArgIndex(fs, lbrace+1); ok { | |
| 187 | + | if 1+argIndex > float64(len(args)-1) { | |
| 188 | + | return nil, evalErrorf("The following format string references more arguments than were supplied: %s", fs) | |
| 189 | + | } | |
| 190 | + | arg := CoerceString(args[1+int(argIndex)]) | |
| 191 | + | if err := e.charge(lbrace - index + len(arg)); err != nil { | |
| 192 | + | return nil, err | |
| 193 | + | } | |
| 194 | + | out.WriteString(fs[index:lbrace]) | |
| 195 | + | out.WriteString(arg) | |
| 196 | + | index = rbrace + 1 | |
| 197 | + | continue | |
| 198 | + | } | |
| 199 | + | } | |
| 200 | + | return nil, evalErrorf("The following format string is invalid: %s", fs) | |
| 201 | + | case rbrace >= 0: | |
| 202 | + | // Escaped right brace. | |
| 203 | + | if charAt(fs, rbrace+1) != '}' { | |
| 204 | + | return nil, evalErrorf("The following format string is invalid: %s", fs) | |
| 205 | + | } | |
| 206 | + | if err := e.charge(rbrace + 1 - index); err != nil { | |
| 207 | + | return nil, err | |
| 208 | + | } | |
| 209 | + | out.WriteString(fs[index : rbrace+1]) | |
| 210 | + | index = rbrace + 2 | |
| 211 | + | default: | |
| 212 | + | if err := e.charge(len(fs) - index); err != nil { | |
| 213 | + | return nil, err | |
| 214 | + | } | |
| 215 | + | out.WriteString(fs[index:]) | |
| 216 | + | index = len(fs) | |
| 217 | + | } | |
| 218 | + | } | |
| 219 | + | return String(normalizeWTF8(out.String())), nil | |
| 220 | + | } | |
| 221 | + | ||
| 222 | + | func indexByteFrom(s string, c byte, from int) int { | |
| 223 | + | if from >= len(s) { | |
| 224 | + | return -1 | |
| 225 | + | } | |
| 226 | + | i := strings.IndexByte(s[from:], c) | |
| 227 | + | if i < 0 { | |
| 228 | + | return -1 | |
| 229 | + | } | |
| 230 | + | return from + i | |
| 231 | + | } | |
| 232 | + | ||
| 233 | + | // charAt returns the byte at i, or 0 past the end (safeCharAt). | |
| 234 | + | func charAt(s string, i int) byte { | |
| 235 | + | if i < len(s) { | |
| 236 | + | return s[i] | |
| 237 | + | } | |
| 238 | + | return 0 | |
| 239 | + | } | |
| 240 | + | ||
| 241 | + | // readArgIndex reads the decimal digits at s[start:]. The index is a float | |
| 242 | + | // because parseInt in the reference yields Infinity for absurdly long digit | |
| 243 | + | // runs, which then fails the argument-count check. | |
| 244 | + | func readArgIndex(s string, start int) (float64, bool) { | |
| 245 | + | n := 0 | |
| 246 | + | for charAt(s, start+n) >= '0' && charAt(s, start+n) <= '9' { | |
| 247 | + | n++ | |
| 248 | + | } | |
| 249 | + | if n == 0 { | |
| 250 | + | return 0, false | |
| 251 | + | } | |
| 252 | + | f, _ := strconv.ParseFloat(s[start:start+n], 64) | |
| 253 | + | if math.IsNaN(f) { | |
| 254 | + | return 0, false | |
| 255 | + | } | |
| 256 | + | return f, true | |
| 257 | + | } | |
A expr/fuzz_test.go +191 -0
| @@ -0,0 +1,191 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "strings" | |
| 8 | + | "testing" | |
| 9 | + | "unicode/utf8" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | // Seeds for the fuzz targets: the corpus of the table tests and a few shapes | |
| 13 | + | // that stress the lexer and the parser. | |
| 14 | + | var fuzzSeeds = []string{ | |
| 15 | + | "", " ", "true", "github.event_name == 'push'", "1 == '1'", "'it''s'", "'unterminated", "a-b", "NaN", "-Infinity", "0x1F", "1e3", ".5", "5.", | |
| 16 | + | "github.event.commits.*.id", "github['x'][0].*", "contains(github.ref, 'main') && !failure()", "format('{0} {{}} {1}', 1, null)", | |
| 17 | + | "fromJSON('[1,{\"a\":null}]')[1].a", "toJSON(fromJSON('{\"b\":1,\"1\":2}'))", "join(matrix.node.*, ', ')", "case(true, 1, 2)", "case(false, 1, true, 2, 3)", | |
| 18 | + | "((((((((((1))))))))))", "!!!!!!true", "1 || 2 && 3 == 4 < 5", "hashFiles('**/*.go', '!vendor/**')", "success() && always()", "\n\t \r", "\u00a0", "'😀' == '\U00010000'", | |
| 19 | + | "=", "&", "|", "!=", "<=", ">=", "()", "(1", "1)", "a.", ".a", "a..b", "a[", "a[]", "a[*]", "*", "a.*.*", "1.2.3", "--1", "1_000", "'\xed\xa0\x80'", | |
| 20 | + | "fromJSON('\"\\ud800\"') == fromJSON('\"\\udc00\"')", "format('{0}{1}', fromJSON('\"\\ud83d\"'), fromJSON('\"\\ude00\"'))", "contains(fromJSON('\"\\ud83d\\ude00\"'), fromJSON('\"\\ude00\"'))", "format('{0}{0}', format('{0}{0}', 'xxxxxxxx'))", "github.event.commits.*.*.*", "format('{0:x}{', 1)", "format('}}{{', 1)", "contains(1,2,3)", "ß == 'SS'", "İ", "'ı' == 'I'", "'\u03a3' < 'z'", | |
| 21 | + | } | |
| 22 | + | ||
| 23 | + | func addSeeds(f *testing.F) { | |
| 24 | + | for _, s := range fuzzSeeds { | |
| 25 | + | f.Add(s) | |
| 26 | + | } | |
| 27 | + | } | |
| 28 | + | ||
| 29 | + | // FuzzLex checks that lexing never panics, that it is deterministic, and | |
| 30 | + | // that successful lexing yields ordered tokens ending with exactly one EOF. | |
| 31 | + | func FuzzLex(f *testing.F) { | |
| 32 | + | addSeeds(f) | |
| 33 | + | f.Fuzz(func(t *testing.T, s string) { | |
| 34 | + | toks, err := Lex(s) | |
| 35 | + | toks2, err2 := Lex(s) | |
| 36 | + | if (err == nil) != (err2 == nil) || len(toks) != len(toks2) { | |
| 37 | + | t.Fatal("lexing is not deterministic") | |
| 38 | + | } | |
| 39 | + | if err != nil { | |
| 40 | + | if _, ok := err.(*Error); !ok { | |
| 41 | + | t.Fatalf("error is %T, not *Error", err) | |
| 42 | + | } | |
| 43 | + | return | |
| 44 | + | } | |
| 45 | + | if len(toks) == 0 || toks[len(toks)-1].Type != TokenEOF { | |
| 46 | + | t.Fatal("token stream does not end with EOF") | |
| 47 | + | } | |
| 48 | + | for i, tk := range toks { | |
| 49 | + | if tk.Type == TokenEOF && i != len(toks)-1 { | |
| 50 | + | t.Fatal("EOF in the middle of the token stream") | |
| 51 | + | } | |
| 52 | + | if tk.Type != TokenEOF && tk.Lexeme == "" { | |
| 53 | + | t.Fatalf("empty lexeme for token %v", tk.Type) | |
| 54 | + | } | |
| 55 | + | if !utf8.ValidString(tk.Lexeme) && utf8.ValidString(s) { | |
| 56 | + | t.Fatalf("invalid UTF-8 lexeme %q from valid input", tk.Lexeme) | |
| 57 | + | } | |
| 58 | + | // The EOF token of the reference starts at a stale offset, so after | |
| 59 | + | // a trailing newline its start column is -1; every other token | |
| 60 | + | // has a column of at least 0. | |
| 61 | + | if tk.Range.Start.Line < 0 || (tk.Type != TokenEOF && tk.Range.Start.Column < 0) || tk.Range.End.Line < tk.Range.Start.Line { | |
| 62 | + | t.Fatalf("bad range %+v", tk.Range) | |
| 63 | + | } | |
| 64 | + | } | |
| 65 | + | }) | |
| 66 | + | } | |
| 67 | + | ||
| 68 | + | // FuzzParse checks that parsing never panics, that errors are *Error and | |
| 69 | + | // that the depth limit holds: no accepted expression nests deeper than the | |
| 70 | + | // parser allows. | |
| 71 | + | func FuzzParse(f *testing.F) { | |
| 72 | + | addSeeds(f) | |
| 73 | + | f.Fuzz(func(t *testing.T, s string) { | |
| 74 | + | node, err := Parse(s, []string{"github", "matrix", "env"}, []FuncInfo{{Name: "hashFiles", MinArgs: 1, MaxArgs: -1}, {Name: "success"}}) | |
| 75 | + | if err != nil { | |
| 76 | + | if _, ok := err.(*Error); !ok { | |
| 77 | + | t.Fatalf("error is %T, not *Error", err) | |
| 78 | + | } | |
| 79 | + | if node != nil { | |
| 80 | + | t.Fatal("node returned with an error") | |
| 81 | + | } | |
| 82 | + | return | |
| 83 | + | } | |
| 84 | + | if node == nil && strings.TrimFunc(s, func(r rune) bool { return r == ' ' || r == '\t' || r == '\r' || r == '\n' }) != "" { | |
| 85 | + | t.Fatalf("empty tree for non-blank input %q", s) | |
| 86 | + | } | |
| 87 | + | if d := nodeDepth(node); d > 2*MaxParserDepth+2 { | |
| 88 | + | t.Fatalf("tree depth %d exceeds the parser limit", d) | |
| 89 | + | } | |
| 90 | + | }) | |
| 91 | + | } | |
| 92 | + | ||
| 93 | + | func nodeDepth(n Node) int { | |
| 94 | + | switch n := n.(type) { | |
| 95 | + | case *Unary: | |
| 96 | + | return 1 + nodeDepth(n.Expr) | |
| 97 | + | case *Binary: | |
| 98 | + | return 1 + max(nodeDepth(n.Left), nodeDepth(n.Right)) | |
| 99 | + | case *Logical: | |
| 100 | + | d := 0 | |
| 101 | + | for _, a := range n.Args { | |
| 102 | + | d = max(d, nodeDepth(a)) | |
| 103 | + | } | |
| 104 | + | return 1 + d | |
| 105 | + | case *Grouping: | |
| 106 | + | return 1 + nodeDepth(n.Group) | |
| 107 | + | case *IndexAccess: | |
| 108 | + | return 1 + max(nodeDepth(n.Expr), nodeDepth(n.Index)) | |
| 109 | + | case *FunctionCall: | |
| 110 | + | d := 0 | |
| 111 | + | for _, a := range n.Args { | |
| 112 | + | d = max(d, nodeDepth(a)) | |
| 113 | + | } | |
| 114 | + | return 1 + d | |
| 115 | + | } | |
| 116 | + | return 1 | |
| 117 | + | } | |
| 118 | + | ||
| 119 | + | // FuzzEvaluate checks that evaluating an arbitrary parsed expression never | |
| 120 | + | // panics, always returns a usable value or an *EvalError, and leaves the | |
| 121 | + | // context untouched. | |
| 122 | + | func FuzzEvaluate(f *testing.F) { | |
| 123 | + | addSeeds(f) | |
| 124 | + | ctx := EvalContext{ | |
| 125 | + | Contexts: map[string]Value{ | |
| 126 | + | "github": mustJSON(f, `{"event_name":"push","ref":"refs/heads/main","event":{"commits":[{"id":"1","added":["a"]},{"id":"2"},{"x":[[1],[2,3]]}],"n":12,"nested":[[1,2],[3]]}}`), | |
| 127 | + | "matrix": mustJSON(f, `{"node":[14,16],"os":"linux","o":{"k":null}}`), | |
| 128 | + | "env": mustJSON(f, `{"A":"","B":"1","C":"x","J":"{\"a\":[1,2]}"}`), | |
| 129 | + | }, | |
| 130 | + | Funcs: map[string]Func{ | |
| 131 | + | "hashfiles": func(_ context.Context, args []Value) (Value, error) { return String("h"), nil }, | |
| 132 | + | "success": func(context.Context, []Value) (Value, error) { return Boolean(true), nil }, | |
| 133 | + | }, | |
| 134 | + | } | |
| 135 | + | before := map[string]string{} | |
| 136 | + | for k, v := range ctx.Contexts { | |
| 137 | + | before[k] = ToJSON(v) | |
| 138 | + | } | |
| 139 | + | f.Fuzz(func(t *testing.T, s string) { | |
| 140 | + | node, err := Parse(s, []string{"github", "matrix", "env"}, []FuncInfo{{Name: "hashFiles", MinArgs: 1, MaxArgs: -1}, {Name: "success"}}) | |
| 141 | + | if err != nil || node == nil { | |
| 142 | + | return | |
| 143 | + | } | |
| 144 | + | v, err := Evaluate(context.Background(), node, ctx) | |
| 145 | + | if err != nil { | |
| 146 | + | if _, ok := err.(*EvalError); !ok { | |
| 147 | + | t.Fatalf("error is %T, not *EvalError", err) | |
| 148 | + | } | |
| 149 | + | return | |
| 150 | + | } | |
| 151 | + | if v == nil { | |
| 152 | + | t.Fatal("nil value without error") | |
| 153 | + | } | |
| 154 | + | _ = CoerceString(v) | |
| 155 | + | _ = Truthy(v) | |
| 156 | + | j := ToJSON(v) | |
| 157 | + | if !utf8.ValidString(j) { | |
| 158 | + | t.Fatalf("toJSON produced invalid UTF-8: %q", j) | |
| 159 | + | } | |
| 160 | + | if _, err := FromJSON([]byte(j)); err != nil { | |
| 161 | + | t.Fatalf("toJSON output does not parse: %v\n%s", err, j) | |
| 162 | + | } | |
| 163 | + | for k, c := range ctx.Contexts { | |
| 164 | + | if ToJSON(c) != before[k] { | |
| 165 | + | t.Fatalf("context %s was modified", k) | |
| 166 | + | } | |
| 167 | + | } | |
| 168 | + | }) | |
| 169 | + | } | |
| 170 | + | ||
| 171 | + | // FuzzFromJSON checks that FromJSON never panics and that whatever it | |
| 172 | + | // accepts prints as JSON that parses back to the same text. | |
| 173 | + | func FuzzFromJSON(f *testing.F) { | |
| 174 | + | for _, s := range []string{`null`, `{"a":[1,2,{"b":null}]}`, `"\ud83d\ude00"`, `{"1":1,"a":2,"01":3}`, `[1e999]`, `-0`, `"\ud800"`, `{"a":1,"A":2}`, `[`, `{"a"`, "\"\x01\""} { | |
| 175 | + | f.Add(s) | |
| 176 | + | } | |
| 177 | + | f.Fuzz(func(t *testing.T, s string) { | |
| 178 | + | v, err := FromJSON([]byte(s)) | |
| 179 | + | if err != nil { | |
| 180 | + | return | |
| 181 | + | } | |
| 182 | + | j := ToJSON(v) | |
| 183 | + | v2, err := FromJSON([]byte(j)) | |
| 184 | + | if err != nil { | |
| 185 | + | t.Fatalf("round trip failed: %v\n%s", err, j) | |
| 186 | + | } | |
| 187 | + | if j2 := ToJSON(v2); j2 != j { | |
| 188 | + | t.Fatalf("round trip changed output:\n%s\n%s", j, j2) | |
| 189 | + | } | |
| 190 | + | }) | |
| 191 | + | } | |
A expr/gen/casing.go +116 -0
| @@ -0,0 +1,116 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | //go:build ignore | |
| 4 | + | ||
| 5 | + | // Command casing reads the JSON produced by casing_dump.mjs on stdin and | |
| 6 | + | // writes casing_tables.go (to the file given with -o, else on stdout). Only the code points whose JavaScript | |
| 7 | + | // mapping differs from the Go simple mapping (unicode.ToUpper/ToLower) are | |
| 8 | + | // emitted: multi-character expansions (U+00DF to "SS") and Unicode version | |
| 9 | + | // differences between Go and the JavaScript engine. | |
| 10 | + | package main | |
| 11 | + | ||
| 12 | + | import ( | |
| 13 | + | "encoding/json" | |
| 14 | + | "flag" | |
| 15 | + | "fmt" | |
| 16 | + | "go/format" | |
| 17 | + | "io" | |
| 18 | + | "os" | |
| 19 | + | "sort" | |
| 20 | + | "strconv" | |
| 21 | + | "unicode" | |
| 22 | + | ) | |
| 23 | + | ||
| 24 | + | type dump struct { | |
| 25 | + | Upper map[string]string `json:"upper"` | |
| 26 | + | Lower map[string]string `json:"lower"` | |
| 27 | + | } | |
| 28 | + | ||
| 29 | + | func main() { | |
| 30 | + | out := flag.String("o", "", "output file (default stdout)") | |
| 31 | + | flag.Parse() | |
| 32 | + | var d dump | |
| 33 | + | if err := json.NewDecoder(os.Stdin).Decode(&d); err != nil { | |
| 34 | + | fmt.Fprintln(os.Stderr, err) | |
| 35 | + | os.Exit(1) | |
| 36 | + | } | |
| 37 | + | // Everything is printed to a pipe, gofmt-ed and then written, so that the | |
| 38 | + | // generated file is already in canonical form. | |
| 39 | + | r, w, err := os.Pipe() | |
| 40 | + | if err != nil { | |
| 41 | + | fatal(err) | |
| 42 | + | } | |
| 43 | + | stdout := os.Stdout | |
| 44 | + | os.Stdout = w | |
| 45 | + | done := make(chan []byte) | |
| 46 | + | go func() { | |
| 47 | + | b, _ := io.ReadAll(r) | |
| 48 | + | done <- b | |
| 49 | + | }() | |
| 50 | + | defer func() { | |
| 51 | + | _ = w.Close() | |
| 52 | + | os.Stdout = stdout | |
| 53 | + | src, err := format.Source(<-done) | |
| 54 | + | if err != nil { | |
| 55 | + | fatal(err) | |
| 56 | + | } | |
| 57 | + | if *out == "" { | |
| 58 | + | _, _ = stdout.Write(src) | |
| 59 | + | return | |
| 60 | + | } | |
| 61 | + | if err := os.WriteFile(*out, src, 0o644); err != nil { | |
| 62 | + | fatal(err) | |
| 63 | + | } | |
| 64 | + | }() | |
| 65 | + | fmt.Println("// SPDX-License-Identifier: MIT") | |
| 66 | + | fmt.Println() | |
| 67 | + | fmt.Println("// Code generated by gen/casing.go from gen/casing_dump.mjs; DO NOT EDIT.") | |
| 68 | + | fmt.Println() | |
| 69 | + | fmt.Println("package expr") | |
| 70 | + | fmt.Println() | |
| 71 | + | fmt.Println("// casingGoUnicodeVersion is the Unicode version of the Go toolchain that generated") | |
| 72 | + | fmt.Println("// the tables below: they are relative to its unicode.ToUpper and unicode.ToLower.") | |
| 73 | + | fmt.Printf("const casingGoUnicodeVersion = %q\n", unicode.Version) | |
| 74 | + | fmt.Println() | |
| 75 | + | emit("jsUpperDelta", "JavaScript String.prototype.toUpperCase", d.Upper, unicode.ToUpper) | |
| 76 | + | emit("jsLowerDelta", "JavaScript String.prototype.toLowerCase (without the Final_Sigma context)", d.Lower, unicode.ToLower) | |
| 77 | + | } | |
| 78 | + | ||
| 79 | + | func emit(name, doc string, m map[string]string, goMap func(rune) rune) { | |
| 80 | + | // Include code points that JavaScript maps to themselves but Go does not. | |
| 81 | + | type entry struct { | |
| 82 | + | cp rune | |
| 83 | + | s string | |
| 84 | + | } | |
| 85 | + | var out []entry | |
| 86 | + | seen := map[rune]bool{} | |
| 87 | + | for k, v := range m { | |
| 88 | + | n, _ := strconv.Atoi(k) | |
| 89 | + | r := rune(n) | |
| 90 | + | seen[r] = true | |
| 91 | + | if string(goMap(r)) != v { | |
| 92 | + | out = append(out, entry{r, v}) | |
| 93 | + | } | |
| 94 | + | } | |
| 95 | + | for r := rune(0); r <= unicode.MaxRune; r++ { | |
| 96 | + | if r >= 0xd800 && r <= 0xdfff || seen[r] { | |
| 97 | + | continue | |
| 98 | + | } | |
| 99 | + | if goMap(r) != r { | |
| 100 | + | out = append(out, entry{r, string(r)}) | |
| 101 | + | } | |
| 102 | + | } | |
| 103 | + | sort.Slice(out, func(i, j int) bool { return out[i].cp < out[j].cp }) | |
| 104 | + | fmt.Printf("// %s maps the code points whose %s result differs from the Go simple mapping.\n", name, doc) | |
| 105 | + | fmt.Printf("var %s = map[rune]string{\n", name) | |
| 106 | + | for _, e := range out { | |
| 107 | + | fmt.Printf("\t0x%04X: %s, // %s\n", e.cp, strconv.Quote(e.s), strconv.QuoteToGraphic(string(e.cp))) | |
| 108 | + | } | |
| 109 | + | fmt.Println("}") | |
| 110 | + | fmt.Println() | |
| 111 | + | } | |
| 112 | + | ||
| 113 | + | func fatal(err error) { | |
| 114 | + | fmt.Fprintln(os.Stderr, err) | |
| 115 | + | os.Exit(1) | |
| 116 | + | } | |
A expr/gen/casing_dump.mjs +21 -0
| @@ -0,0 +1,21 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | // | |
| 3 | + | // Dumps the JavaScript toUpperCase/toLowerCase mapping of every code point | |
| 4 | + | // whose mapping is not the identity, as JSON on stdout. It is the input of | |
| 5 | + | // casing.go, which turns it into ../casing_tables.go: | |
| 6 | + | // | |
| 7 | + | // go generate ./expr (runs: node gen/casing_dump.mjs | go run gen/casing.go -o casing_tables.go) | |
| 8 | + | // | |
| 9 | + | // The mapping is that of the JavaScript engine running the script (Node/V8), | |
| 10 | + | // which is what @actions/expressions uses for case-insensitive comparisons. | |
| 11 | + | const upper = {}; | |
| 12 | + | const lower = {}; | |
| 13 | + | for (let cp = 0; cp <= 0x10ffff; cp++) { | |
| 14 | + | if (cp >= 0xd800 && cp <= 0xdfff) continue; | |
| 15 | + | const s = String.fromCodePoint(cp); | |
| 16 | + | const u = s.toUpperCase(); | |
| 17 | + | const l = s.toLowerCase(); | |
| 18 | + | if (u !== s) upper[cp] = u; | |
| 19 | + | if (l !== s) lower[cp] = l; | |
| 20 | + | } | |
| 21 | + | process.stdout.write(JSON.stringify({ upper, lower })); | |
A expr/gen/golden.mjs +69 -0
| @@ -0,0 +1,69 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | // | |
| 3 | + | // Writes the golden files of TestToJSONGolden: for each input testdata/tojson/NAME.json | |
| 4 | + | // it writes NAME.golden, which is what the reference implementation | |
| 5 | + | // (@actions/expressions) prints for toJSON(fromJSON(input)), that is | |
| 6 | + | // JSON.stringify(JSON.parse(input, reviver), replacer, " "). | |
| 7 | + | // | |
| 8 | + | // node gen/golden.mjs /path/to/@actions/expressions/dist testdata/tojson | |
| 9 | + | // | |
| 10 | + | // New inputs are added by hand to that directory (and listed in INPUTS below | |
| 11 | + | // when they are generated rather than written out). | |
| 12 | + | ||
| 13 | + | import { pathToFileURL } from "node:url"; | |
| 14 | + | import path from "node:path"; | |
| 15 | + | import fs from "node:fs"; | |
| 16 | + | ||
| 17 | + | const dist = path.resolve(process.argv[2]); | |
| 18 | + | const dir = process.argv[3]; | |
| 19 | + | const { data } = await import(pathToFileURL(path.join(dist, "index.js")).href); | |
| 20 | + | ||
| 21 | + | const INPUTS = { | |
| 22 | + | null: "null", | |
| 23 | + | true: "true", | |
| 24 | + | false: "false", | |
| 25 | + | zero: "0", | |
| 26 | + | negzero: "-0", | |
| 27 | + | int: "42", | |
| 28 | + | negative: "-17", | |
| 29 | + | float: "1.5", | |
| 30 | + | exp_small: "1e-7", | |
| 31 | + | exp_large: "1e21", | |
| 32 | + | exp_20: "1e20", | |
| 33 | + | exp_pos: "1E2", | |
| 34 | + | precision: "0.30000000000000004", | |
| 35 | + | big_int: "12345678901234567890", | |
| 36 | + | max_double: "1.7976931348623157e308", | |
| 37 | + | min_double: "5e-324", | |
| 38 | + | overflow: "1e999", | |
| 39 | + | string_empty: '""', | |
| 40 | + | string_plain: '"hello"', | |
| 41 | + | string_escapes: '"quote \\" backslash \\\\ slash \\/ nl \\n cr \\r tab \\t bs \\b ff \\f"', | |
| 42 | + | string_control: '"\\u0000\\u0001\\u001f\\u007f\\u0080"', | |
| 43 | + | string_unicode: '"h\\u00e9llo \\u4e16\\u754c \\ud83d\\ude00 \\u2028\\u2029"', | |
| 44 | + | string_lone_surrogate: '"\\ud800"', | |
| 45 | + | array_empty: "[]", | |
| 46 | + | array_flat: "[1,2,3]", | |
| 47 | + | array_mixed: '[null,true,"a",1.5,[],{}]', | |
| 48 | + | array_nested: "[[1,[2,[3,[]]]],[]]", | |
| 49 | + | object_empty: "{}", | |
| 50 | + | object_flat: '{"a":1,"b":"two","c":null}', | |
| 51 | + | object_nested: '{"a":{"b":{"c":{"d":[1,{"e":2}]}}},"f":[]}', | |
| 52 | + | object_order: '{"b":1,"a":2,"c":3}', | |
| 53 | + | object_numeric_keys: '{"b":1,"10":2,"2":3,"a":4,"01":5,"1":6,"4294967294":7,"4294967295":8,"-1":9}', | |
| 54 | + | object_case_dup: '{"a":1,"A":2,"b":3,"B":4}', | |
| 55 | + | object_dup_key: '{"a":1,"b":2,"a":3}', | |
| 56 | + | object_empty_key: '{"":1,"a":2}', | |
| 57 | + | object_key_escapes: '{"quote\\"key":1,"new\\nline":2,"\\u00e9":3}', | |
| 58 | + | github_event: '{"action":"opened","number":1,"pull_request":{"id":1,"draft":false,"labels":[{"name":"bug","color":"d73a4a"},{"name":"docs","color":null}],"head":{"ref":"feat/x","sha":"abc123"}},"repository":{"full_name":"octo/repo","private":true,"topics":[]}}', | |
| 59 | + | }; | |
| 60 | + | ||
| 61 | + | fs.mkdirSync(dir, { recursive: true }); | |
| 62 | + | for (const [name, text] of Object.entries(INPUTS)) { | |
| 63 | + | fs.writeFileSync(path.join(dir, name + ".json"), text + "\n"); | |
| 64 | + | } | |
| 65 | + | for (const f of fs.readdirSync(dir).filter(f => f.endsWith(".json")).sort()) { | |
| 66 | + | const text = fs.readFileSync(path.join(dir, f), "utf8"); | |
| 67 | + | const value = JSON.parse(text, data.reviver); | |
| 68 | + | fs.writeFileSync(path.join(dir, f.replace(/\.json$/, ".golden")), JSON.stringify(value, data.replacer, " ")); | |
| 69 | + | } | |
A expr/gen/oracle.mjs +366 -0
| @@ -0,0 +1,366 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | // | |
| 3 | + | // Generates testdata/oracle.json.gz: the results of the reference implementation | |
| 4 | + | // (@actions/expressions 0.3.61, MIT, GitHub) on a corpus of expressions. The Go | |
| 5 | + | // tests (oracle_test.go) replay the corpus and compare token by token what | |
| 6 | + | // GitHub's JavaScript library does with what this package does. | |
| 7 | + | // | |
| 8 | + | // node gen/oracle.mjs /path/to/@actions/expressions/dist [/path/to/actions-expressions-X.tgz] \\ | |
| 9 | + | // | gzip -9 -n > testdata/oracle.json.gz | |
| 10 | + | // | |
| 11 | + | // The optional tarball is the npm package the dist directory came from: its | |
| 12 | + | // sha512 (base64, the "integrity" field of the npm registry) is recorded next | |
| 13 | + | // to the Node version in the output, so the provenance of the oracle is | |
| 14 | + | // checkable. | |
| 15 | + | // | |
| 16 | + | // The corpus is hand-written edge cases plus a seeded pseudo-random walk over | |
| 17 | + | // the grammar and over mutations of it, so the file is reproducible. | |
| 18 | + | ||
| 19 | + | import { createHash } from "node:crypto"; | |
| 20 | + | import { readFileSync } from "node:fs"; | |
| 21 | + | import { pathToFileURL } from "node:url"; | |
| 22 | + | import path from "node:path"; | |
| 23 | + | ||
| 24 | + | const dist = path.resolve(process.argv[2] ?? "node_modules/@actions/expressions/dist"); | |
| 25 | + | const ex = await import(pathToFileURL(path.join(dist, "index.js")).href); | |
| 26 | + | const { Lexer, Parser, Evaluator, data } = ex; | |
| 27 | + | ||
| 28 | + | const pkgJson = JSON.parse(readFileSync(path.join(dist, "..", "package.json"), "utf8")); | |
| 29 | + | const integrity = process.argv[3] | |
| 30 | + | ? "sha512-" + createHash("sha512").update(readFileSync(process.argv[3])).digest("base64") | |
| 31 | + | : ""; | |
| 32 | + | const reference = { | |
| 33 | + | package: `${pkgJson.name} ${pkgJson.version}`, | |
| 34 | + | integrity, | |
| 35 | + | node: process.version, | |
| 36 | + | v8: process.versions.v8, | |
| 37 | + | unicode: process.versions.unicode, | |
| 38 | + | }; | |
| 39 | + | ||
| 40 | + | // ---- contexts ------------------------------------------------------------- | |
| 41 | + | ||
| 42 | + | const contexts = { | |
| 43 | + | github: { | |
| 44 | + | event_name: "push", | |
| 45 | + | ref: "refs/heads/main", | |
| 46 | + | sha: "0123abc", | |
| 47 | + | actor: "Octocat", | |
| 48 | + | repository: "octo/Hello-World", | |
| 49 | + | run_number: 7, | |
| 50 | + | run_id: "1658821493", | |
| 51 | + | event: { | |
| 52 | + | number: 12, | |
| 53 | + | commits: [ | |
| 54 | + | { id: "1", message: "fix: bug", author: { name: "Ann", email: "a@x" }, added: ["a.txt"] }, | |
| 55 | + | { id: "2", message: "feat: thing", author: { name: "Bob" }, added: [] }, | |
| 56 | + | { id: "3", message: "Chore", added: ["b", "c"] }, | |
| 57 | + | ], | |
| 58 | + | head_commit: { message: "hello", timestamp: null }, | |
| 59 | + | labels: [{ name: "bug" }, { name: "Docs" }, { name: "help wanted" }], | |
| 60 | + | pull_request: { draft: false, title: "A title", labels: [{ name: "x" }] }, | |
| 61 | + | nested: [[1, 2], [3], [], [4, 5, 6]], | |
| 62 | + | }, | |
| 63 | + | "weird-key": 1, | |
| 64 | + | "a b": "space", | |
| 65 | + | }, | |
| 66 | + | env: { FOO: "bar", EMPTY: "", NUM: "42", TRUE: "true", JSONTEXT: '{"a":[1,2,3]}', Mixed: "MiXeD", Float: "1.50", Hex: "0x1F" }, | |
| 67 | + | matrix: { | |
| 68 | + | os: "ubuntu", | |
| 69 | + | node: [14, 16, 18], | |
| 70 | + | include: [{ os: "windows", node: 20 }, { os: "mac", experimental: true }], | |
| 71 | + | v: 1.5, | |
| 72 | + | flag: true, | |
| 73 | + | nothing: null, | |
| 74 | + | ver: "3.10", | |
| 75 | + | big: 12345678901234567890, | |
| 76 | + | neg: -0.25, | |
| 77 | + | tiny: 1e-7, | |
| 78 | + | zero: 0, | |
| 79 | + | obj: { k: "v" }, | |
| 80 | + | }, | |
| 81 | + | inputs: { name: "World", count: 3, deploy: false, list: ["a", "b"], empty: {} }, | |
| 82 | + | steps: { "10": { outputs: { r: "ten" } }, "2": { outputs: { r: "two" } }, build: { outputs: { r: "b", Other: "o" }, outcome: "success" } }, | |
| 83 | + | needs: {}, | |
| 84 | + | vars: { A: "1", B: "two" }, | |
| 85 | + | }; | |
| 86 | + | const ctxNames = Object.keys(contexts); | |
| 87 | + | ||
| 88 | + | const extFuncs = [ | |
| 89 | + | { name: "success", description: "", minArgs: 0, maxArgs: 0, call: () => new data.BooleanData(true) }, | |
| 90 | + | { name: "always", description: "", minArgs: 0, maxArgs: 0, call: () => new data.BooleanData(true) }, | |
| 91 | + | { | |
| 92 | + | name: "hashFiles", | |
| 93 | + | description: "", | |
| 94 | + | minArgs: 1, | |
| 95 | + | maxArgs: 255, | |
| 96 | + | call: (...a) => new data.StringData("hash:" + a.map(x => x.coerceString()).join("|")), | |
| 97 | + | }, | |
| 98 | + | ]; | |
| 99 | + | const extInfos = extFuncs.map(f => ({ name: f.name, description: f.description, minArgs: f.minArgs, maxArgs: f.maxArgs })); | |
| 100 | + | const funcMap = new Map(extFuncs.map(f => [f.name.toLowerCase(), f])); | |
| 101 | + | ||
| 102 | + | const rootDict = new data.Dictionary( | |
| 103 | + | ...Object.keys(contexts).map(k => ({ key: k, value: JSON.parse(JSON.stringify(contexts[k]), data.reviver) })) | |
| 104 | + | ); | |
| 105 | + | ||
| 106 | + | // ---- running the reference ------------------------------------------------- | |
| 107 | + | ||
| 108 | + | function kindOf(v) { | |
| 109 | + | switch (v.kind) { | |
| 110 | + | case data.Kind.Null: return "null"; | |
| 111 | + | case data.Kind.Boolean: return "boolean"; | |
| 112 | + | case data.Kind.Number: return "number"; | |
| 113 | + | case data.Kind.String: return "string"; | |
| 114 | + | case data.Kind.Array: return "array"; | |
| 115 | + | case data.Kind.Dictionary: return "object"; | |
| 116 | + | } | |
| 117 | + | return "?"; | |
| 118 | + | } | |
| 119 | + | ||
| 120 | + | function run(expr) { | |
| 121 | + | const out = { e: expr }; | |
| 122 | + | let tokens; | |
| 123 | + | try { | |
| 124 | + | tokens = new Lexer(expr).lex().tokens; | |
| 125 | + | } catch (e) { | |
| 126 | + | if (!(e instanceof Error) || e instanceof TypeError || e instanceof RangeError) { out.skip = String(e); return out; } | |
| 127 | + | out.lexErr = e.message; | |
| 128 | + | return out; | |
| 129 | + | } | |
| 130 | + | out.tokens = tokens.map(t => [t.type, t.lexeme, t.range.start.line, t.range.start.column, t.range.end.line, t.range.end.column, | |
| 131 | + | t.type === 17 ? (Number.isNaN(t.value) ? "NaN" : String(t.value)) : t.type === 18 ? t.value : null]); | |
| 132 | + | let ast; | |
| 133 | + | try { | |
| 134 | + | ast = new Parser(tokens, ctxNames, extInfos).parse(); | |
| 135 | + | } catch (e) { | |
| 136 | + | if (e instanceof TypeError || e instanceof RangeError) { out.skip = String(e); return out; } | |
| 137 | + | out.parseErr = e.message; | |
| 138 | + | if (e.pos) out.pos = [e.pos.line, e.pos.column]; | |
| 139 | + | return out; | |
| 140 | + | } | |
| 141 | + | if (ast === undefined) { out.empty = true; return out; } | |
| 142 | + | let res; | |
| 143 | + | try { | |
| 144 | + | res = new Evaluator(ast, rootDict, funcMap).evaluate(); | |
| 145 | + | } catch (e) { | |
| 146 | + | if (e instanceof TypeError || e instanceof RangeError) { out.skip = String(e); return out; } | |
| 147 | + | out.evalErr = e.message; | |
| 148 | + | return out; | |
| 149 | + | } | |
| 150 | + | if (res === undefined) { out.skip = "undefined result"; return out; } | |
| 151 | + | const str = res.coerceString(); | |
| 152 | + | out.r = { k: kindOf(res), s: str, j: JSON.stringify(res, data.replacer, " ") }; | |
| 153 | + | if (!str.isWellFormed()) { | |
| 154 | + | // A string holding a lone surrogate cannot travel through JSON text and | |
| 155 | + | // UTF-8: give its UTF-16 code units instead. | |
| 156 | + | out.r.s = ""; | |
| 157 | + | out.r.u = Array.from({ length: str.length }, (_, i) => str.charCodeAt(i)); | |
| 158 | + | } | |
| 159 | + | return out; | |
| 160 | + | } | |
| 161 | + | ||
| 162 | + | // ---- corpus ---------------------------------------------------------------- | |
| 163 | + | ||
| 164 | + | const corpus = []; | |
| 165 | + | const add = (...xs) => corpus.push(...xs); | |
| 166 | + | ||
| 167 | + | // literals, numbers | |
| 168 | + | add("true", "false", "null", "NaN", "Infinity", "-Infinity", "+Infinity", "1", "-1", "+1", "1.5", ".5", "-.5", "5.", "1e3", "1E-3", "1e21", "1e-7", | |
| 169 | + | "0x10", "0X1f", "0o17", "0b101", "-0x10", "0x", "0b2", "1_000", "1e", "1e+", "--1", "+-1", "-", "+", ".", "..", "1.2.3", "0.1", "0.30000000000000004", | |
| 170 | + | "100000000000000000000", "1000000000000000000000", "123456789012345678", "-0", "0", "00", "007", "1e308", "1e309", "-1e309", "4.9e-324", | |
| 171 | + | "0.000001", "0.0000001", "123456789.123456789", "0.1234567890123456789", "9.9999999999999995", "0.9999999999999999", "0.99999999999999999", | |
| 172 | + | "1.0000000000000002", "2.5", "0.5", "1.005", "1.45", "8.345", "0.000000000000005", "0.0000000000000005", "0.0000000000000004999", | |
| 173 | + | "1e15", "1e16", "123456789012345.6", "12345678901234567.5"); | |
| 174 | + | for (const n of ["0.1", "0.5", "1.5", "2.5", "1e21", "1e-7", "1e300", "-1e-7", "0.000001", "123456789012345678901", "5e-324", "1.7976931348623157e308", | |
| 175 | + | "0.30000000000000004", "1.005", "0.1234567890123456", "0.12345678901234567", "1234567.1234567890123456", "-0.0000000000000005", "0.0000000000000005", | |
| 176 | + | "NaN", "Infinity", "-Infinity", "100", "1e20", "123e-20", "99999999999999999999", "0.000000000000001", "0.0000000000000011", "4503599627370497.5", | |
| 177 | + | "0.45", "0.55", "-1.5", "1.0000000000000005", "1.0000000000000004", "10.000000000000005"]) { | |
| 178 | + | add(`format('{0}', ${n})`, `toJSON(${n})`, `${n} == '${n}'`, `'${n}' == ${n}`, `join(fromJSON('[${n}]'), ',')`); | |
| 179 | + | } | |
| 180 | + | ||
| 181 | + | // strings | |
| 182 | + | add("'a'", "''", "''''", "'it''s'", "'a''b''c'", "'''a'", "'a'''", "'a''''b'", "'unterminated", "'", "'a'b", "'a' 'b'", "'multi\nline'", | |
| 183 | + | "'\u00e9'", "'\u{1F600}'", "'\u{1F600}' == '\u{1F600}'", "\"dq\"", "'tab\there'"); | |
| 184 | + | ||
| 185 | + | // whitespace & lexing oddities | |
| 186 | + | add(" true ", "\ttrue\r\n", "\n\ntrue", "\u00a0true", "true\u00a0", "tr\u00a0ue", "\vtrue", "true\u2028", "true\ufeff", "a", "github", "GITHUB", "GitHub.Event_Name", | |
| 187 | + | "github .event_name", "github. event_name", "github.\nevent_name", "github\n.event_name", "github.event_name.", "github..event_name", ".github", "github.", | |
| 188 | + | "github[", "github[]", "github['event_name'", "github['event_name']]", "github[0]", "github[-1]", "github[1.5]", "github['0']", "github[true]", "github[null]", | |
| 189 | + | "github[nan]", "github[NaN]", "github[Infinity]", "github[github]", "github[github.event]", "github['EVENT_NAME']", "github[ 'event_name' ]", | |
| 190 | + | "=", "==", "===", "= =", "&", "&&", "&&&", "|", "||", "|||", "!", "!!", "!!!true", "!=", "!==", "!==true", "<", "<=", "<=>", ">", ">=", ">==", | |
| 191 | + | "1 = 1", "1 & 1", "1 | 1", "1 &&", "&& 1", "1 && && 1", "1 ||", "|| 1", "@", "#", "$", "1 + 1", "1 - 1", "1 * 1", "1 / 1", "a-b", "a_b", "a1", "1a", "_a", "-a", "a-", | |
| 192 | + | "github-", "github-x", "github.event-name", "github.event_name-x", "github.0", "github.1a", "github.-x", "github.true", "github.null", "github.NaN", "github.Infinity", | |
| 193 | + | "github.false", "env.TRUE", "env.true", "env.null", "(", ")", "()", "(())", "(1", "1)", "((1)", "(1))", "(1)(2)", "1 2", "1,2", ",", "1,", "[", "]", "[]", "[1]", "{}", | |
| 194 | + | "*", "*.a", "1.*", "'a'.b", "'a'[0]", "(github).event_name", "(github)['event_name']", "(github.event).number", "(1).a", "(1)[0]", "1[0]", "true.a", "null.a", "null[0]", | |
| 195 | + | "true[0]", "'abc'[0]", "'abc'.length", "(('a'))[0]", "toJSON(1)[0]", "fromJSON('[1,2]')[1]", "fromJSON('{\"a\":1}').a", "fromJSON('{\"a\":1}')['A']", "fromJSON('[[1]]')[0][0]"); | |
| 196 | + | ||
| 197 | + | // operators, coercions | |
| 198 | + | const vals = ["null", "true", "false", "0", "1", "-1", "0.5", "NaN", "Infinity", "''", "'0'", "'1'", "' 1 '", "'abc'", "'ABC'", "'a'", "'b'", "'true'", "'null'", "'NaN'", "'Infinity'", "'0x10'", "'1e2'", "'-1'", "'1.0'", "'\u00df'", "'SS'", "'ss'", "'\u0131'", "'I'", "'i'", "'\u0130'", "'\u03c3'", "'\u03c2'", "'\u03a3'", "'\ufb03'", "'\u212a'", "'k'", "'K'", "'\u{1F600}'", "'\ue000'", "'\u{10000}'", "github", "github.event.commits", "github.event.commits[0]", "env", "matrix.node", "matrix.obj", "inputs.empty", "fromJSON('[]')", "fromJSON('{}')", "github.event.commits.*", "github.event.nope.*", "matrix.nothing", "env.NUM", "env.EMPTY"]; | |
| 199 | + | for (const a of vals) { | |
| 200 | + | add(`!${a}`, `!!${a}`, `${a} || 'dflt'`, `${a} && 'then'`, `format('{0}', ${a})`, `toJSON(${a})`, `join(${a})`, `join(${a}, '-')`, `fromJSON(${a})`, `case(${a}, 'x', 'y', 'z')`, `case(${a} == ${a}, 1, 2, 3)`); | |
| 201 | + | for (const b of vals) { | |
| 202 | + | for (const op of ["==", "!=", "<", "<=", ">", ">="]) add(`${a} ${op} ${b}`); | |
| 203 | + | add(`contains(${a}, ${b})`, `startsWith(${a}, ${b})`, `endsWith(${a}, ${b})`); | |
| 204 | + | } | |
| 205 | + | } | |
| 206 | + | ||
| 207 | + | // precedence and chains | |
| 208 | + | add("true || false && false", "(true || false) && false", "false && true || true", "1 == 1 == true", "1 < 2 == true", "true == 1 < 2", "!1 == 2", "!(1 == 2)", "!1 < 2", "1 < !2", "!true == false", | |
| 209 | + | "1 || 2 || 3", "0 || '' || null", "0 || '' || 'x'", "1 && 2 && 3", "1 && 0 && 3", "null && 1", "null || null", "'a' && 'b' || 'c'", "'' && 'b' || 'c'", "a && b", "true && (false || true)", | |
| 210 | + | "1 == 1 && 2 == 2 || 3 == 4", "1 != 1 || 2 != 2", "github.event_name == 'push' && contains(github.ref, 'main')", "github.event_name != 'push' || !contains(github.ref, 'dev')", | |
| 211 | + | "contains(github.event.commits.*.message, 'fix: bug')", "contains(github.event.commits.*.message, 'FIX: BUG')", "contains(toJSON(github.event.commits), 'Bob')", | |
| 212 | + | "github.event.commits[0].author.name", "github.event.commits[1].author.email", "github.event.commits[2].author.name", "github.event.commits[3]", "github.event.commits[-1]", "github.event.commits['1'].id", | |
| 213 | + | "github.event.commits[1.9].id", "github.event.commits['1.9'].id", "github.event.commits[' 1 '].id", "github.event.commits[true]", "github.event.commits[false].id", "github.event.commits[null].id", | |
| 214 | + | "github.event.commits[1e0].id", "github.event.commits[0x1].id", "github.event.commits[Infinity]", "github.event.commits[-0].id", "github.event.commits[1e400]", "github.event.commits[1e20]", | |
| 215 | + | "github.event.commits.*", "github.event.commits.*.id", "github.event.commits.*.author", "github.event.commits.*.author.name", "github.event.commits.*.author.email", "github.event.commits.*.added", | |
| 216 | + | "github.event.commits.*.added.*", "github.event.commits.*.added[0]", "github.event.commits.*.added[1]", "github.event.commits.*.*", "github.event.commits[*].id", "github.event.commits[*]", | |
| 217 | + | "github.event.commits[*][*]", "github.event.commits.*[0]", "github.event.commits.*['id']", "github.event.commits.*.author.*", "github.*", "github.event.*", "github.*.number", "github.*.*", "github.event.*.*", | |
| 218 | + | "github.event.nested.*", "github.event.nested.*.*", "github.event.nested[*][0]", "github.event.nested.*[1]", "github.event.nested.*.*.*", "github.event.nested[3][1]", "github.event.nested[2][0]", | |
| 219 | + | "github.event.labels.*.name", "join(github.event.labels.*.name, ', ')", "join(github.event.labels.*.name)", "toJSON(github.event.labels.*.name)", "toJSON(github.event.labels.*)", | |
| 220 | + | "github.event.head_commit.*", "github.event.head_commit.message", "github.event.head_commit.timestamp", "github.event.head_commit.timestamp.x", "github.event.head_commit.timestamp.*", | |
| 221 | + | "github.event.pull_request.draft", "github.event.pull_request.draft == false", "!github.event.pull_request.draft", "github.event.pull_request.labels.*.name", | |
| 222 | + | "github['weird-key']", "github.weird-key", "github['a b']", "github.a b", "github.EVENT.NUMBER", "github.Event['Number']", "github.event.number == 12", "github.event.number == '12'", "github.event.number > 5", | |
| 223 | + | "github.event.number > '5'", "github.event.number > 'x'", "github.run_number", "github.run_id", "github.run_id == 1658821493", "github.run_id > 1658821492", | |
| 224 | + | "env.FOO", "env.foo", "env.Mixed == 'mixed'", "env.NUM == 42", "env.NUM > 41", "env.NUM + 1", "env.Float == 1.5", "env.Hex == 31", "env.TRUE == true", "env.EMPTY == null", "env.EMPTY == false", "env.EMPTY", | |
| 225 | + | "fromJSON(env.JSONTEXT).a[1]", "fromJSON(env.JSONTEXT).a.*", "toJSON(fromJSON(env.JSONTEXT))", | |
| 226 | + | "matrix.os", "matrix.node", "matrix.node[0]", "matrix.node.*", "matrix.include.*.os", "matrix.include[0].node", "matrix.include[1].experimental", "matrix.v", "matrix.flag", "matrix.nothing", "matrix.neg", "matrix.tiny", | |
| 227 | + | "matrix.big", "matrix.big == 12345678901234567890", "format('{0}', matrix.big)", "format('{0} {1}', matrix.tiny, matrix.neg)", "matrix.ver", "matrix.ver == 3.1", "matrix.ver == '3.10'", "matrix.zero || 'z'", | |
| 228 | + | "matrix.obj", "matrix.obj == matrix.obj", "matrix.obj != matrix.obj", "matrix.node == matrix.node", "github.event.commits == github.event.commits", "github.event.commits.* == github.event.commits.*", | |
| 229 | + | "matrix.obj.k", "matrix.obj['k']", "matrix.obj[0]", "matrix.obj.*", "matrix.node['0']", "matrix.node[0] == 14", "matrix.node[0] < matrix.node[1]", "contains(matrix.node, 16)", "contains(matrix.node, '16')", "contains(matrix.node, 17)", | |
| 230 | + | "contains(matrix.include.*.os, 'MAC')", "inputs.name", "inputs.count", "inputs.count == 3", "inputs.deploy", "inputs.deploy == false", "!inputs.deploy", "inputs.list", "inputs.list.*", "inputs.empty", "inputs.empty.*", "inputs.empty.a", | |
| 231 | + | "inputs.missing", "inputs.missing.deeper", "inputs.missing[0]", "inputs.missing.*", "inputs.missing.*.x", "inputs['missing']['x']", "steps.build.outputs.r", "steps.BUILD.OUTPUTS.OTHER", "steps.build.outcome == 'success'", "steps.build.outputs.*", | |
| 232 | + | "steps.10.outputs.r", "steps['10'].outputs.r", "steps[10].outputs.r", "steps[2].outputs.r", "steps.*", "toJSON(steps)", "toJSON(steps.*.outputs)", "steps.*.outputs.r", "needs", "needs.x", "needs.x.result", "needs.*", "toJSON(needs)", "toJSON(needs.*)", | |
| 233 | + | "vars.A", "vars.B", "vars.A == 1", "vars.C || 'default'", "toJSON(vars)", "toJSON(github.event.commits[0])", "toJSON(github.event)", "toJSON(contains)", "toJSON(github)", "toJSON(matrix)", "toJSON(inputs)", "toJSON(env)", | |
| 234 | + | "toJSON(github.event.commits.*.added)", "toJSON(github.event.nested)", "toJSON(github.event.nested.*)", "toJSON(github.event.nested.*.*)", "toJSON(matrix.big)", "toJSON(matrix.tiny)", "toJSON(matrix.neg)", "toJSON(matrix.nothing)", "toJSON(inputs.empty)", "toJSON(inputs.list)"); | |
| 235 | + | ||
| 236 | + | // functions: arity and behaviours | |
| 237 | + | add("contains()", "contains(1)", "contains(1,2)", "contains(1,2,3)", "contains('a','b',)", "contains(,)", "contains('abc','B')", "contains('abc','')", "contains('', '')", "contains('', 'a')", "contains(null, '')", "contains(null, null)", "contains(1234, 23)", "contains(true, 'RU')", "contains(github, 'a')", | |
| 238 | + | "contains('a', github)", "contains(github.event.commits, 'x')", "contains('\u0130', 'i')", "contains('I', '\u0131')", "contains('\u00df', 'ss')", "contains('SS', '\u00df')", "contains('\u03a3', '\u03c3')", "contains('\u03c2', '\u03c3')", "contains('A\u03a3', 'a\u03c2')", "contains('\u212a', 'k')", | |
| 239 | + | "contains(fromJSON('[1,\"2\",null]'), null)", "contains(fromJSON('[1,\"2\",null]'), '')", "contains(fromJSON('[1,\"2\",null]'), 2)", "contains(fromJSON('[]'), null)", "contains(fromJSON('[[1]]'), fromJSON('[1]'))", "contains(github.event.commits.*.id, 2)", "contains(github.event.commits.*.id, '2')", | |
| 240 | + | "startsWith()", "startsWith('a')", "startsWith('abc','AB')", "startsWith('abc','')", "startsWith('', '')", "startsWith(null, '')", "startsWith(123, 12)", "startsWith(github, 'a')", "startsWith('\u0131', 'I')", "startsWith('\u0131', '\u0131')", "startsWith('i', '\u0131')", "startsWith('\u00df', 'ss')", "startsWith('SS', '\u00df')", | |
| 241 | + | "endsWith('abc','BC')", "endsWith('abc','')", "endsWith(null, 'a')", "endsWith(1.50, '5')", "endsWith(1.50, 5)", "endsWith('\u00df', 'S')", "endsWith('abc', 'abcd')", | |
| 242 | + | "format()", "format('a')", "format('')", "format(null)", "format(1)", "format(true)", "format('{0}')", "format('{0}', 1)", "format('{0}', 1, 2)", "format('{1}', 1)", "format('{1}', 1, 2)", "format('{0}{0}', 'x')", "format('{0}{1}{0}', 'a', 'b')", "format('{{')", "format('}}')", "format('{{}}')", "format('{{0}}', 1)", | |
| 243 | + | "format('{{{0}}}', 1)", "format('{', 1)", "format('}', 1)", "format('{}', 1)", "format('{ 0}', 1)", "format('{0 }', 1)", "format('{0:x}', 1)", "format('{0,5}', 1)", "format('{0 anything}', 1)", "format('{00}', 1)", "format('{01}', 1, 2)", "format('{-1}', 1)", "format('{a}', 1)", "format('{0', 1)", "format('0}', 1)", "format('{{0}', 1)", "format('{0}}', 1)", | |
| 244 | + | "format('{0}}}', 1)", "format('a{0}b{1}c', 1, 2)", "format('{0}', null)", "format('{0}', true)", "format('{0}', github)", "format('{0}', github.event.commits)", "format('{0}', matrix.node.*)", "format('{12345678901234567890123}', 1)", "format('{99999999999999999999999999999999999}', 1)", "format('{0}\n{0}', 'a')", "format('x}', 1)", "format('}{', 1)", "format('{{{{', 1)", "format('{}}', 1)", "format('{0}{{', 1)", "format('{{{0}', 1)", | |
| 245 | + | "format('{0}', 'a', 'b', 'c')", "format('{3}', 0, 1, 2, 3)", "format('{2}', 0, 1, 2, 3)", "format('{255}', 1)", "format('\u{1F600}{0}\u{1F600}', 'x')", "format('{0}', '\u{1F600}')", "format('{ {0}', 1)", "format(' {0} ', 1)", | |
| 246 | + | "fromJSON()", "fromJSON('1','2')", "fromJSON('')", "fromJSON(' ')", "fromJSON('\n')", "fromJSON(null)", "fromJSON('null')", "fromJSON('true')", "fromJSON('false')", "fromJSON('1')", "fromJSON('-1.5e2')", "fromJSON('\"s\"')", "fromJSON('[1,2,3]')", "fromJSON('{\"a\":1}')", "fromJSON('{\"a\":1,\"A\":2}')", "fromJSON('{\"a\":1,\"a\":2}')", "toJSON(fromJSON('{\"a\":1,\"A\":2}'))", "toJSON(fromJSON('{\"a\":1,\"a\":2,\"b\":3}'))", | |
| 247 | + | "toJSON(fromJSON('{\"b\":1,\"2\":2,\"1\":3,\"a\":4,\"01\":5,\"4294967294\":6,\"4294967295\":7,\"-1\":8}'))", "toJSON(fromJSON('{\"\":1}'))", "toJSON(fromJSON('{\"a\":{\"b\":{\"c\":[]}}}'))", "fromJSON('{')", "fromJSON('{\"a\"')", "fromJSON('[1,]')", "fromJSON('[1 2]')", "fromJSON('{a:1}')", "fromJSON(\"{'a':1}\")", "fromJSON('01')", "fromJSON('1.')", "fromJSON('.5')", "fromJSON('+1')", "fromJSON('NaN')", "fromJSON('Infinity')", "fromJSON('undefined')", "fromJSON('nul')", "fromJSON('truee')", | |
| 248 | + | "fromJSON('1 2')", "fromJSON(' 1 ')", "fromJSON('\"\\u0041\"')", "fromJSON('\"\\ud83d\\ude00\"')", "fromJSON('\"\\x\"')", "fromJSON('\"a\nb\"')", "fromJSON('\"\\/\"')", "fromJSON('1e999')", "fromJSON('-1e999')", "fromJSON('-0')", "toJSON(fromJSON('-0'))", "toJSON(fromJSON('1e999'))", "toJSON(fromJSON('1E2'))", "toJSON(fromJSON('0.1e1'))", "toJSON(fromJSON('123456789012345678901234567890'))", | |
| 249 | + | "toJSON(fromJSON('[1e21,1e-7,0.000001,100,1.5,-2.5e-10]'))", "toJSON(fromJSON('\"\\u0000\\u001f\\u007f\\u2028\\\"\\\\\"'))", "toJSON('\u0000')", "toJSON('\u001f\u007f\u0080\u2028\u2029\ufeff')", "toJSON('\"\\')", "toJSON('a\nb\tc\rd\be\ff')", "toJSON('\u00e9\u00df\u{1F600}')", "toJSON(fromJSON('[]'))", "toJSON(fromJSON('{}'))", "toJSON(fromJSON('[[],{}]'))", "toJSON(fromJSON('{\"a\":[],\"b\":{}}'))", "toJSON(fromJSON('[null]'))", | |
| 250 | + | "toJSON()", "toJSON(1,2)", "toJSON(null)", "toJSON(true)", "toJSON(1)", "toJSON(NaN)", "toJSON(Infinity)", "toJSON(-Infinity)", "toJSON(-0)", "toJSON('x')", "toJSON('')", "toJSON(1e21)", "toJSON(1e-7)", "toJSON(0.1)", "toJSON(123456789012345680000)", | |
| 251 | + | "join()", "join('a')", "join('a','b')", "join(null)", "join(1.50)", "join(true, 'x')", "join(github)", "join(github, 'x')", "join(fromJSON('[]'))", "join(fromJSON('[1,2,3]'))", "join(fromJSON('[1,2,3]'), '')", "join(fromJSON('[1,2,3]'), ', ')", "join(fromJSON('[1,2,3]'), 1)", "join(fromJSON('[1,2,3]'), null)", "join(fromJSON('[1,2,3]'), true)", "join(fromJSON('[1,2,3]'), github)", "join(fromJSON('[1,2,3]'), fromJSON('[]'))", "join(fromJSON('[1,2,3]'), NaN)", | |
| 252 | + | "join(fromJSON('[null,true,\"a\",1.5,[1],{}]'))", "join(matrix.node, '.')", "join(matrix.node.*, '.')", "join(github.event.commits.*.id, '/')", "join(1,2,3)", "join(fromJSON('[\"a\",\"b\"]'), '\u{1F600}')", | |
| 253 | + | "case()", "case(true)", "case(true, 1)", "case(true, 1, 2)", "case(false, 1, 2)", "case(true, 1, false, 2, 3)", "case(false, 1, true, 2, 3)", "case(false, 1, false, 2, 3)", "case(false, 1, false, 2, 3, 4)", "case(false, 1, false, 2, 3, 4, 5)", "case(1, 2, 3)", "case('true', 2, 3)", "case(null, 2, 3)", "case(false, 2, 3, 4)", "case(true, 2, 1, 3)", "case(true, 2, null, 3)", "case(false, 1, 'x', 3)", | |
| 254 | + | "case(true, github.event_name, 'other')", "case(github.event_name == 'push', 'a', github.event_name == 'pr', 'b', 'c')", "CASE(true, 1, 2)", "case(true, matrix.node, 2)", "toJSON(case(true, matrix.node.*, 2))", "case(1 == 1, 'a', 'b')", "case(!0, 'a', 'b')", "case(fromJSON('true'), 'a', 'b')", | |
| 255 | + | "CONTAINS('a','A')", "Contains('a','A')", "STARTSWITH('a','A')", "endswith('a','A')", "FORMAT('{0}',1)", "fromjson('1')", "FROMJSON('1')", "tojson(1)", "TOJSON(1)", "JOIN(fromJSON('[1]'))", "unknown()", "unknown(1)", "Constantine()", "notafunction", "success", "success()", "success(1)", "always()", "ALWAYS()", "hashFiles()", "hashFiles('a')", "hashFiles('a','b')", "hashFiles('**/*.go', '!x')", "HASHFILES('a')", "failure()", "cancelled()", | |
| 256 | + | "success() && hashFiles('a') != ''", "contains(github.ref, 'main') && success()", "always() || false", "hashFiles(github.sha, null, 1.5)", "hashFiles(1,2,3,4,5,6,7,8,9,10)", "format()", "format(1,2,3,4)"); | |
| 257 | + | add("hashFiles(" + Array.from({ length: 255 }, (_, i) => `'${i}'`).join(",") + ")", "hashFiles(" + Array.from({ length: 256 }, (_, i) => `'${i}'`).join(",") + ")"); | |
| 258 | + | add("format('{0}'" + ", 1".repeat(254) + ")", "format('{0}'" + ", 1".repeat(255) + ")", "case(" + Array.from({ length: 101 }, (_, i) => i < 100 ? (i % 2 == 0 ? "false" : "1") : "'end'").join(",") + ")"); | |
| 259 | + | add("contains('a', 'b',)", "contains(1,)", "contains(,1)", "contains('a' 'b')", "contains('a';'b')", "contains(('a'), ('b'))", "contains(contains('a','a'), true)", "format(format('{{0}}'), 5)", "toJSON(toJSON(1))", "fromJSON(toJSON(github.event.commits[0])).author.name", "fromJSON(fromJSON('\"[1]\"'))[0]", "fromJSON('[1,2]')[fromJSON('1')]", | |
| 260 | + | "join(fromJSON(toJSON(github.event.labels.*.name)), '+')", "toJSON(fromJSON(toJSON(github.event.commits)))", "format('{0}', toJSON(null))", "fromJSON(format('[{0},{1}]', 1, 2))[1]", "fromJSON('[{\"a\":1},{\"a\":2},{\"b\":3}]').*.a", "toJSON(fromJSON('[{\"a\":1},{\"a\":2},{\"b\":3}]').*.a)", "toJSON(fromJSON('[{\"a\":[1,2]},{\"a\":[3]}]').*.a.*)", "toJSON(fromJSON('[{\"a\":[1,2]},{\"a\":[3]}]').*.a[0])", "toJSON(fromJSON('[[1,2],[3]]').*[0])", "toJSON(fromJSON('[[1,2],[3]]')[*][1])", | |
| 261 | + | "toJSON(fromJSON('{\"a\":{\"x\":1},\"b\":{\"x\":2}}').*.x)", "toJSON(fromJSON('{\"a\":{\"x\":1},\"b\":{\"x\":2}}')[*].x)", "toJSON(fromJSON('{\"a\":{\"x\":1},\"b\":2}').*.x)", "toJSON(fromJSON('[1,2]').*.x)", "toJSON(fromJSON('1').*)", "toJSON(null.*)", "toJSON('abc'.*)", "toJSON(fromJSON('[]').*.a)", "toJSON(fromJSON('[]').*)", "toJSON(fromJSON('{}').*)"); | |
| 262 | + | ||
| 263 | + | // lone surrogates (kept as WTF-8 by the Go port) | |
| 264 | + | { | |
| 265 | + | const H = "fromJSON('\"\\ud800\"')", L = "fromJSON('\"\\udc00\"')", H2 = "fromJSON('\"\\ud83d\"')", L2 = "fromJSON('\"\\ude00\"')"; | |
| 266 | + | const lone = [H, L, H2, L2, "'\u{1F600}'", "'\ue000'", "'\uffff'", "'a'", "''", "'\ufffd'", "'\u{10000}'"]; | |
| 267 | + | for (const a of lone) { | |
| 268 | + | add(`toJSON(${a})`, `format('{0}', ${a})`, `join(${a})`, `${a} || 'd'`, `startsWith(${a}, ${a})`, `case(true, ${a}, 1)`); | |
| 269 | + | for (const b of lone) { | |
| 270 | + | for (const op of ["==", "!=", "<", "<=", ">", ">="]) add(`${a} ${op} ${b}`); | |
| 271 | + | add(`contains(${a}, ${b})`, `startsWith(${a}, ${b})`, `endsWith(${a}, ${b})`, `format('{0}{1}', ${a}, ${b})`, `join(fromJSON('[]'), ${a}) == ''`, | |
| 272 | + | `toJSON(fromJSON(toJSON(${a})))`, `format('{0}', ${a}) == format('{0}', ${b})`); | |
| 273 | + | } | |
| 274 | + | } | |
| 275 | + | add(`${H} == ${L}`, `${H} != ${L}`, `${H} < ${L}`, `${H} > ${L}`, `contains('x${"\u{1F600}"}', ${H2})`, `contains('x\u{1F600}', ${L2})`, `startsWith('\u{1F600}', ${H2})`, `endsWith('\u{1F600}', ${L2})`, | |
| 276 | + | `format('{0}{1}', ${H2}, ${L2}) == '\u{1F600}'`, `format('{0}{1}', ${L2}, ${H2}) == '\u{1F600}'`, `join(fromJSON('["\\ud83d","\\ude00"]'), '') == '\u{1F600}'`, | |
| 277 | + | `join(fromJSON('["\\ud83d","\\ude00"]'), '')`, `format('{0}{1}', ${H2}, ${L2})`, `format('{0}{1}{0}', ${H2}, ${L2})`, `toJSON(format('{0}{1}', ${H2}, ${L2}))`, | |
| 278 | + | `toJSON(fromJSON('{"\\ud800":1,"\\ud800x":2,"a\\udc00":3}'))`, `fromJSON('{"\\ud800":1}')[${H}]`, `fromJSON('{"\\ud800":1}')[${L}]`, `toJSON(fromJSON('["\\ud800\\ud800","\\udc00\\ud800"]'))`, | |
| 279 | + | `${H} == '\ufffd'`, `'\ufffd' == ${H}`, `'\ufffd' < ${H}`, `${H} < '\ufffd'`, `${H} < '\u{1F600}'`, `'\u{1F600}' < ${H}`, `${L} < '\ue000'`, `'\ue000' < ${L}`, | |
| 280 | + | `fromJSON('"\\ud800\\u0041"') == 'A'`, `contains(fromJSON('"A\\ud800"'), 'a')`, `startsWith(fromJSON('"a\\ud800"'), 'A')`, `endsWith(fromJSON('"\\ud800a"'), 'A')`, | |
| 281 | + | `contains(fromJSON('"\\ud800\\udc00"'), ${H})`, `contains(fromJSON('"\\ud800\\udc00"'), ${L})`, `startsWith(fromJSON('"\\ud800\\udc00"'), ${H})`, `endsWith(fromJSON('"\\ud800\\udc00"'), ${L})`, | |
| 282 | + | `contains(fromJSON('["\\ud800"]'), ${H})`, `contains(fromJSON('["\\ud800"]'), ${L})`, `fromJSON('"\\ud800"') == fromJSON('"\\ud800"')`); | |
| 283 | + | } | |
| 284 | + | ||
| 285 | + | // depth and length limits | |
| 286 | + | const nest = (n, open, body, close) => open.repeat(n) + body + close.repeat(n); | |
| 287 | + | for (const n of [1, 10, 24, 25, 26, 48, 49, 50, 51, 52, 100, 200]) { | |
| 288 | + | add(nest(n, "(", "1", ")"), "!".repeat(n) + "true", nest(n, "format(", "'x'", ")"), "github" + ".event".repeat(n), "github" + "[0]".repeat(n), "github" + "['event']".repeat(n), | |
| 289 | + | "1" + " || 1".repeat(n), "1" + " && 1".repeat(n), "1" + " == 1".repeat(n), nest(n, "(1 || ", "1", ")"), nest(n, "!(", "1", ")"), nest(n, "(1 && ", "1", ")"), nest(n, "github[", "0", "]"), nest(n, "contains(1, ", "1", ")")); | |
| 290 | + | } | |
| 291 | + | for (const n of [20990, 21000, 21001, 22000]) { | |
| 292 | + | add("'" + "a".repeat(n - 2) + "'", " ".repeat(n) + "1", "1" + " ".repeat(n - 1), "'" + "\u{1F600}".repeat(Math.floor((n - 2) / 2)) + "'", "'" + "\u{1F600}".repeat(Math.floor((n - 3) / 2)) + "'"); | |
| 293 | + | } | |
| 294 | + | ||
| 295 | + | // positions and error messages | |
| 296 | + | for (const e of ["1 +", "a b", "github github", "github.event_name ==", "== 1", "(github.event_name", "github.event_name)", "contains('a',", "contains('a'", "contains('a' 'b')", "contains(", "github[0", "github[0)", | |
| 297 | + | "1 ==\n 2 ===\n 3", "true &&\n false ||\n nope", "\n\n bogus", "'a'\n 'b'\n zzz", "true\n&\nfalse", "foo(", "foo", "github.event_name\n .\n ", " ) ", "1 !", "1 !!", "1 ! 1", "!", "! ", "!(", "!)", "(!)", "[1]", "github.[0]", "github.*.", "github.*.*.", "github[*", "github[*]]", "github[* ]", "github[1 2]", "github[,]", | |
| 298 | + | "1 && ( 2 ||", "1\t2", "x\ty", "github\r\n.event", "a\u00a0b", "'x' =", "'x' = 'y'", "!= 1", ">", "<<", ">>", "1 <> 2", "1 >< 2", "1 =< 2", "1 => 2", "contains(github.event_name, 'push'", "contains(github.event_name 'push')", "$ {{ 1 }}", "${{ 1 }}", "${{", "}}", "{{ 1 }}", "{ 1 }", "1 }", "a.b.c", "github.a.b.c.d", "git hub", "github github.ref"]) add(e); | |
| 299 | + | ||
| 300 | + | // ---- seeded random ---------------------------------------------------------- | |
| 301 | + | ||
| 302 | + | function mulberry32(a) { | |
| 303 | + | return function () { | |
| 304 | + | a |= 0; a = (a + 0x6d2b79f5) | 0; | |
| 305 | + | let t = Math.imul(a ^ (a >>> 15), 1 | a); | |
| 306 | + | t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t; | |
| 307 | + | return ((t ^ (t >>> 14)) >>> 0) / 4294967296; | |
| 308 | + | }; | |
| 309 | + | } | |
| 310 | + | const rnd = mulberry32(20260512); | |
| 311 | + | const pick = a => a[Math.floor(rnd() * a.length)]; | |
| 312 | + | const lits = ["true", "false", "null", "0", "1", "2", "42", "-1", "1.5", "0.1", "NaN", "Infinity", "''", "'a'", "'A'", "'abc'", "'1'", "'0'", "'true'", "'x y'", "'it''s'", "'{0}'", "'[1,2]'", "'{\"a\":1}'", "'\u00df'", "'\u0131'", "'\u03a3'", "'\u{1F600}'"]; | |
| 313 | + | const paths = ["github", "github.event_name", "github.event.commits", "github.event.commits.*", "github.event.commits.*.id", "github.event.commits[0]", "github.event.commits[1].author", "github.event.commits[1].author.name", "github.event.labels.*.name", "github.event.nested.*.*", "github.event.nested[3]", "github.ref", "github.nope", "github.event.nope.deep", | |
| 314 | + | "env", "env.FOO", "env.NUM", "env.EMPTY", "matrix", "matrix.node", "matrix.node.*", "matrix.node[1]", "matrix.v", "matrix.flag", "matrix.nothing", "matrix.include.*.os", "matrix.obj", "inputs", "inputs.count", "inputs.deploy", "inputs.list.*", "inputs.empty", "steps.build.outputs.r", "steps.*.outputs", "steps['10']", "needs", "needs.*", "vars.A", "vars.B"]; | |
| 315 | + | function gen(d) { | |
| 316 | + | const r = rnd(); | |
| 317 | + | if (d <= 0 || r < 0.3) return rnd() < 0.5 ? pick(lits) : pick(paths); | |
| 318 | + | if (r < 0.4) return pick(paths) + (rnd() < 0.5 ? `[${gen(d - 1)}]` : rnd() < 0.5 ? ".*" : "[*]"); | |
| 319 | + | if (r < 0.5) return `!${gen(d - 1)}`; | |
| 320 | + | if (r < 0.58) return `(${gen(d - 1)})`; | |
| 321 | + | if (r < 0.7) return `${gen(d - 1)} ${pick(["==", "!=", "<", "<=", ">", ">="])} ${gen(d - 1)}`; | |
| 322 | + | if (r < 0.78) return `${gen(d - 1)} ${pick(["&&", "||"])} ${gen(d - 1)}`; | |
| 323 | + | const fn = pick([["contains", 2], ["startsWith", 2], ["endsWith", 2], ["format", 3], ["fromJSON", 1], ["toJSON", 1], ["join", 2], ["case", 3], ["hashFiles", 2], ["success", 0]]); | |
| 324 | + | const n = fn[0] === "format" ? 1 + Math.floor(rnd() * 3) : fn[0] === "case" ? pick([1, 3, 5]) : fn[0] === "join" ? 1 + Math.floor(rnd() * 2) : fn[1]; | |
| 325 | + | const args = Array.from({ length: n }, () => gen(d - 1)); | |
| 326 | + | if (fn[0] === "format" && rnd() < 0.7) args[0] = pick(["'{0}'", "'{0}{1}'", "'a{1}b{0}c'", "'{{{0}}}'", "'{0:x}'", "'{2}'"]); | |
| 327 | + | if (fn[0] === "case" && rnd() < 0.7) for (let i = 0; i + 1 < args.length; i += 2) args[i] = pick(["true", "false", "1 == 1", "!0", "github.event_name == 'push'"]); | |
| 328 | + | if (fn[0] === "fromJSON" && rnd() < 0.5) args[0] = pick(["'[1,2]'", "'{\"a\":{\"b\":1}}'", "'null'", "'\"x\"'", "'1e3'", "'{'", "''", "'[{\"a\":1},{\"a\":2}]'", "toJSON(" + gen(d - 1) + ")"]); | |
| 329 | + | return `${fn[0]}(${args.join(", ")})`; | |
| 330 | + | } | |
| 331 | + | const alphabet = ["(", ")", "[", "]", ".", ",", "!", "=", "&", "|", "<", ">", "*", "'", " ", "\n", "-", "+", "0", "1", "a", "github", "true", "null", "\u00e9", "\u{1F600}", "''", "&&", "||", "==", "!=", ">=", "<="]; | |
| 332 | + | function mutate(s) { | |
| 333 | + | const chars = Array.from(s); | |
| 334 | + | const k = 1 + Math.floor(rnd() * 3); | |
| 335 | + | for (let i = 0; i < k; i++) { | |
| 336 | + | const at = Math.floor(rnd() * (chars.length + 1)); | |
| 337 | + | switch (Math.floor(rnd() * 3)) { | |
| 338 | + | case 0: chars.splice(at, 0, pick(alphabet)); break; | |
| 339 | + | case 1: chars.splice(at, 1); break; | |
| 340 | + | default: chars.splice(at, 1, pick(alphabet)); | |
| 341 | + | } | |
| 342 | + | } | |
| 343 | + | return chars.join(""); | |
| 344 | + | } | |
| 345 | + | for (let i = 0; i < 2500; i++) add(gen(1 + Math.floor(rnd() * 4))); | |
| 346 | + | for (let i = 0; i < 1500; i++) add(mutate(gen(1 + Math.floor(rnd() * 3)))); | |
| 347 | + | // random doubles as literals | |
| 348 | + | for (let i = 0; i < 400; i++) { | |
| 349 | + | const digits = Array.from({ length: 1 + Math.floor(rnd() * 20) }, () => Math.floor(rnd() * 10)).join(""); | |
| 350 | + | const dot = Math.floor(rnd() * (digits.length + 1)); | |
| 351 | + | let lit = digits.slice(0, dot) + (dot < digits.length || rnd() < 0.3 ? "." : "") + digits.slice(dot); | |
| 352 | + | if (lit === "." || lit === "") lit = "0.5"; | |
| 353 | + | if (rnd() < 0.3) lit += "e" + (rnd() < 0.5 ? "-" : "") + Math.floor(rnd() * 30); | |
| 354 | + | add(`format('{0}', ${lit})`, `toJSON(${lit})`); | |
| 355 | + | } | |
| 356 | + | ||
| 357 | + | const seen = new Set(); | |
| 358 | + | const cases = []; | |
| 359 | + | for (const e of corpus) { | |
| 360 | + | if (seen.has(e)) continue; | |
| 361 | + | seen.add(e); | |
| 362 | + | cases.push(run(e)); | |
| 363 | + | } | |
| 364 | + | const skipped = cases.filter(c => c.skip).length; | |
| 365 | + | console.error(`cases=${cases.length} skipped(JS crash)=${skipped}`); | |
| 366 | + | process.stdout.write(JSON.stringify({ reference, contexts, cases: cases.filter(c => !c.skip) }) + "\n"); | |
A expr/golden_test.go +48 -0
| @@ -0,0 +1,48 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "os" | |
| 7 | + | "path/filepath" | |
| 8 | + | "strings" | |
| 9 | + | "testing" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | // TestToJSONGolden checks toJSON(fromJSON(x)) against the output of the | |
| 13 | + | // reference implementation for each testdata/tojson/NAME.json. The .golden | |
| 14 | + | // files are generated by gen/golden.mjs. | |
| 15 | + | func TestToJSONGolden(t *testing.T) { | |
| 16 | + | inputs, err := filepath.Glob("testdata/tojson/*.json") | |
| 17 | + | if err != nil || len(inputs) < 30 { | |
| 18 | + | t.Fatalf("golden inputs: %v (%d found)", err, len(inputs)) | |
| 19 | + | } | |
| 20 | + | for _, in := range inputs { | |
| 21 | + | name := strings.TrimSuffix(filepath.Base(in), ".json") | |
| 22 | + | t.Run(name, func(t *testing.T) { | |
| 23 | + | text, err := os.ReadFile(in) | |
| 24 | + | if err != nil { | |
| 25 | + | t.Fatal(err) | |
| 26 | + | } | |
| 27 | + | want, err := os.ReadFile(strings.TrimSuffix(in, ".json") + ".golden") | |
| 28 | + | if err != nil { | |
| 29 | + | t.Fatal(err) | |
| 30 | + | } | |
| 31 | + | v, err := FromJSON(text) | |
| 32 | + | if err != nil { | |
| 33 | + | t.Fatal(err) | |
| 34 | + | } | |
| 35 | + | if got := ToJSON(v); got != string(want) { | |
| 36 | + | t.Errorf("toJSON mismatch\n got: %q\nwant: %q", got, want) | |
| 37 | + | } | |
| 38 | + | // The golden output is itself JSON that must round-trip. | |
| 39 | + | v2, err := FromJSON(want) | |
| 40 | + | if err != nil { | |
| 41 | + | t.Fatalf("golden output does not parse: %v", err) | |
| 42 | + | } | |
| 43 | + | if got := ToJSON(v2); got != string(want) { | |
| 44 | + | t.Errorf("toJSON is not stable\n got: %q\nwant: %q", got, want) | |
| 45 | + | } | |
| 46 | + | }) | |
| 47 | + | } | |
| 48 | + | } | |
A expr/json.go +556 -0
| @@ -0,0 +1,556 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "errors" | |
| 7 | + | "fmt" | |
| 8 | + | "math" | |
| 9 | + | "sort" | |
| 10 | + | "strconv" | |
| 11 | + | "strings" | |
| 12 | + | "unicode/utf8" | |
| 13 | + | ) | |
| 14 | + | ||
| 15 | + | // MaxJSONDepth bounds the nesting that FromJSON accepts and that ToJSON and | |
| 16 | + | // EncodeJSON write. JavaScript engines fail with a stack overflow somewhere | |
| 17 | + | // in that region; fromJSON reports it like any other parse error. The bound | |
| 18 | + | // also stops the encoder on cyclic values, which can be built through | |
| 19 | + | // [Array.Append]. | |
| 20 | + | const MaxJSONDepth = 1000 | |
| 21 | + | ||
| 22 | + | // FromJSON parses JSON text into a [Value], like the fromJSON function. | |
| 23 | + | // | |
| 24 | + | // The grammar is strict RFC 8259 JSON as JSON.parse accepts it. Numbers are | |
| 25 | + | // float64 (1e999 is +Inf). Objects become [*Dictionary] values: keys that | |
| 26 | + | // are array indices ("0", "42") come first in ascending order, then the rest | |
| 27 | + | // in document order, as JavaScript enumerates object keys; a repeated key | |
| 28 | + | // keeps its first position and its last value; keys that differ only by case | |
| 29 | + | // collapse to the first. | |
| 30 | + | func FromJSON(data []byte) (Value, error) { | |
| 31 | + | s := string(data) | |
| 32 | + | if !utf8.ValidString(s) { | |
| 33 | + | s = string([]rune(s)) // invalid bytes become U+FFFD, as in a JS string | |
| 34 | + | } | |
| 35 | + | p := &jsonParser{s: s} | |
| 36 | + | p.skipSpace() | |
| 37 | + | v, err := p.value(0) | |
| 38 | + | if err != nil { | |
| 39 | + | return nil, err | |
| 40 | + | } | |
| 41 | + | p.skipSpace() | |
| 42 | + | if p.i < len(p.s) { | |
| 43 | + | return nil, p.unexpected() | |
| 44 | + | } | |
| 45 | + | return v, nil | |
| 46 | + | } | |
| 47 | + | ||
| 48 | + | type jsonParser struct { | |
| 49 | + | s string | |
| 50 | + | i int | |
| 51 | + | } | |
| 52 | + | ||
| 53 | + | func (p *jsonParser) unexpected() error { | |
| 54 | + | if p.i >= len(p.s) { | |
| 55 | + | return errors.New("unexpected end of JSON input") | |
| 56 | + | } | |
| 57 | + | r, _ := utf8.DecodeRuneInString(p.s[p.i:]) | |
| 58 | + | return fmt.Errorf("unexpected token %q in JSON at position %d", r, p.i) | |
| 59 | + | } | |
| 60 | + | ||
| 61 | + | func (p *jsonParser) skipSpace() { | |
| 62 | + | for p.i < len(p.s) { | |
| 63 | + | switch p.s[p.i] { | |
| 64 | + | case ' ', '\t', '\n', '\r': | |
| 65 | + | p.i++ | |
| 66 | + | default: | |
| 67 | + | return | |
| 68 | + | } | |
| 69 | + | } | |
| 70 | + | } | |
| 71 | + | ||
| 72 | + | func (p *jsonParser) literal(word string, v Value) (Value, error) { | |
| 73 | + | if !strings.HasPrefix(p.s[p.i:], word) { | |
| 74 | + | // Advance to the first mismatching byte for the error position. | |
| 75 | + | for j := 0; j < len(word) && p.i < len(p.s) && p.s[p.i] == word[j]; j++ { | |
| 76 | + | p.i++ | |
| 77 | + | } | |
| 78 | + | return nil, p.unexpected() | |
| 79 | + | } | |
| 80 | + | p.i += len(word) | |
| 81 | + | return v, nil | |
| 82 | + | } | |
| 83 | + | ||
| 84 | + | func (p *jsonParser) value(depth int) (Value, error) { | |
| 85 | + | if depth > MaxJSONDepth { | |
| 86 | + | return nil, errors.New("maximum JSON nesting depth exceeded") | |
| 87 | + | } | |
| 88 | + | if p.i >= len(p.s) { | |
| 89 | + | return nil, p.unexpected() | |
| 90 | + | } | |
| 91 | + | switch c := p.s[p.i]; { | |
| 92 | + | case c == '{': | |
| 93 | + | return p.object(depth) | |
| 94 | + | case c == '[': | |
| 95 | + | return p.array(depth) | |
| 96 | + | case c == '"': | |
| 97 | + | s, err := p.str() | |
| 98 | + | if err != nil { | |
| 99 | + | return nil, err | |
| 100 | + | } | |
| 101 | + | return String(s), nil | |
| 102 | + | case c == 't': | |
| 103 | + | return p.literal("true", Boolean(true)) | |
| 104 | + | case c == 'f': | |
| 105 | + | return p.literal("false", Boolean(false)) | |
| 106 | + | case c == 'n': | |
| 107 | + | return p.literal("null", Null{}) | |
| 108 | + | case c == '-' || c >= '0' && c <= '9': | |
| 109 | + | return p.number() | |
| 110 | + | } | |
| 111 | + | return nil, p.unexpected() | |
| 112 | + | } | |
| 113 | + | ||
| 114 | + | func (p *jsonParser) number() (Value, error) { | |
| 115 | + | start := p.i | |
| 116 | + | if p.s[p.i] == '-' { | |
| 117 | + | p.i++ | |
| 118 | + | } | |
| 119 | + | digits := func() int { | |
| 120 | + | n := 0 | |
| 121 | + | for p.i < len(p.s) && p.s[p.i] >= '0' && p.s[p.i] <= '9' { | |
| 122 | + | p.i++ | |
| 123 | + | n++ | |
| 124 | + | } | |
| 125 | + | return n | |
| 126 | + | } | |
| 127 | + | switch { | |
| 128 | + | case p.i < len(p.s) && p.s[p.i] == '0': | |
| 129 | + | p.i++ | |
| 130 | + | case digits() == 0: | |
| 131 | + | return nil, p.unexpected() | |
| 132 | + | } | |
| 133 | + | if p.i < len(p.s) && p.s[p.i] == '.' { | |
| 134 | + | p.i++ | |
| 135 | + | if digits() == 0 { | |
| 136 | + | return nil, p.unexpected() | |
| 137 | + | } | |
| 138 | + | } | |
| 139 | + | if p.i < len(p.s) && (p.s[p.i] == 'e' || p.s[p.i] == 'E') { | |
| 140 | + | p.i++ | |
| 141 | + | if p.i < len(p.s) && (p.s[p.i] == '+' || p.s[p.i] == '-') { | |
| 142 | + | p.i++ | |
| 143 | + | } | |
| 144 | + | if digits() == 0 { | |
| 145 | + | return nil, p.unexpected() | |
| 146 | + | } | |
| 147 | + | } | |
| 148 | + | f, _ := strconv.ParseFloat(p.s[start:p.i], 64) // range errors give ±Inf | |
| 149 | + | return Number(f), nil | |
| 150 | + | } | |
| 151 | + | ||
| 152 | + | // str reads a string literal; p.i is on the opening quote. | |
| 153 | + | func (p *jsonParser) str() (string, error) { | |
| 154 | + | p.i++ | |
| 155 | + | var b strings.Builder | |
| 156 | + | for { | |
| 157 | + | if p.i >= len(p.s) { | |
| 158 | + | return "", errors.New("unterminated string in JSON") | |
| 159 | + | } | |
| 160 | + | c := p.s[p.i] | |
| 161 | + | switch { | |
| 162 | + | case c == '"': | |
| 163 | + | p.i++ | |
| 164 | + | return b.String(), nil | |
| 165 | + | case c < 0x20: | |
| 166 | + | return "", p.unexpected() | |
| 167 | + | case c != '\\': | |
| 168 | + | b.WriteByte(c) | |
| 169 | + | p.i++ | |
| 170 | + | continue | |
| 171 | + | } | |
| 172 | + | p.i++ | |
| 173 | + | if p.i >= len(p.s) { | |
| 174 | + | return "", errors.New("unterminated string in JSON") | |
| 175 | + | } | |
| 176 | + | e := p.s[p.i] | |
| 177 | + | p.i++ | |
| 178 | + | switch e { | |
| 179 | + | case '"', '\\', '/': | |
| 180 | + | b.WriteByte(e) | |
| 181 | + | case 'b': | |
| 182 | + | b.WriteByte('\b') | |
| 183 | + | case 'f': | |
| 184 | + | b.WriteByte('\f') | |
| 185 | + | case 'n': | |
| 186 | + | b.WriteByte('\n') | |
| 187 | + | case 'r': | |
| 188 | + | b.WriteByte('\r') | |
| 189 | + | case 't': | |
| 190 | + | b.WriteByte('\t') | |
| 191 | + | case 'u': | |
| 192 | + | r, err := p.hex4() | |
| 193 | + | if err != nil { | |
| 194 | + | return "", err | |
| 195 | + | } | |
| 196 | + | if r >= 0xD800 && r < 0xDC00 && strings.HasPrefix(p.s[p.i:], `\u`) { | |
| 197 | + | save := p.i | |
| 198 | + | p.i += 2 | |
| 199 | + | lo, err := p.hex4() | |
| 200 | + | if err != nil { | |
| 201 | + | return "", err | |
| 202 | + | } | |
| 203 | + | if lo >= 0xDC00 && lo < 0xE000 { | |
| 204 | + | r = 0x10000 + (r-0xD800)<<10 + (lo - 0xDC00) | |
| 205 | + | } else { | |
| 206 | + | p.i = save // not a pair; the second escape is read on its own | |
| 207 | + | } | |
| 208 | + | } | |
| 209 | + | // A lone surrogate cannot be a Go rune: writeRune keeps it in | |
| 210 | + | // its three-byte WTF-8 form so that toJSON writes it back as | |
| 211 | + | // the same \uXXXX escape, like JSON.stringify. | |
| 212 | + | writeRune(&b, r) | |
| 213 | + | default: | |
| 214 | + | p.i-- | |
| 215 | + | return "", p.unexpected() | |
| 216 | + | } | |
| 217 | + | } | |
| 218 | + | } | |
| 219 | + | ||
| 220 | + | func (p *jsonParser) hex4() (rune, error) { | |
| 221 | + | if p.i+4 > len(p.s) { | |
| 222 | + | p.i = len(p.s) | |
| 223 | + | return 0, p.unexpected() | |
| 224 | + | } | |
| 225 | + | var r rune | |
| 226 | + | for k := 0; k < 4; k++ { | |
| 227 | + | c := p.s[p.i] | |
| 228 | + | var d byte | |
| 229 | + | switch { | |
| 230 | + | case c >= '0' && c <= '9': | |
| 231 | + | d = c - '0' | |
| 232 | + | case c >= 'a' && c <= 'f': | |
| 233 | + | d = c - 'a' + 10 | |
| 234 | + | case c >= 'A' && c <= 'F': | |
| 235 | + | d = c - 'A' + 10 | |
| 236 | + | default: | |
| 237 | + | return 0, p.unexpected() | |
| 238 | + | } | |
| 239 | + | r = r<<4 | rune(d) | |
| 240 | + | p.i++ | |
| 241 | + | } | |
| 242 | + | return r, nil | |
| 243 | + | } | |
| 244 | + | ||
| 245 | + | func (p *jsonParser) array(depth int) (Value, error) { | |
| 246 | + | p.i++ | |
| 247 | + | arr := &Array{} | |
| 248 | + | p.skipSpace() | |
| 249 | + | if p.i < len(p.s) && p.s[p.i] == ']' { | |
| 250 | + | p.i++ | |
| 251 | + | return arr, nil | |
| 252 | + | } | |
| 253 | + | for { | |
| 254 | + | p.skipSpace() | |
| 255 | + | v, err := p.value(depth + 1) | |
| 256 | + | if err != nil { | |
| 257 | + | return nil, err | |
| 258 | + | } | |
| 259 | + | arr.items = append(arr.items, v) | |
| 260 | + | p.skipSpace() | |
| 261 | + | if p.i >= len(p.s) { | |
| 262 | + | return nil, p.unexpected() | |
| 263 | + | } | |
| 264 | + | switch p.s[p.i] { | |
| 265 | + | case ',': | |
| 266 | + | p.i++ | |
| 267 | + | case ']': | |
| 268 | + | p.i++ | |
| 269 | + | return arr, nil | |
| 270 | + | default: | |
| 271 | + | return nil, p.unexpected() | |
| 272 | + | } | |
| 273 | + | } | |
| 274 | + | } | |
| 275 | + | ||
| 276 | + | func (p *jsonParser) object(depth int) (Value, error) { | |
| 277 | + | p.i++ | |
| 278 | + | var keys []string | |
| 279 | + | vals := map[string]Value{} | |
| 280 | + | p.skipSpace() | |
| 281 | + | if p.i < len(p.s) && p.s[p.i] == '}' { | |
| 282 | + | p.i++ | |
| 283 | + | return NewDictionary(), nil | |
| 284 | + | } | |
| 285 | + | for { | |
| 286 | + | p.skipSpace() | |
| 287 | + | if p.i >= len(p.s) || p.s[p.i] != '"' { | |
| 288 | + | return nil, p.unexpected() | |
| 289 | + | } | |
| 290 | + | k, err := p.str() | |
| 291 | + | if err != nil { | |
| 292 | + | return nil, err | |
| 293 | + | } | |
| 294 | + | p.skipSpace() | |
| 295 | + | if p.i >= len(p.s) || p.s[p.i] != ':' { | |
| 296 | + | return nil, p.unexpected() | |
| 297 | + | } | |
| 298 | + | p.i++ | |
| 299 | + | p.skipSpace() | |
| 300 | + | v, err := p.value(depth + 1) | |
| 301 | + | if err != nil { | |
| 302 | + | return nil, err | |
| 303 | + | } | |
| 304 | + | if _, dup := vals[k]; !dup { | |
| 305 | + | keys = append(keys, k) | |
| 306 | + | } | |
| 307 | + | vals[k] = v // last value wins, at the first position | |
| 308 | + | p.skipSpace() | |
| 309 | + | if p.i >= len(p.s) { | |
| 310 | + | return nil, p.unexpected() | |
| 311 | + | } | |
| 312 | + | switch p.s[p.i] { | |
| 313 | + | case ',': | |
| 314 | + | p.i++ | |
| 315 | + | continue | |
| 316 | + | case '}': | |
| 317 | + | p.i++ | |
| 318 | + | default: | |
| 319 | + | return nil, p.unexpected() | |
| 320 | + | } | |
| 321 | + | break | |
| 322 | + | } | |
| 323 | + | d := NewDictionary() | |
| 324 | + | for _, k := range jsKeyOrder(keys) { | |
| 325 | + | d.Add(k, vals[k]) | |
| 326 | + | } | |
| 327 | + | return d, nil | |
| 328 | + | } | |
| 329 | + | ||
| 330 | + | // isArrayIndex reports whether k is the canonical decimal form of an integer | |
| 331 | + | // in [0, 2^32-2], the keys JavaScript enumerates before all others. | |
| 332 | + | func isArrayIndex(k string) bool { | |
| 333 | + | if k == "" || len(k) > 10 || (len(k) > 1 && k[0] == '0') { | |
| 334 | + | return false | |
| 335 | + | } | |
| 336 | + | for i := 0; i < len(k); i++ { | |
| 337 | + | if k[i] < '0' || k[i] > '9' { | |
| 338 | + | return false | |
| 339 | + | } | |
| 340 | + | } | |
| 341 | + | n, _ := strconv.ParseUint(k, 10, 64) | |
| 342 | + | return n < math.MaxUint32 | |
| 343 | + | } | |
| 344 | + | ||
| 345 | + | // jsKeyOrder reorders keys the way Object.keys enumerates a JavaScript | |
| 346 | + | // object: array indices ascending, then the others in their given order. | |
| 347 | + | func jsKeyOrder(keys []string) []string { | |
| 348 | + | var idx, rest []string | |
| 349 | + | for _, k := range keys { | |
| 350 | + | if isArrayIndex(k) { | |
| 351 | + | idx = append(idx, k) | |
| 352 | + | } else { | |
| 353 | + | rest = append(rest, k) | |
| 354 | + | } | |
| 355 | + | } | |
| 356 | + | if len(idx) == 0 { | |
| 357 | + | return keys | |
| 358 | + | } | |
| 359 | + | sort.Slice(idx, func(i, j int) bool { | |
| 360 | + | a, _ := strconv.ParseUint(idx[i], 10, 64) | |
| 361 | + | b, _ := strconv.ParseUint(idx[j], 10, 64) | |
| 362 | + | return a < b | |
| 363 | + | }) | |
| 364 | + | return append(idx, rest...) | |
| 365 | + | } | |
| 366 | + | ||
| 367 | + | // ToJSON renders v as the toJSON function does: JSON.stringify(v, replacer, | |
| 368 | + | // " "), that is two-space indentation, "key": value pairs, "[]" and "{}" | |
| 369 | + | // for empty containers, and numbers as JavaScript prints them (NaN and the | |
| 370 | + | // infinities as null, -0 as 0, 1e21 as 1e+21). Dictionary keys that are array | |
| 371 | + | // indices come first in ascending order, as when GitHub's replacer builds a | |
| 372 | + | // JavaScript object. Control characters, quotes and backslashes in strings | |
| 373 | + | // are escaped; everything else, including non-ASCII text, is written as is. | |
| 374 | + | // Lone surrogates, which FromJSON keeps in WTF-8 form, are written as \uXXXX. | |
| 375 | + | // | |
| 376 | + | // ToJSON suits the finite values that FromJSON and Evaluate produce. A value | |
| 377 | + | // nested deeper than [MaxJSONDepth], which only a cyclic value built by hand | |
| 378 | + | // can be, makes it return "null"; use [EncodeJSON] to get an error instead | |
| 379 | + | // and to bound the output. | |
| 380 | + | func ToJSON(v Value) string { | |
| 381 | + | s, err := EncodeJSON(v, -1) | |
| 382 | + | if err != nil { | |
| 383 | + | return "null" | |
| 384 | + | } | |
| 385 | + | return s | |
| 386 | + | } | |
| 387 | + | ||
| 388 | + | // EncodeJSON is [ToJSON] with limits: it fails when v is nested deeper than | |
| 389 | + | // [MaxJSONDepth] and, when maxBytes is not negative, as soon as the output | |
| 390 | + | // would exceed maxBytes bytes, without building the rest. | |
| 391 | + | func EncodeJSON(v Value, maxBytes int) (string, error) { | |
| 392 | + | w := &jsonWriter{limit: maxBytes} | |
| 393 | + | w.value(v, 0) | |
| 394 | + | if w.err != nil { | |
| 395 | + | return "", w.err | |
| 396 | + | } | |
| 397 | + | return w.b.String(), nil | |
| 398 | + | } | |
| 399 | + | ||
| 400 | + | var ( | |
| 401 | + | errJSONTooDeep = errors.New("value nested too deeply to encode as JSON (a cycle?)") | |
| 402 | + | errJSONTooBig = errors.New("JSON output exceeds the size limit") | |
| 403 | + | ) | |
| 404 | + | ||
| 405 | + | type jsonWriter struct { | |
| 406 | + | b strings.Builder | |
| 407 | + | limit int // negative: unlimited | |
| 408 | + | err error | |
| 409 | + | } | |
| 410 | + | ||
| 411 | + | // grow reports whether n more bytes may be written, recording the error | |
| 412 | + | // otherwise. | |
| 413 | + | func (w *jsonWriter) grow(n int) bool { | |
| 414 | + | if w.err != nil { | |
| 415 | + | return false | |
| 416 | + | } | |
| 417 | + | if w.limit >= 0 && n > w.limit-w.b.Len() { | |
| 418 | + | w.err = errJSONTooBig | |
| 419 | + | return false | |
| 420 | + | } | |
| 421 | + | return true | |
| 422 | + | } | |
| 423 | + | ||
| 424 | + | func (w *jsonWriter) str(s string) { | |
| 425 | + | if w.grow(len(s)) { | |
| 426 | + | w.b.WriteString(s) | |
| 427 | + | } | |
| 428 | + | } | |
| 429 | + | ||
| 430 | + | func (w *jsonWriter) indent(depth int) { | |
| 431 | + | if w.grow(2 * depth) { | |
| 432 | + | for i := 0; i < depth; i++ { | |
| 433 | + | w.b.WriteString(" ") | |
| 434 | + | } | |
| 435 | + | } | |
| 436 | + | } | |
| 437 | + | ||
| 438 | + | func (w *jsonWriter) value(v Value, depth int) { | |
| 439 | + | if w.err != nil { | |
| 440 | + | return | |
| 441 | + | } | |
| 442 | + | if depth > MaxJSONDepth { | |
| 443 | + | w.err = errJSONTooDeep | |
| 444 | + | return | |
| 445 | + | } | |
| 446 | + | switch x := normalize(v).(type) { | |
| 447 | + | case Null: | |
| 448 | + | w.str("null") | |
| 449 | + | case Boolean: | |
| 450 | + | if x { | |
| 451 | + | w.str("true") | |
| 452 | + | } else { | |
| 453 | + | w.str("false") | |
| 454 | + | } | |
| 455 | + | case Number: | |
| 456 | + | f := float64(x) | |
| 457 | + | if math.IsNaN(f) || math.IsInf(f, 0) { | |
| 458 | + | w.str("null") | |
| 459 | + | } else { | |
| 460 | + | w.str(jsNumberToString(f)) | |
| 461 | + | } | |
| 462 | + | case String: | |
| 463 | + | w.jsonString(string(x)) | |
| 464 | + | case *Array: | |
| 465 | + | if len(x.items) == 0 { | |
| 466 | + | w.str("[]") | |
| 467 | + | return | |
| 468 | + | } | |
| 469 | + | w.str("[\n") | |
| 470 | + | for i, it := range x.items { | |
| 471 | + | w.indent(depth + 1) | |
| 472 | + | w.value(it, depth+1) | |
| 473 | + | if i < len(x.items)-1 { | |
| 474 | + | w.str(",") | |
| 475 | + | } | |
| 476 | + | w.str("\n") | |
| 477 | + | if w.err != nil { | |
| 478 | + | return | |
| 479 | + | } | |
| 480 | + | } | |
| 481 | + | w.indent(depth) | |
| 482 | + | w.str("]") | |
| 483 | + | case *Dictionary: | |
| 484 | + | if len(x.keys) == 0 { | |
| 485 | + | w.str("{}") | |
| 486 | + | return | |
| 487 | + | } | |
| 488 | + | keys := jsKeyOrder(x.keys) | |
| 489 | + | w.str("{\n") | |
| 490 | + | for i, k := range keys { | |
| 491 | + | val, _ := x.Get(k) | |
| 492 | + | w.indent(depth + 1) | |
| 493 | + | w.jsonString(k) | |
| 494 | + | w.str(": ") | |
| 495 | + | w.value(val, depth+1) | |
| 496 | + | if i < len(keys)-1 { | |
| 497 | + | w.str(",") | |
| 498 | + | } | |
| 499 | + | w.str("\n") | |
| 500 | + | if w.err != nil { | |
| 501 | + | return | |
| 502 | + | } | |
| 503 | + | } | |
| 504 | + | w.indent(depth) | |
| 505 | + | w.str("}") | |
| 506 | + | } | |
| 507 | + | } | |
| 508 | + | ||
| 509 | + | const hexDigits = "0123456789abcdef" | |
| 510 | + | ||
| 511 | + | func (w *jsonWriter) jsonString(s string) { | |
| 512 | + | if !w.grow(len(s) + 2) { | |
| 513 | + | return | |
| 514 | + | } | |
| 515 | + | w.b.WriteByte('"') | |
| 516 | + | for i := 0; i < len(s); { | |
| 517 | + | r, n := decodeRune(s[i:]) | |
| 518 | + | i += n | |
| 519 | + | var esc string | |
| 520 | + | switch { | |
| 521 | + | case r == '"': | |
| 522 | + | esc = `\"` | |
| 523 | + | case r == '\\': | |
| 524 | + | esc = `\\` | |
| 525 | + | case r == '\b': | |
| 526 | + | esc = `\b` | |
| 527 | + | case r == '\f': | |
| 528 | + | esc = `\f` | |
| 529 | + | case r == '\n': | |
| 530 | + | esc = `\n` | |
| 531 | + | case r == '\r': | |
| 532 | + | esc = `\r` | |
| 533 | + | case r == '\t': | |
| 534 | + | esc = `\t` | |
| 535 | + | case r < 0x20: | |
| 536 | + | esc = `\u00` + string([]byte{hexDigits[r>>4], hexDigits[r&0xF]}) | |
| 537 | + | case r >= 0xD800 && r < 0xE000: | |
| 538 | + | // A lone surrogate (WTF-8, see FromJSON). | |
| 539 | + | esc = `\u` + string([]byte{hexDigits[r>>12], hexDigits[r>>8&0xF], hexDigits[r>>4&0xF], hexDigits[r&0xF]}) | |
| 540 | + | } | |
| 541 | + | if esc != "" { | |
| 542 | + | if !w.grow(len(esc)) { | |
| 543 | + | return | |
| 544 | + | } | |
| 545 | + | w.b.WriteString(esc) | |
| 546 | + | continue | |
| 547 | + | } | |
| 548 | + | if !w.grow(utf8.RuneLen(r)) { | |
| 549 | + | return | |
| 550 | + | } | |
| 551 | + | w.b.WriteRune(r) // other invalid UTF-8 comes out as U+FFFD | |
| 552 | + | } | |
| 553 | + | if w.grow(1) { | |
| 554 | + | w.b.WriteByte('"') | |
| 555 | + | } | |
| 556 | + | } | |
A expr/json_test.go +266 -0
| @@ -0,0 +1,266 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "math" | |
| 7 | + | "strings" | |
| 8 | + | "testing" | |
| 9 | + | ) | |
| 10 | + | ||
| 11 | + | func TestFromJSONValid(t *testing.T) { | |
| 12 | + | tests := []struct { | |
| 13 | + | in string | |
| 14 | + | want string // ToJSON of the result | |
| 15 | + | }{ | |
| 16 | + | {"null", "null"}, | |
| 17 | + | {" true ", "true"}, | |
| 18 | + | {"\t\r\n false \n", "false"}, | |
| 19 | + | {"0", "0"}, | |
| 20 | + | {"-0", "0"}, | |
| 21 | + | {"-12.5e1", "-125"}, | |
| 22 | + | {"1E2", "100"}, | |
| 23 | + | {"0.1e1", "1"}, | |
| 24 | + | {`"a\u00e9\n"`, `"aé\n"`}, | |
| 25 | + | {`"\ud83d\ude00"`, `"😀"`}, | |
| 26 | + | {`"\ud800"`, `"\ud800"`}, | |
| 27 | + | {`"\udc00x"`, `"\udc00x"`}, | |
| 28 | + | {`"\ud800\u0041"`, `"\ud800A"`}, | |
| 29 | + | {`"\/"`, `"/"`}, | |
| 30 | + | {`[]`, `[]`}, | |
| 31 | + | {`{}`, `{}`}, | |
| 32 | + | {`[ 1 , [ ] , { } ]`, "[\n 1,\n [],\n {}\n]"}, | |
| 33 | + | {`{"a":1,"A":2}`, "{\n \"a\": 2\n}"}, // see below: the last value of an exact duplicate wins, case variants collapse to the first | |
| 34 | + | {`{"a":1,"a":2}`, "{\n \"a\": 2\n}"}, | |
| 35 | + | {`{"b":1,"1":2,"a":3,"0":4}`, "{\n \"0\": 4,\n \"1\": 2,\n \"b\": 1,\n \"a\": 3\n}"}, | |
| 36 | + | } | |
| 37 | + | for _, tc := range tests { | |
| 38 | + | if tc.in == `{"a":1,"A":2}` { | |
| 39 | + | // "a" and "A" are distinct JSON keys but the same dictionary key: | |
| 40 | + | // the first one wins. | |
| 41 | + | tc.want = "{\n \"a\": 1\n}" | |
| 42 | + | } | |
| 43 | + | v, err := FromJSON([]byte(tc.in)) | |
| 44 | + | if err != nil { | |
| 45 | + | t.Errorf("FromJSON(%q): %v", tc.in, err) | |
| 46 | + | continue | |
| 47 | + | } | |
| 48 | + | if got := ToJSON(v); got != tc.want { | |
| 49 | + | t.Errorf("FromJSON(%q) -> %q, want %q", tc.in, got, tc.want) | |
| 50 | + | } | |
| 51 | + | } | |
| 52 | + | } | |
| 53 | + | ||
| 54 | + | func TestFromJSONInvalid(t *testing.T) { | |
| 55 | + | for _, in := range []string{ | |
| 56 | + | "", " ", "{", "}", "[", "]", `{"a"`, `{"a":`, `{"a":1`, `{"a":1,}`, `{,}`, `{a:1}`, `{'a':1}`, `{"a" 1}`, `[1,]`, `[,1]`, `[1 2]`, | |
| 57 | + | "01", "1.", ".5", "+1", "-", "1e", "1e+", "--1", "0x10", "NaN", "Infinity", "-Infinity", "undefined", "nul", "truee", "tru", "True", | |
| 58 | + | `"abc`, `"a` + "\n" + `b"`, "\"\x01\"", `"\x"`, `"\u12"`, `"\u12g4"`, `"\`, "1 2", "[] []", "{} x", `"a""b"`, "\x00", "\u00a01", "1\u00a0", "\ufeff1", | |
| 59 | + | } { | |
| 60 | + | if v, err := FromJSON([]byte(in)); err == nil { | |
| 61 | + | t.Errorf("FromJSON(%q) = %v, want error", in, ToJSON(v)) | |
| 62 | + | } | |
| 63 | + | } | |
| 64 | + | } | |
| 65 | + | ||
| 66 | + | func TestFromJSONNumbers(t *testing.T) { | |
| 67 | + | for in, want := range map[string]float64{ | |
| 68 | + | "1e999": math.Inf(1), "-1e999": math.Inf(-1), "1e-999": 0, "123456789012345678901234567890": 1.2345678901234568e29, "5e-324": 5e-324, | |
| 69 | + | } { | |
| 70 | + | v, err := FromJSON([]byte(in)) | |
| 71 | + | if err != nil { | |
| 72 | + | t.Errorf("FromJSON(%q): %v", in, err) | |
| 73 | + | continue | |
| 74 | + | } | |
| 75 | + | if got := float64(v.(Number)); got != want { | |
| 76 | + | t.Errorf("FromJSON(%q) = %v, want %v", in, got, want) | |
| 77 | + | } | |
| 78 | + | } | |
| 79 | + | v, _ := FromJSON([]byte("-0")) | |
| 80 | + | if !math.Signbit(float64(v.(Number))) { | |
| 81 | + | t.Error("-0 lost its sign") | |
| 82 | + | } | |
| 83 | + | } | |
| 84 | + | ||
| 85 | + | func TestFromJSONInvalidUTF8(t *testing.T) { | |
| 86 | + | v, err := FromJSON([]byte("\"a\xffb\"")) | |
| 87 | + | if err != nil { | |
| 88 | + | t.Fatal(err) | |
| 89 | + | } | |
| 90 | + | if s := string(v.(String)); s != "a\ufffdb" { | |
| 91 | + | t.Errorf("got %q", s) | |
| 92 | + | } | |
| 93 | + | } | |
| 94 | + | ||
| 95 | + | func TestFromJSONDeepNesting(t *testing.T) { | |
| 96 | + | deep := strings.Repeat("[", 900) + strings.Repeat("]", 900) | |
| 97 | + | if _, err := FromJSON([]byte(deep)); err != nil { | |
| 98 | + | t.Errorf("depth 900: %v", err) | |
| 99 | + | } | |
| 100 | + | tooDeep := strings.Repeat("[", MaxJSONDepth+10) + strings.Repeat("]", MaxJSONDepth+10) | |
| 101 | + | if _, err := FromJSON([]byte(tooDeep)); err == nil { | |
| 102 | + | t.Error("absurd depth accepted") | |
| 103 | + | } | |
| 104 | + | } | |
| 105 | + | ||
| 106 | + | func TestToJSONFormatting(t *testing.T) { | |
| 107 | + | d := NewDictionary() | |
| 108 | + | d.Add("name", String("x")) | |
| 109 | + | d.Add("list", NewArray(Number(1), Null{}, Boolean(true), NewArray(), NewDictionary())) | |
| 110 | + | inner := NewDictionary() | |
| 111 | + | inner.Add("k", String("v")) | |
| 112 | + | d.Add("obj", inner) | |
| 113 | + | want := `{ | |
| 114 | + | "name": "x", | |
| 115 | + | "list": [ | |
| 116 | + | 1, | |
| 117 | + | null, | |
| 118 | + | true, | |
| 119 | + | [], | |
| 120 | + | {} | |
| 121 | + | ], | |
| 122 | + | "obj": { | |
| 123 | + | "k": "v" | |
| 124 | + | } | |
| 125 | + | }` | |
| 126 | + | if got := ToJSON(d); got != want { | |
| 127 | + | t.Errorf("got\n%s\nwant\n%s", got, want) | |
| 128 | + | } | |
| 129 | + | tests := []struct { | |
| 130 | + | v Value | |
| 131 | + | want string | |
| 132 | + | }{ | |
| 133 | + | {nil, "null"}, | |
| 134 | + | {Null{}, "null"}, | |
| 135 | + | {Number(math.NaN()), "null"}, | |
| 136 | + | {Number(math.Inf(1)), "null"}, | |
| 137 | + | {Number(math.Inf(-1)), "null"}, | |
| 138 | + | {Number(math.Copysign(0, -1)), "0"}, | |
| 139 | + | {Number(1e21), "1e+21"}, | |
| 140 | + | {Number(1e-7), "1e-7"}, | |
| 141 | + | {Number(123456789), "123456789"}, | |
| 142 | + | {String("\u0000\u001f\u007f\u0080\u2028"), `"\u0000\u001f` + "\u007f\u0080\u2028" + `"`}, | |
| 143 | + | {String("\"\\\b\f\n\r\t"), `"\"\\\b\f\n\r\t"`}, | |
| 144 | + | {String("</script>&'"), `"</script>&'"`}, | |
| 145 | + | {String("\xff"), "\"\ufffd\""}, | |
| 146 | + | {(*Array)(nil), "null"}, | |
| 147 | + | {(*Dictionary)(nil), "null"}, | |
| 148 | + | {NewArray(nil), "[\n null\n]"}, | |
| 149 | + | } | |
| 150 | + | for _, tc := range tests { | |
| 151 | + | if got := ToJSON(tc.v); got != tc.want { | |
| 152 | + | t.Errorf("ToJSON(%#v) = %q, want %q", tc.v, got, tc.want) | |
| 153 | + | } | |
| 154 | + | } | |
| 155 | + | } | |
| 156 | + | ||
| 157 | + | func TestDictionary(t *testing.T) { | |
| 158 | + | d := NewDictionary() | |
| 159 | + | if !d.Add("Name", String("first")) { | |
| 160 | + | t.Fatal("Add failed") | |
| 161 | + | } | |
| 162 | + | if d.Add("NAME", String("second")) { | |
| 163 | + | t.Error("case variant was added") | |
| 164 | + | } | |
| 165 | + | d.Add("other", Number(1)) | |
| 166 | + | if v, ok := d.Get("nAmE"); !ok || v.(String) != "first" { | |
| 167 | + | t.Errorf("Get = %v, %v", v, ok) | |
| 168 | + | } | |
| 169 | + | if _, ok := d.Get("missing"); ok { | |
| 170 | + | t.Error("Get of missing key succeeded") | |
| 171 | + | } | |
| 172 | + | if got := strings.Join(d.Keys(), ","); got != "Name,other" { | |
| 173 | + | t.Errorf("Keys = %s", got) | |
| 174 | + | } | |
| 175 | + | if d.Len() != 2 || len(d.Values()) != 2 || len(d.Pairs()) != 2 || d.Pairs()[0].Key != "Name" { | |
| 176 | + | t.Errorf("Len/Values/Pairs inconsistent: %d", d.Len()) | |
| 177 | + | } | |
| 178 | + | // Non-ASCII keys fold like JavaScript's toLowerCase. | |
| 179 | + | d.Add("ΑΣ", String("sigma")) | |
| 180 | + | if _, ok := d.Get("ας"); !ok { | |
| 181 | + | t.Error("final sigma key not found") | |
| 182 | + | } | |
| 183 | + | var zero Dictionary | |
| 184 | + | zero.Add("a", Null{}) | |
| 185 | + | if _, ok := zero.Get("A"); !ok { | |
| 186 | + | t.Error("zero Dictionary is not usable") | |
| 187 | + | } | |
| 188 | + | var nilDict *Dictionary | |
| 189 | + | if _, ok := nilDict.Get("a"); ok || nilDict.Len() != 0 || nilDict.Keys() != nil || nilDict.Values() != nil || nilDict.Pairs() != nil { | |
| 190 | + | t.Error("nil Dictionary is not empty") | |
| 191 | + | } | |
| 192 | + | } | |
| 193 | + | ||
| 194 | + | func TestArray(t *testing.T) { | |
| 195 | + | a := NewArray(String("x"), nil) | |
| 196 | + | if a.Len() != 2 || a.At(0).(String) != "x" || a.At(1).Kind() != KindNull || a.At(2).Kind() != KindNull || a.At(-1).Kind() != KindNull { | |
| 197 | + | t.Errorf("At/Len wrong: %v", a) | |
| 198 | + | } | |
| 199 | + | a.Append(Number(3)) | |
| 200 | + | if a.Len() != 3 || a.Filtered() { | |
| 201 | + | t.Error("Append/Filtered") | |
| 202 | + | } | |
| 203 | + | items := a.Items() | |
| 204 | + | items[0] = Null{} | |
| 205 | + | if a.At(0).Kind() != KindString { | |
| 206 | + | t.Error("Items does not copy") | |
| 207 | + | } | |
| 208 | + | var nilArr *Array | |
| 209 | + | if nilArr.Len() != 0 || nilArr.Items() != nil || nilArr.At(0).Kind() != KindNull || nilArr.Filtered() { | |
| 210 | + | t.Error("nil Array is not empty") | |
| 211 | + | } | |
| 212 | + | src := []Value{String("a")} | |
| 213 | + | b := NewArray(src...) | |
| 214 | + | src[0] = String("changed") | |
| 215 | + | if b.At(0).(String) != "a" { | |
| 216 | + | t.Error("NewArray does not copy") | |
| 217 | + | } | |
| 218 | + | } | |
| 219 | + | ||
| 220 | + | func TestCoerceAndTruthy(t *testing.T) { | |
| 221 | + | for _, tc := range []struct { | |
| 222 | + | v Value | |
| 223 | + | s string | |
| 224 | + | truthy bool | |
| 225 | + | }{ | |
| 226 | + | {nil, "", false}, | |
| 227 | + | {Null{}, "", false}, | |
| 228 | + | {Boolean(true), "true", true}, | |
| 229 | + | {Boolean(false), "false", false}, | |
| 230 | + | {Number(0), "0", false}, | |
| 231 | + | {Number(math.Copysign(0, -1)), "0", false}, | |
| 232 | + | {Number(math.NaN()), "NaN", false}, | |
| 233 | + | {Number(2), "2", true}, | |
| 234 | + | {Number(math.Inf(-1)), "-Infinity", true}, | |
| 235 | + | {String(""), "", false}, | |
| 236 | + | {String("0"), "0", true}, | |
| 237 | + | {String("false"), "false", true}, | |
| 238 | + | {NewArray(), "Array", true}, | |
| 239 | + | {NewDictionary(), "Object", true}, | |
| 240 | + | {(*Array)(nil), "", false}, | |
| 241 | + | } { | |
| 242 | + | if got := CoerceString(tc.v); got != tc.s { | |
| 243 | + | t.Errorf("CoerceString(%#v) = %q, want %q", tc.v, got, tc.s) | |
| 244 | + | } | |
| 245 | + | if got := Truthy(tc.v); got != tc.truthy { | |
| 246 | + | t.Errorf("Truthy(%#v) = %v, want %v", tc.v, got, tc.truthy) | |
| 247 | + | } | |
| 248 | + | } | |
| 249 | + | if KindDictionary.String() != "Object" || Kind(99).String() != "unknown" { | |
| 250 | + | t.Error("Kind.String") | |
| 251 | + | } | |
| 252 | + | } | |
| 253 | + | ||
| 254 | + | func TestToNumber(t *testing.T) { | |
| 255 | + | for _, tc := range []struct { | |
| 256 | + | v Value | |
| 257 | + | want float64 | |
| 258 | + | }{ | |
| 259 | + | {nil, 0}, {Null{}, 0}, {Boolean(true), 1}, {Boolean(false), 0}, {Number(2.5), 2.5}, {String(" 7 "), 7}, {String("x"), math.NaN()}, {NewArray(), math.NaN()}, {NewDictionary(), math.NaN()}, | |
| 260 | + | } { | |
| 261 | + | got := toNumber(tc.v) | |
| 262 | + | if got != tc.want && !(math.IsNaN(got) && math.IsNaN(tc.want)) { | |
| 263 | + | t.Errorf("toNumber(%#v) = %v, want %v", tc.v, got, tc.want) | |
| 264 | + | } | |
| 265 | + | } | |
| 266 | + | } | |
A expr/lexer.go +361 -0
| @@ -0,0 +1,361 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "fmt" | |
| 7 | + | "math" | |
| 8 | + | "strings" | |
| 9 | + | "unicode/utf16" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | // TokenType is the kind of a lexical token. | |
| 13 | + | type TokenType int | |
| 14 | + | ||
| 15 | + | // The token types, in the order of the reference implementation. | |
| 16 | + | const ( | |
| 17 | + | TokenUnknown TokenType = iota | |
| 18 | + | TokenLeftParen | |
| 19 | + | TokenRightParen | |
| 20 | + | TokenLeftBracket | |
| 21 | + | TokenRightBracket | |
| 22 | + | TokenComma | |
| 23 | + | TokenDot | |
| 24 | + | TokenBang | |
| 25 | + | TokenBangEqual | |
| 26 | + | TokenEqualEqual | |
| 27 | + | TokenGreater | |
| 28 | + | TokenGreaterEqual | |
| 29 | + | TokenLess | |
| 30 | + | TokenLessEqual | |
| 31 | + | TokenAnd | |
| 32 | + | TokenOr | |
| 33 | + | TokenStar | |
| 34 | + | TokenNumber | |
| 35 | + | TokenString | |
| 36 | + | TokenIdentifier | |
| 37 | + | TokenTrue | |
| 38 | + | TokenFalse | |
| 39 | + | TokenNull | |
| 40 | + | TokenEOF | |
| 41 | + | ) | |
| 42 | + | ||
| 43 | + | var tokenTypeNames = [...]string{ | |
| 44 | + | "UNKNOWN", "LEFT_PAREN", "RIGHT_PAREN", "LEFT_BRACKET", "RIGHT_BRACKET", | |
| 45 | + | "COMMA", "DOT", "BANG", "BANG_EQUAL", "EQUAL_EQUAL", "GREATER", | |
| 46 | + | "GREATER_EQUAL", "LESS", "LESS_EQUAL", "AND", "OR", "STAR", "NUMBER", | |
| 47 | + | "STRING", "IDENTIFIER", "TRUE", "FALSE", "NULL", "EOF", | |
| 48 | + | } | |
| 49 | + | ||
| 50 | + | func (t TokenType) String() string { | |
| 51 | + | if t >= 0 && int(t) < len(tokenTypeNames) { | |
| 52 | + | return tokenTypeNames[t] | |
| 53 | + | } | |
| 54 | + | return fmt.Sprintf("TokenType(%d)", int(t)) | |
| 55 | + | } | |
| 56 | + | ||
| 57 | + | // Range is the extent of a token. Both positions are on the line where the | |
| 58 | + | // token ends, as in the reference implementation. | |
| 59 | + | type Range struct { | |
| 60 | + | Start, End Position | |
| 61 | + | } | |
| 62 | + | ||
| 63 | + | // Token is a lexical token. | |
| 64 | + | type Token struct { | |
| 65 | + | Type TokenType | |
| 66 | + | Lexeme string | |
| 67 | + | Range Range | |
| 68 | + | // Value is the number of a TokenNumber, as a [Number], or the unescaped | |
| 69 | + | // text of a TokenString, as a [String]; nil for other tokens. | |
| 70 | + | Value Value | |
| 71 | + | } | |
| 72 | + | ||
| 73 | + | // tokenString is the text an error message shows for a token. | |
| 74 | + | func tokenString(t Token) string { | |
| 75 | + | switch t.Type { | |
| 76 | + | case TokenEOF: | |
| 77 | + | return "EOF" | |
| 78 | + | case TokenString: | |
| 79 | + | return string(t.Value.(String)) | |
| 80 | + | } | |
| 81 | + | return t.Lexeme | |
| 82 | + | } | |
| 83 | + | ||
| 84 | + | // Lex splits an expression into tokens. The result always ends with a | |
| 85 | + | // TokenEOF token. | |
| 86 | + | func Lex(input string) ([]Token, error) { | |
| 87 | + | if exceedsUTF16Len(input, MaxExpressionLength) { | |
| 88 | + | // The reference throws a bare Error("ErrorExceededMaxLength") here, | |
| 89 | + | // not the descriptive ExpressionError the parser would build. The | |
| 90 | + | // length is checked before the input is converted so that a huge | |
| 91 | + | // input costs no allocation. | |
| 92 | + | return nil, &Error{Type: ErrorExceededMaxLength, Message: "ErrorExceededMaxLength"} | |
| 93 | + | } | |
| 94 | + | units := utf16.Encode([]rune(input)) | |
| 95 | + | lx := &lexer{input: string(utf16.Decode(units)), units: units} | |
| 96 | + | if err := lx.lex(); err != nil { | |
| 97 | + | return nil, err | |
| 98 | + | } | |
| 99 | + | return lx.tokens, nil | |
| 100 | + | } | |
| 101 | + | ||
| 102 | + | // lexer is a line-by-line port of Lexer in lexer.js. It works on UTF-16 code | |
| 103 | + | // units so that positions match the reference. | |
| 104 | + | type lexer struct { | |
| 105 | + | input string | |
| 106 | + | units []uint16 | |
| 107 | + | start int | |
| 108 | + | offset int | |
| 109 | + | line int | |
| 110 | + | lastLineOffset int | |
| 111 | + | tokens []Token | |
| 112 | + | } | |
| 113 | + | ||
| 114 | + | func (l *lexer) lex() error { | |
| 115 | + | for !l.atEnd() { | |
| 116 | + | l.start = l.offset | |
| 117 | + | c := l.next() | |
| 118 | + | var err error | |
| 119 | + | switch c { | |
| 120 | + | case '(': | |
| 121 | + | l.addToken(TokenLeftParen, nil) | |
| 122 | + | case ')': | |
| 123 | + | l.addToken(TokenRightParen, nil) | |
| 124 | + | case '[': | |
| 125 | + | l.addToken(TokenLeftBracket, nil) | |
| 126 | + | case ']': | |
| 127 | + | l.addToken(TokenRightBracket, nil) | |
| 128 | + | case ',': | |
| 129 | + | l.addToken(TokenComma, nil) | |
| 130 | + | case '.': | |
| 131 | + | p := l.previous() | |
| 132 | + | if p != TokenIdentifier && p != TokenRightBracket && p != TokenRightParen && p != TokenStar { | |
| 133 | + | err = l.consumeNumber() | |
| 134 | + | } else { | |
| 135 | + | l.addToken(TokenDot, nil) | |
| 136 | + | } | |
| 137 | + | case '-', '+': | |
| 138 | + | err = l.consumeNumber() | |
| 139 | + | case '!': | |
| 140 | + | l.addToken(l.pick('=', TokenBangEqual, TokenBang), nil) | |
| 141 | + | case '=': | |
| 142 | + | if !l.match('=') { | |
| 143 | + | // Illegal; continue reading until a boundary and report it. | |
| 144 | + | err = l.consumeIdentifier() | |
| 145 | + | break | |
| 146 | + | } | |
| 147 | + | l.addToken(TokenEqualEqual, nil) | |
| 148 | + | case '<': | |
| 149 | + | l.addToken(l.pick('=', TokenLessEqual, TokenLess), nil) | |
| 150 | + | case '>': | |
| 151 | + | l.addToken(l.pick('=', TokenGreaterEqual, TokenGreater), nil) | |
| 152 | + | case '&': | |
| 153 | + | if !l.match('&') { | |
| 154 | + | err = l.consumeIdentifier() | |
| 155 | + | break | |
| 156 | + | } | |
| 157 | + | l.addToken(TokenAnd, nil) | |
| 158 | + | case '|': | |
| 159 | + | if !l.match('|') { | |
| 160 | + | err = l.consumeIdentifier() | |
| 161 | + | break | |
| 162 | + | } | |
| 163 | + | l.addToken(TokenOr, nil) | |
| 164 | + | case '*': | |
| 165 | + | l.addToken(TokenStar, nil) | |
| 166 | + | case ' ', '\r', '\t': | |
| 167 | + | // Whitespace is ignored. Only these three characters and the | |
| 168 | + | // newline are: other Unicode spaces (U+00A0, \v, \f...) are not | |
| 169 | + | // skipped here and fall through to the identifier rule. | |
| 170 | + | case '\n': | |
| 171 | + | l.line++ | |
| 172 | + | l.lastLineOffset = l.offset | |
| 173 | + | case '\'': | |
| 174 | + | err = l.consumeString() | |
| 175 | + | default: | |
| 176 | + | if c >= '0' && c <= '9' { | |
| 177 | + | err = l.consumeNumber() | |
| 178 | + | } else { | |
| 179 | + | err = l.consumeIdentifier() | |
| 180 | + | } | |
| 181 | + | } | |
| 182 | + | if err != nil { | |
| 183 | + | return err | |
| 184 | + | } | |
| 185 | + | } | |
| 186 | + | l.tokens = append(l.tokens, Token{Type: TokenEOF, Range: l.rng()}) | |
| 187 | + | return nil | |
| 188 | + | } | |
| 189 | + | ||
| 190 | + | func (l *lexer) pos() Position { | |
| 191 | + | return Position{Line: l.line, Column: l.start - l.lastLineOffset} | |
| 192 | + | } | |
| 193 | + | ||
| 194 | + | func (l *lexer) rng() Range { | |
| 195 | + | return Range{ | |
| 196 | + | Start: l.pos(), | |
| 197 | + | End: Position{Line: l.line, Column: l.offset - l.lastLineOffset}, | |
| 198 | + | } | |
| 199 | + | } | |
| 200 | + | ||
| 201 | + | func (l *lexer) atEnd() bool { return l.offset >= len(l.units) } | |
| 202 | + | ||
| 203 | + | // peek returns the next unit without consuming it, or 0 at the end. | |
| 204 | + | func (l *lexer) peek() uint16 { | |
| 205 | + | if l.atEnd() { | |
| 206 | + | return 0 | |
| 207 | + | } | |
| 208 | + | return l.units[l.offset] | |
| 209 | + | } | |
| 210 | + | ||
| 211 | + | func (l *lexer) peekNext() uint16 { | |
| 212 | + | if l.offset+1 >= len(l.units) { | |
| 213 | + | return 0 | |
| 214 | + | } | |
| 215 | + | return l.units[l.offset+1] | |
| 216 | + | } | |
| 217 | + | ||
| 218 | + | func (l *lexer) previous() TokenType { | |
| 219 | + | if len(l.tokens) == 0 { | |
| 220 | + | return TokenEOF | |
| 221 | + | } | |
| 222 | + | return l.tokens[len(l.tokens)-1].Type | |
| 223 | + | } | |
| 224 | + | ||
| 225 | + | func (l *lexer) next() uint16 { | |
| 226 | + | c := l.units[l.offset] | |
| 227 | + | l.offset++ | |
| 228 | + | return c | |
| 229 | + | } | |
| 230 | + | ||
| 231 | + | func (l *lexer) match(expected uint16) bool { | |
| 232 | + | if l.atEnd() || l.units[l.offset] != expected { | |
| 233 | + | return false | |
| 234 | + | } | |
| 235 | + | l.offset++ | |
| 236 | + | return true | |
| 237 | + | } | |
| 238 | + | ||
| 239 | + | // pick consumes expected if it is next and returns yes, otherwise no. | |
| 240 | + | func (l *lexer) pick(expected uint16, yes, no TokenType) TokenType { | |
| 241 | + | if l.match(expected) { | |
| 242 | + | return yes | |
| 243 | + | } | |
| 244 | + | return no | |
| 245 | + | } | |
| 246 | + | ||
| 247 | + | func (l *lexer) substring(from, to int) string { | |
| 248 | + | return string(utf16.Decode(l.units[from:to])) | |
| 249 | + | } | |
| 250 | + | ||
| 251 | + | func (l *lexer) addToken(t TokenType, v Value) { | |
| 252 | + | l.tokens = append(l.tokens, Token{ | |
| 253 | + | Type: t, | |
| 254 | + | Lexeme: l.substring(l.start, l.offset), | |
| 255 | + | Range: l.rng(), | |
| 256 | + | Value: v, | |
| 257 | + | }) | |
| 258 | + | } | |
| 259 | + | ||
| 260 | + | func (l *lexer) unexpected(lexeme string) *Error { | |
| 261 | + | return &Error{ | |
| 262 | + | Type: ErrorUnexpectedSymbol, | |
| 263 | + | Message: fmt.Sprintf("Unexpected symbol: '%s'. Located at position %d within expression: %s", lexeme, l.start+1, l.input), | |
| 264 | + | Pos: l.pos(), | |
| 265 | + | Token: lexeme, | |
| 266 | + | } | |
| 267 | + | } | |
| 268 | + | ||
| 269 | + | func (l *lexer) consumeNumber() error { | |
| 270 | + | for !l.atEnd() && (!isBoundary(l.peek()) || l.peek() == '.') { | |
| 271 | + | l.next() | |
| 272 | + | } | |
| 273 | + | lexeme := l.substring(l.start, l.offset) | |
| 274 | + | v := jsToNumber(lexeme) | |
| 275 | + | if math.IsNaN(v) { | |
| 276 | + | return l.unexpected(lexeme) | |
| 277 | + | } | |
| 278 | + | l.addToken(TokenNumber, Number(v)) | |
| 279 | + | return nil | |
| 280 | + | } | |
| 281 | + | ||
| 282 | + | func (l *lexer) consumeString() error { | |
| 283 | + | for (l.peek() != '\'' || l.peekNext() == '\'') && !l.atEnd() { | |
| 284 | + | if l.peek() == '\n' { | |
| 285 | + | l.line++ | |
| 286 | + | } | |
| 287 | + | if l.peek() == '\'' && l.peekNext() == '\'' { | |
| 288 | + | // Escaped quote: consume the first of the pair here. | |
| 289 | + | l.next() | |
| 290 | + | } | |
| 291 | + | l.next() | |
| 292 | + | } | |
| 293 | + | if l.atEnd() { | |
| 294 | + | return l.unexpected(l.substring(l.start, len(l.units))) | |
| 295 | + | } | |
| 296 | + | l.next() // closing quote | |
| 297 | + | value := l.substring(l.start+1, l.offset-1) | |
| 298 | + | // The reference calls value.replace("''", "'") with a string pattern, | |
| 299 | + | // which replaces only the FIRST doubled quote: 'a''b''c' is a'b''c. | |
| 300 | + | // GitHub's C# lexer unescapes every pair. Followed as written; see | |
| 301 | + | // PORTING.md. | |
| 302 | + | value = strings.Replace(value, "''", "'", 1) | |
| 303 | + | l.addToken(TokenString, String(value)) | |
| 304 | + | return nil | |
| 305 | + | } | |
| 306 | + | ||
| 307 | + | func (l *lexer) consumeIdentifier() error { | |
| 308 | + | for !l.atEnd() && !isBoundary(l.peek()) { | |
| 309 | + | l.next() | |
| 310 | + | } | |
| 311 | + | tokenType := TokenIdentifier | |
| 312 | + | var tokenValue Value | |
| 313 | + | lexeme := l.substring(l.start, l.offset) | |
| 314 | + | if l.previous() != TokenDot { | |
| 315 | + | switch lexeme { | |
| 316 | + | case "true": | |
| 317 | + | tokenType = TokenTrue | |
| 318 | + | case "false": | |
| 319 | + | tokenType = TokenFalse | |
| 320 | + | case "null": | |
| 321 | + | tokenType = TokenNull | |
| 322 | + | case "NaN": | |
| 323 | + | tokenType = TokenNumber | |
| 324 | + | tokenValue = Number(nan) | |
| 325 | + | case "Infinity": | |
| 326 | + | tokenType = TokenNumber | |
| 327 | + | tokenValue = Number(posInf) | |
| 328 | + | } | |
| 329 | + | } | |
| 330 | + | if !isLegalIdentifier(lexeme) { | |
| 331 | + | return l.unexpected(lexeme) | |
| 332 | + | } | |
| 333 | + | l.addToken(tokenType, tokenValue) | |
| 334 | + | return nil | |
| 335 | + | } | |
| 336 | + | ||
| 337 | + | // isBoundary reports whether c ends a number or identifier. | |
| 338 | + | func isBoundary(c uint16) bool { | |
| 339 | + | switch c { | |
| 340 | + | case '(', '[', ')', ']', ',', '.', '!', '>', '<', '=', '&', '|': | |
| 341 | + | return true | |
| 342 | + | } | |
| 343 | + | return isJSSpace(rune(c)) | |
| 344 | + | } | |
| 345 | + | ||
| 346 | + | // isLegalIdentifier reports whether s matches [A-Za-z_][A-Za-z0-9_-]*. | |
| 347 | + | func isLegalIdentifier(s string) bool { | |
| 348 | + | if s == "" { | |
| 349 | + | return false | |
| 350 | + | } | |
| 351 | + | for i := 0; i < len(s); i++ { | |
| 352 | + | c := s[i] | |
| 353 | + | switch { | |
| 354 | + | case c >= 'a' && c <= 'z', c >= 'A' && c <= 'Z', c == '_': | |
| 355 | + | case i > 0 && (c >= '0' && c <= '9' || c == '-'): | |
| 356 | + | default: | |
| 357 | + | return false | |
| 358 | + | } | |
| 359 | + | } | |
| 360 | + | return true | |
| 361 | + | } | |
A expr/limits_test.go +438 -0
| @@ -0,0 +1,438 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "errors" | |
| 8 | + | "fmt" | |
| 9 | + | "strings" | |
| 10 | + | "testing" | |
| 11 | + | "unicode" | |
| 12 | + | ) | |
| 13 | + | ||
| 14 | + | func mustParse(t testing.TB, src string, ctxNames []string, funcs []FuncInfo) Node { | |
| 15 | + | t.Helper() | |
| 16 | + | n, err := Parse(src, ctxNames, funcs) | |
| 17 | + | if err != nil { | |
| 18 | + | t.Fatalf("Parse(%.60q): %v", src, err) | |
| 19 | + | } | |
| 20 | + | return n | |
| 21 | + | } | |
| 22 | + | ||
| 23 | + | func evalWith(t testing.TB, src string, ec EvalContext) (Value, error) { | |
| 24 | + | t.Helper() | |
| 25 | + | return Evaluate(context.Background(), mustParse(t, src, []string{"big", "github"}, nil), ec) | |
| 26 | + | } | |
| 27 | + | ||
| 28 | + | func asEvalError(t testing.TB, err error) *EvalError { | |
| 29 | + | t.Helper() | |
| 30 | + | var ee *EvalError | |
| 31 | + | if !errors.As(err, &ee) { | |
| 32 | + | t.Fatalf("error %v (%T) is not an *EvalError", err, err) | |
| 33 | + | } | |
| 34 | + | return ee | |
| 35 | + | } | |
| 36 | + | ||
| 37 | + | // An expression of a few hundred bytes that would build a gigabyte string | |
| 38 | + | // without a budget: each level of format doubles the text. | |
| 39 | + | func doublingExpr(levels int) string { | |
| 40 | + | e := "'x'" | |
| 41 | + | for i := 0; i < levels; i++ { | |
| 42 | + | e = "format('{0}{0}', " + e + ")" | |
| 43 | + | } | |
| 44 | + | return e | |
| 45 | + | } | |
| 46 | + | ||
| 47 | + | func TestOutputBudgetFormat(t *testing.T) { | |
| 48 | + | // The doubling chain charges every intermediate string: 19 levels cost | |
| 49 | + | // 2^20-2 bytes in total, which just fits the default budget of 1 MiB; | |
| 50 | + | // 20 levels do not. | |
| 51 | + | if _, err := evalWith(t, doublingExpr(19), EvalContext{}); err != nil { | |
| 52 | + | t.Fatalf("19 doublings: %v", err) | |
| 53 | + | } | |
| 54 | + | _, err := evalWith(t, doublingExpr(20), EvalContext{}) | |
| 55 | + | if err == nil { | |
| 56 | + | t.Fatal("a chain of 20 doublings fits the budget?") | |
| 57 | + | } | |
| 58 | + | if ee := asEvalError(t, err); !strings.Contains(ee.Message, "exceeds the limit of 1048576 bytes") { | |
| 59 | + | t.Errorf("message = %q", ee.Message) | |
| 60 | + | } | |
| 61 | + | if v, err := evalWith(t, doublingExpr(10), EvalContext{}); err != nil || len(CoerceString(v)) != 1024 { | |
| 62 | + | t.Errorf("10 doublings: %v, %v", v, err) | |
| 63 | + | } | |
| 64 | + | // Absurd depth is cut off long before memory is. | |
| 65 | + | if _, err := evalWith(t, doublingExpr(40), EvalContext{}); err == nil { | |
| 66 | + | t.Error("40 doublings succeeded") | |
| 67 | + | } | |
| 68 | + | // The limit can be raised, lowered or removed. | |
| 69 | + | if _, err := evalWith(t, doublingExpr(12), EvalContext{MaxResultBytes: 4096}); err == nil { | |
| 70 | + | t.Error("small limit not applied") | |
| 71 | + | } | |
| 72 | + | if v, err := evalWith(t, doublingExpr(20), EvalContext{MaxResultBytes: -1}); err != nil || len(CoerceString(v)) != 1<<20 { | |
| 73 | + | t.Errorf("unlimited: %v", err) | |
| 74 | + | } | |
| 75 | + | if v, err := evalWith(t, doublingExpr(12), EvalContext{MaxResultBytes: 1 << 20}); err != nil || len(CoerceString(v)) != 4096 { | |
| 76 | + | t.Errorf("explicit limit: %v", err) | |
| 77 | + | } | |
| 78 | + | // A single oversized argument. | |
| 79 | + | big := String(strings.Repeat("y", 100)) | |
| 80 | + | _, err = evalWith(t, "format('{0}{0}', big)", EvalContext{Contexts: map[string]Value{"big": big}, MaxResultBytes: 150}) | |
| 81 | + | if err == nil { | |
| 82 | + | t.Error("format over the limit succeeded") | |
| 83 | + | } | |
| 84 | + | if _, err = evalWith(t, "format('{0}{0}', big)", EvalContext{Contexts: map[string]Value{"big": big}, MaxResultBytes: 200}); err != nil { | |
| 85 | + | t.Errorf("format at the limit failed: %v", err) | |
| 86 | + | } | |
| 87 | + | } | |
| 88 | + | ||
| 89 | + | func bigArray(n int) *Array { | |
| 90 | + | a := NewArray() | |
| 91 | + | for i := 0; i < n; i++ { | |
| 92 | + | a.Append(String("item")) | |
| 93 | + | } | |
| 94 | + | return a | |
| 95 | + | } | |
| 96 | + | ||
| 97 | + | func TestOutputBudgetJoin(t *testing.T) { | |
| 98 | + | ec := EvalContext{Contexts: map[string]Value{"big": bigArray(1000)}} | |
| 99 | + | // 1000 items of 4 bytes and 999 separators of 2: about 6 KB. | |
| 100 | + | if _, err := evalWith(t, "join(big, ', ')", ec); err != nil { | |
| 101 | + | t.Fatal(err) | |
| 102 | + | } | |
| 103 | + | ec.MaxResultBytes = 5000 | |
| 104 | + | if _, err := evalWith(t, "join(big, ', ')", ec); err == nil { | |
| 105 | + | t.Error("join over the limit succeeded") | |
| 106 | + | } | |
| 107 | + | // A huge separator is charged too. | |
| 108 | + | ec = EvalContext{Contexts: map[string]Value{"big": bigArray(1000), "github": String(strings.Repeat("s", 2000))}, MaxResultBytes: 1 << 20} | |
| 109 | + | if _, err := evalWith(t, "join(big, github)", ec); err == nil { // 999 * 2000 bytes of separators | |
| 110 | + | t.Error("a 2 MB join fits a 1 MiB budget?") | |
| 111 | + | } | |
| 112 | + | } | |
| 113 | + | ||
| 114 | + | func TestOutputBudgetToJSON(t *testing.T) { | |
| 115 | + | ec := EvalContext{Contexts: map[string]Value{"big": bigArray(1000)}} | |
| 116 | + | if _, err := evalWith(t, "toJSON(big)", ec); err != nil { | |
| 117 | + | t.Fatal(err) | |
| 118 | + | } | |
| 119 | + | ec.MaxResultBytes = 1000 | |
| 120 | + | _, err := evalWith(t, "toJSON(big)", ec) | |
| 121 | + | if err == nil { | |
| 122 | + | t.Fatal("toJSON over the limit succeeded") | |
| 123 | + | } | |
| 124 | + | if ee := asEvalError(t, err); !strings.Contains(ee.Message, "exceeds the limit of 1000 bytes") { | |
| 125 | + | t.Errorf("message = %q", ee.Message) | |
| 126 | + | } | |
| 127 | + | // The budget is shared: two toJSON of 400 bytes each exceed 700. | |
| 128 | + | ec.Contexts = map[string]Value{"big": bigArray(40)} | |
| 129 | + | ec.MaxResultBytes = 700 | |
| 130 | + | if _, err := evalWith(t, "toJSON(big)", ec); err != nil { | |
| 131 | + | t.Fatalf("one toJSON: %v", err) | |
| 132 | + | } | |
| 133 | + | if _, err := evalWith(t, "toJSON(big) && toJSON(big)", ec); err == nil { | |
| 134 | + | t.Error("budget is not shared between calls") | |
| 135 | + | } | |
| 136 | + | } | |
| 137 | + | ||
| 138 | + | func TestOutputBudgetFilter(t *testing.T) { | |
| 139 | + | // A filter over 100 arrays of 100 items, then a second flattening: the | |
| 140 | + | // elements built are charged. | |
| 141 | + | outer := NewArray() | |
| 142 | + | for i := 0; i < 100; i++ { | |
| 143 | + | outer.Append(bigArray(100)) | |
| 144 | + | } | |
| 145 | + | ec := EvalContext{Contexts: map[string]Value{"big": outer}} | |
| 146 | + | if _, err := evalWith(t, "big.*.*", ec); err != nil { | |
| 147 | + | t.Fatalf("filter within budget: %v", err) | |
| 148 | + | } | |
| 149 | + | ec.MaxResultBytes = 100 * 16 // the first .* costs 1600, the second 160000 | |
| 150 | + | if _, err := evalWith(t, "big.*.*", ec); err == nil { | |
| 151 | + | t.Error("filter over the budget succeeded") | |
| 152 | + | } | |
| 153 | + | // A fan-out through shared arrays: an array of 1000 references to one | |
| 154 | + | // array of 1000 items yields a million elements after two stars. | |
| 155 | + | shared := bigArray(1000) | |
| 156 | + | fan := NewArray() | |
| 157 | + | for i := 0; i < 1000; i++ { | |
| 158 | + | fan.Append(shared) | |
| 159 | + | } | |
| 160 | + | ec = EvalContext{Contexts: map[string]Value{"big": fan}} | |
| 161 | + | _, err := evalWith(t, "big.*.*.*", ec) | |
| 162 | + | if err == nil { | |
| 163 | + | t.Fatal("fan-out filter succeeded") | |
| 164 | + | } | |
| 165 | + | _ = asEvalError(t, err) | |
| 166 | + | // Lookups that do not build arrays are free. | |
| 167 | + | if _, err := evalWith(t, "big[0][0]", EvalContext{Contexts: map[string]Value{"big": fan}, MaxResultBytes: 1}); err != nil { | |
| 168 | + | t.Errorf("indexing charged the budget: %v", err) | |
| 169 | + | } | |
| 170 | + | } | |
| 171 | + | ||
| 172 | + | func TestEvaluateContextCancellation(t *testing.T) { | |
| 173 | + | ctx, cancel := context.WithCancel(context.Background()) | |
| 174 | + | cancel() | |
| 175 | + | n := mustParse(t, "1 == 1", nil, nil) | |
| 176 | + | _, err := Evaluate(ctx, n, EvalContext{}) | |
| 177 | + | ee := asEvalError(t, err) | |
| 178 | + | if !errors.Is(err, context.Canceled) || ee.Unwrap() != context.Canceled { | |
| 179 | + | t.Errorf("err = %v, want context.Canceled", err) | |
| 180 | + | } | |
| 181 | + | ||
| 182 | + | // Cancelled from inside a function while a big filter and contains loop run. | |
| 183 | + | arr := bigArray(1 << 16) | |
| 184 | + | ctx, cancel = context.WithCancel(context.Background()) | |
| 185 | + | defer cancel() | |
| 186 | + | ec := EvalContext{ | |
| 187 | + | Contexts: map[string]Value{"big": arr, "github": String("g")}, | |
| 188 | + | Funcs: map[string]Func{"stop": func(context.Context, []Value) (Value, error) { | |
| 189 | + | cancel() | |
| 190 | + | return Null{}, nil | |
| 191 | + | }}, | |
| 192 | + | MaxResultBytes: -1, | |
| 193 | + | } | |
| 194 | + | node := mustParse(t, "stop() || contains(big, 'nope') || join(big, ',')", []string{"big", "github"}, []FuncInfo{{Name: "stop"}}) | |
| 195 | + | _, err = Evaluate(ctx, node, ec) | |
| 196 | + | if !errors.Is(err, context.Canceled) { | |
| 197 | + | t.Errorf("cancel inside evaluation: err = %v", err) | |
| 198 | + | } | |
| 199 | + | ||
| 200 | + | // A deadline during a long loop. | |
| 201 | + | ctx, cancel2 := context.WithTimeout(context.Background(), 0) | |
| 202 | + | defer cancel2() | |
| 203 | + | <-ctx.Done() | |
| 204 | + | if _, err = Evaluate(ctx, mustParse(t, "contains(big, 'nope')", []string{"big"}, nil), EvalContext{Contexts: map[string]Value{"big": arr}}); !errors.Is(err, context.DeadlineExceeded) { | |
| 205 | + | t.Errorf("deadline: err = %v", err) | |
| 206 | + | } | |
| 207 | + | // A nil context behaves as Background. | |
| 208 | + | //nolint:staticcheck // deliberately passing nil | |
| 209 | + | if v, err := Evaluate(nil, n, EvalContext{}); err != nil || v != Boolean(true) { | |
| 210 | + | t.Errorf("nil context: %v, %v", v, err) | |
| 211 | + | } | |
| 212 | + | } | |
| 213 | + | ||
| 214 | + | func TestFuncReceivesContext(t *testing.T) { | |
| 215 | + | type key struct{} | |
| 216 | + | ctx := context.WithValue(context.Background(), key{}, "marker") | |
| 217 | + | var got any | |
| 218 | + | ec := EvalContext{Funcs: map[string]Func{"probe": func(c context.Context, _ []Value) (Value, error) { | |
| 219 | + | got = c.Value(key{}) | |
| 220 | + | return Null{}, nil | |
| 221 | + | }}} | |
| 222 | + | if _, err := Evaluate(ctx, mustParse(t, "probe()", nil, []FuncInfo{{Name: "probe"}}), ec); err != nil { | |
| 223 | + | t.Fatal(err) | |
| 224 | + | } | |
| 225 | + | if got != "marker" { | |
| 226 | + | t.Errorf("function did not receive the evaluation context: %v", got) | |
| 227 | + | } | |
| 228 | + | } | |
| 229 | + | ||
| 230 | + | func TestHandBuiltASTArity(t *testing.T) { | |
| 231 | + | call := func(name string, n int) Node { | |
| 232 | + | args := make([]Node, n) | |
| 233 | + | for i := range args { | |
| 234 | + | args[i] = &Literal{Value: String("x")} | |
| 235 | + | } | |
| 236 | + | return &FunctionCall{Name: Token{Type: TokenIdentifier, Lexeme: name}, Args: args} | |
| 237 | + | } | |
| 238 | + | for _, tc := range []struct { | |
| 239 | + | name string | |
| 240 | + | n int | |
| 241 | + | want string | |
| 242 | + | }{ | |
| 243 | + | {"contains", 0, "Too few parameters supplied: 'contains'"}, | |
| 244 | + | {"contains", 1, "Too few parameters supplied: 'contains'"}, | |
| 245 | + | {"contains", 3, "Too many parameters supplied: 'contains'"}, | |
| 246 | + | {"startsWith", 1, "Too few parameters supplied: 'startsWith'"}, | |
| 247 | + | {"endsWith", 0, "Too few parameters supplied: 'endsWith'"}, | |
| 248 | + | {"join", 0, "Too few parameters supplied: 'join'"}, | |
| 249 | + | {"join", 3, "Too many parameters supplied: 'join'"}, | |
| 250 | + | {"toJSON", 0, "Too few parameters supplied: 'toJSON'"}, | |
| 251 | + | {"fromJSON", 2, "Too many parameters supplied: 'fromJSON'"}, | |
| 252 | + | {"format", 0, "Too few parameters supplied: 'format'"}, | |
| 253 | + | {"format", 256, "Too many parameters supplied: 'format'"}, | |
| 254 | + | {"case", 0, "Too few parameters supplied: 'case'"}, | |
| 255 | + | {"case", 2, "Too few parameters supplied: 'case'"}, | |
| 256 | + | {"case", 4, "Even number of parameters supplied, requires an odd number of parameters: 'case'"}, | |
| 257 | + | {"nosuch", 1, "function nosuch is not implemented"}, | |
| 258 | + | } { | |
| 259 | + | var err error | |
| 260 | + | func() { | |
| 261 | + | defer func() { | |
| 262 | + | if r := recover(); r != nil { | |
| 263 | + | t.Errorf("%s(%d args) panicked: %v", tc.name, tc.n, r) | |
| 264 | + | } | |
| 265 | + | }() | |
| 266 | + | _, err = Evaluate(context.Background(), call(tc.name, tc.n), EvalContext{}) | |
| 267 | + | }() | |
| 268 | + | if ee := (&EvalError{}); !errors.As(err, &ee) || ee.Message != tc.want { | |
| 269 | + | t.Errorf("%s(%d args): err = %v, want %q", tc.name, tc.n, err, tc.want) | |
| 270 | + | } | |
| 271 | + | } | |
| 272 | + | // Other malformed trees must not panic either. | |
| 273 | + | for _, n := range []Node{&Star{}, &Logical{}, &Binary{Left: &Literal{}, Right: &Literal{}}, &IndexAccess{Expr: &Literal{}, Index: &Literal{}}, &Literal{Value: nil}, &Unary{Expr: &Literal{}}} { | |
| 274 | + | if _, err := Evaluate(context.Background(), n, EvalContext{}); err != nil { | |
| 275 | + | _ = asEvalError(t, err) | |
| 276 | + | } | |
| 277 | + | } | |
| 278 | + | } | |
| 279 | + | ||
| 280 | + | func TestEvalDepthCap(t *testing.T) { | |
| 281 | + | var n Node = &Literal{Value: Boolean(true)} | |
| 282 | + | for i := 0; i < maxEvalDepth+10; i++ { | |
| 283 | + | n = &Grouping{Group: n} | |
| 284 | + | } | |
| 285 | + | _, err := Evaluate(context.Background(), n, EvalContext{}) | |
| 286 | + | if ee := asEvalError(t, err); !strings.Contains(ee.Message, "deeper than") { | |
| 287 | + | t.Errorf("message = %q", ee.Message) | |
| 288 | + | } | |
| 289 | + | } | |
| 290 | + | ||
| 291 | + | func TestExtensionErrorsAreWrapped(t *testing.T) { | |
| 292 | + | sentinel := errors.New("kaput") | |
| 293 | + | ec := EvalContext{Funcs: map[string]Func{ | |
| 294 | + | "boom": func(context.Context, []Value) (Value, error) { return nil, sentinel }, | |
| 295 | + | "wrapped": func(context.Context, []Value) (Value, error) { return nil, fmt.Errorf("while hashing: %w", sentinel) }, | |
| 296 | + | "already": func(context.Context, []Value) (Value, error) { return nil, &EvalError{Message: "mine"} }, | |
| 297 | + | }} | |
| 298 | + | infos := []FuncInfo{{Name: "boom"}, {Name: "wrapped"}, {Name: "already"}} | |
| 299 | + | for src, msg := range map[string]string{"boom()": "kaput", "wrapped()": "while hashing: kaput", "already()": "mine"} { | |
| 300 | + | _, err := Evaluate(context.Background(), mustParse(t, src, nil, infos), ec) | |
| 301 | + | ee := asEvalError(t, err) | |
| 302 | + | if ee.Message != msg { | |
| 303 | + | t.Errorf("%s: message %q, want %q", src, ee.Message, msg) | |
| 304 | + | } | |
| 305 | + | if src != "already()" && (!errors.Is(err, sentinel) || ee.Unwrap() == nil) { | |
| 306 | + | t.Errorf("%s: sentinel not reachable with errors.Is", src) | |
| 307 | + | } | |
| 308 | + | } | |
| 309 | + | } | |
| 310 | + | ||
| 311 | + | func cyclic() *Array { | |
| 312 | + | a := NewArray() | |
| 313 | + | a.Append(a) | |
| 314 | + | return a | |
| 315 | + | } | |
| 316 | + | ||
| 317 | + | func TestToJSONCycle(t *testing.T) { | |
| 318 | + | a := cyclic() | |
| 319 | + | if got := ToJSON(a); got != "null" { | |
| 320 | + | t.Errorf("ToJSON(cycle) = %.40q, want null", got) | |
| 321 | + | } | |
| 322 | + | if _, err := EncodeJSON(a, -1); err == nil || !strings.Contains(err.Error(), "nested too deeply") { | |
| 323 | + | t.Errorf("EncodeJSON(cycle) err = %v", err) | |
| 324 | + | } | |
| 325 | + | d := NewDictionary() | |
| 326 | + | d.Add("self", d) | |
| 327 | + | if _, err := EncodeJSON(d, -1); err == nil { | |
| 328 | + | t.Error("dictionary cycle encoded") | |
| 329 | + | } | |
| 330 | + | // Evaluated: toJSON(cycle) is an *EvalError, not a stack overflow. | |
| 331 | + | _, err := evalWith(t, "toJSON(big)", EvalContext{Contexts: map[string]Value{"big": a}}) | |
| 332 | + | _ = asEvalError(t, err) | |
| 333 | + | // Everything within MaxJSONDepth still works, and the cut is exact. | |
| 334 | + | deep := Value(Number(1)) | |
| 335 | + | for i := 0; i < MaxJSONDepth; i++ { | |
| 336 | + | deep = NewArray(deep) | |
| 337 | + | } | |
| 338 | + | if _, err := EncodeJSON(deep, -1); err != nil { | |
| 339 | + | t.Errorf("depth %d: %v", MaxJSONDepth, err) | |
| 340 | + | } | |
| 341 | + | if _, err := EncodeJSON(NewArray(deep), -1); err == nil { | |
| 342 | + | t.Errorf("depth %d accepted", MaxJSONDepth+1) | |
| 343 | + | } | |
| 344 | + | } | |
| 345 | + | ||
| 346 | + | func TestEncodeJSONLimit(t *testing.T) { | |
| 347 | + | a := bigArray(100) | |
| 348 | + | full, err := EncodeJSON(a, -1) | |
| 349 | + | if err != nil { | |
| 350 | + | t.Fatal(err) | |
| 351 | + | } | |
| 352 | + | if got, err := EncodeJSON(a, len(full)); err != nil || got != full { | |
| 353 | + | t.Errorf("exact limit: %v", err) | |
| 354 | + | } | |
| 355 | + | if _, err := EncodeJSON(a, len(full)-1); err == nil { | |
| 356 | + | t.Error("one byte over the limit accepted") | |
| 357 | + | } | |
| 358 | + | if _, err := EncodeJSON(String(strings.Repeat("a", 100)), 50); err == nil { | |
| 359 | + | t.Error("long string over the limit accepted") | |
| 360 | + | } | |
| 361 | + | if got, err := EncodeJSON(Null{}, 0); err == nil { | |
| 362 | + | t.Errorf("limit 0: got %q", got) | |
| 363 | + | } | |
| 364 | + | } | |
| 365 | + | ||
| 366 | + | func TestOverLongInputIsRejectedCheaply(t *testing.T) { | |
| 367 | + | huge := strings.Repeat("a", 64<<20) | |
| 368 | + | allocs := testing.AllocsPerRun(3, func() { | |
| 369 | + | if _, err := Lex(huge); err == nil { | |
| 370 | + | t.Fatal("64 MiB expression accepted") | |
| 371 | + | } | |
| 372 | + | }) | |
| 373 | + | if allocs > 3 { | |
| 374 | + | t.Errorf("rejecting a 64 MiB expression allocated %v times", allocs) | |
| 375 | + | } | |
| 376 | + | if exceedsUTF16Len("😀", 1) != true || exceedsUTF16Len("😀", 2) || exceedsUTF16Len("ab", 2) || !exceedsUTF16Len("abc", 2) || exceedsUTF16Len("", 0) { | |
| 377 | + | t.Error("exceedsUTF16Len") | |
| 378 | + | } | |
| 379 | + | if exceedsUTF16Len("\xff\xff", 2) || !exceedsUTF16Len("\xff\xff\xff", 2) { | |
| 380 | + | t.Error("invalid bytes count as one unit each") | |
| 381 | + | } | |
| 382 | + | } | |
| 383 | + | ||
| 384 | + | // The case tables are generated relative to the Unicode version of the Go | |
| 385 | + | // toolchain: another toolchain needs them regenerated. | |
| 386 | + | func TestCasingTablesMatchGoUnicodeVersion(t *testing.T) { | |
| 387 | + | if unicode.Version != casingGoUnicodeVersion { | |
| 388 | + | t.Errorf("casing_tables.go was generated with Go's Unicode %s but this toolchain has %s: regenerate with `go generate ./expr` (needs node) and commit the result", | |
| 389 | + | casingGoUnicodeVersion, unicode.Version) | |
| 390 | + | } | |
| 391 | + | } | |
| 392 | + | ||
| 393 | + | func TestWTF8Helpers(t *testing.T) { | |
| 394 | + | hi := "\xed\xa0\x80" // U+D800 | |
| 395 | + | lo := "\xed\xb0\x80" // U+DC00 | |
| 396 | + | if !hasWTF8(hi) || hasWTF8("abc") || hasWTF8("é\xed") { | |
| 397 | + | t.Error("hasWTF8") | |
| 398 | + | } | |
| 399 | + | if r, w := decodeRune(hi); r != 0xD800 || w != 3 { | |
| 400 | + | t.Errorf("decodeRune = %U, %d", r, w) | |
| 401 | + | } | |
| 402 | + | for _, f := range []func(string) string{jsToUpper, toUpperSpecial, jsToLower, normalizeWTF8} { | |
| 403 | + | if got := f("a" + hi + "b"); !strings.Contains(got, hi) || strings.Contains(got, "\ufffd") { | |
| 404 | + | t.Errorf("lone surrogate not preserved: %q", got) | |
| 405 | + | } | |
| 406 | + | } | |
| 407 | + | if got := normalizeWTF8("\xed\xa0\xbd\xed\xb8\x80"); got != "😀" { | |
| 408 | + | t.Errorf("surrogate pair not joined: %q", got) | |
| 409 | + | } | |
| 410 | + | if got := normalizeWTF8(lo + hi); got != lo+hi { | |
| 411 | + | t.Errorf("reversed pair joined: %q", got) | |
| 412 | + | } | |
| 413 | + | if foldKey(hi) != hi || foldKey("A"+hi) != "a"+hi { | |
| 414 | + | t.Error("foldKey") | |
| 415 | + | } | |
| 416 | + | if compareUTF16(hi, "\ue000") >= 0 || compareUTF16(lo, "\ue000") >= 0 || compareUTF16(hi, "\U00010000") >= 0 { | |
| 417 | + | t.Error("lone surrogates sort before U+E000 and before supplementary characters starting with the same unit") | |
| 418 | + | } | |
| 419 | + | if !jsIncludes("\U0001F600", "\xed\xa0\xbd") || !jsHasPrefix("\U0001F600", "\xed\xa0\xbd") || !jsHasSuffix("\U0001F600", "\xed\xb8\x80") { | |
| 420 | + | t.Error("search by lone surrogate inside a supplementary character") | |
| 421 | + | } | |
| 422 | + | } | |
| 423 | + | ||
| 424 | + | func TestLoneSurrogateExpressions(t *testing.T) { | |
| 425 | + | // The cases the review asked for; the oracle covers many more. | |
| 426 | + | runEvalCases(t, []evalCase{ | |
| 427 | + | {`fromJSON('"\ud800"') == fromJSON('"\udc00"')`, "Boolean:false"}, | |
| 428 | + | {`fromJSON('"\ud800"') == fromJSON('"\ud800"')`, "Boolean:true"}, | |
| 429 | + | {"fromJSON('\"\\ud800\"') == '\ufffd'", "Boolean:false"}, | |
| 430 | + | {`fromJSON('"\ud800"') < fromJSON('"\udc00"')`, "Boolean:true"}, | |
| 431 | + | {"fromJSON('\"\\ud800\"') < '\ue000'", "Boolean:true"}, | |
| 432 | + | {"fromJSON('\"\\ud800\"') < '\U00010000'", "Boolean:true"}, | |
| 433 | + | {`contains(fromJSON('"a\ud800"'), fromJSON('"\ud800"'))`, "Boolean:true"}, | |
| 434 | + | {`contains(fromJSON('"\ud83d\ude00"'), fromJSON('"\ud83d"'))`, "Boolean:true"}, | |
| 435 | + | {`format('{0}{1}', fromJSON('"\ud83d"'), fromJSON('"\ude00"')) == '😀'`, "Boolean:true"}, | |
| 436 | + | {`toJSON(fromJSON('"\ud800"'))`, `String:"\ud800"`}, | |
| 437 | + | }) | |
| 438 | + | } | |
A expr/number.go +180 -0
| @@ -0,0 +1,180 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "math" | |
| 7 | + | "math/big" | |
| 8 | + | "strconv" | |
| 9 | + | "strings" | |
| 10 | + | ) | |
| 11 | + | ||
| 12 | + | var ( | |
| 13 | + | nan = math.NaN() | |
| 14 | + | posInf = math.Inf(1) | |
| 15 | + | ) | |
| 16 | + | ||
| 17 | + | // jsNumberToString is Number.prototype.toString: the shortest decimal that | |
| 18 | + | // round-trips, in positional notation for exponents from -7 to 20 and in | |
| 19 | + | // exponent notation ("1e+21", "1.5e-7") beyond. | |
| 20 | + | func jsNumberToString(x float64) string { | |
| 21 | + | switch { | |
| 22 | + | case math.IsNaN(x): | |
| 23 | + | return "NaN" | |
| 24 | + | case x == 0: | |
| 25 | + | return "0" | |
| 26 | + | case math.IsInf(x, 1): | |
| 27 | + | return "Infinity" | |
| 28 | + | case math.IsInf(x, -1): | |
| 29 | + | return "-Infinity" | |
| 30 | + | } | |
| 31 | + | sign := "" | |
| 32 | + | if x < 0 { | |
| 33 | + | sign = "-" | |
| 34 | + | x = -x | |
| 35 | + | } | |
| 36 | + | s := strconv.FormatFloat(x, 'e', -1, 64) // d.ddde±xx | |
| 37 | + | mant, expStr, _ := strings.Cut(s, "e") | |
| 38 | + | exp, _ := strconv.Atoi(expStr) | |
| 39 | + | digits := strings.Replace(mant, ".", "", 1) | |
| 40 | + | k := len(digits) | |
| 41 | + | n := exp + 1 | |
| 42 | + | switch { | |
| 43 | + | case k <= n && n <= 21: | |
| 44 | + | return sign + digits + strings.Repeat("0", n-k) | |
| 45 | + | case 0 < n && n <= 21: | |
| 46 | + | return sign + digits[:n] + "." + digits[n:] | |
| 47 | + | case -6 < n && n <= 0: | |
| 48 | + | return sign + "0." + strings.Repeat("0", -n) + digits | |
| 49 | + | } | |
| 50 | + | e := n - 1 | |
| 51 | + | esign := "+" | |
| 52 | + | if e < 0 { | |
| 53 | + | esign = "-" | |
| 54 | + | e = -e | |
| 55 | + | } | |
| 56 | + | if k == 1 { | |
| 57 | + | return sign + digits + "e" + esign + strconv.Itoa(e) | |
| 58 | + | } | |
| 59 | + | return sign + digits[:1] + "." + digits[1:] + "e" + esign + strconv.Itoa(e) | |
| 60 | + | } | |
| 61 | + | ||
| 62 | + | var pow10to15 = new(big.Int).Exp(big.NewInt(10), big.NewInt(15), nil) | |
| 63 | + | ||
| 64 | + | // numberToCoercedString is NumberData.coerceString of the reference | |
| 65 | + | // implementation: (+x.toFixed(15)).toString(). Rounding to 15 decimals | |
| 66 | + | // removes binary noise (0.1+0.2 is "0.3") and ties round half up like | |
| 67 | + | // toFixed. | |
| 68 | + | func numberToCoercedString(x float64) string { | |
| 69 | + | if x == 0 { | |
| 70 | + | return "0" | |
| 71 | + | } | |
| 72 | + | if math.IsNaN(x) || math.IsInf(x, 0) { | |
| 73 | + | return jsNumberToString(x) | |
| 74 | + | } | |
| 75 | + | a := math.Abs(x) | |
| 76 | + | // toFixed returns Number::toString from 1e21 up, and for integers the | |
| 77 | + | // fixed form parses back to the same number. | |
| 78 | + | if a >= 1e21 || a == math.Trunc(a) { | |
| 79 | + | return jsNumberToString(x) | |
| 80 | + | } | |
| 81 | + | r := new(big.Rat).SetFloat64(a) | |
| 82 | + | r.Mul(r, new(big.Rat).SetInt(pow10to15)) | |
| 83 | + | q, rem := new(big.Int).QuoRem(r.Num(), r.Denom(), new(big.Int)) | |
| 84 | + | if rem.Lsh(rem, 1).Cmp(r.Denom()) >= 0 { | |
| 85 | + | q.Add(q, big.NewInt(1)) | |
| 86 | + | } | |
| 87 | + | digits := q.String() | |
| 88 | + | if len(digits) < 16 { | |
| 89 | + | digits = strings.Repeat("0", 16-len(digits)) + digits | |
| 90 | + | } | |
| 91 | + | fixed := digits[:len(digits)-15] + "." + digits[len(digits)-15:] | |
| 92 | + | v, _ := strconv.ParseFloat(fixed, 64) | |
| 93 | + | if x < 0 { | |
| 94 | + | v = -v | |
| 95 | + | } | |
| 96 | + | return jsNumberToString(v) | |
| 97 | + | } | |
| 98 | + | ||
| 99 | + | // jsToNumber is Number(string): the StringToNumber algorithm of ECMAScript. | |
| 100 | + | // Surrounding whitespace is ignored, the empty string is 0, and the accepted | |
| 101 | + | // forms are decimal literals with optional sign, fraction and exponent, | |
| 102 | + | // [+-]Infinity, and unsigned 0x, 0o and 0b integers. Everything else, including | |
| 103 | + | // digit separators and "inf" or "nan", is NaN. | |
| 104 | + | func jsToNumber(s string) float64 { | |
| 105 | + | s = jsTrim(s) | |
| 106 | + | if s == "" { | |
| 107 | + | return 0 | |
| 108 | + | } | |
| 109 | + | if len(s) > 2 && s[0] == '0' { | |
| 110 | + | base := 0 | |
| 111 | + | switch s[1] { | |
| 112 | + | case 'x', 'X': | |
| 113 | + | base = 16 | |
| 114 | + | case 'o', 'O': | |
| 115 | + | base = 8 | |
| 116 | + | case 'b', 'B': | |
| 117 | + | base = 2 | |
| 118 | + | } | |
| 119 | + | if base != 0 { | |
| 120 | + | n, ok := new(big.Int).SetString(s[2:], base) | |
| 121 | + | if !ok || strings.ContainsAny(s[2:], "_+-") { | |
| 122 | + | return nan | |
| 123 | + | } | |
| 124 | + | f, _ := new(big.Float).SetInt(n).Float64() | |
| 125 | + | return f | |
| 126 | + | } | |
| 127 | + | } | |
| 128 | + | body := s | |
| 129 | + | neg := false | |
| 130 | + | if body[0] == '+' || body[0] == '-' { | |
| 131 | + | neg = body[0] == '-' | |
| 132 | + | body = body[1:] | |
| 133 | + | } | |
| 134 | + | if body == "Infinity" { | |
| 135 | + | if neg { | |
| 136 | + | return math.Inf(-1) | |
| 137 | + | } | |
| 138 | + | return math.Inf(1) | |
| 139 | + | } | |
| 140 | + | if !isDecimalLiteral(body) { | |
| 141 | + | return nan | |
| 142 | + | } | |
| 143 | + | v, _ := strconv.ParseFloat(body, 64) // out of range yields ±Inf, which is what JS does too | |
| 144 | + | if neg { | |
| 145 | + | v = -v | |
| 146 | + | } | |
| 147 | + | return v | |
| 148 | + | } | |
| 149 | + | ||
| 150 | + | // isDecimalLiteral reports whether s matches | |
| 151 | + | // (digits [. digits?] | . digits) ([eE] [+-]? digits)?. | |
| 152 | + | func isDecimalLiteral(s string) bool { | |
| 153 | + | i := 0 | |
| 154 | + | digits := func() int { | |
| 155 | + | start := i | |
| 156 | + | for i < len(s) && s[i] >= '0' && s[i] <= '9' { | |
| 157 | + | i++ | |
| 158 | + | } | |
| 159 | + | return i - start | |
| 160 | + | } | |
| 161 | + | intDigits := digits() | |
| 162 | + | fracDigits := 0 | |
| 163 | + | if i < len(s) && s[i] == '.' { | |
| 164 | + | i++ | |
| 165 | + | fracDigits = digits() | |
| 166 | + | } | |
| 167 | + | if intDigits == 0 && fracDigits == 0 { | |
| 168 | + | return false | |
| 169 | + | } | |
| 170 | + | if i < len(s) && (s[i] == 'e' || s[i] == 'E') { | |
| 171 | + | i++ | |
| 172 | + | if i < len(s) && (s[i] == '+' || s[i] == '-') { | |
| 173 | + | i++ | |
| 174 | + | } | |
| 175 | + | if digits() == 0 { | |
| 176 | + | return false | |
| 177 | + | } | |
| 178 | + | } | |
| 179 | + | return i == len(s) | |
| 180 | + | } | |
A expr/oracle_test.go +296 -0
| @@ -0,0 +1,296 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "compress/gzip" | |
| 7 | + | "context" | |
| 8 | + | "encoding/json" | |
| 9 | + | "errors" | |
| 10 | + | "fmt" | |
| 11 | + | "math" | |
| 12 | + | "os" | |
| 13 | + | "strings" | |
| 14 | + | "testing" | |
| 15 | + | ) | |
| 16 | + | ||
| 17 | + | // The oracle file holds what the reference implementation (GitHub's | |
| 18 | + | // @actions/expressions 0.3.61, run under Node) does on about 33 000 | |
| 19 | + | // expressions: hand-picked edge cases and a seeded random walk over the | |
| 20 | + | // grammar. It is generated by gen/oracle.mjs. TestOracle replays it and | |
| 21 | + | // demands identical tokens, error messages and positions, result kinds, | |
| 22 | + | // string coercions and toJSON output. | |
| 23 | + | ||
| 24 | + | type oracleFile struct { | |
| 25 | + | Reference oracleReference `json:"reference"` | |
| 26 | + | Contexts map[string]json.RawMessage `json:"contexts"` | |
| 27 | + | Cases []oracleCase `json:"cases"` | |
| 28 | + | } | |
| 29 | + | ||
| 30 | + | // oracleReference records where the oracle comes from. | |
| 31 | + | type oracleReference struct { | |
| 32 | + | Package string `json:"package"` | |
| 33 | + | Integrity string `json:"integrity"` | |
| 34 | + | Node string `json:"node"` | |
| 35 | + | V8 string `json:"v8"` | |
| 36 | + | Unicode string `json:"unicode"` | |
| 37 | + | } | |
| 38 | + | ||
| 39 | + | type oracleCase struct { | |
| 40 | + | E string `json:"e"` | |
| 41 | + | Tokens [][]any `json:"tokens"` | |
| 42 | + | LexErr *string `json:"lexErr"` | |
| 43 | + | ParseErr *string `json:"parseErr"` | |
| 44 | + | Pos []int `json:"pos"` | |
| 45 | + | Empty bool `json:"empty"` | |
| 46 | + | EvalErr *string `json:"evalErr"` | |
| 47 | + | R *oracleResult `json:"r"` | |
| 48 | + | } | |
| 49 | + | ||
| 50 | + | type oracleResult struct { | |
| 51 | + | K string `json:"k"` | |
| 52 | + | S string `json:"s"` | |
| 53 | + | J string `json:"j"` | |
| 54 | + | // U holds the UTF-16 code units of the string form when it contains a | |
| 55 | + | // lone surrogate and S is therefore empty. | |
| 56 | + | U []uint16 `json:"u"` | |
| 57 | + | } | |
| 58 | + | ||
| 59 | + | var oracleFuncInfos = []FuncInfo{ | |
| 60 | + | {Name: "success", MinArgs: 0, MaxArgs: 0}, | |
| 61 | + | {Name: "always", MinArgs: 0, MaxArgs: 0}, | |
| 62 | + | {Name: "hashFiles", MinArgs: 1, MaxArgs: 255}, | |
| 63 | + | } | |
| 64 | + | ||
| 65 | + | var oracleFuncs = map[string]Func{ | |
| 66 | + | "success": func(context.Context, []Value) (Value, error) { return Boolean(true), nil }, | |
| 67 | + | "always": func(context.Context, []Value) (Value, error) { return Boolean(true), nil }, | |
| 68 | + | "hashFiles": func(_ context.Context, args []Value) (Value, error) { | |
| 69 | + | s := "hash:" | |
| 70 | + | for i, a := range args { | |
| 71 | + | if i > 0 { | |
| 72 | + | s += "|" | |
| 73 | + | } | |
| 74 | + | s += CoerceString(a) | |
| 75 | + | } | |
| 76 | + | return String(s), nil | |
| 77 | + | }, | |
| 78 | + | } | |
| 79 | + | ||
| 80 | + | func kindName(v Value) string { | |
| 81 | + | switch v.Kind() { | |
| 82 | + | case KindNull: | |
| 83 | + | return "null" | |
| 84 | + | case KindBoolean: | |
| 85 | + | return "boolean" | |
| 86 | + | case KindNumber: | |
| 87 | + | return "number" | |
| 88 | + | case KindString: | |
| 89 | + | return "string" | |
| 90 | + | case KindArray: | |
| 91 | + | return "array" | |
| 92 | + | } | |
| 93 | + | return "object" | |
| 94 | + | } | |
| 95 | + | ||
| 96 | + | func loadOracle(tb testing.TB) (*oracleFile, map[string]Value, []string) { | |
| 97 | + | tb.Helper() | |
| 98 | + | f, err := os.Open("testdata/oracle.json.gz") | |
| 99 | + | if err != nil { | |
| 100 | + | tb.Fatal(err) | |
| 101 | + | } | |
| 102 | + | defer func() { _ = f.Close() }() | |
| 103 | + | zr, err := gzip.NewReader(f) | |
| 104 | + | if err != nil { | |
| 105 | + | tb.Fatal(err) | |
| 106 | + | } | |
| 107 | + | var of oracleFile | |
| 108 | + | if err := json.NewDecoder(zr).Decode(&of); err != nil { | |
| 109 | + | tb.Fatal(err) | |
| 110 | + | } | |
| 111 | + | ctx := map[string]Value{} | |
| 112 | + | var names []string | |
| 113 | + | for name, raw := range of.Contexts { | |
| 114 | + | v, err := FromJSON(raw) | |
| 115 | + | if err != nil { | |
| 116 | + | tb.Fatalf("context %s: %v", name, err) | |
| 117 | + | } | |
| 118 | + | ctx[name] = v | |
| 119 | + | names = append(names, name) | |
| 120 | + | } | |
| 121 | + | return &of, ctx, names | |
| 122 | + | } | |
| 123 | + | ||
| 124 | + | // oracleRun carries the state shared by the checks of one oracle case. | |
| 125 | + | type oracleRun struct { | |
| 126 | + | t *testing.T | |
| 127 | + | ec EvalContext | |
| 128 | + | ctxNames []string | |
| 129 | + | failures int | |
| 130 | + | c oracleCase | |
| 131 | + | } | |
| 132 | + | ||
| 133 | + | func (r *oracleRun) fail(format string, args ...any) { | |
| 134 | + | r.failures++ | |
| 135 | + | if r.failures <= 40 { | |
| 136 | + | e := r.c.E | |
| 137 | + | if len(e) > 200 { | |
| 138 | + | e = e[:200] + "..." | |
| 139 | + | } | |
| 140 | + | r.t.Errorf("%q: %s", e, fmt.Sprintf(format, args...)) | |
| 141 | + | } | |
| 142 | + | } | |
| 143 | + | ||
| 144 | + | // checkLex compares the tokens and reports whether the case lexes. | |
| 145 | + | func (r *oracleRun) checkLex() bool { | |
| 146 | + | c := r.c | |
| 147 | + | tokens, err := Lex(c.E) | |
| 148 | + | if c.LexErr != nil { | |
| 149 | + | switch { | |
| 150 | + | case err == nil: | |
| 151 | + | r.fail("lex: want error %q, got none", *c.LexErr) | |
| 152 | + | case err.Error() != *c.LexErr: | |
| 153 | + | r.fail("lex error: want %q, got %q", *c.LexErr, err.Error()) | |
| 154 | + | } | |
| 155 | + | return false | |
| 156 | + | } | |
| 157 | + | if err != nil { | |
| 158 | + | r.fail("lex: unexpected error %v", err) | |
| 159 | + | return false | |
| 160 | + | } | |
| 161 | + | if len(tokens) != len(c.Tokens) { | |
| 162 | + | r.fail("lex: %d tokens, want %d", len(tokens), len(c.Tokens)) | |
| 163 | + | return false | |
| 164 | + | } | |
| 165 | + | for i, tk := range tokens { | |
| 166 | + | w := c.Tokens[i] | |
| 167 | + | got := []any{float64(tk.Type), tk.Lexeme, float64(tk.Range.Start.Line), float64(tk.Range.Start.Column), float64(tk.Range.End.Line), float64(tk.Range.End.Column)} | |
| 168 | + | for j := range got { | |
| 169 | + | if got[j] != w[j] { | |
| 170 | + | r.fail("token %d field %d: got %v, want %v", i, j, got[j], w[j]) | |
| 171 | + | } | |
| 172 | + | } | |
| 173 | + | var gv any | |
| 174 | + | switch tk.Type { | |
| 175 | + | case TokenNumber: | |
| 176 | + | f := float64(tk.Value.(Number)) | |
| 177 | + | if math.IsNaN(f) { | |
| 178 | + | gv = "NaN" | |
| 179 | + | } else { | |
| 180 | + | gv = jsNumberToString(f) | |
| 181 | + | } | |
| 182 | + | case TokenString: | |
| 183 | + | gv = string(tk.Value.(String)) | |
| 184 | + | } | |
| 185 | + | if gv != w[6] { | |
| 186 | + | r.fail("token %d value: got %v, want %v", i, gv, w[6]) | |
| 187 | + | } | |
| 188 | + | } | |
| 189 | + | return true | |
| 190 | + | } | |
| 191 | + | ||
| 192 | + | // checkParse compares the parse outcome and returns the tree when the case | |
| 193 | + | // parses to something to evaluate. | |
| 194 | + | func (r *oracleRun) checkParse() Node { | |
| 195 | + | c := r.c | |
| 196 | + | node, err := Parse(c.E, r.ctxNames, oracleFuncInfos) | |
| 197 | + | if c.ParseErr != nil { | |
| 198 | + | var pe *Error | |
| 199 | + | switch { | |
| 200 | + | case err == nil: | |
| 201 | + | r.fail("parse: want error %q, got none", *c.ParseErr) | |
| 202 | + | case err.Error() != *c.ParseErr: | |
| 203 | + | r.fail("parse error: want %q, got %q", *c.ParseErr, err.Error()) | |
| 204 | + | case !errors.As(err, &pe): | |
| 205 | + | r.fail("parse error has type %T", err) | |
| 206 | + | case c.Pos != nil && (pe.Pos.Line != c.Pos[0] || pe.Pos.Column != c.Pos[1]): | |
| 207 | + | r.fail("parse error position: got %v, want %v", pe.Pos, c.Pos) | |
| 208 | + | } | |
| 209 | + | return nil | |
| 210 | + | } | |
| 211 | + | if err != nil { | |
| 212 | + | r.fail("parse: unexpected error %v", err) | |
| 213 | + | return nil | |
| 214 | + | } | |
| 215 | + | if c.Empty { | |
| 216 | + | if node != nil { | |
| 217 | + | r.fail("parse: want empty") | |
| 218 | + | } | |
| 219 | + | return nil | |
| 220 | + | } | |
| 221 | + | return node | |
| 222 | + | } | |
| 223 | + | ||
| 224 | + | func (r *oracleRun) checkEval(node Node) { | |
| 225 | + | c := r.c | |
| 226 | + | v, err := Evaluate(context.Background(), node, r.ec) | |
| 227 | + | if c.EvalErr != nil { | |
| 228 | + | switch { | |
| 229 | + | case err == nil: | |
| 230 | + | r.fail("eval: want error %q, got %s", *c.EvalErr, CoerceString(v)) | |
| 231 | + | case err.Error() != *c.EvalErr: | |
| 232 | + | r.fail("eval error: want %q, got %q", *c.EvalErr, err.Error()) | |
| 233 | + | } | |
| 234 | + | return | |
| 235 | + | } | |
| 236 | + | if err != nil { | |
| 237 | + | r.fail("eval: unexpected error %v", err) | |
| 238 | + | return | |
| 239 | + | } | |
| 240 | + | if k := kindName(v); k != c.R.K { | |
| 241 | + | r.fail("result kind: got %s, want %s", k, c.R.K) | |
| 242 | + | } | |
| 243 | + | if c.R.U != nil { | |
| 244 | + | if got := utf16Units(CoerceString(v)); !unitsEqual(got, c.R.U) { | |
| 245 | + | r.fail("result string units: got %04x, want %04x", got, c.R.U) | |
| 246 | + | } | |
| 247 | + | } else if s := CoerceString(v); s != c.R.S { | |
| 248 | + | r.fail("result string: got %q, want %q", s, c.R.S) | |
| 249 | + | } | |
| 250 | + | if j := ToJSON(v); j != c.R.J { | |
| 251 | + | r.fail("result JSON: got %q, want %q", j, c.R.J) | |
| 252 | + | } | |
| 253 | + | } | |
| 254 | + | ||
| 255 | + | func TestOracle(t *testing.T) { | |
| 256 | + | of, ctxValues, ctxNames := loadOracle(t) | |
| 257 | + | if len(of.Cases) < 30000 { | |
| 258 | + | t.Fatalf("oracle has only %d cases", len(of.Cases)) | |
| 259 | + | } | |
| 260 | + | ref := of.Reference | |
| 261 | + | if !strings.HasPrefix(ref.Integrity, "sha512-") { | |
| 262 | + | t.Errorf("oracle does not record the integrity of the reference package: %+v", ref) | |
| 263 | + | } | |
| 264 | + | t.Logf("reference: %s, integrity %s, generated with node %s (V8 %s, Unicode %s)", ref.Package, ref.Integrity, ref.Node, ref.V8, ref.Unicode) | |
| 265 | + | ||
| 266 | + | run := &oracleRun{t: t, ctxNames: ctxNames, ec: EvalContext{Contexts: ctxValues, Funcs: oracleFuncs, MaxResultBytes: -1}} | |
| 267 | + | var counts struct{ lexErr, parseErr, evalErr, ok, empty int } | |
| 268 | + | for _, c := range of.Cases { | |
| 269 | + | run.c = c | |
| 270 | + | switch { | |
| 271 | + | case c.LexErr != nil: | |
| 272 | + | counts.lexErr++ | |
| 273 | + | case c.ParseErr != nil: | |
| 274 | + | counts.parseErr++ | |
| 275 | + | case c.Empty: | |
| 276 | + | counts.empty++ | |
| 277 | + | case c.EvalErr != nil: | |
| 278 | + | counts.evalErr++ | |
| 279 | + | default: | |
| 280 | + | counts.ok++ | |
| 281 | + | } | |
| 282 | + | if !run.checkLex() { | |
| 283 | + | continue | |
| 284 | + | } | |
| 285 | + | node := run.checkParse() | |
| 286 | + | if node == nil { | |
| 287 | + | continue | |
| 288 | + | } | |
| 289 | + | run.checkEval(node) | |
| 290 | + | } | |
| 291 | + | t.Logf("%d cases: %d lex errors, %d parse errors, %d empty, %d eval errors, %d values; %d failures", | |
| 292 | + | len(of.Cases), counts.lexErr, counts.parseErr, counts.empty, counts.evalErr, counts.ok, run.failures) | |
| 293 | + | if run.failures > 0 { | |
| 294 | + | t.Fatalf("%d oracle mismatches", run.failures) | |
| 295 | + | } | |
| 296 | + | } | |
A expr/parser.go +376 -0
| @@ -0,0 +1,376 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import "math" | |
| 6 | + | ||
| 7 | + | // FuncInfo describes a function that expressions may call besides the | |
| 8 | + | // built-in ones (case, contains, endsWith, format, fromJSON, join, | |
| 9 | + | // startsWith, toJSON). The parser checks names and argument counts against | |
| 10 | + | // it; the implementation is supplied at evaluation time in | |
| 11 | + | // [EvalContext.Funcs]. | |
| 12 | + | type FuncInfo struct { | |
| 13 | + | // Name is matched case-insensitively. | |
| 14 | + | Name string | |
| 15 | + | // MinArgs and MaxArgs bound the number of arguments. A negative MaxArgs | |
| 16 | + | // means no upper bound. | |
| 17 | + | MinArgs int | |
| 18 | + | MaxArgs int | |
| 19 | + | } | |
| 20 | + | ||
| 21 | + | // builtinFuncs are the functions of the reference's wellKnownFunctions, keyed | |
| 22 | + | // by lower-cased name. They take precedence over caller-supplied FuncInfo for | |
| 23 | + | // validation. | |
| 24 | + | var builtinFuncs = map[string]FuncInfo{ | |
| 25 | + | "case": {"case", 3, math.MaxInt}, | |
| 26 | + | "contains": {"contains", 2, 2}, | |
| 27 | + | "endswith": {"endsWith", 2, 2}, | |
| 28 | + | "format": {"format", 1, 255}, | |
| 29 | + | "fromjson": {"fromJson", 1, 1}, | |
| 30 | + | "join": {"join", 1, 2}, | |
| 31 | + | "startswith": {"startsWith", 2, 2}, | |
| 32 | + | "tojson": {"toJson", 1, 1}, | |
| 33 | + | } | |
| 34 | + | ||
| 35 | + | // Parse parses the text of one expression (the inside of ${{ }}). | |
| 36 | + | // | |
| 37 | + | // contexts lists the context names the expression may reference, such as | |
| 38 | + | // "github" or "matrix"; funcs lists the extra functions it may call. Names | |
| 39 | + | // are case-insensitive. A reference to any other context or function is an | |
| 40 | + | // error, as is a call with a wrong number of arguments. | |
| 41 | + | // | |
| 42 | + | // The returned error is an [*Error]. An expression with no tokens, such as | |
| 43 | + | // "" or " ", is not an error: the reference returns nothing for it, and Parse | |
| 44 | + | // returns a nil [Node], which [Evaluate] rejects. | |
| 45 | + | func Parse(expr string, contexts []string, funcs []FuncInfo) (Node, error) { | |
| 46 | + | tokens, err := Lex(expr) | |
| 47 | + | if err != nil { | |
| 48 | + | return nil, err | |
| 49 | + | } | |
| 50 | + | p := &parser{ | |
| 51 | + | tokens: tokens, | |
| 52 | + | contexts: make(map[string]bool, len(contexts)), | |
| 53 | + | funcs: make(map[string]FuncInfo, len(funcs)), | |
| 54 | + | } | |
| 55 | + | for _, c := range contexts { | |
| 56 | + | p.contexts[foldKey(c)] = true | |
| 57 | + | } | |
| 58 | + | for _, f := range funcs { | |
| 59 | + | p.funcs[foldKey(f.Name)] = f | |
| 60 | + | } | |
| 61 | + | return p.parse() | |
| 62 | + | } | |
| 63 | + | ||
| 64 | + | // parser is a port of Parser in parser.js: recursive descent, with the same | |
| 65 | + | // precedence (lowest first): ||, &&, == !=, < <= > >=, !, then index and | |
| 66 | + | // call. | |
| 67 | + | type parser struct { | |
| 68 | + | tokens []Token | |
| 69 | + | offset int | |
| 70 | + | depth int | |
| 71 | + | contexts map[string]bool | |
| 72 | + | funcs map[string]FuncInfo | |
| 73 | + | } | |
| 74 | + | ||
| 75 | + | func (p *parser) parse() (Node, error) { | |
| 76 | + | if p.atEnd() { | |
| 77 | + | return nil, nil | |
| 78 | + | } | |
| 79 | + | n, err := p.expression() | |
| 80 | + | if err != nil { | |
| 81 | + | return nil, err | |
| 82 | + | } | |
| 83 | + | if !p.atEnd() { | |
| 84 | + | return nil, newTokenError(ErrorUnexpectedSymbol, p.peek()) | |
| 85 | + | } | |
| 86 | + | return n, nil | |
| 87 | + | } | |
| 88 | + | ||
| 89 | + | func (p *parser) expression() (Node, error) { | |
| 90 | + | if err := p.incrDepth(); err != nil { | |
| 91 | + | return nil, err | |
| 92 | + | } | |
| 93 | + | n, err := p.logicalOr() | |
| 94 | + | p.depth-- | |
| 95 | + | return n, err | |
| 96 | + | } | |
| 97 | + | ||
| 98 | + | func (p *parser) logicalOr() (Node, error) { | |
| 99 | + | n, err := p.logicalAnd() | |
| 100 | + | if err != nil || !p.check(TokenOr) { | |
| 101 | + | return n, err | |
| 102 | + | } | |
| 103 | + | return p.logicalChain(TokenOr, n, p.logicalAnd) | |
| 104 | + | } | |
| 105 | + | ||
| 106 | + | func (p *parser) logicalAnd() (Node, error) { | |
| 107 | + | n, err := p.equality() | |
| 108 | + | if err != nil || !p.check(TokenAnd) { | |
| 109 | + | return n, err | |
| 110 | + | } | |
| 111 | + | return p.logicalChain(TokenAnd, n, p.equality) | |
| 112 | + | } | |
| 113 | + | ||
| 114 | + | // logicalChain parses the rest of a flat chain of operator operands, the | |
| 115 | + | // first of which is already parsed. | |
| 116 | + | func (p *parser) logicalChain(op TokenType, first Node, operand func() (Node, error)) (Node, error) { | |
| 117 | + | if err := p.incrDepth(); err != nil { | |
| 118 | + | return nil, err | |
| 119 | + | } | |
| 120 | + | logical := &Logical{Operator: p.peek(), Args: []Node{first}} | |
| 121 | + | for p.match(op) { | |
| 122 | + | right, err := operand() | |
| 123 | + | if err != nil { | |
| 124 | + | return nil, err | |
| 125 | + | } | |
| 126 | + | logical.Args = append(logical.Args, right) | |
| 127 | + | } | |
| 128 | + | p.depth-- | |
| 129 | + | return logical, nil | |
| 130 | + | } | |
| 131 | + | ||
| 132 | + | func (p *parser) equality() (Node, error) { | |
| 133 | + | n, err := p.comparison() | |
| 134 | + | if err != nil { | |
| 135 | + | return nil, err | |
| 136 | + | } | |
| 137 | + | for p.match(TokenBangEqual, TokenEqualEqual) { | |
| 138 | + | op := p.previous() | |
| 139 | + | right, err := p.comparison() | |
| 140 | + | if err != nil { | |
| 141 | + | return nil, err | |
| 142 | + | } | |
| 143 | + | n = &Binary{Left: n, Operator: op, Right: right} | |
| 144 | + | } | |
| 145 | + | return n, nil | |
| 146 | + | } | |
| 147 | + | ||
| 148 | + | func (p *parser) comparison() (Node, error) { | |
| 149 | + | n, err := p.unary() | |
| 150 | + | if err != nil { | |
| 151 | + | return nil, err | |
| 152 | + | } | |
| 153 | + | for p.match(TokenGreater, TokenGreaterEqual, TokenLess, TokenLessEqual) { | |
| 154 | + | op := p.previous() | |
| 155 | + | right, err := p.unary() | |
| 156 | + | if err != nil { | |
| 157 | + | return nil, err | |
| 158 | + | } | |
| 159 | + | n = &Binary{Left: n, Operator: op, Right: right} | |
| 160 | + | } | |
| 161 | + | return n, nil | |
| 162 | + | } | |
| 163 | + | ||
| 164 | + | func (p *parser) unary() (Node, error) { | |
| 165 | + | if p.match(TokenBang) { | |
| 166 | + | if err := p.incrDepth(); err != nil { | |
| 167 | + | return nil, err | |
| 168 | + | } | |
| 169 | + | op := p.previous() | |
| 170 | + | operand, err := p.unary() | |
| 171 | + | if err != nil { | |
| 172 | + | return nil, err | |
| 173 | + | } | |
| 174 | + | p.depth-- | |
| 175 | + | return &Unary{Operator: op, Expr: operand}, nil | |
| 176 | + | } | |
| 177 | + | return p.index() | |
| 178 | + | } | |
| 179 | + | ||
| 180 | + | func (p *parser) index() (Node, error) { | |
| 181 | + | n, err := p.call() | |
| 182 | + | if err != nil { | |
| 183 | + | return nil, err | |
| 184 | + | } | |
| 185 | + | depthIncreased := 0 | |
| 186 | + | switch n.(type) { | |
| 187 | + | case *Grouping, *FunctionCall, *ContextAccess: | |
| 188 | + | loop: | |
| 189 | + | for { | |
| 190 | + | switch { | |
| 191 | + | case p.match(TokenLeftBracket): | |
| 192 | + | var idx Node | |
| 193 | + | if p.match(TokenStar) { | |
| 194 | + | idx = &Star{} | |
| 195 | + | } else if idx, err = p.expression(); err != nil { | |
| 196 | + | return nil, err | |
| 197 | + | } | |
| 198 | + | if err := p.consume(TokenRightBracket, ErrorUnexpectedSymbol); err != nil { | |
| 199 | + | return nil, err | |
| 200 | + | } | |
| 201 | + | if err := p.incrDepth(); err != nil { | |
| 202 | + | return nil, err | |
| 203 | + | } | |
| 204 | + | depthIncreased++ | |
| 205 | + | n = &IndexAccess{Expr: n, Index: idx} | |
| 206 | + | case p.match(TokenDot): | |
| 207 | + | if err := p.incrDepth(); err != nil { | |
| 208 | + | return nil, err | |
| 209 | + | } | |
| 210 | + | depthIncreased++ | |
| 211 | + | switch { | |
| 212 | + | case p.match(TokenIdentifier): | |
| 213 | + | prop := p.previous() | |
| 214 | + | n = &IndexAccess{Expr: n, Index: &Literal{Value: String(prop.Lexeme), Token: prop}} | |
| 215 | + | case p.match(TokenStar): | |
| 216 | + | n = &IndexAccess{Expr: n, Index: &Star{}} | |
| 217 | + | default: | |
| 218 | + | return nil, newTokenError(ErrorUnexpectedSymbol, p.peek()) | |
| 219 | + | } | |
| 220 | + | default: | |
| 221 | + | break loop | |
| 222 | + | } | |
| 223 | + | } | |
| 224 | + | } | |
| 225 | + | p.depth -= depthIncreased | |
| 226 | + | return n, nil | |
| 227 | + | } | |
| 228 | + | ||
| 229 | + | func (p *parser) call() (Node, error) { | |
| 230 | + | if !p.check(TokenIdentifier) { | |
| 231 | + | return p.primary() | |
| 232 | + | } | |
| 233 | + | ident := p.next() | |
| 234 | + | if !p.match(TokenLeftParen) { | |
| 235 | + | if !p.contexts[foldKey(ident.Lexeme)] { | |
| 236 | + | return nil, newTokenError(ErrorUnrecognizedContext, ident) | |
| 237 | + | } | |
| 238 | + | return &ContextAccess{Name: ident}, nil | |
| 239 | + | } | |
| 240 | + | var args []Node | |
| 241 | + | for !p.match(TokenRightParen) { | |
| 242 | + | arg, err := p.expression() | |
| 243 | + | if err != nil { | |
| 244 | + | return nil, err | |
| 245 | + | } | |
| 246 | + | args = append(args, arg) | |
| 247 | + | if !p.check(TokenRightParen) { | |
| 248 | + | // A trailing comma before ")" is accepted, as in the reference. | |
| 249 | + | if err := p.consume(TokenComma, ErrorUnexpectedSymbol); err != nil { | |
| 250 | + | return nil, err | |
| 251 | + | } | |
| 252 | + | } | |
| 253 | + | } | |
| 254 | + | if err := p.validateFunction(ident, len(args)); err != nil { | |
| 255 | + | return nil, err | |
| 256 | + | } | |
| 257 | + | return &FunctionCall{Name: ident, Args: args}, nil | |
| 258 | + | } | |
| 259 | + | ||
| 260 | + | // validateFunction checks that the function exists and accepts argCount | |
| 261 | + | // arguments (funcs.js validateFunction). | |
| 262 | + | func (p *parser) validateFunction(ident Token, argCount int) error { | |
| 263 | + | name := foldKey(ident.Lexeme) | |
| 264 | + | f, ok := builtinFuncs[name] | |
| 265 | + | if !ok { | |
| 266 | + | f, ok = p.funcs[name] | |
| 267 | + | if !ok { | |
| 268 | + | return newTokenError(ErrorUnrecognizedFunction, ident) | |
| 269 | + | } | |
| 270 | + | } | |
| 271 | + | if t, bad := arityError(name, f, argCount); bad { | |
| 272 | + | return newTokenError(t, ident) | |
| 273 | + | } | |
| 274 | + | return nil | |
| 275 | + | } | |
| 276 | + | ||
| 277 | + | // arityError reports whether argCount is wrong for f, and how. name is the | |
| 278 | + | // lower-cased function name: case also needs an odd number of arguments. | |
| 279 | + | func arityError(name string, f FuncInfo, argCount int) (ErrorType, bool) { | |
| 280 | + | switch { | |
| 281 | + | case argCount < f.MinArgs: | |
| 282 | + | return ErrorTooFewParameters, true | |
| 283 | + | case f.MaxArgs >= 0 && argCount > f.MaxArgs: | |
| 284 | + | return ErrorTooManyParameters, true | |
| 285 | + | case name == "case" && argCount%2 == 0: | |
| 286 | + | return ErrorEvenParameters, true | |
| 287 | + | } | |
| 288 | + | return 0, false | |
| 289 | + | } | |
| 290 | + | ||
| 291 | + | func (p *parser) primary() (Node, error) { | |
| 292 | + | switch { | |
| 293 | + | case p.match(TokenFalse): | |
| 294 | + | return &Literal{Value: Boolean(false), Token: p.previous()}, nil | |
| 295 | + | case p.match(TokenTrue): | |
| 296 | + | return &Literal{Value: Boolean(true), Token: p.previous()}, nil | |
| 297 | + | case p.match(TokenNull): | |
| 298 | + | return &Literal{Value: Null{}, Token: p.previous()}, nil | |
| 299 | + | case p.match(TokenNumber): | |
| 300 | + | return &Literal{Value: p.previous().Value, Token: p.previous()}, nil | |
| 301 | + | case p.match(TokenString): | |
| 302 | + | return &Literal{Value: p.previous().Value, Token: p.previous()}, nil | |
| 303 | + | case p.match(TokenLeftParen): | |
| 304 | + | n, err := p.expression() | |
| 305 | + | if err != nil { | |
| 306 | + | return nil, err | |
| 307 | + | } | |
| 308 | + | if p.atEnd() { | |
| 309 | + | // Back up to the last token before the EOF. | |
| 310 | + | return nil, newTokenError(ErrorUnexpectedEndOfExpression, p.previous()) | |
| 311 | + | } | |
| 312 | + | if err := p.consume(TokenRightParen, ErrorUnexpectedSymbol); err != nil { | |
| 313 | + | return nil, err | |
| 314 | + | } | |
| 315 | + | return &Grouping{Group: n}, nil | |
| 316 | + | case p.atEnd(): | |
| 317 | + | return nil, newTokenError(ErrorUnexpectedEndOfExpression, p.previous()) | |
| 318 | + | } | |
| 319 | + | return nil, newTokenError(ErrorUnexpectedSymbol, p.peek()) | |
| 320 | + | } | |
| 321 | + | ||
| 322 | + | // match consumes the next token if it has one of the given types. | |
| 323 | + | func (p *parser) match(types ...TokenType) bool { | |
| 324 | + | for _, t := range types { | |
| 325 | + | if p.check(t) { | |
| 326 | + | p.next() | |
| 327 | + | return true | |
| 328 | + | } | |
| 329 | + | } | |
| 330 | + | return false | |
| 331 | + | } | |
| 332 | + | ||
| 333 | + | func (p *parser) check(t TokenType) bool { | |
| 334 | + | if p.atEnd() { | |
| 335 | + | return false | |
| 336 | + | } | |
| 337 | + | return p.peek().Type == t | |
| 338 | + | } | |
| 339 | + | ||
| 340 | + | func (p *parser) atEnd() bool { return p.peek().Type == TokenEOF } | |
| 341 | + | ||
| 342 | + | func (p *parser) next() Token { | |
| 343 | + | if !p.atEnd() { | |
| 344 | + | p.offset++ | |
| 345 | + | } | |
| 346 | + | return p.previous() | |
| 347 | + | } | |
| 348 | + | ||
| 349 | + | func (p *parser) peek() Token { return p.tokens[p.offset] } | |
| 350 | + | ||
| 351 | + | // previous returns the token before the current one. At offset 0 the | |
| 352 | + | // reference reads tokens[-1], which is undefined and crashes; here it | |
| 353 | + | // returns the zero Token. That is unreachable: previous is only used after | |
| 354 | + | // a token was consumed. | |
| 355 | + | func (p *parser) previous() Token { | |
| 356 | + | if p.offset == 0 { | |
| 357 | + | return Token{} | |
| 358 | + | } | |
| 359 | + | return p.tokens[p.offset-1] | |
| 360 | + | } | |
| 361 | + | ||
| 362 | + | func (p *parser) consume(t TokenType, errType ErrorType) error { | |
| 363 | + | if p.check(t) { | |
| 364 | + | p.next() | |
| 365 | + | return nil | |
| 366 | + | } | |
| 367 | + | return newTokenError(errType, p.peek()) | |
| 368 | + | } | |
| 369 | + | ||
| 370 | + | func (p *parser) incrDepth() error { | |
| 371 | + | p.depth++ | |
| 372 | + | if p.depth > MaxParserDepth { | |
| 373 | + | return newTokenError(ErrorExceededMaxDepth, p.peek()) | |
| 374 | + | } | |
| 375 | + | return nil | |
| 376 | + | } | |
A expr/testdata/SOURCE +11 -0
| @@ -0,0 +1,11 @@ | |||
| 1 | + | oracle.json.gz and tojson/*.golden are outputs of @actions/expressions 0.3.61 | |
| 2 | + | (https://github.com/actions/languageservices, MIT, Copyright GitHub, Inc.) run | |
| 3 | + | on expressions and JSON inputs written for this repository. They are generated | |
| 4 | + | by ../gen/oracle.mjs and ../gen/golden.mjs. See ../../THIRD_PARTY.md. | |
| 5 | + | ||
| 6 | + | Reference package: @actions/expressions 0.3.61 | |
| 7 | + | integrity (sha512 of actions-expressions-0.3.61.tgz, computed locally in | |
| 8 | + | the format of the npm registry "dist.integrity" field): | |
| 9 | + | sha512-Ig+tXELvTjlqxQAWBQnLNn2dPpNraLMdSnT+vDzDPmJtbNUhIAH/Z9kVT8pZ8UtJi9vUGs8hwHE8JFJmbOnLEA== | |
| 10 | + | Generated with: node v25.2.1 (V8 14.1.146.11-node.14, Unicode 16.0) | |
| 11 | + | The same values are in the "reference" field of oracle.json.gz. | |
A expr/testdata/oracle.json.gz +0 -0 binary file: no text diff
binary file: no text diff
A expr/testdata/tojson/array_empty.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | []⏎̸ | |
A expr/testdata/tojson/array_empty.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | [] | |
A expr/testdata/tojson/array_flat.golden +5 -0
| @@ -0,0 +1,5 @@ | |||
| 1 | + | [ | |
| 2 | + | 1, | |
| 3 | + | 2, | |
| 4 | + | 3 | |
| 5 | + | ]⏎̸ | |
A expr/testdata/tojson/array_flat.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | [1,2,3] | |
A expr/testdata/tojson/array_mixed.golden +8 -0
| @@ -0,0 +1,8 @@ | |||
| 1 | + | [ | |
| 2 | + | null, | |
| 3 | + | true, | |
| 4 | + | "a", | |
| 5 | + | 1.5, | |
| 6 | + | [], | |
| 7 | + | {} | |
| 8 | + | ]⏎̸ | |
A expr/testdata/tojson/array_mixed.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | [null,true,"a",1.5,[],{}] | |
A expr/testdata/tojson/array_nested.golden +13 -0
| @@ -0,0 +1,13 @@ | |||
| 1 | + | [ | |
| 2 | + | [ | |
| 3 | + | 1, | |
| 4 | + | [ | |
| 5 | + | 2, | |
| 6 | + | [ | |
| 7 | + | 3, | |
| 8 | + | [] | |
| 9 | + | ] | |
| 10 | + | ] | |
| 11 | + | ], | |
| 12 | + | [] | |
| 13 | + | ]⏎̸ | |
A expr/testdata/tojson/array_nested.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | [[1,[2,[3,[]]]],[]] | |
A expr/testdata/tojson/big_int.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 12345678901234567000⏎̸ | |
A expr/testdata/tojson/big_int.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 12345678901234567890 | |
A expr/testdata/tojson/exp_20.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 100000000000000000000⏎̸ | |
A expr/testdata/tojson/exp_20.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1e20 | |
A expr/testdata/tojson/exp_large.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1e+21⏎̸ | |
A expr/testdata/tojson/exp_large.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1e21 | |
A expr/testdata/tojson/exp_pos.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 100⏎̸ | |
A expr/testdata/tojson/exp_pos.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1E2 | |
A expr/testdata/tojson/exp_small.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1e-7⏎̸ | |
A expr/testdata/tojson/exp_small.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1e-7 | |
A expr/testdata/tojson/false.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | false⏎̸ | |
A expr/testdata/tojson/false.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | false | |
A expr/testdata/tojson/float.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1.5⏎̸ | |
A expr/testdata/tojson/float.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1.5 | |
A expr/testdata/tojson/github_event.golden +27 -0
| @@ -0,0 +1,27 @@ | |||
| 1 | + | { | |
| 2 | + | "action": "opened", | |
| 3 | + | "number": 1, | |
| 4 | + | "pull_request": { | |
| 5 | + | "id": 1, | |
| 6 | + | "draft": false, | |
| 7 | + | "labels": [ | |
| 8 | + | { | |
| 9 | + | "name": "bug", | |
| 10 | + | "color": "d73a4a" | |
| 11 | + | }, | |
| 12 | + | { | |
| 13 | + | "name": "docs", | |
| 14 | + | "color": null | |
| 15 | + | } | |
| 16 | + | ], | |
| 17 | + | "head": { | |
| 18 | + | "ref": "feat/x", | |
| 19 | + | "sha": "abc123" | |
| 20 | + | } | |
| 21 | + | }, | |
| 22 | + | "repository": { | |
| 23 | + | "full_name": "octo/repo", | |
| 24 | + | "private": true, | |
| 25 | + | "topics": [] | |
| 26 | + | } | |
| 27 | + | }⏎̸ | |
A expr/testdata/tojson/github_event.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"action":"opened","number":1,"pull_request":{"id":1,"draft":false,"labels":[{"name":"bug","color":"d73a4a"},{"name":"docs","color":null}],"head":{"ref":"feat/x","sha":"abc123"}},"repository":{"full_name":"octo/repo","private":true,"topics":[]}} | |
A expr/testdata/tojson/int.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 42⏎̸ | |
A expr/testdata/tojson/int.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 42 | |
A expr/testdata/tojson/max_double.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1.7976931348623157e+308⏎̸ | |
A expr/testdata/tojson/max_double.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1.7976931348623157e308 | |
A expr/testdata/tojson/min_double.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 5e-324⏎̸ | |
A expr/testdata/tojson/min_double.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 5e-324 | |
A expr/testdata/tojson/negative.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | -17⏎̸ | |
A expr/testdata/tojson/negative.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | -17 | |
A expr/testdata/tojson/negzero.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 0⏎̸ | |
A expr/testdata/tojson/negzero.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | -0 | |
A expr/testdata/tojson/null.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | null⏎̸ | |
A expr/testdata/tojson/null.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | null | |
A expr/testdata/tojson/object_case_dup.golden +4 -0
| @@ -0,0 +1,4 @@ | |||
| 1 | + | { | |
| 2 | + | "a": 1, | |
| 3 | + | "b": 3 | |
| 4 | + | }⏎̸ | |
A expr/testdata/tojson/object_case_dup.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"a":1,"A":2,"b":3,"B":4} | |
A expr/testdata/tojson/object_dup_key.golden +4 -0
| @@ -0,0 +1,4 @@ | |||
| 1 | + | { | |
| 2 | + | "a": 3, | |
| 3 | + | "b": 2 | |
| 4 | + | }⏎̸ | |
A expr/testdata/tojson/object_dup_key.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"a":1,"b":2,"a":3} | |
A expr/testdata/tojson/object_empty.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {}⏎̸ | |
A expr/testdata/tojson/object_empty.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {} | |
A expr/testdata/tojson/object_empty_key.golden +4 -0
| @@ -0,0 +1,4 @@ | |||
| 1 | + | { | |
| 2 | + | "": 1, | |
| 3 | + | "a": 2 | |
| 4 | + | }⏎̸ | |
A expr/testdata/tojson/object_empty_key.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"":1,"a":2} | |
A expr/testdata/tojson/object_flat.golden +5 -0
| @@ -0,0 +1,5 @@ | |||
| 1 | + | { | |
| 2 | + | "a": 1, | |
| 3 | + | "b": "two", | |
| 4 | + | "c": null | |
| 5 | + | }⏎̸ | |
A expr/testdata/tojson/object_flat.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"a":1,"b":"two","c":null} | |
A expr/testdata/tojson/object_key_escapes.golden +5 -0
| @@ -0,0 +1,5 @@ | |||
| 1 | + | { | |
| 2 | + | "quote\"key": 1, | |
| 3 | + | "new\nline": 2, | |
| 4 | + | "é": 3 | |
| 5 | + | }⏎̸ | |
A expr/testdata/tojson/object_key_escapes.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"quote\"key":1,"new\nline":2,"\u00e9":3} | |
A expr/testdata/tojson/object_nested.golden +15 -0
| @@ -0,0 +1,15 @@ | |||
| 1 | + | { | |
| 2 | + | "a": { | |
| 3 | + | "b": { | |
| 4 | + | "c": { | |
| 5 | + | "d": [ | |
| 6 | + | 1, | |
| 7 | + | { | |
| 8 | + | "e": 2 | |
| 9 | + | } | |
| 10 | + | ] | |
| 11 | + | } | |
| 12 | + | } | |
| 13 | + | }, | |
| 14 | + | "f": [] | |
| 15 | + | }⏎̸ | |
A expr/testdata/tojson/object_nested.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"a":{"b":{"c":{"d":[1,{"e":2}]}}},"f":[]} | |
A expr/testdata/tojson/object_numeric_keys.golden +11 -0
| @@ -0,0 +1,11 @@ | |||
| 1 | + | { | |
| 2 | + | "1": 6, | |
| 3 | + | "2": 3, | |
| 4 | + | "10": 2, | |
| 5 | + | "4294967294": 7, | |
| 6 | + | "b": 1, | |
| 7 | + | "a": 4, | |
| 8 | + | "01": 5, | |
| 9 | + | "4294967295": 8, | |
| 10 | + | "-1": 9 | |
| 11 | + | }⏎̸ | |
A expr/testdata/tojson/object_numeric_keys.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"b":1,"10":2,"2":3,"a":4,"01":5,"1":6,"4294967294":7,"4294967295":8,"-1":9} | |
A expr/testdata/tojson/object_order.golden +5 -0
| @@ -0,0 +1,5 @@ | |||
| 1 | + | { | |
| 2 | + | "b": 1, | |
| 3 | + | "a": 2, | |
| 4 | + | "c": 3 | |
| 5 | + | }⏎̸ | |
A expr/testdata/tojson/object_order.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | {"b":1,"a":2,"c":3} | |
A expr/testdata/tojson/overflow.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | null⏎̸ | |
A expr/testdata/tojson/overflow.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 1e999 | |
A expr/testdata/tojson/precision.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 0.30000000000000004⏎̸ | |
A expr/testdata/tojson/precision.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 0.30000000000000004 | |
A expr/testdata/tojson/string_control.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "\u0000\u0001\u001f<U+007F><U+0080>"⏎̸ | |
A expr/testdata/tojson/string_control.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "\u0000\u0001\u001f\u007f\u0080" | |
A expr/testdata/tojson/string_empty.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | ""⏎̸ | |
A expr/testdata/tojson/string_empty.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "" | |
A expr/testdata/tojson/string_escapes.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "quote \" backslash \\ slash / nl \n cr \r tab \t bs \b ff \f"⏎̸ | |
A expr/testdata/tojson/string_escapes.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "quote \" backslash \\ slash \/ nl \n cr \r tab \t bs \b ff \f" | |
A expr/testdata/tojson/string_lone_surrogate.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "\ud800"⏎̸ | |
A expr/testdata/tojson/string_lone_surrogate.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "\ud800" | |
A expr/testdata/tojson/string_plain.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "hello"⏎̸ | |
A expr/testdata/tojson/string_plain.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "hello" | |
A expr/testdata/tojson/string_unicode.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "héllo 世界 😀 <U+2028><U+2029>"⏎̸ | |
A expr/testdata/tojson/string_unicode.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | "h\u00e9llo \u4e16\u754c \ud83d\ude00 \u2028\u2029" | |
A expr/testdata/tojson/true.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | true⏎̸ | |
A expr/testdata/tojson/true.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | true | |
A expr/testdata/tojson/zero.golden +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 0⏎̸ | |
A expr/testdata/tojson/zero.json +1 -0
| @@ -0,0 +1,1 @@ | |||
| 1 | + | 0 | |
A expr/testutil_test.go +62 -0
| @@ -0,0 +1,62 @@ | |||
| 1 | + | // SPDX-License-Identifier: MIT | |
| 2 | + | ||
| 3 | + | package expr | |
| 4 | + | ||
| 5 | + | import ( | |
| 6 | + | "context" | |
| 7 | + | "testing" | |
| 8 | + | ) | |
| 9 | + | ||
| 10 | + | // mustJSON parses JSON text into a Value or fails the test. | |
| 11 | + | func mustJSON(tb testing.TB, s string) Value { | |
| 12 | + | tb.Helper() | |
| 13 | + | v, err := FromJSON([]byte(s)) | |
| 14 | + | if err != nil { | |
| 15 | + | tb.Fatalf("FromJSON(%q): %v", s, err) | |
| 16 | + | } | |
| 17 | + | return v | |
| 18 | + | } | |
| 19 | + | ||
| 20 | + | // testContexts is the environment of the table tests. | |
| 21 | + | func testContexts(tb testing.TB) map[string]Value { | |
| 22 | + | tb.Helper() | |
| 23 | + | return map[string]Value{ | |
| 24 | + | "github": mustJSON(tb, `{ | |
| 25 | + | "event_name": "push", | |
| 26 | + | "ref": "refs/heads/main", | |
| 27 | + | "actor": "Octocat", | |
| 28 | + | "run_number": 7, | |
| 29 | + | "event": { | |
| 30 | + | "number": 12, | |
| 31 | + | "commits": [ | |
| 32 | + | {"id": "1", "message": "fix: bug", "author": {"name": "Ann"}, "added": ["a.txt"]}, | |
| 33 | + | {"id": "2", "message": "feat: thing", "author": {"name": "Bob"}, "added": []}, | |
| 34 | + | {"id": "3", "message": "Chore", "added": ["b", "c"]} | |
| 35 | + | ], | |
| 36 | + | "issue": {"labels": [{"name": "bug"}, {"name": "Docs"}]}, | |
| 37 | + | "nested": [[1, 2], [3], [], [4, 5, 6]] | |
| 38 | + | } | |
| 39 | + | }`), | |
| 40 | + | "env": mustJSON(tb, `{"FOO": "bar", "EMPTY": "", "NUM": "42", "Mixed": "MiXeD"}`), | |
| 41 | + | "matrix": mustJSON(tb, `{"os": "ubuntu", "node": [14, 16, 18], "nothing": null, "ver": "3.10", "obj": {"k": "v"}}`), | |
| 42 | + | "inputs": mustJSON(tb, `{"name": "World", "count": 3, "deploy": false, "empty": {}}`), | |
| 43 | + | "steps": mustJSON(tb, `{"10": {"outputs": {"r": "ten"}}, "2": {"outputs": {"r": "two"}}}`), | |
| 44 | + | } | |
| 45 | + | } | |
| 46 | + | ||
| 47 | + | var testContextNames = []string{"github", "env", "matrix", "inputs", "steps"} | |
| 48 | + | ||
| 49 | + | // evalString parses and evaluates src against the test contexts and returns | |
| 50 | + | // the result as "kind:string" or the error text prefixed by "ERR ". | |
| 51 | + | func evalString(tb testing.TB, src string) string { | |
| 52 | + | tb.Helper() | |
| 53 | + | node, err := Parse(src, testContextNames, nil) | |
| 54 | + | if err != nil { | |
| 55 | + | return "PARSE " + err.Error() | |
| 56 | + | } | |
| 57 | + | v, err := Evaluate(context.Background(), node, EvalContext{Contexts: testContexts(tb)}) | |
| 58 | + | if err != nil { | |
| 59 | + | return "ERR " + err.Error() | |
| 60 | + | } | |
| 61 | + | return v.Kind().String() + ":" + CoerceString(v) | |
| 62 | + | } | |