detached 900025e

fix(expr): read fromJSON with Json.NET's leniency, keep document order and write toJSON like the runner

commit
900025edf732e24aed3f5ea615545bd02cda189c
author
brice · · 2026-10-12 18:01 UTC
parent
75fcba3
signature
GPG VerifiedCD88F9E564E0102646FB6077A1A07C7C7C3E349D · arch-gpg Verified for the committer email
10 files · +286 -217
Split
UnifiedSplit
M expr/PORTING.md +37 -16
@@ -140,22 +140,43 @@ whether to keep the JS behaviour or follow the runner.
140140 is therefore a parse error. Same in the C# lexer (`LexicalAnalyzer.
141141 TryGetNextToken`: `c == '-' || c == '+' || digit` starts `ReadNumberToken`),
142142 checked by the differential run.
143-- [ ] **Object key order from `fromJSON`**: array-index keys first, ascending
144- (JavaScript `Object.keys`), then insertion order. `Json.NET` keeps
145- document order. The same ordering is applied by `toJSON` (the JS
146- replacer builds a plain object). Check against the runner for
147- `toJSON(fromJSON('{"b":1,"1":2}'))`.
148-- [ ] **Duplicate JSON keys**: last value wins at the first position (JS);
149- keys that differ only by case collapse to the first.
150-- [ ] **`fromJSON` errors**: the JS message is
151- `Error parsing JSON when evaluating fromJson` with the V8 `SyntaxError`
152- as cause; here the cause is a Go error with similar but not identical
153- wording (only the outer message is stable). `empty input` for blank
154- input. C# messages differ; check.
155-- [ ] **`toJSON`**: `JSON.stringify(v, replacer, " ")`: two-space indent,
156- `[]` and `{}` for empties, `NaN`/`Infinity` as `null`, `-0` as `0`,
157- no escaping of `<`, `>`, `&`, U+2028/U+2029; lone surrogates as
158- `\udXXX`. Json.NET differs on some of these (`Infinity`, `-0`, escapes).
143+- [x] **Object key order from `fromJSON`**: resolved, follows the C#. Keys keep
144+ the document order (`JTokenExtensions.ToPipelineContextData` iterates
145+ `JObject.Properties()` into `DictionaryContextData`, a list); the JS
146+ `Object.keys` order (array-index keys first) is gone, from `FromJSON` and
147+ from `toJSON`. `toJSON(fromJSON('{"b":1,"1":2}'))` is `b` then `1`.
148+- [x] **Duplicate JSON keys**: resolved, follows the C#. `subContext[property.Name] =
149+ ...` on a `DictionaryContextData` whose index is `OrdinalIgnoreCase`
150+ (`DictionaryContextData.this[]` setter): the last value wins, at the
151+ position and with the spelling of the first. `{"a":1,"A":2}` is
152+ `{"a": 2}` (the JS keeps `1`). Json.NET itself also merges exact
153+ duplicates (last wins).
154+- [x] **`fromJSON` errors and grammar**: resolved, follows the C#. The reader is
155+ Json.NET's (`Sdk/Functions/FromJson.cs`: `JToken.ReadFrom` on a
156+ `JsonTextReader` with `DateParseHandling.None`,
157+ `FloatParseHandling.Double`), which is lenient: single-quoted strings,
158+ unquoted names, trailing commas, `[,1]` (an empty element is `""`),
159+ `NaN`/`Infinity`/`-Infinity`/`undefined`, `.5`, `5.`, `01`, `0x1F`, raw
160+ control characters in strings, anything after the first value, and
161+ `MaxDepth` 64. An unpaired surrogate escape becomes U+FFFD. Blank input
162+ is a `JsonReaderException` (here: "empty input", kept as the outer
163+ message; the Json.NET message texts are not reproduced, only that the
164+ call fails). Not ported: `/* */` and `//` comments (they are read as a
165+ value by the runner, i.e. `/*x*/1` is the string `x`) and
166+ `new Date(1)` constructors; both are errors here.
167+- [x] **`toJSON`**: resolved, follows the C#. `Sdk/Functions/ToJson.cs` builds
168+ the text itself: two-space indentation, `[]`/`{}` for empties, numbers
169+ through `ConvertToString` (so `NaN`, `Infinity` and `-Infinity` appear
170+ bare, `1E+21`, 15 significant digits, `-0` is `0`) and strings through
171+ `JsonUtility.ToString` (Json.NET defaults: `\b \f \n \r \t`, `\u00XX`
172+ for other controls, `\u0085 \u2028 \u2029`, no escaping of `< > & '`, and
173+ non-ASCII as is). Checked against the runner's code
174+ (`testdata/tojson/*.runner`). The JS `null` for non-finite numbers, the
175+ shortest round-trip form and the raw U+2028 are not followed.
176+ Caveat: `JsonUtility` is stubbed in `gen/runner/Stubs.cs` with
177+ `JsonConvert.SerializeObject`; the real one passes the runner's
178+ `VssJsonMediaTypeFormatter` settings, which do not change string
179+ escaping.
159180 - [ ] **Index access**: `a[b]` where `b` is a number uses `floor(b)` and
160181 rejects NaN, negatives and infinities; where `b` is a string it is a key
161182 or a decimal index (`'1'`, `' 1 '`, `'1.9'` all index 1 of an array).
@@ -140,22 +140,43 @@ whether to keep the JS behaviour or follow the runner.
140 is therefore a parse error. Same in the C# lexer (`LexicalAnalyzer.140 is therefore a parse error. Same in the C# lexer (`LexicalAnalyzer.
141 TryGetNextToken`: `c == '-' || c == '+' || digit` starts `ReadNumberToken`),141 TryGetNextToken`: `c == '-' || c == '+' || digit` starts `ReadNumberToken`),
142 checked by the differential run.142 checked by the differential run.
143-- [ ] **Object key order from `fromJSON`**: array-index keys first, ascending143+- [x] **Object key order from `fromJSON`**: resolved, follows the C#. Keys keep
144- (JavaScript `Object.keys`), then insertion order. `Json.NET` keeps144+ the document order (`JTokenExtensions.ToPipelineContextData` iterates
145- document order. The same ordering is applied by `toJSON` (the JS145+ `JObject.Properties()` into `DictionaryContextData`, a list); the JS
146- replacer builds a plain object). Check against the runner for146+ `Object.keys` order (array-index keys first) is gone, from `FromJSON` and
147- `toJSON(fromJSON('{"b":1,"1":2}'))`.147+ from `toJSON`. `toJSON(fromJSON('{"b":1,"1":2}'))` is `b` then `1`.
148-- [ ] **Duplicate JSON keys**: last value wins at the first position (JS);148+- [x] **Duplicate JSON keys**: resolved, follows the C#. `subContext[property.Name] =
149- keys that differ only by case collapse to the first.149+ ...` on a `DictionaryContextData` whose index is `OrdinalIgnoreCase`
150-- [ ] **`fromJSON` errors**: the JS message is150+ (`DictionaryContextData.this[]` setter): the last value wins, at the
151- `Error parsing JSON when evaluating fromJson` with the V8 `SyntaxError`151+ position and with the spelling of the first. `{"a":1,"A":2}` is
152- as cause; here the cause is a Go error with similar but not identical152+ `{"a": 2}` (the JS keeps `1`). Json.NET itself also merges exact
153- wording (only the outer message is stable). `empty input` for blank153+ duplicates (last wins).
154- input. C# messages differ; check.154+- [x] **`fromJSON` errors and grammar**: resolved, follows the C#. The reader is
155-- [ ] **`toJSON`**: `JSON.stringify(v, replacer, " ")`: two-space indent,155+ Json.NET's (`Sdk/Functions/FromJson.cs`: `JToken.ReadFrom` on a
156- `[]` and `{}` for empties, `NaN`/`Infinity` as `null`, `-0` as `0`,156+ `JsonTextReader` with `DateParseHandling.None`,
157- no escaping of `<`, `>`, `&`, U+2028/U+2029; lone surrogates as157+ `FloatParseHandling.Double`), which is lenient: single-quoted strings,
158- `\udXXX`. Json.NET differs on some of these (`Infinity`, `-0`, escapes).158+ unquoted names, trailing commas, `[,1]` (an empty element is `""`),
159+ `NaN`/`Infinity`/`-Infinity`/`undefined`, `.5`, `5.`, `01`, `0x1F`, raw
160+ control characters in strings, anything after the first value, and
161+ `MaxDepth` 64. An unpaired surrogate escape becomes U+FFFD. Blank input
162+ is a `JsonReaderException` (here: "empty input", kept as the outer
163+ message; the Json.NET message texts are not reproduced, only that the
164+ call fails). Not ported: `/* */` and `//` comments (they are read as a
165+ value by the runner, i.e. `/*x*/1` is the string `x`) and
166+ `new Date(1)` constructors; both are errors here.
167+- [x] **`toJSON`**: resolved, follows the C#. `Sdk/Functions/ToJson.cs` builds
168+ the text itself: two-space indentation, `[]`/`{}` for empties, numbers
169+ through `ConvertToString` (so `NaN`, `Infinity` and `-Infinity` appear
170+ bare, `1E+21`, 15 significant digits, `-0` is `0`) and strings through
171+ `JsonUtility.ToString` (Json.NET defaults: `\b \f \n \r \t`, `\u00XX`
172+ for other controls, `\u0085 \u2028 \u2029`, no escaping of `< > & '`, and
173+ non-ASCII as is). Checked against the runner's code
174+ (`testdata/tojson/*.runner`). The JS `null` for non-finite numbers, the
175+ shortest round-trip form and the raw U+2028 are not followed.
176+ Caveat: `JsonUtility` is stubbed in `gen/runner/Stubs.cs` with
177+ `JsonConvert.SerializeObject`; the real one passes the runner's
178+ `VssJsonMediaTypeFormatter` settings, which do not change string
179+ escaping.
159 - [ ] **Index access**: `a[b]` where `b` is a number uses `floor(b)` and180 - [ ] **Index access**: `a[b]` where `b` is a number uses `floor(b)` and
160 rejects NaN, negatives and infinities; where `b` is a string it is a key181 rejects NaN, negatives and infinities; where `b` is a string it is a key
161 or a decimal index (`'1'`, `' 1 '`, `'1.9'` all index 1 of an array).182 or a decimal index (`'1'`, `' 1 '`, `'1.9'` all index 1 of an array).
M expr/data.go +13 -0
@@ -160,6 +160,19 @@ func (d *Dictionary) Add(key string, v Value) bool {
160160 return true
161161 }
162162
163+// set stores v under key: a key that differs only by case from an existing one
164+// replaces its value and keeps its spelling and position.
165+func (d *Dictionary) set(key string, v Value) {
166+ if d.index == nil {
167+ d.index = map[string]int{}
168+ }
169+ if i, ok := d.index[foldKey(key)]; ok {
170+ d.vals[i] = v
171+ return
172+ }
173+ d.Add(key, v)
174+}
175+
163176 // Get returns the value stored under key, compared case-insensitively.
164177 func (d *Dictionary) Get(key string) (Value, bool) {
165178 if d == nil {
@@ -160,6 +160,19 @@ func (d *Dictionary) Add(key string, v Value) bool {
160 return true160 return true
161 }161 }
162 162
163+// set stores v under key: a key that differs only by case from an existing one
164+// replaces its value and keeps its spelling and position.
165+func (d *Dictionary) set(key string, v Value) {
166+ if d.index == nil {
167+ d.index = map[string]int{}
168+ }
169+ if i, ok := d.index[foldKey(key)]; ok {
170+ d.vals[i] = v
171+ return
172+ }
173+ d.Add(key, v)
174+}
175+
163 // Get returns the value stored under key, compared case-insensitively.176 // Get returns the value stored under key, compared case-insensitively.
164 func (d *Dictionary) Get(key string) (Value, bool) {177 func (d *Dictionary) Get(key string) (Value, bool) {
165 if d == nil {178 if d == nil {
M expr/eval_test.go +10 -6
@@ -328,7 +328,9 @@ func TestEvalFilters(t *testing.T) {
328328 {"toJSON(matrix.node.*)", "String:[\n 14,\n 16,\n 18\n]"},
329329 {"toJSON(matrix.nothing.*)", "String:[]"},
330330 {"toJSON(inputs.empty.*)", "String:[]"},
331- {"toJSON(steps.*.outputs.r)", "String:[\n \"two\",\n \"ten\"\n]"},
331+ // Object members keep the order of the document (the JavaScript
332+ // reference puts array-index keys first).
333+ {"toJSON(steps.*.outputs.r)", "String:[\n \"ten\",\n \"two\"\n]"},
332334 {"toJSON(fromJSON('{\"a\":{\"x\":1},\"b\":{\"x\":2}}').*.x)", "String:[\n 1,\n 2\n]"},
333335 {"toJSON(fromJSON('[1,2]').*.x)", "String:[]"},
334336 {"toJSON(fromJSON('[{\"a\":[1,2]},{\"a\":[3]}]').*.a.*)", "String:[\n 1,\n 2,\n 3\n]"},
@@ -425,11 +427,13 @@ func TestEvalBuiltinFunctions(t *testing.T) {
425427 {"fromJSON('\n')", "ERR empty input"},
426428 {"fromJSON(null)", "ERR empty input"},
427429 {"fromJSON('{')", "ERR Error parsing JSON when evaluating fromJson"},
428- {"fromJSON('[1,]')", "ERR Error parsing JSON when evaluating fromJson"},
429- {"fromJSON('{a:1}')", "ERR Error parsing JSON when evaluating fromJson"},
430- {"fromJSON('01')", "ERR Error parsing JSON when evaluating fromJson"},
431- {"fromJSON('NaN')", "ERR Error parsing JSON when evaluating fromJson"},
432- {"fromJSON('1 2')", "ERR Error parsing JSON when evaluating fromJson"},
430+ // Json.NET is lenient: trailing commas, unquoted names, leading zeros,
431+ // NaN and Infinity and anything after the first value are accepted.
432+ {"fromJSON('[1,]')", "Array:Array"},
433+ {"fromJSON('{a:1}').a", "Number:1"},
434+ {"fromJSON('01')", "Number:1"},
435+ {"fromJSON('NaN')", "Number:NaN"},
436+ {"fromJSON('1 2')", "Number:1"}, // only the first value is read
433437 {"fromJSON(' 1 ')", "Number:1"},
434438 {"fromJSON('-1.5e2')", "Number:-150"},
435439 {"fromJSON('1e999')", "Number:Infinity"},
@@ -328,7 +328,9 @@ func TestEvalFilters(t *testing.T) {
328 {"toJSON(matrix.node.*)", "String:[\n 14,\n 16,\n 18\n]"},328 {"toJSON(matrix.node.*)", "String:[\n 14,\n 16,\n 18\n]"},
329 {"toJSON(matrix.nothing.*)", "String:[]"},329 {"toJSON(matrix.nothing.*)", "String:[]"},
330 {"toJSON(inputs.empty.*)", "String:[]"},330 {"toJSON(inputs.empty.*)", "String:[]"},
331- {"toJSON(steps.*.outputs.r)", "String:[\n \"two\",\n \"ten\"\n]"},331+ // Object members keep the order of the document (the JavaScript
332+ // reference puts array-index keys first).
333+ {"toJSON(steps.*.outputs.r)", "String:[\n \"ten\",\n \"two\"\n]"},
332 {"toJSON(fromJSON('{\"a\":{\"x\":1},\"b\":{\"x\":2}}').*.x)", "String:[\n 1,\n 2\n]"},334 {"toJSON(fromJSON('{\"a\":{\"x\":1},\"b\":{\"x\":2}}').*.x)", "String:[\n 1,\n 2\n]"},
333 {"toJSON(fromJSON('[1,2]').*.x)", "String:[]"},335 {"toJSON(fromJSON('[1,2]').*.x)", "String:[]"},
334 {"toJSON(fromJSON('[{\"a\":[1,2]},{\"a\":[3]}]').*.a.*)", "String:[\n 1,\n 2,\n 3\n]"},336 {"toJSON(fromJSON('[{\"a\":[1,2]},{\"a\":[3]}]').*.a.*)", "String:[\n 1,\n 2,\n 3\n]"},
@@ -425,11 +427,13 @@ func TestEvalBuiltinFunctions(t *testing.T) {
425 {"fromJSON('\n')", "ERR empty input"},427 {"fromJSON('\n')", "ERR empty input"},
426 {"fromJSON(null)", "ERR empty input"},428 {"fromJSON(null)", "ERR empty input"},
427 {"fromJSON('{')", "ERR Error parsing JSON when evaluating fromJson"},429 {"fromJSON('{')", "ERR Error parsing JSON when evaluating fromJson"},
428- {"fromJSON('[1,]')", "ERR Error parsing JSON when evaluating fromJson"},430+ // Json.NET is lenient: trailing commas, unquoted names, leading zeros,
429- {"fromJSON('{a:1}')", "ERR Error parsing JSON when evaluating fromJson"},431+ // NaN and Infinity and anything after the first value are accepted.
430- {"fromJSON('01')", "ERR Error parsing JSON when evaluating fromJson"},432+ {"fromJSON('[1,]')", "Array:Array"},
431- {"fromJSON('NaN')", "ERR Error parsing JSON when evaluating fromJson"},433+ {"fromJSON('{a:1}').a", "Number:1"},
432- {"fromJSON('1 2')", "ERR Error parsing JSON when evaluating fromJson"},434+ {"fromJSON('01')", "Number:1"},
435+ {"fromJSON('NaN')", "Number:NaN"},
436+ {"fromJSON('1 2')", "Number:1"}, // only the first value is read
433 {"fromJSON(' 1 ')", "Number:1"},437 {"fromJSON(' 1 ')", "Number:1"},
434 {"fromJSON('-1.5e2')", "Number:-150"},438 {"fromJSON('-1.5e2')", "Number:-150"},
435 {"fromJSON('1e999')", "Number:Infinity"},439 {"fromJSON('1e999')", "Number:Infinity"},
M expr/json.go +157 -163
@@ -6,9 +6,9 @@ import (
66 "errors"
77 "fmt"
88 "math"
9- "sort"
109 "strconv"
1110 "strings"
11+ "unicode"
1212 "unicode/utf8"
1313 )
1414
@@ -21,30 +21,37 @@ const MaxJSONDepth = 1000
2121
2222 // FromJSON parses JSON text into a [Value], like the fromJSON function.
2323 //
24-// The grammar is strict RFC 8259 JSON as JSON.parse accepts it. Numbers are
25-// float64 (1e999 is +Inf). Objects become [*Dictionary] values: keys that
26-// are array indices ("0", "42") come first in ascending order, then the rest
27-// in document order, as JavaScript enumerates object keys; a repeated key
28-// keeps its first position and its last value; keys that differ only by case
29-// collapse to the first.
24+// The reader is Json.NET's, as in the runner (Sdk/Functions/FromJson.cs:
25+// JToken.ReadFrom on a JsonTextReader, then ToPipelineContextData), so it
26+// accepts more than RFC 8259:
27+//
28+// - only the first value is read, anything after it is ignored;
29+// - strings may use single quotes and may contain raw control characters,
30+// object names may be unquoted (letters, digits, _ and $);
31+// - a trailing comma is accepted, an empty array element is an empty string;
32+// - NaN, Infinity and -Infinity are numbers, numbers may be written .5, 5.,
33+// 1.e2, 01 or 0x1F (hexadecimal, no sign), and undefined is an empty
34+// string;
35+// - nesting is limited to 64 levels (JsonReader.MaxDepth).
36+//
37+// Not ported (they fail here): comments and constructors (new Date(1)).
38+// Numbers are float64 (1e999 is +Inf). Objects become [*Dictionary] values in
39+// document order; a repeated key, compared case-insensitively, keeps the
40+// spelling and position of its first occurrence and the value of its last.
41+// An unpaired surrogate escape reads as U+FFFD.
3042 func FromJSON(data []byte) (Value, error) {
3143 s := string(data)
3244 if !utf8.ValidString(s) {
33- s = string([]rune(s)) // invalid bytes become U+FFFD, as in a JS string
45+ s = string([]rune(s)) // invalid bytes become U+FFFD
3446 }
3547 p := &jsonParser{s: s}
3648 p.skipSpace()
37- v, err := p.value(0)
38- if err != nil {
39- return nil, err
40- }
41- p.skipSpace()
42- if p.i < len(p.s) {
43- return nil, p.unexpected()
44- }
45- return v, nil
49+ return p.value(0)
4650 }
4751
52+// jsonReaderMaxDepth is Json.NET's JsonReader.MaxDepth default.
53+const jsonReaderMaxDepth = 64
54+
4855 type jsonParser struct {
4956 s string
5057 i int
@@ -55,102 +62,108 @@ func (p *jsonParser) unexpected() error {
5562 return errors.New("unexpected end of JSON input")
5663 }
5764 r, _ := utf8.DecodeRuneInString(p.s[p.i:])
58- return fmt.Errorf("unexpected token %q in JSON at position %d", r, p.i)
65+ return fmt.Errorf("unexpected character %q in JSON at position %d", r, p.i)
5966 }
6067
6168 func (p *jsonParser) skipSpace() {
6269 for p.i < len(p.s) {
63- switch p.s[p.i] {
64- case ' ', '\t', '\n', '\r':
65- p.i++
66- default:
70+ r, w := utf8.DecodeRuneInString(p.s[p.i:])
71+ if !isNetSpace(r) {
6772 return
6873 }
74+ p.i += w
6975 }
7076 }
7177
72-func (p *jsonParser) literal(word string, v Value) (Value, error) {
73- if !strings.HasPrefix(p.s[p.i:], word) {
74- // Advance to the first mismatching byte for the error position.
75- for j := 0; j < len(word) && p.i < len(p.s) && p.s[p.i] == word[j]; j++ {
76- p.i++
78+// isJSONSeparator reports whether c ends a literal or a number.
79+func isJSONSeparator(r rune) bool {
80+ switch r {
81+ case ',', '}', ']', ')', '/':
82+ return true
83+ }
84+ return isNetSpace(r)
85+}
86+
87+// token returns the run of characters up to the next separator.
88+func (p *jsonParser) token() string {
89+ start := p.i
90+ for p.i < len(p.s) {
91+ r, w := utf8.DecodeRuneInString(p.s[p.i:])
92+ if isJSONSeparator(r) {
93+ break
7794 }
78- return nil, p.unexpected()
95+ p.i += w
7996 }
80- p.i += len(word)
81- return v, nil
97+ return p.s[start:p.i]
8298 }
8399
84100 func (p *jsonParser) value(depth int) (Value, error) {
85- if depth > MaxJSONDepth {
86- return nil, errors.New("maximum JSON nesting depth exceeded")
87- }
88101 if p.i >= len(p.s) {
89102 return nil, p.unexpected()
90103 }
91104 switch c := p.s[p.i]; {
92105 case c == '{':
93- return p.object(depth)
106+ return p.object(depth + 1)
94107 case c == '[':
95- return p.array(depth)
96- case c == '"':
108+ return p.array(depth + 1)
109+ case c == '"' || c == '\'':
97110 s, err := p.str()
98111 if err != nil {
99112 return nil, err
100113 }
101114 return String(s), nil
102- case c == 't':
103- return p.literal("true", Boolean(true))
104- case c == 'f':
105- return p.literal("false", Boolean(false))
106- case c == 'n':
107- return p.literal("null", Null{})
108- case c == '-' || c >= '0' && c <= '9':
115+ case c == '-' || c == '.' || c >= '0' && c <= '9':
109116 return p.number()
110117 }
118+ start := p.i
119+ switch word := p.token(); word {
120+ case "true":
121+ return Boolean(true), nil
122+ case "false":
123+ return Boolean(false), nil
124+ case "null":
125+ return Null{}, nil
126+ case "undefined":
127+ return String(""), nil
128+ case "NaN":
129+ return Number(nan), nil
130+ case "Infinity":
131+ return Number(posInf), nil
132+ }
133+ p.i = start
111134 return nil, p.unexpected()
112135 }
113136
137+// number reads a number token: decimal with optional sign, fraction and
138+// exponent (a digit is required in the mantissa), -Infinity, or an unsigned
139+// 0x hexadecimal integer.
114140 func (p *jsonParser) number() (Value, error) {
115141 start := p.i
116- if p.s[p.i] == '-' {
117- p.i++
118- }
119- digits := func() int {
120- n := 0
121- for p.i < len(p.s) && p.s[p.i] >= '0' && p.s[p.i] <= '9' {
122- p.i++
123- n++
142+ tok := p.token()
143+ if tok == "-Infinity" {
144+ return Number(math.Inf(-1)), nil
145+ }
146+ if len(tok) > 2 && tok[0] == '0' && (tok[1] == 'x' || tok[1] == 'X') {
147+ if len(tok) <= 18 {
148+ if n, err := strconv.ParseUint(tok[2:], 16, 64); err == nil && !strings.ContainsAny(tok[2:], "+-_") {
149+ return Number(float64(int64(n))), nil
150+ }
124151 }
125- return n
126- }
127- switch {
128- case p.i < len(p.s) && p.s[p.i] == '0':
129- p.i++
130- case digits() == 0:
131- return nil, p.unexpected()
152+ p.i = start
153+ return nil, fmt.Errorf("input string %q is not a valid number", tok)
132154 }
133- if p.i < len(p.s) && p.s[p.i] == '.' {
134- p.i++
135- if digits() == 0 {
136- return nil, p.unexpected()
137- }
155+ body := strings.TrimPrefix(tok, "-")
156+ if !isDecimalLiteral(body) {
157+ p.i = start
158+ return nil, fmt.Errorf("input string %q is not a valid number", tok)
138159 }
139- if p.i < len(p.s) && (p.s[p.i] == 'e' || p.s[p.i] == 'E') {
140- p.i++
141- if p.i < len(p.s) && (p.s[p.i] == '+' || p.s[p.i] == '-') {
142- p.i++
143- }
144- if digits() == 0 {
145- return nil, p.unexpected()
146- }
147- }
148- f, _ := strconv.ParseFloat(p.s[start:p.i], 64) // range errors give ±Inf
160+ f, _ := strconv.ParseFloat(tok, 64) // range errors give ±Inf
149161 return Number(f), nil
150162 }
151163
152164 // str reads a string literal; p.i is on the opening quote.
153165 func (p *jsonParser) str() (string, error) {
166+ quote := p.s[p.i]
154167 p.i++
155168 var b strings.Builder
156169 for {
@@ -158,13 +171,11 @@ func (p *jsonParser) str() (string, error) {
158171 return "", errors.New("unterminated string in JSON")
159172 }
160173 c := p.s[p.i]
161- switch {
162- case c == '"':
174+ if c == quote {
163175 p.i++
164176 return b.String(), nil
165- case c < 0x20:
166- return "", p.unexpected()
167- case c != '\\':
177+ }
178+ if c != '\\' {
168179 b.WriteByte(c)
169180 p.i++
170181 continue
@@ -176,7 +187,7 @@ func (p *jsonParser) str() (string, error) {
176187 e := p.s[p.i]
177188 p.i++
178189 switch e {
179- case '"', '\\', '/':
190+ case '"', '\'', '\\', '/':
180191 b.WriteByte(e)
181192 case 'b':
182193 b.WriteByte('\b')
@@ -206,13 +217,13 @@ func (p *jsonParser) str() (string, error) {
206217 p.i = save // not a pair; the second escape is read on its own
207218 }
208219 }
209- // A lone surrogate cannot be a Go rune: writeRune keeps it in
210- // its three-byte WTF-8 form so that toJSON writes it back as
211- // the same \uXXXX escape, like JSON.stringify.
212- writeRune(&b, r)
220+ if r >= 0xD800 && r < 0xE000 {
221+ r = utf8.RuneError // an unpaired surrogate
222+ }
223+ b.WriteRune(r)
213224 default:
214- p.i--
215- return "", p.unexpected()
225+ p.i -= 2
226+ return "", fmt.Errorf("bad JSON escape sequence \\%c at position %d", e, p.i)
216227 }
217228 }
218229 }
@@ -243,16 +254,27 @@ func (p *jsonParser) hex4() (rune, error) {
243254 }
244255
245256 func (p *jsonParser) array(depth int) (Value, error) {
257+ if depth > jsonReaderMaxDepth {
258+ return nil, fmt.Errorf("the reader's MaxDepth of %d has been exceeded", jsonReaderMaxDepth)
259+ }
246260 p.i++
247261 arr := &Array{}
248- p.skipSpace()
249- if p.i < len(p.s) && p.s[p.i] == ']' {
250- p.i++
251- return arr, nil
252- }
253262 for {
254263 p.skipSpace()
255- v, err := p.value(depth + 1)
264+ if p.i >= len(p.s) {
265+ return nil, p.unexpected()
266+ }
267+ switch p.s[p.i] {
268+ case ']':
269+ p.i++
270+ return arr, nil
271+ case ',':
272+ // An empty element is "undefined", which reads as an empty string.
273+ p.i++
274+ arr.items = append(arr.items, String(""))
275+ continue
276+ }
277+ v, err := p.value(depth)
256278 if err != nil {
257279 return nil, err
258280 }
@@ -265,29 +287,50 @@ func (p *jsonParser) array(depth int) (Value, error) {
265287 case ',':
266288 p.i++
267289 case ']':
268- p.i++
269- return arr, nil
270290 default:
271291 return nil, p.unexpected()
272292 }
273293 }
274294 }
275295
296+// propertyName reads a quoted or unquoted object name.
297+func (p *jsonParser) propertyName() (string, error) {
298+ if p.s[p.i] == '"' || p.s[p.i] == '\'' {
299+ return p.str()
300+ }
301+ start := p.i
302+ for p.i < len(p.s) {
303+ r, w := utf8.DecodeRuneInString(p.s[p.i:])
304+ if r == ':' || isNetSpace(r) {
305+ break
306+ }
307+ if !unicode.IsLetter(r) && !unicode.IsDigit(r) && r != '_' && r != '$' {
308+ return "", p.unexpected()
309+ }
310+ p.i += w
311+ }
312+ if p.i == start {
313+ return "", p.unexpected()
314+ }
315+ return p.s[start:p.i], nil
316+}
317+
276318 func (p *jsonParser) object(depth int) (Value, error) {
277- p.i++
278- var keys []string
279- vals := map[string]Value{}
280- p.skipSpace()
281- if p.i < len(p.s) && p.s[p.i] == '}' {
282- p.i++
283- return NewDictionary(), nil
319+ if depth > jsonReaderMaxDepth {
320+ return nil, fmt.Errorf("the reader's MaxDepth of %d has been exceeded", jsonReaderMaxDepth)
284321 }
322+ p.i++
323+ d := NewDictionary()
285324 for {
286325 p.skipSpace()
287- if p.i >= len(p.s) || p.s[p.i] != '"' {
326+ if p.i >= len(p.s) {
288327 return nil, p.unexpected()
289328 }
290- k, err := p.str()
329+ if p.s[p.i] == '}' {
330+ p.i++
331+ return d, nil
332+ }
333+ k, err := p.propertyName()
291334 if err != nil {
292335 return nil, err
293336 }
@@ -297,14 +340,11 @@ func (p *jsonParser) object(depth int) (Value, error) {
297340 }
298341 p.i++
299342 p.skipSpace()
300- v, err := p.value(depth + 1)
343+ v, err := p.value(depth)
301344 if err != nil {
302345 return nil, err
303346 }
304- if _, dup := vals[k]; !dup {
305- keys = append(keys, k)
306- }
307- vals[k] = v // last value wins, at the first position
347+ d.set(k, v)
308348 p.skipSpace()
309349 if p.i >= len(p.s) {
310350 return nil, p.unexpected()
@@ -312,66 +352,21 @@ func (p *jsonParser) object(depth int) (Value, error) {
312352 switch p.s[p.i] {
313353 case ',':
314354 p.i++
315- continue
316355 case '}':
317- p.i++
318356 default:
319357 return nil, p.unexpected()
320358 }
321- break
322- }
323- d := NewDictionary()
324- for _, k := range jsKeyOrder(keys) {
325- d.Add(k, vals[k])
326- }
327- return d, nil
328-}
329-
330-// isArrayIndex reports whether k is the canonical decimal form of an integer
331-// in [0, 2^32-2], the keys JavaScript enumerates before all others.
332-func isArrayIndex(k string) bool {
333- if k == "" || len(k) > 10 || (len(k) > 1 && k[0] == '0') {
334- return false
335- }
336- for i := 0; i < len(k); i++ {
337- if k[i] < '0' || k[i] > '9' {
338- return false
339- }
340- }
341- n, _ := strconv.ParseUint(k, 10, 64)
342- return n < math.MaxUint32
343-}
344-
345-// jsKeyOrder reorders keys the way Object.keys enumerates a JavaScript
346-// object: array indices ascending, then the others in their given order.
347-func jsKeyOrder(keys []string) []string {
348- var idx, rest []string
349- for _, k := range keys {
350- if isArrayIndex(k) {
351- idx = append(idx, k)
352- } else {
353- rest = append(rest, k)
354- }
355- }
356- if len(idx) == 0 {
357- return keys
358359 }
359- sort.Slice(idx, func(i, j int) bool {
360- a, _ := strconv.ParseUint(idx[i], 10, 64)
361- b, _ := strconv.ParseUint(idx[j], 10, 64)
362- return a < b
363- })
364- return append(idx, rest...)
365360 }
366361
367-// ToJSON renders v as the toJSON function does: JSON.stringify(v, replacer,
368-// " "), that is two-space indentation, "key": value pairs, "[]" and "{}"
369-// for empty containers, and numbers as JavaScript prints them (NaN and the
370-// infinities as null, -0 as 0, 1e21 as 1e+21). Dictionary keys that are array
371-// indices come first in ascending order, as when GitHub's replacer builds a
372-// JavaScript object. Control characters, quotes and backslashes in strings
373-// are escaped; everything else, including non-ASCII text, is written as is.
374-// Lone surrogates, which FromJSON keeps in WTF-8 form, are written as \uXXXX.
362+// ToJSON renders v as the toJSON function does (Sdk/Functions/ToJson.cs): two
363+// space indentation, "key": value pairs in the dictionary's order, "[]" and
364+// "{}" for empty containers. A number is written as its string form
365+// (numberToCoercedString: at most 15 significant digits, so NaN and the
366+// infinities are written as such). Strings are escaped as Json.NET does by
367+// default: quotes, backslashes, \b \f \n \r \t, the other control characters
368+// as \u00xx, and U+0085, U+2028 and U+2029 as \uxxxx; everything else,
369+// including non-ASCII text, is written as is.
375370 //
376371 // ToJSON suits the finite values that FromJSON and Evaluate produce. A value
377372 // nested deeper than [MaxJSONDepth], which only a cyclic value built by hand
@@ -485,7 +480,7 @@ func (w *jsonWriter) value(v Value, depth int) {
485480 w.str("{}")
486481 return
487482 }
488- keys := jsKeyOrder(x.keys)
483+ keys := x.keys
489484 w.str("{\n")
490485 for i, k := range keys {
491486 val, _ := x.Get(k)
@@ -534,8 +529,7 @@ func (w *jsonWriter) jsonString(s string) {
534529 esc = `\t`
535530 case r < 0x20:
536531 esc = `\u00` + string([]byte{hexDigits[r>>4], hexDigits[r&0xF]})
537- case r >= 0xD800 && r < 0xE000:
538- // A lone surrogate (WTF-8, see FromJSON).
532+ case r == 0x85 || r == 0x2028 || r == 0x2029 || r >= 0xD800 && r < 0xE000:
539533 esc = `\u` + string([]byte{hexDigits[r>>12], hexDigits[r>>8&0xF], hexDigits[r>>4&0xF], hexDigits[r&0xF]})
540534 }
541535 if esc != "" {
@@ -6,9 +6,9 @@ import (
6 "errors"6 "errors"
7 "fmt"7 "fmt"
8 "math"8 "math"
9- "sort"
10 "strconv"9 "strconv"
11 "strings"10 "strings"
11+ "unicode"
12 "unicode/utf8"12 "unicode/utf8"
13 )13 )
14 14
@@ -21,30 +21,37 @@ const MaxJSONDepth = 1000
21 21
22 // FromJSON parses JSON text into a [Value], like the fromJSON function.22 // FromJSON parses JSON text into a [Value], like the fromJSON function.
23 //23 //
24-// The grammar is strict RFC 8259 JSON as JSON.parse accepts it. Numbers are24+// The reader is Json.NET's, as in the runner (Sdk/Functions/FromJson.cs:
25-// float64 (1e999 is +Inf). Objects become [*Dictionary] values: keys that25+// JToken.ReadFrom on a JsonTextReader, then ToPipelineContextData), so it
26-// are array indices ("0", "42") come first in ascending order, then the rest26+// accepts more than RFC 8259:
27-// in document order, as JavaScript enumerates object keys; a repeated key27+//
28-// keeps its first position and its last value; keys that differ only by case28+// - only the first value is read, anything after it is ignored;
29-// collapse to the first.29+// - strings may use single quotes and may contain raw control characters,
30+// object names may be unquoted (letters, digits, _ and $);
31+// - a trailing comma is accepted, an empty array element is an empty string;
32+// - NaN, Infinity and -Infinity are numbers, numbers may be written .5, 5.,
33+// 1.e2, 01 or 0x1F (hexadecimal, no sign), and undefined is an empty
34+// string;
35+// - nesting is limited to 64 levels (JsonReader.MaxDepth).
36+//
37+// Not ported (they fail here): comments and constructors (new Date(1)).
38+// Numbers are float64 (1e999 is +Inf). Objects become [*Dictionary] values in
39+// document order; a repeated key, compared case-insensitively, keeps the
40+// spelling and position of its first occurrence and the value of its last.
41+// An unpaired surrogate escape reads as U+FFFD.
30 func FromJSON(data []byte) (Value, error) {42 func FromJSON(data []byte) (Value, error) {
31 s := string(data)43 s := string(data)
32 if !utf8.ValidString(s) {44 if !utf8.ValidString(s) {
33- s = string([]rune(s)) // invalid bytes become U+FFFD, as in a JS string45+ s = string([]rune(s)) // invalid bytes become U+FFFD
34 }46 }
35 p := &jsonParser{s: s}47 p := &jsonParser{s: s}
36 p.skipSpace()48 p.skipSpace()
37- v, err := p.value(0)49+ return p.value(0)
38- if err != nil {
39- return nil, err
40- }
41- p.skipSpace()
42- if p.i < len(p.s) {
43- return nil, p.unexpected()
44- }
45- return v, nil
46 }50 }
47 51
52+// jsonReaderMaxDepth is Json.NET's JsonReader.MaxDepth default.
53+const jsonReaderMaxDepth = 64
54+
48 type jsonParser struct {55 type jsonParser struct {
49 s string56 s string
50 i int57 i int
@@ -55,102 +62,108 @@ func (p *jsonParser) unexpected() error {
55 return errors.New("unexpected end of JSON input")62 return errors.New("unexpected end of JSON input")
56 }63 }
57 r, _ := utf8.DecodeRuneInString(p.s[p.i:])64 r, _ := utf8.DecodeRuneInString(p.s[p.i:])
58- return fmt.Errorf("unexpected token %q in JSON at position %d", r, p.i)65+ return fmt.Errorf("unexpected character %q in JSON at position %d", r, p.i)
59 }66 }
60 67
61 func (p *jsonParser) skipSpace() {68 func (p *jsonParser) skipSpace() {
62 for p.i < len(p.s) {69 for p.i < len(p.s) {
63- switch p.s[p.i] {70+ r, w := utf8.DecodeRuneInString(p.s[p.i:])
64- case ' ', '\t', '\n', '\r':71+ if !isNetSpace(r) {
65- p.i++
66- default:
67 return72 return
68 }73 }
74+ p.i += w
69 }75 }
70 }76 }
71 77
72-func (p *jsonParser) literal(word string, v Value) (Value, error) {78+// isJSONSeparator reports whether c ends a literal or a number.
73- if !strings.HasPrefix(p.s[p.i:], word) {79+func isJSONSeparator(r rune) bool {
74- // Advance to the first mismatching byte for the error position.80+ switch r {
75- for j := 0; j < len(word) && p.i < len(p.s) && p.s[p.i] == word[j]; j++ {81+ case ',', '}', ']', ')', '/':
76- p.i++82+ return true
83+ }
84+ return isNetSpace(r)
85+}
86+
87+// token returns the run of characters up to the next separator.
88+func (p *jsonParser) token() string {
89+ start := p.i
90+ for p.i < len(p.s) {
91+ r, w := utf8.DecodeRuneInString(p.s[p.i:])
92+ if isJSONSeparator(r) {
93+ break
77 }94 }
78- return nil, p.unexpected()95+ p.i += w
79 }96 }
80- p.i += len(word)97+ return p.s[start:p.i]
81- return v, nil
82 }98 }
83 99
84 func (p *jsonParser) value(depth int) (Value, error) {100 func (p *jsonParser) value(depth int) (Value, error) {
85- if depth > MaxJSONDepth {
86- return nil, errors.New("maximum JSON nesting depth exceeded")
87- }
88 if p.i >= len(p.s) {101 if p.i >= len(p.s) {
89 return nil, p.unexpected()102 return nil, p.unexpected()
90 }103 }
91 switch c := p.s[p.i]; {104 switch c := p.s[p.i]; {
92 case c == '{':105 case c == '{':
93- return p.object(depth)106+ return p.object(depth + 1)
94 case c == '[':107 case c == '[':
95- return p.array(depth)108+ return p.array(depth + 1)
96- case c == '"':109+ case c == '"' || c == '\'':
97 s, err := p.str()110 s, err := p.str()
98 if err != nil {111 if err != nil {
99 return nil, err112 return nil, err
100 }113 }
101 return String(s), nil114 return String(s), nil
102- case c == 't':115+ case c == '-' || c == '.' || c >= '0' && c <= '9':
103- return p.literal("true", Boolean(true))
104- case c == 'f':
105- return p.literal("false", Boolean(false))
106- case c == 'n':
107- return p.literal("null", Null{})
108- case c == '-' || c >= '0' && c <= '9':
109 return p.number()116 return p.number()
110 }117 }
118+ start := p.i
119+ switch word := p.token(); word {
120+ case "true":
121+ return Boolean(true), nil
122+ case "false":
123+ return Boolean(false), nil
124+ case "null":
125+ return Null{}, nil
126+ case "undefined":
127+ return String(""), nil
128+ case "NaN":
129+ return Number(nan), nil
130+ case "Infinity":
131+ return Number(posInf), nil
132+ }
133+ p.i = start
111 return nil, p.unexpected()134 return nil, p.unexpected()
112 }135 }
113 136
137+// number reads a number token: decimal with optional sign, fraction and
138+// exponent (a digit is required in the mantissa), -Infinity, or an unsigned
139+// 0x hexadecimal integer.
114 func (p *jsonParser) number() (Value, error) {140 func (p *jsonParser) number() (Value, error) {
115 start := p.i141 start := p.i
116- if p.s[p.i] == '-' {142+ tok := p.token()
117- p.i++143+ if tok == "-Infinity" {
118- }144+ return Number(math.Inf(-1)), nil
119- digits := func() int {145+ }
120- n := 0146+ if len(tok) > 2 && tok[0] == '0' && (tok[1] == 'x' || tok[1] == 'X') {
121- for p.i < len(p.s) && p.s[p.i] >= '0' && p.s[p.i] <= '9' {147+ if len(tok) <= 18 {
122- p.i++148+ if n, err := strconv.ParseUint(tok[2:], 16, 64); err == nil && !strings.ContainsAny(tok[2:], "+-_") {
123- n++149+ return Number(float64(int64(n))), nil
150+ }
124 }151 }
125- return n152+ p.i = start
126- }153+ return nil, fmt.Errorf("input string %q is not a valid number", tok)
127- switch {
128- case p.i < len(p.s) && p.s[p.i] == '0':
129- p.i++
130- case digits() == 0:
131- return nil, p.unexpected()
132 }154 }
133- if p.i < len(p.s) && p.s[p.i] == '.' {155+ body := strings.TrimPrefix(tok, "-")
134- p.i++156+ if !isDecimalLiteral(body) {
135- if digits() == 0 {157+ p.i = start
136- return nil, p.unexpected()158+ return nil, fmt.Errorf("input string %q is not a valid number", tok)
137- }
138 }159 }
139- if p.i < len(p.s) && (p.s[p.i] == 'e' || p.s[p.i] == 'E') {160+ f, _ := strconv.ParseFloat(tok, 64) // range errors give ±Inf
140- p.i++
141- if p.i < len(p.s) && (p.s[p.i] == '+' || p.s[p.i] == '-') {
142- p.i++
143- }
144- if digits() == 0 {
145- return nil, p.unexpected()
146- }
147- }
148- f, _ := strconv.ParseFloat(p.s[start:p.i], 64) // range errors give ±Inf
149 return Number(f), nil161 return Number(f), nil
150 }162 }
151 163
152 // str reads a string literal; p.i is on the opening quote.164 // str reads a string literal; p.i is on the opening quote.
153 func (p *jsonParser) str() (string, error) {165 func (p *jsonParser) str() (string, error) {
166+ quote := p.s[p.i]
154 p.i++167 p.i++
155 var b strings.Builder168 var b strings.Builder
156 for {169 for {
@@ -158,13 +171,11 @@ func (p *jsonParser) str() (string, error) {
158 return "", errors.New("unterminated string in JSON")171 return "", errors.New("unterminated string in JSON")
159 }172 }
160 c := p.s[p.i]173 c := p.s[p.i]
161- switch {174+ if c == quote {
162- case c == '"':
163 p.i++175 p.i++
164 return b.String(), nil176 return b.String(), nil
165- case c < 0x20:177+ }
166- return "", p.unexpected()178+ if c != '\\' {
167- case c != '\\':
168 b.WriteByte(c)179 b.WriteByte(c)
169 p.i++180 p.i++
170 continue181 continue
@@ -176,7 +187,7 @@ func (p *jsonParser) str() (string, error) {
176 e := p.s[p.i]187 e := p.s[p.i]
177 p.i++188 p.i++
178 switch e {189 switch e {
179- case '"', '\\', '/':190+ case '"', '\'', '\\', '/':
180 b.WriteByte(e)191 b.WriteByte(e)
181 case 'b':192 case 'b':
182 b.WriteByte('\b')193 b.WriteByte('\b')
@@ -206,13 +217,13 @@ func (p *jsonParser) str() (string, error) {
206 p.i = save // not a pair; the second escape is read on its own217 p.i = save // not a pair; the second escape is read on its own
207 }218 }
208 }219 }
209- // A lone surrogate cannot be a Go rune: writeRune keeps it in220+ if r >= 0xD800 && r < 0xE000 {
210- // its three-byte WTF-8 form so that toJSON writes it back as221+ r = utf8.RuneError // an unpaired surrogate
211- // the same \uXXXX escape, like JSON.stringify.222+ }
212- writeRune(&b, r)223+ b.WriteRune(r)
213 default:224 default:
214- p.i--225+ p.i -= 2
215- return "", p.unexpected()226+ return "", fmt.Errorf("bad JSON escape sequence \\%c at position %d", e, p.i)
216 }227 }
217 }228 }
218 }229 }
@@ -243,16 +254,27 @@ func (p *jsonParser) hex4() (rune, error) {
243 }254 }
244 255
245 func (p *jsonParser) array(depth int) (Value, error) {256 func (p *jsonParser) array(depth int) (Value, error) {
257+ if depth > jsonReaderMaxDepth {
258+ return nil, fmt.Errorf("the reader's MaxDepth of %d has been exceeded", jsonReaderMaxDepth)
259+ }
246 p.i++260 p.i++
247 arr := &Array{}261 arr := &Array{}
248- p.skipSpace()
249- if p.i < len(p.s) && p.s[p.i] == ']' {
250- p.i++
251- return arr, nil
252- }
253 for {262 for {
254 p.skipSpace()263 p.skipSpace()
255- v, err := p.value(depth + 1)264+ if p.i >= len(p.s) {
265+ return nil, p.unexpected()
266+ }
267+ switch p.s[p.i] {
268+ case ']':
269+ p.i++
270+ return arr, nil
271+ case ',':
272+ // An empty element is "undefined", which reads as an empty string.
273+ p.i++
274+ arr.items = append(arr.items, String(""))
275+ continue
276+ }
277+ v, err := p.value(depth)
256 if err != nil {278 if err != nil {
257 return nil, err279 return nil, err
258 }280 }
@@ -265,29 +287,50 @@ func (p *jsonParser) array(depth int) (Value, error) {
265 case ',':287 case ',':
266 p.i++288 p.i++
267 case ']':289 case ']':
268- p.i++
269- return arr, nil
270 default:290 default:
271 return nil, p.unexpected()291 return nil, p.unexpected()
272 }292 }
273 }293 }
274 }294 }
275 295
296+// propertyName reads a quoted or unquoted object name.
297+func (p *jsonParser) propertyName() (string, error) {
298+ if p.s[p.i] == '"' || p.s[p.i] == '\'' {
299+ return p.str()
300+ }
301+ start := p.i
302+ for p.i < len(p.s) {
303+ r, w := utf8.DecodeRuneInString(p.s[p.i:])
304+ if r == ':' || isNetSpace(r) {
305+ break
306+ }
307+ if !unicode.IsLetter(r) && !unicode.IsDigit(r) && r != '_' && r != '$' {
308+ return "", p.unexpected()
309+ }
310+ p.i += w
311+ }
312+ if p.i == start {
313+ return "", p.unexpected()
314+ }
315+ return p.s[start:p.i], nil
316+}
317+
276 func (p *jsonParser) object(depth int) (Value, error) {318 func (p *jsonParser) object(depth int) (Value, error) {
277- p.i++319+ if depth > jsonReaderMaxDepth {
278- var keys []string320+ return nil, fmt.Errorf("the reader's MaxDepth of %d has been exceeded", jsonReaderMaxDepth)
279- vals := map[string]Value{}
280- p.skipSpace()
281- if p.i < len(p.s) && p.s[p.i] == '}' {
282- p.i++
283- return NewDictionary(), nil
284 }321 }
322+ p.i++
323+ d := NewDictionary()
285 for {324 for {
286 p.skipSpace()325 p.skipSpace()
287- if p.i >= len(p.s) || p.s[p.i] != '"' {326+ if p.i >= len(p.s) {
288 return nil, p.unexpected()327 return nil, p.unexpected()
289 }328 }
290- k, err := p.str()329+ if p.s[p.i] == '}' {
330+ p.i++
331+ return d, nil
332+ }
333+ k, err := p.propertyName()
291 if err != nil {334 if err != nil {
292 return nil, err335 return nil, err
293 }336 }
@@ -297,14 +340,11 @@ func (p *jsonParser) object(depth int) (Value, error) {
297 }340 }
298 p.i++341 p.i++
299 p.skipSpace()342 p.skipSpace()
300- v, err := p.value(depth + 1)343+ v, err := p.value(depth)
301 if err != nil {344 if err != nil {
302 return nil, err345 return nil, err
303 }346 }
304- if _, dup := vals[k]; !dup {347+ d.set(k, v)
305- keys = append(keys, k)
306- }
307- vals[k] = v // last value wins, at the first position
308 p.skipSpace()348 p.skipSpace()
309 if p.i >= len(p.s) {349 if p.i >= len(p.s) {
310 return nil, p.unexpected()350 return nil, p.unexpected()
@@ -312,66 +352,21 @@ func (p *jsonParser) object(depth int) (Value, error) {
312 switch p.s[p.i] {352 switch p.s[p.i] {
313 case ',':353 case ',':
314 p.i++354 p.i++
315- continue
316 case '}':355 case '}':
317- p.i++
318 default:356 default:
319 return nil, p.unexpected()357 return nil, p.unexpected()
320 }358 }
321- break
322- }
323- d := NewDictionary()
324- for _, k := range jsKeyOrder(keys) {
325- d.Add(k, vals[k])
326- }
327- return d, nil
328-}
329-
330-// isArrayIndex reports whether k is the canonical decimal form of an integer
331-// in [0, 2^32-2], the keys JavaScript enumerates before all others.
332-func isArrayIndex(k string) bool {
333- if k == "" || len(k) > 10 || (len(k) > 1 && k[0] == '0') {
334- return false
335- }
336- for i := 0; i < len(k); i++ {
337- if k[i] < '0' || k[i] > '9' {
338- return false
339- }
340- }
341- n, _ := strconv.ParseUint(k, 10, 64)
342- return n < math.MaxUint32
343-}
344-
345-// jsKeyOrder reorders keys the way Object.keys enumerates a JavaScript
346-// object: array indices ascending, then the others in their given order.
347-func jsKeyOrder(keys []string) []string {
348- var idx, rest []string
349- for _, k := range keys {
350- if isArrayIndex(k) {
351- idx = append(idx, k)
352- } else {
353- rest = append(rest, k)
354- }
355- }
356- if len(idx) == 0 {
357- return keys
358 }359 }
359- sort.Slice(idx, func(i, j int) bool {
360- a, _ := strconv.ParseUint(idx[i], 10, 64)
361- b, _ := strconv.ParseUint(idx[j], 10, 64)
362- return a < b
363- })
364- return append(idx, rest...)
365 }360 }
366 361
367-// ToJSON renders v as the toJSON function does: JSON.stringify(v, replacer,362+// ToJSON renders v as the toJSON function does (Sdk/Functions/ToJson.cs): two
368-// " "), that is two-space indentation, "key": value pairs, "[]" and "{}"363+// space indentation, "key": value pairs in the dictionary's order, "[]" and
369-// for empty containers, and numbers as JavaScript prints them (NaN and the364+// "{}" for empty containers. A number is written as its string form
370-// infinities as null, -0 as 0, 1e21 as 1e+21). Dictionary keys that are array365+// (numberToCoercedString: at most 15 significant digits, so NaN and the
371-// indices come first in ascending order, as when GitHub's replacer builds a366+// infinities are written as such). Strings are escaped as Json.NET does by
372-// JavaScript object. Control characters, quotes and backslashes in strings367+// default: quotes, backslashes, \b \f \n \r \t, the other control characters
373-// are escaped; everything else, including non-ASCII text, is written as is.368+// as \u00xx, and U+0085, U+2028 and U+2029 as \uxxxx; everything else,
374-// Lone surrogates, which FromJSON keeps in WTF-8 form, are written as \uXXXX.369+// including non-ASCII text, is written as is.
375 //370 //
376 // ToJSON suits the finite values that FromJSON and Evaluate produce. A value371 // ToJSON suits the finite values that FromJSON and Evaluate produce. A value
377 // nested deeper than [MaxJSONDepth], which only a cyclic value built by hand372 // nested deeper than [MaxJSONDepth], which only a cyclic value built by hand
@@ -485,7 +480,7 @@ func (w *jsonWriter) value(v Value, depth int) {
485 w.str("{}")480 w.str("{}")
486 return481 return
487 }482 }
488- keys := jsKeyOrder(x.keys)483+ keys := x.keys
489 w.str("{\n")484 w.str("{\n")
490 for i, k := range keys {485 for i, k := range keys {
491 val, _ := x.Get(k)486 val, _ := x.Get(k)
@@ -534,8 +529,7 @@ func (w *jsonWriter) jsonString(s string) {
534 esc = `\t`529 esc = `\t`
535 case r < 0x20:530 case r < 0x20:
536 esc = `\u00` + string([]byte{hexDigits[r>>4], hexDigits[r&0xF]})531 esc = `\u00` + string([]byte{hexDigits[r>>4], hexDigits[r&0xF]})
537- case r >= 0xD800 && r < 0xE000:532+ case r == 0x85 || r == 0x2028 || r == 0x2029 || r >= 0xD800 && r < 0xE000:
538- // A lone surrogate (WTF-8, see FromJSON).
539 esc = `\u` + string([]byte{hexDigits[r>>12], hexDigits[r>>8&0xF], hexDigits[r>>4&0xF], hexDigits[r&0xF]})533 esc = `\u` + string([]byte{hexDigits[r>>12], hexDigits[r>>8&0xF], hexDigits[r>>4&0xF], hexDigits[r&0xF]})
540 }534 }
541 if esc != "" {535 if esc != "" {
M expr/json_test.go +43 -21
@@ -8,6 +8,9 @@ import (
88 "testing"
99 )
1010
11+// The accepted and rejected inputs below were established by running
12+// fromJSON through the runner's own code (gen/runner): Json.NET's reader, not
13+// JSON.parse.
1114 func TestFromJSONValid(t *testing.T) {
1215 tests := []struct {
1316 in string
@@ -17,29 +20,41 @@ func TestFromJSONValid(t *testing.T) {
1720 {" true ", "true"},
1821 {"\t\r\n false \n", "false"},
1922 {"0", "0"},
20- {"-0", "0"},
2123 {"-12.5e1", "-125"},
2224 {"1E2", "100"},
2325 {"0.1e1", "1"},
2426 {`"a\u00e9\n"`, `"aé\n"`},
2527 {`"\ud83d\ude00"`, `"😀"`},
26- {`"\ud800"`, `"\ud800"`},
27- {`"\udc00x"`, `"\udc00x"`},
28- {`"\ud800\u0041"`, `"\ud800A"`},
2928 {`"\/"`, `"/"`},
3029 {`[]`, `[]`},
3130 {`{}`, `{}`},
3231 {`[ 1 , [ ] , { } ]`, "[\n 1,\n [],\n {}\n]"},
33- {`{"a":1,"A":2}`, "{\n \"a\": 2\n}"}, // see below: the last value of an exact duplicate wins, case variants collapse to the first
32+ // An unpaired surrogate escape becomes U+FFFD.
33+ {`"\ud800"`, "\"\ufffd\""},
34+ {`"\udc00x"`, "\"\ufffdx\""},
35+ {`"\ud800\u0041"`, "\"\ufffdA\""},
36+ // Repeated keys: the last value, at the first position and spelling;
37+ // keys compare case-insensitively.
38+ {`{"a":1,"A":2}`, "{\n \"a\": 2\n}"},
3439 {`{"a":1,"a":2}`, "{\n \"a\": 2\n}"},
35- {`{"b":1,"1":2,"a":3,"0":4}`, "{\n \"0\": 4,\n \"1\": 2,\n \"b\": 1,\n \"a\": 3\n}"},
40+ // Document order, array-index keys included.
41+ {`{"b":1,"1":2,"a":3,"0":4}`, "{\n \"b\": 1,\n \"1\": 2,\n \"a\": 3,\n \"0\": 4\n}"},
42+ // Leniency of Json.NET.
43+ {`{"a":1,}`, "{\n \"a\": 1\n}"},
44+ {`{a:1,$b:2,c_d:3}`, "{\n \"a\": 1,\n \"$b\": 2,\n \"c_d\": 3\n}"},
45+ {`{'a':'x\'y'}`, "{\n \"a\": \"x'y\"\n}"},
46+ {`[1,]`, "[\n 1\n]"},
47+ {`[,1]`, "[\n \"\",\n 1\n]"},
48+ {`[1,,2]`, "[\n 1,\n \"\",\n 2\n]"},
49+ {"01", "1"}, {"1.", "1"}, {".5", "0.5"}, {"-.5", "-0.5"}, {"1.e2", "100"},
50+ {"0x10", "16"}, {"0X1f", "31"},
51+ {"NaN", "NaN"}, {"Infinity", "Infinity"}, {"-Infinity", "-Infinity"},
52+ {"undefined", `""`},
53+ {"1 2", "1"}, {"[] []", "[]"}, {"{} x", "{}"}, {`"a""b"`, `"a"`}, {"true false", "true"},
54+ {"\u00a01", "1"}, {"1\u00a0", "1"}, {"\u00851", "1"},
55+ {"\"a\nb\"", `"a\nb"`}, {"\"\x01\"", `"\u0001"`},
3656 }
3757 for _, tc := range tests {
38- if tc.in == `{"a":1,"A":2}` {
39- // "a" and "A" are distinct JSON keys but the same dictionary key:
40- // the first one wins.
41- tc.want = "{\n \"a\": 1\n}"
42- }
4358 v, err := FromJSON([]byte(tc.in))
4459 if err != nil {
4560 t.Errorf("FromJSON(%q): %v", tc.in, err)
@@ -53,9 +68,9 @@ func TestFromJSONValid(t *testing.T) {
5368
5469 func TestFromJSONInvalid(t *testing.T) {
5570 for _, in := range []string{
56- "", " ", "{", "}", "[", "]", `{"a"`, `{"a":`, `{"a":1`, `{"a":1,}`, `{,}`, `{a:1}`, `{'a':1}`, `{"a" 1}`, `[1,]`, `[,1]`, `[1 2]`,
57- "01", "1.", ".5", "+1", "-", "1e", "1e+", "--1", "0x10", "NaN", "Infinity", "-Infinity", "undefined", "nul", "truee", "tru", "True",
58- `"abc`, `"a` + "\n" + `b"`, "\"\x01\"", `"\x"`, `"\u12"`, `"\u12g4"`, `"\`, "1 2", "[] []", "{} x", `"a""b"`, "\x00", "\u00a01", "1\u00a0", "\ufeff1",
71+ "", " ", "{", "}", "[", "]", `{"a"`, `{"a":`, `{"a":1`, `{,}`, `{"a" 1}`, `[1 2]`,
72+ "+1", "-", "1e", "1e+", "--1", "-0x10", "nul", "truee", "tru", "True", "TRUE", "NULL", "nan", "+Infinity", "1_2",
73+ `"abc`, `"\x"`, `"\u12"`, `"\u12g4"`, `"\`, "\x00", "\ufeff1", "@", "new Foo",
5974 } {
6075 if v, err := FromJSON([]byte(in)); err == nil {
6176 t.Errorf("FromJSON(%q) = %v, want error", in, ToJSON(v))
@@ -80,6 +95,9 @@ func TestFromJSONNumbers(t *testing.T) {
8095 if !math.Signbit(float64(v.(Number))) {
8196 t.Error("-0 lost its sign")
8297 }
98+ if got := ToJSON(v); got != "0" {
99+ t.Errorf("ToJSON(-0) = %q", got)
100+ }
83101 }
84102
85103 func TestFromJSONInvalidUTF8(t *testing.T) {
@@ -93,13 +111,17 @@ func TestFromJSONInvalidUTF8(t *testing.T) {
93111 }
94112
95113 func TestFromJSONDeepNesting(t *testing.T) {
96- deep := strings.Repeat("[", 900) + strings.Repeat("]", 900)
97- if _, err := FromJSON([]byte(deep)); err != nil {
98- t.Errorf("depth 900: %v", err)
114+ // Json.NET's reader allows 64 levels (JsonReader.MaxDepth).
115+ for _, n := range []int{1, 63, 64} {
116+ if _, err := FromJSON([]byte(strings.Repeat("[", n) + "1" + strings.Repeat("]", n))); err != nil {
117+ t.Errorf("depth %d: %v", n, err)
118+ }
99119 }
100- tooDeep := strings.Repeat("[", MaxJSONDepth+10) + strings.Repeat("]", MaxJSONDepth+10)
101- if _, err := FromJSON([]byte(tooDeep)); err == nil {
102- t.Error("absurd depth accepted")
120+ for _, open := range []string{"[", `{"a":`} {
121+ close := map[string]string{"[": "]", `{"a":`: "}"}[open]
122+ if _, err := FromJSON([]byte(strings.Repeat(open, 65) + "1" + strings.Repeat(close, 65))); err == nil {
123+ t.Errorf("depth 65 of %q accepted", open)
124+ }
103125 }
104126 }
105127
@@ -141,7 +163,7 @@ func TestToJSONFormatting(t *testing.T) {
141163 {Number(1e21), "1E+21"},
142164 {Number(1e-7), "1E-07"},
143165 {Number(123456789), "123456789"},
144- {String("\u0000\u001f\u007f\u0080\u2028"), `"\u0000\u001f` + "\u007f\u0080\u2028" + `"`},
166+ {String("\u0000\u001f\u007f\u0080\u2028"), `"\u0000\u001f` + "\u007f\u0080" + `\u2028"`},
145167 {String("\"\\\b\f\n\r\t"), `"\"\\\b\f\n\r\t"`},
146168 {String("</script>&'"), `"</script>&'"`},
147169 {String("\xff"), "\"\ufffd\""},
@@ -8,6 +8,9 @@ import (
8 "testing"8 "testing"
9 )9 )
10 10
11+// The accepted and rejected inputs below were established by running
12+// fromJSON through the runner's own code (gen/runner): Json.NET's reader, not
13+// JSON.parse.
11 func TestFromJSONValid(t *testing.T) {14 func TestFromJSONValid(t *testing.T) {
12 tests := []struct {15 tests := []struct {
13 in string16 in string
@@ -17,29 +20,41 @@ func TestFromJSONValid(t *testing.T) {
17 {" true ", "true"},20 {" true ", "true"},
18 {"\t\r\n false \n", "false"},21 {"\t\r\n false \n", "false"},
19 {"0", "0"},22 {"0", "0"},
20- {"-0", "0"},
21 {"-12.5e1", "-125"},23 {"-12.5e1", "-125"},
22 {"1E2", "100"},24 {"1E2", "100"},
23 {"0.1e1", "1"},25 {"0.1e1", "1"},
24 {`"a\u00e9\n"`, `"aé\n"`},26 {`"a\u00e9\n"`, `"aé\n"`},
25 {`"\ud83d\ude00"`, `"😀"`},27 {`"\ud83d\ude00"`, `"😀"`},
26- {`"\ud800"`, `"\ud800"`},
27- {`"\udc00x"`, `"\udc00x"`},
28- {`"\ud800\u0041"`, `"\ud800A"`},
29 {`"\/"`, `"/"`},28 {`"\/"`, `"/"`},
30 {`[]`, `[]`},29 {`[]`, `[]`},
31 {`{}`, `{}`},30 {`{}`, `{}`},
32 {`[ 1 , [ ] , { } ]`, "[\n 1,\n [],\n {}\n]"},31 {`[ 1 , [ ] , { } ]`, "[\n 1,\n [],\n {}\n]"},
33- {`{"a":1,"A":2}`, "{\n \"a\": 2\n}"}, // see below: the last value of an exact duplicate wins, case variants collapse to the first32+ // An unpaired surrogate escape becomes U+FFFD.
33+ {`"\ud800"`, "\"\ufffd\""},
34+ {`"\udc00x"`, "\"\ufffdx\""},
35+ {`"\ud800\u0041"`, "\"\ufffdA\""},
36+ // Repeated keys: the last value, at the first position and spelling;
37+ // keys compare case-insensitively.
38+ {`{"a":1,"A":2}`, "{\n \"a\": 2\n}"},
34 {`{"a":1,"a":2}`, "{\n \"a\": 2\n}"},39 {`{"a":1,"a":2}`, "{\n \"a\": 2\n}"},
35- {`{"b":1,"1":2,"a":3,"0":4}`, "{\n \"0\": 4,\n \"1\": 2,\n \"b\": 1,\n \"a\": 3\n}"},40+ // Document order, array-index keys included.
41+ {`{"b":1,"1":2,"a":3,"0":4}`, "{\n \"b\": 1,\n \"1\": 2,\n \"a\": 3,\n \"0\": 4\n}"},
42+ // Leniency of Json.NET.
43+ {`{"a":1,}`, "{\n \"a\": 1\n}"},
44+ {`{a:1,$b:2,c_d:3}`, "{\n \"a\": 1,\n \"$b\": 2,\n \"c_d\": 3\n}"},
45+ {`{'a':'x\'y'}`, "{\n \"a\": \"x'y\"\n}"},
46+ {`[1,]`, "[\n 1\n]"},
47+ {`[,1]`, "[\n \"\",\n 1\n]"},
48+ {`[1,,2]`, "[\n 1,\n \"\",\n 2\n]"},
49+ {"01", "1"}, {"1.", "1"}, {".5", "0.5"}, {"-.5", "-0.5"}, {"1.e2", "100"},
50+ {"0x10", "16"}, {"0X1f", "31"},
51+ {"NaN", "NaN"}, {"Infinity", "Infinity"}, {"-Infinity", "-Infinity"},
52+ {"undefined", `""`},
53+ {"1 2", "1"}, {"[] []", "[]"}, {"{} x", "{}"}, {`"a""b"`, `"a"`}, {"true false", "true"},
54+ {"\u00a01", "1"}, {"1\u00a0", "1"}, {"\u00851", "1"},
55+ {"\"a\nb\"", `"a\nb"`}, {"\"\x01\"", `"\u0001"`},
36 }56 }
37 for _, tc := range tests {57 for _, tc := range tests {
38- if tc.in == `{"a":1,"A":2}` {
39- // "a" and "A" are distinct JSON keys but the same dictionary key:
40- // the first one wins.
41- tc.want = "{\n \"a\": 1\n}"
42- }
43 v, err := FromJSON([]byte(tc.in))58 v, err := FromJSON([]byte(tc.in))
44 if err != nil {59 if err != nil {
45 t.Errorf("FromJSON(%q): %v", tc.in, err)60 t.Errorf("FromJSON(%q): %v", tc.in, err)
@@ -53,9 +68,9 @@ func TestFromJSONValid(t *testing.T) {
53 68
54 func TestFromJSONInvalid(t *testing.T) {69 func TestFromJSONInvalid(t *testing.T) {
55 for _, in := range []string{70 for _, in := range []string{
56- "", " ", "{", "}", "[", "]", `{"a"`, `{"a":`, `{"a":1`, `{"a":1,}`, `{,}`, `{a:1}`, `{'a':1}`, `{"a" 1}`, `[1,]`, `[,1]`, `[1 2]`,71+ "", " ", "{", "}", "[", "]", `{"a"`, `{"a":`, `{"a":1`, `{,}`, `{"a" 1}`, `[1 2]`,
57- "01", "1.", ".5", "+1", "-", "1e", "1e+", "--1", "0x10", "NaN", "Infinity", "-Infinity", "undefined", "nul", "truee", "tru", "True",72+ "+1", "-", "1e", "1e+", "--1", "-0x10", "nul", "truee", "tru", "True", "TRUE", "NULL", "nan", "+Infinity", "1_2",
58- `"abc`, `"a` + "\n" + `b"`, "\"\x01\"", `"\x"`, `"\u12"`, `"\u12g4"`, `"\`, "1 2", "[] []", "{} x", `"a""b"`, "\x00", "\u00a01", "1\u00a0", "\ufeff1",73+ `"abc`, `"\x"`, `"\u12"`, `"\u12g4"`, `"\`, "\x00", "\ufeff1", "@", "new Foo",
59 } {74 } {
60 if v, err := FromJSON([]byte(in)); err == nil {75 if v, err := FromJSON([]byte(in)); err == nil {
61 t.Errorf("FromJSON(%q) = %v, want error", in, ToJSON(v))76 t.Errorf("FromJSON(%q) = %v, want error", in, ToJSON(v))
@@ -80,6 +95,9 @@ func TestFromJSONNumbers(t *testing.T) {
80 if !math.Signbit(float64(v.(Number))) {95 if !math.Signbit(float64(v.(Number))) {
81 t.Error("-0 lost its sign")96 t.Error("-0 lost its sign")
82 }97 }
98+ if got := ToJSON(v); got != "0" {
99+ t.Errorf("ToJSON(-0) = %q", got)
100+ }
83 }101 }
84 102
85 func TestFromJSONInvalidUTF8(t *testing.T) {103 func TestFromJSONInvalidUTF8(t *testing.T) {
@@ -93,13 +111,17 @@ func TestFromJSONInvalidUTF8(t *testing.T) {
93 }111 }
94 112
95 func TestFromJSONDeepNesting(t *testing.T) {113 func TestFromJSONDeepNesting(t *testing.T) {
96- deep := strings.Repeat("[", 900) + strings.Repeat("]", 900)114+ // Json.NET's reader allows 64 levels (JsonReader.MaxDepth).
97- if _, err := FromJSON([]byte(deep)); err != nil {115+ for _, n := range []int{1, 63, 64} {
98- t.Errorf("depth 900: %v", err)116+ if _, err := FromJSON([]byte(strings.Repeat("[", n) + "1" + strings.Repeat("]", n))); err != nil {
117+ t.Errorf("depth %d: %v", n, err)
118+ }
99 }119 }
100- tooDeep := strings.Repeat("[", MaxJSONDepth+10) + strings.Repeat("]", MaxJSONDepth+10)120+ for _, open := range []string{"[", `{"a":`} {
101- if _, err := FromJSON([]byte(tooDeep)); err == nil {121+ close := map[string]string{"[": "]", `{"a":`: "}"}[open]
102- t.Error("absurd depth accepted")122+ if _, err := FromJSON([]byte(strings.Repeat(open, 65) + "1" + strings.Repeat(close, 65))); err == nil {
123+ t.Errorf("depth 65 of %q accepted", open)
124+ }
103 }125 }
104 }126 }
105 127
@@ -141,7 +163,7 @@ func TestToJSONFormatting(t *testing.T) {
141 {Number(1e21), "1E+21"},163 {Number(1e21), "1E+21"},
142 {Number(1e-7), "1E-07"},164 {Number(1e-7), "1E-07"},
143 {Number(123456789), "123456789"},165 {Number(123456789), "123456789"},
144- {String("\u0000\u001f\u007f\u0080\u2028"), `"\u0000\u001f` + "\u007f\u0080\u2028" + `"`},166+ {String("\u0000\u001f\u007f\u0080\u2028"), `"\u0000\u001f` + "\u007f\u0080" + `\u2028"`},
145 {String("\"\\\b\f\n\r\t"), `"\"\\\b\f\n\r\t"`},167 {String("\"\\\b\f\n\r\t"), `"\"\\\b\f\n\r\t"`},
146 {String("</script>&'"), `"</script>&'"`},168 {String("</script>&'"), `"</script>&'"`},
147 {String("\xff"), "\"\ufffd\""},169 {String("\xff"), "\"\ufffd\""},
M expr/limits_test.go +9 -11
@@ -422,17 +422,15 @@ func TestWTF8Helpers(t *testing.T) {
422422 }
423423
424424 func TestLoneSurrogateExpressions(t *testing.T) {
425- // The cases the review asked for; the oracle covers many more.
425+ // Json.NET turns an unpaired surrogate escape into U+FFFD, so the
426+ // runner has no lone surrogates (JavaScript strings do: see PORTING.md).
426427 runEvalCases(t, []evalCase{
427- {`fromJSON('"\ud800"') == fromJSON('"\udc00"')`, "Boolean:false"},
428- {`fromJSON('"\ud800"') == fromJSON('"\ud800"')`, "Boolean:true"},
429- {"fromJSON('\"\\ud800\"') == '\ufffd'", "Boolean:false"},
430- {`fromJSON('"\ud800"') < fromJSON('"\udc00"')`, "Boolean:true"},
431- {"fromJSON('\"\\ud800\"') < '\ue000'", "Boolean:true"},
432- {"fromJSON('\"\\ud800\"') < '\U00010000'", "Boolean:true"},
433- {`contains(fromJSON('"a\ud800"'), fromJSON('"\ud800"'))`, "Boolean:true"},
434- {`contains(fromJSON('"\ud83d\ude00"'), fromJSON('"\ud83d"'))`, "Boolean:true"},
435- {`format('{0}{1}', fromJSON('"\ud83d"'), fromJSON('"\ude00"')) == '😀'`, "Boolean:true"},
436- {`toJSON(fromJSON('"\ud800"'))`, `String:"\ud800"`},
428+ {`fromJSON('"\ud800"') == fromJSON('"\udc00"')`, "Boolean:true"},
429+ {"fromJSON('\"\\ud800\"') == '\ufffd'", "Boolean:true"},
430+ {`fromJSON('"\ud800"') < fromJSON('"\udc00"')`, "Boolean:false"},
431+ {`contains(fromJSON('"\ud83d\ude00"'), fromJSON('"\ud83d"'))`, "Boolean:false"},
432+ {`fromJSON('"\ud83d\ude00"') == '😀'`, "Boolean:true"},
433+ {`format('{0}{1}', fromJSON('"\ud83d"'), fromJSON('"\ude00"')) == '😀'`, "Boolean:false"},
434+ {`toJSON(fromJSON('"\ud800"'))`, "String:\"\ufffd\""},
437435 })
438436 }
@@ -422,17 +422,15 @@ func TestWTF8Helpers(t *testing.T) {
422 }422 }
423 423
424 func TestLoneSurrogateExpressions(t *testing.T) {424 func TestLoneSurrogateExpressions(t *testing.T) {
425- // The cases the review asked for; the oracle covers many more.425+ // Json.NET turns an unpaired surrogate escape into U+FFFD, so the
426+ // runner has no lone surrogates (JavaScript strings do: see PORTING.md).
426 runEvalCases(t, []evalCase{427 runEvalCases(t, []evalCase{
427- {`fromJSON('"\ud800"') == fromJSON('"\udc00"')`, "Boolean:false"},428+ {`fromJSON('"\ud800"') == fromJSON('"\udc00"')`, "Boolean:true"},
428- {`fromJSON('"\ud800"') == fromJSON('"\ud800"')`, "Boolean:true"},429+ {"fromJSON('\"\\ud800\"') == '\ufffd'", "Boolean:true"},
429- {"fromJSON('\"\\ud800\"') == '\ufffd'", "Boolean:false"},430+ {`fromJSON('"\ud800"') < fromJSON('"\udc00"')`, "Boolean:false"},
430- {`fromJSON('"\ud800"') < fromJSON('"\udc00"')`, "Boolean:true"},431+ {`contains(fromJSON('"\ud83d\ude00"'), fromJSON('"\ud83d"'))`, "Boolean:false"},
431- {"fromJSON('\"\\ud800\"') < '\ue000'", "Boolean:true"},432+ {`fromJSON('"\ud83d\ude00"') == '😀'`, "Boolean:true"},
432- {"fromJSON('\"\\ud800\"') < '\U00010000'", "Boolean:true"},433+ {`format('{0}{1}', fromJSON('"\ud83d"'), fromJSON('"\ude00"')) == '😀'`, "Boolean:false"},
433- {`contains(fromJSON('"a\ud800"'), fromJSON('"\ud800"'))`, "Boolean:true"},434+ {`toJSON(fromJSON('"\ud800"'))`, "String:\"\ufffd\""},
434- {`contains(fromJSON('"\ud83d\ude00"'), fromJSON('"\ud83d"'))`, "Boolean:true"},
435- {`format('{0}{1}', fromJSON('"\ud83d"'), fromJSON('"\ude00"')) == '😀'`, "Boolean:true"},
436- {`toJSON(fromJSON('"\ud800"'))`, `String:"\ud800"`},
437 })435 })
438 }436 }
A expr/testdata/tojson/object_case_dup.runner +4 -0
@@ -0,0 +1,4 @@
1+{
2+ "a": 2,
3+ "b": 4
4+}⏎̸
@@ -0,0 +1,4 @@
1+{
2+ "a": 2,
3+ "b": 4
4+}⏎̸
A expr/testdata/tojson/object_numeric_keys.runner +11 -0
@@ -0,0 +1,11 @@
1+{
2+ "b": 1,
3+ "10": 2,
4+ "2": 3,
5+ "a": 4,
6+ "01": 5,
7+ "1": 6,
8+ "4294967294": 7,
9+ "4294967295": 8,
10+ "-1": 9
11+}⏎̸
@@ -0,0 +1,11 @@
1+{
2+ "b": 1,
3+ "10": 2,
4+ "2": 3,
5+ "a": 4,
6+ "01": 5,
7+ "1": 6,
8+ "4294967294": 7,
9+ "4294967295": 8,
10+ "-1": 9
11+}⏎̸
A expr/testdata/tojson/string_lone_surrogate.runner +1 -0
@@ -0,0 +1,1 @@
1+"�"⏎̸
@@ -0,0 +1,1 @@
1+"�"⏎̸
A expr/testdata/tojson/string_unicode.runner +1 -0
@@ -0,0 +1,1 @@
1+"héllo 世界 😀 \u2028\u2029"⏎̸
@@ -0,0 +1,1 @@
1+"héllo 世界 😀 \u2028\u2029"⏎̸

Keyboard shortcuts

Single-key shortcuts are on. ctrl k and esc work whatever this says.

ctrl k :
go to a section, branch, tag, file or repository
?
this list
j k
move the row cursor down / up
enter
open the row under the cursor; on Files, a directory opens or closes in place
↑ ↓ ← → home end
in the file tree (focus on a row): previous / next row, close / open a directory or go to its parent / first entry, first / last row. They never open a file
l h
on Files: open a directory or a file / close a directory or go up to its parent, then to the parent page. Elsewhere: into the side panel (on a merge request, its files) / back to the content; there j k move between its controls
g g G
first / last row
/
filter the list; on Files, find a file anywhere in the repository (esc clears, enter opens the best match)
b
switch branch or tag
g s g f g c g b g m g i g r g a
tabs: summary, files, commits, branches & tags, merge requests, issues, releases, actions (when available)
g h
home
w
wide mode: use the whole width of the screen, or not (remembered)
y
copy a permalink pinned to the commit
d u x
diff: split / unified / collapse or expand all
e E
diff: the whole file in view / every file, with the lines the hunks hide
n
merge request: next note in the diff
esc
close a dialog, leave a field

Branches 1

Tags 3

↑↓ moveenter openesc close