Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .claude/agents/usertalk-engineer.md
Original file line number Diff line number Diff line change
Expand Up @@ -287,7 +287,15 @@ else {
- **Why**: The parser expects consistent indentation even for blank lines
- **Best Practice**: UserTalk code is typically compact without blank lines for visual separation

3. **Test Data Isolation - Use system.temp, NEVER system table**
3. **Inline Closing Braces (Canonical Style)**
- ✅ CORRECT: `try { ... ; doThing ()};` — `};` inlined at end of last statement
- ✅ CORRECT: `if cond { yes ()}` newline `else { no ()};` — `}` inlined before else
- ✅ CORRECT: `on f (x) { return (x * 2)}` — function body `}` inlined
- ❌ NON-CANONICAL: closing `}` on its own line, lone `};` on its own line
- **Why**: The textifier (outline → text round-trip) inlines `}` and `};` at the end of the last statement of a block. Code written in C-style multi-line layout is not what production source looks like and may get reformatted during round-trip.
- **Where it matters**: writing/editing `.ut` files, ODB scripts via the protocol. yaml integration test `script:` blocks tolerate C-style layout because the protocol normalizes newlines (#624, #628, #635), but production-style is still preferred.

4. **Test Data Isolation - Use system.temp, NEVER system table**
- ❌ WRONG: `system.verbs.tcp.test.foo = "bar"` (modifies system table!)
- ✅ CORRECT: `new(tableType, @system.temp.tcpTest); system.temp.tcpTest.foo = "bar"`
- **Cleanup**: Always `delete(@system.temp.tcpTest)` at end of test
Expand Down
21 changes: 20 additions & 1 deletion docs/usertalk/CLAUDE_PRIMER.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ This primer's job: stop you from probing-and-experimenting for basic idioms duri
| `for x in collection` always yields values | Two forms: `for v in listOrRecordValue` yields **values** (works on records and lists). `for adrM in @table` yields **addresses** (works on tables; pass the address, not the value). Dereference with `adrM^`. |
| `/* comment */` works | **Not a UserTalk comment.** It silently parses as a division-multiplication expression and can corrupt your script's return value. Use `//` or `«…»`. |
| Verb calls: `f(x)` | UserLand style: **space before `(`**: `f (x)`. Both compile; only the spaced form is idiomatic. |
| Closing brace on its own line | Idiomatically **inline at the end of the last statement**: `... ; return x}`. This matches outline ↔ text round-trip. |
| Closing brace on its own line | Canonical UT **inlines `}` and `};` at the end of the last statement of a block** — never on its own line. `try { ... ; doThing ()};` not `try {\n ...\n};`. See section 2.6. |
| Method-style verb evolution | UserTalk is dispatch-by-name through the ODB. The "verb" `string.upper` is a script at `system.verbs.builtins.string.upper` — a glue script that calls a C kernel implementation. |

If something feels surprising, check this table before assuming. The rest of the primer expands each row.
Expand Down Expand Up @@ -101,6 +101,25 @@ For tables: pass the address (`@examples.colors`), not the value (`examples.colo

`nameOf (adrMember^)` gets the member name; `nameOf (adrMember)` gets the local variable's name (`"adrMember"`), which is rarely what you want.

### 2.6 Brace and `};` placement (canonical style)

UserTalk's outline ↔ text round-trip puts the closing `}` (and `};` when a statement separator is needed) **inline at the end of the last statement of a block**, not on its own line:

```
try { //comment OK here
scheduler.shutdown ()}; // '};' inlined — '}' closes try, ';' terminates statement
if cond {
yes ()} // '}' inline; no ';' because 'else' follows
else {
no ()}; // '};' inline at end of else
on f (x) {
return (x * 2)} // function body closes inline
```

You almost never see `\n}` or `\n};` or a lone `;` on a line by itself in textified UT. If you do, you're looking at hand-written C-style source, not canonical output from the textifier.

This matters when writing UserTalk for production (`.ut` files, ODB scripts) — match the inline convention. In yaml integration test `script:` blocks, C-style multi-line is OK because the protocol normalizes newlines (#624 / #628 / #635), but inline-canonical style is still preferred for parity.

---

## 3. Failure-mode decoder — when you see this error
Expand Down
99 changes: 82 additions & 17 deletions frontier-cli/repl_variables.c
Original file line number Diff line number Diff line change
Expand Up @@ -300,18 +300,24 @@ hdlhashtable repl_get_variables_table(void) {
* statement's value.
*
* Rules:
* - Insert ';' before every \r unless the preceding non-whitespace byte
* is already ';' or '{' (existing sep, or block-open like `if x {\n`).
* A trailing ';' before '}' is harmless: UserTalk accepts `stmt; }` inside
* a bracketedstatementlist, so no '}' guard is needed here.
* - Same guard applies to every standalone \n (\n not part of \r\n).
* - Insert ';' before every \r/\n unless the preceding non-whitespace
* byte is already ';' or '{' or '\0' (existing sep, block-open like
* `if x {\n`, or start-of-script).
* - '}' guard with lookahead: when prev is '}', peek past whitespace
* and comments for the next real token. If it's 'else'/';'/'}'/EOF,
* suppress the ';' — those tokens either attach to the '}' (else),
* close further (}}, EOF), or already provide the separator (;).
* Other followers (return, local, identifier, ...) are new statements
* and need an explicit ';'. Without this, `try { }\nelse { }` is split
* into a standalone try and an orphan else, producing a parse error
* that the with-wrapper masks as a truthy default. Issue #635.
* - Never double-insert ';' (';' guard prevents ;;).
* - \r\n pairs are treated as a single separator (\n is consumed after \r).
*
* The result is a freshly-malloc'd C string that the caller owns.
* Returns NULL on allocation failure.
*
* Issue #624.
* Issues #624, #635.
*/
static char *normalize_newlines_to_semicolons(const char *script) {
if (script == NULL)
Expand All @@ -330,20 +336,79 @@ static char *normalize_newlines_to_semicolons(const char *script) {
while (*src != '\0') {
unsigned char c = (unsigned char)*src;

if (c == '\r') {
if (prev_nonws != ';' && prev_nonws != '{' && prev_nonws != '\0')
if (c == '\r' || c == '\n') {
/* Decide whether this newline needs a ';' inserted before it.
*
* UserTalk grammar requires ';' between statements at top level.
* Bare \r/\n are whitespace to the scanner, so multi-line scripts
* without explicit ';' fail to parse. We insert one synthetically.
*
* Guards (no ';' inserted):
* - prev is ';' — already separated
* - prev is '{' — start of block, next is first statement
* - prev is '}' AND next non-whitespace is 'else'/';'/'}' — the
* '}' closes a block; inserting ';' before 'else' would split
* it from its if/try clause. Other followers (return, local,
* identifier, ...) ARE statements and need an explicit ';'.
* - prev is '\0' — start of script
*/
boolean need_sep = (prev_nonws != ';' && prev_nonws != '{' && prev_nonws != '\0');
if (need_sep && prev_nonws == '}') {
/* Peek past inter-token noise — whitespace AND comments — to
* find the next real token after the '}'. A '//' or '«...»'
* comment between '}' and 'else' on its own line must not hide
* the 'else' from the lookahead (#635 P1).
*
* UTF-8: '«' is 0xC2 0xAB. We match the leading byte 0xC2
* followed by 0xAB; both bytes are well-formed UTF-8 and any
* non-comment 0xC2 byte (continuation of a different char) is
* followed by a non-0xAB byte, so the check is unambiguous. */
const char *peek = src;
for (;;) {
while (*peek == ' ' || *peek == '\t' || *peek == '\r' || *peek == '\n')
peek++;
if (peek[0] == '/' && peek[1] == '/') {
while (*peek != '\0' && *peek != '\r' && *peek != '\n')
peek++;
/* Stop AT the \r/\n. The outer loop's whitespace
* pass on the next iteration will skip it. */
continue;
}
if ((unsigned char)peek[0] == 0xC2 && (unsigned char)peek[1] == 0xAB) {
peek += 2;
/* Walk to the closing '»' (0xC2 0xBB). End-of-string
* exit is safe — the malformed source will fail to
* compile after normalization, but we stop scanning. */
while (*peek != '\0' &&
!((unsigned char)peek[0] == 0xC2 && (unsigned char)peek[1] == 0xBB))
peek++;
if (*peek != '\0')
peek += 2; /* skip the closing '»' */
continue;
}
break;
}
/* Tokens that attach to or close the '}' without needing ';'. */
if (*peek == '\0' || *peek == '}' || *peek == ';' ||
(peek[0] == 'e' && peek[1] == 'l' && peek[2] == 's' && peek[3] == 'e' &&
(peek[4] == '\0' || peek[4] == ' ' || peek[4] == '\t' ||
peek[4] == '\r' || peek[4] == '\n' || peek[4] == '{'))) {
need_sep = false;
}
}
if (need_sep)
*dst++ = ';';
*dst++ = '\r';
*dst++ = (char)c;
src++;
/* '\0' != '\n', so end-of-string input never advances past the terminator. */
if (*src == '\n')
/* CRLF: silently consume the \n after an emitted \r so we don't
* produce two visible separators. Matches the pre-#628 contract. */
if (c == '\r' && *src == '\n')
src++;
prev_nonws = ';';
} else if (c == '\n') {
if (prev_nonws != ';' && prev_nonws != '{' && prev_nonws != '\0')
*dst++ = ';';
*dst++ = '\n';
src++;
/* prev_nonws is now ';' regardless of whether we emitted one.
* The newline acts as a statement-boundary signal even when the
* '}' guard suppressed the explicit ';' — subsequent guards key
* off "we just finished a statement", which is the truth either
* way. Don't try to mirror the emitted-byte history here. */
prev_nonws = ';';
} else {
*dst++ = (char)c;
Expand Down
84 changes: 84 additions & 0 deletions tests/integration/test_cases/protocol_eval_compile_errors.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -133,3 +133,87 @@ tests:
result:
value: "1"
type: "long"

- name: "protocol script/eval - try/else across newlines does not get split by ';' (#624)"
description: "When a try { } is followed by an else { }, the closing '}' of the try is followed by a newline before 'else'. Without a '}' in the guard, normalize_newlines_to_semicolons inserts a ';' between '}' and 'else', which splits the try from its else clause. The parser sees a standalone try (executes silently) followed by an orphan 'else' keyword, producing a parse error that the with-wrapper silently masks as 'true'. The fix is to treat '}' the same as ';' and '{' — newlines that follow it must not gain a ';'."
protocol_ops:
- op: "script/eval"
params:
expression: "local (result = \"ok\");\ntry {\n local (x = 1 + 1)}\nelse {\n result = \"error\"};\nreturn result"
validate:
success: true
result:
value: "ok"
type: "string"

- name: "protocol script/eval - try/else recovers from real error (#624)"
description: "Companion to the previous test: when the try body DOES throw, the else block runs and result becomes 'caught'. Verifies the }-before-else fix doesn't break the error-recovery path."
protocol_ops:
- op: "script/eval"
params:
expression: "local (result = \"ok\");\ntry {\n local (x = 1 / 0)}\nelse {\n result = \"caught\"};\nreturn result"
validate:
success: true
result:
value: "caught"
type: "string"

- name: "protocol script/eval - if/else across newlines (#624)"
description: "Same shape as try/else: an if's closing '}' followed by newline then 'else' must not gain a ';'. Locks in the same fix for if/else."
protocol_ops:
- op: "script/eval"
params:
expression: "local (result = \"none\");\nif false {\n result = \"yes\"}\nelse {\n result = \"no\"};\nreturn result"
validate:
success: true
result:
value: "no"
type: "string"

- name: "protocol script/eval - comment on own line between } and else (#635)"
description: "A '//' line comment between the closing '}' of a try/if body and the 'else' on a separate line must not break the composite statement. The peek-past-noise loop needs to skip line comments to find 'else'. Without this, the normalizer sees '/' instead of 'else' and inserts ';' before the comment line — splitting try from else (same failure mode as the original #635 P0)."
protocol_ops:
- op: "script/eval"
params:
expression: "local (result = \"ok\");\ntry {\n local (x = 1 / 0)}\n// recover gracefully\nelse {\n result = \"caught\"};\nreturn result"
validate:
success: true
result:
value: "caught"
type: "string"

- name: "protocol script/eval - canonical inline '};' between try and else (#635)"
description: "Canonical UT source inlines '};' at end of last statement of a block. 'try { body };\nelse { ... }' is the production-shape case (vs. the C-style multi-line cases in the previous tests). Verifies the normalizer handles the inline form correctly."
protocol_ops:
- op: "script/eval"
params:
expression: "local (result = \"ok\"); try { local (x = 1 / 0)}\nelse { result = \"caught\"}; return result"
validate:
success: true
result:
value: "caught"
type: "string"

- name: "protocol script/eval - nested try/else across newlines (#635)"
description: "Two levels of nested try/else, each spanning a newline. Locks in correct lookahead behaviour through stacked '}}' transitions. Inner try catches, outer try sees no error and passes through."
protocol_ops:
- op: "script/eval"
params:
expression: "local (result = \"start\");\ntry {\n try {\n local (x = 1 / 0)}\n else {\n result = \"inner\"}}\nelse {\n result = \"outer\"};\nreturn result"
validate:
success: true
result:
value: "inner"
type: "string"

- name: "protocol script/eval - case/else across newlines (#635)"
description: "case statements use the same caseheader casebody elsetoken bracketedstatementlist grammar as try/else and if/else. The '}\\nelse' boundary should behave identically. Locks in the third UserTalk construct that exercises the }-else lookahead so future refactors can't silently regress it."
protocol_ops:
- op: "script/eval"
params:
expression: "local (n = 2);\nlocal (result = \"\");\ncase n {\n 1 { result = \"one\"};\n 2 { result = \"two\"}}\nelse {\n result = \"other\"};\nreturn result"
validate:
success: true
result:
value: "two"
type: "string"