Skip to content

Heuristic JCL/COBOL detection and syntax highlighting in the raw record view#28

Merged
Tannex merged 2 commits into
mainfrom
issue-21-syntax-highlighting
Jul 19, 2026
Merged

Heuristic JCL/COBOL detection and syntax highlighting in the raw record view#28
Tannex merged 2 commits into
mainfrom
issue-21-syntax-highlighting

Conversation

@Tannex

@Tannex Tannex commented Jul 19, 2026

Copy link
Copy Markdown
Owner

Implements #21. Stacked on PR #27 (issue-19-table-polish) — merge that first; this PR's own diff is commit d7da6a2.

What

  • Detection (internal/cqt/syntax.go): classifies a data set/member's decoded records as JCL, COBOL, or plain.
    • JCL: ≥60% of sampled non-blank lines start with // or /* (min 2).
    • COBOL: scored signals — division headers (strong), PIC/PICTURE clauses, column-7 comment indicators, and a numeric sequence area on most lines (supporting signal only, so flat data files with leading record ids stay plain).
    • Anything below the bar renders exactly as before.
  • Caching: the verdict is computed once per loaded page — cached on the workspace keyed by record-cache size, reset alongside rawLongest on refresh/reset. Never per keystroke or per render frame; content type can't change mid-file, so a 120-line prefix sample carries the verdict.
  • Highlighting (raw record view only): pure per-line tokenizers using the existing consolePalette, matching the table-polish tones:
    • JCL: comments/delimiters muted, //NAME cyan, operations (JOB/EXEC/DD/…) amber, KEYWORD= parameter keys and quoted literals green, in-stream data plain.
    • COBOL: sequence area muted, column-7 comment lines muted wholesale, keywords/verbs cyan, PIC + picture strings amber, quoted literals green.
  • Highlighted lines strip back to their exact input (ansi.Strip(highlight(l)) == l), so the viewport's ANSI-aware ansi.Cut keeps horizontal panning and narrow terminals safe. The cursor row stays untinted so the selected-row style renders uniformly (same rule as the table cell tints).
  • Demo mode: DEMO.JCLLIB members serve a realistic JCL job and DEMO.SOURCE.COBOL members a COBOL program, exercising every token class offline (and flowing through the edit-mode text path too).

Screenshots (demo mode, via scripts/tui-shot)

JCL (120×40) COBOL (120×40) JCL narrow (70×24)
jcl cobol narrow

Decisions

  • Highlighting applies to the raw record view only. The editor (bubbles textarea) renders its own content, so reusing the highlighter there is not trivial — noted as a possible follow-up.
  • Keyword sets are deliberately partial (common operations/verbs/clauses); a full reserved-word list would add noise, not signal.

Testing

go build ./... && go vet ./... && go test ./... all pass, gofmt clean. New tests cover detection (JCL, COBOL, flat-data and mostly-plain negatives), both highlighters' token classes with content round-trip guarantees, once-per-page cache behavior (content swap without count change is not re-detected; a new page is), raw-view integration with an untinted cursor row, and unchanged plain rendering.

Closes #21

🤖 Generated with Claude Code

@Tannex
Tannex force-pushed the issue-19-table-polish branch from f145dcb to 801c9d3 Compare July 19, 2026 10:06
@Tannex
Tannex force-pushed the issue-21-syntax-highlighting branch from d7da6a2 to 162669c Compare July 19, 2026 10:06
@Tannex
Tannex force-pushed the issue-19-table-polish branch from 801c9d3 to f9dacca Compare July 19, 2026 10:10
@Tannex
Tannex force-pushed the issue-21-syntax-highlighting branch from 162669c to cd5cd88 Compare July 19, 2026 10:10
@Tannex Tannex linked an issue Jul 19, 2026 that may be closed by this pull request
@Tannex
Tannex force-pushed the issue-19-table-polish branch from f9dacca to 7c57bb0 Compare July 19, 2026 11:31
@Tannex
Tannex force-pushed the issue-21-syntax-highlighting branch from cd5cd88 to a757197 Compare July 19, 2026 11:32

@Tannex Tannex left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit heavy on in-line comments, could indicate the code needs refactoring.

Could use better support for COBOL keywords, but we can do that another time.

LGTM;

@Tannex
Tannex force-pushed the issue-19-table-polish branch from 7c57bb0 to 1981841 Compare July 19, 2026 13:34
Tannex and others added 2 commits July 19, 2026 15:34
The raw record view now classifies decoded records once per loaded page
(never per keystroke or render frame) as JCL, COBOL, or plain via cheap
heuristics: JCL by the //-statement ratio, COBOL by scored signals
(division headers, PIC clauses, column-7 comments, sequence areas) so
flat data files with leading numeric ids stay plain.

Detected sources get subtle consolePalette tints: JCL statement names,
operations, keyword parameters, and comments; COBOL sequence areas,
comment lines, keywords, PIC clauses, and literals. Highlighted lines
strip back to their exact input, so the viewport's ANSI-aware cut keeps
horizontal panning and narrow terminals safe, and the cursor row stays
untinted so the selected style renders uniformly.

Demo mode serves a real JCL job from DEMO.JCLLIB and a COBOL program
from DEMO.SOURCE.COBOL so every token class is visible offline.

Closes #21

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extract named helpers (isJCLPrefixed, containsCOBOLDivision,
containsPICClause, blankOrDigits, takeQuoted, splitAtSpace, wordEnd) and
COBOL column constants so the scanning code reads without per-line
comments. No behavior change; existing syntax tests pass unmodified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Tannex
Tannex force-pushed the issue-21-syntax-highlighting branch from 9e94ea4 to ba96a8a Compare July 19, 2026 13:34
Base automatically changed from issue-19-table-polish to main July 19, 2026 13:56
@Tannex
Tannex merged commit 28f9adb into main Jul 19, 2026
1 check passed
@Tannex
Tannex deleted the issue-21-syntax-highlighting branch July 19, 2026 13:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Heuristic detection of JCL / COBOL source and syntax/keyword highlighting

1 participant