Fix BASIC round-trip corruption and scripting-subsystem bugs#4
Merged
Conversation
Tokenizer/detokenizer (every BASIC save/load passes through these): - modtok: DATA statements are now copied literally until ':' (outside quotes) or end of line, matching real CRUNCH. Previously keywords inside DATA items were tokenized — "DATA MONDAY" LISTed correctly but READ returned garbage token bytes. Also '?' now tokenizes to PRINT ($99) as in stock BASIC instead of becoming a runtime SYNTAX ERROR. - moddet: quote-mode tracking added (same rule as ROM LIST — '"' toggles, reset per line). String contents are raw PETSCII where shifted letters/graphics are >= $80; without the flag, PRINT"<shift-A>" detokenized as PRINT"ATN"... and corrupted the text. - Both modules (and modsct) now carry a link-time .assert that code+data end below the staging buffer — the modtok change pushed code past the old $C400 staging start, which the assert caught immediately; staging moved to $C480. Script tokenizer (modsct): - The REU parameter block and include table were fixed addresses that the assembled binary had grown past — the first DMA overwrote the tail of the keyword table. They are now linker-reserved inside the image (plus the same staging .assert), so overlap is impossible. - INCLUDE now stops tokenization with an INCLUDE UNSUPPORTED status instead of feeding the broken include pipeline (which loaded files over the running editor's code and never spliced the pool into the deployed script). Refusing at tokenize time fully disables the feature; the script-runner manual documents it as reserved. - Mirrors the DATA-literal and '?' fixes. Script runner (modscr) and handler (modscrh): - modules.asm stashed the module load address at $38/$39 — BASIC's MEMSIZ hi byte and CURLIN lo byte — so the script handoff's CLR copied a garbage string-heap top into FRETOP and the first string operation failed. The stash now lives in the $0228/$0229 param area, and modscr additionally establishes MEMSIZ=$A000 before the handoff CLR. - verify_script sanity-clamps the metadata length to $0002-$97FF before it drives a DMA over $0801+ (stale REU contents can pass the 2-byte magic check; REU length 0 means a 64 KB transfer). deploy_script also guarantees the $00 at $0800 that the TXTPTR=TXTTAB-1 convention needs. - ONERR resume now uses the GOTO convention (TXTPTR = line_start - 1, jmp NEWSTT). NEWSTT does not CHRGET; the old "+3" convention resumed 4 bytes inside the statement for target lines < 256 and looped SYNTAX ERROR for the rest. The resume also resets the stack to BASIC's $FA baseline like the stock error handler, fixing the stack leak on every caught error. - The IGONE wedge's pass-through re-derives CHRGET's flags via CHRGOT instead of a bare SEC: the range tests destroyed Z, so legal empty statements (trailing ':', '::') dispatched into the LET path and raised spurious SYNTAX ERRORs. - hnd_read_str_arg is bounded to hnd_cmd_buf's 64 bytes; a 66+ char string literal used to overwrite the dispatch code that follows the buffer. Long literals truncate but parsing stays in sync. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AEpe9vgM7nhKqiYP4Qw68n
…earch
Editor:
- Downward viewport re-anchor counted CONTENT_ROWS-1 lines back, so
crossing the bottom edge jumped the view two lines, then zero — jerky
scrolling. It now counts CONTENT_ROWS and scrolls exactly one line.
- Shifted letters and graphics (PETSCII >= $A0) are typeable: the input
filter rejected everything >= $80, but only $80-$9F are control codes,
and the renderer already displayed the higher range fine.
- F7 quit restores RPTFLG: BASIC cold start doesn't touch it, so the
editor's all-keys-repeat setting leaked into the BASIC session.
Disk utility (moddsk):
- F4 no longer pages forever into empty pages: the skip phase set
DSK_TRUNCATED ("more entries after this page") merely because a
previous page existed. It's now set only when the entry cap is hit.
- The page-skip count is 16-bit; 8-bit math wrapped at page 3 (300
entries) on large CMD/SD2IEC directories.
- Cursor-down at the bottom row no longer leaves the selection
highlight off-screen for one keypress (boundary off-by-one).
- The legend read "F2=PG+ F4=PG-" but the handlers (and the editor's
own convention) are F2=back/F4=forward — swapped to match the code.
- Format now reports failure: the status-channel reply was drained
unread, so 26,WRITE PROTECT ON looked identical to success. The DOS
code is captured and shown as "FORMAT ERROR NN". The command also
drops the spurious ",P" suffix (a LOAD/SAVE file-type suffix that
some DOSes reject with 30,SYNTAX ERROR) and requires the 2-char ID
CBM DOS expects.
- Header no longer documents D=delete / R=rename handlers that were
never implemented (the '@' raw command channel covers those).
Module loader (modules.asm):
- Removed a stray `lda (LPTR,x)` in the popup draw loop: X holds the
previous character after the first iteration, so it dereferenced a
random zero-page pointer every loop — a read landing on $DC0D/$DD0D
would clear pending CIA interrupt flags.
- A failed module LOAD no longer stacks two blocking messages
("MODULE NOT FOUND" then "MODULE ERROR", ~3 s total).
- MODASM's rich-error display is gated on MODASM having actually run:
five other modules LOAD their code over $C007, so a coincidental $FF
there (or a stale flag from an earlier assembly) displayed garbage as
an assembler error for unrelated failures.
- The post-call BASIC-ROM re-bank decision reads the loader's own
stash instead of LPTR, which the module just had every opportunity to
clobber — a clobber would have left BASIC ROM banked out for good.
Search & replace (modsfr):
- Case-insensitive search folds shifted PETSCII letters ($C1-$DA, the
normal encoding for SHIFT+letter) in addition to $61-$7A, so it
actually matches shifted text.
- The find/replace fields accept printable PETSCII >= $A0 (matching the
editor's new input rule), and the field renderer maps the $A0-$FF
ranges to the correct screen codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AEpe9vgM7nhKqiYP4Qw68n
All changes verified against a full cc65 build; module staging margins re-checked (the link-time asserts guard them). Tokenizers (modtok + modsct try_keyword — dominant cost of saving): - Single-char operators (* + - / < = > ^) dispatch through an 8-entry table instead of walking the whole ~600-byte keyword table. - Characters that can't start a keyword (digits, space, punctuation, shifted PETSCII) skip the scan entirely — previously every source byte, including every space, walked the full table. - The table walk itself rejects entries on their first character and is Y-indexed with one pointer advance per entry, replacing a 16-bit INC plus PHA/PLA flag dance per BYTE. Saving a large BASIC file drops from minutes toward seconds (~10x on the scan). - The now-unreachable 1-char kwtab entries are kept as documentation. Colorizer (col_match_kw callers): - Operators dispatch directly to a one-cell keyword paint. They occur several times on nearly every line, and each previously cost the 8-byte gap-aware lookahead fill plus a scan of up to ~50 entries (~2K cycles per '='). Word-start letters still use the table scan, which already exits on a first-character mismatch. Editor: - New shared row40[] offset table (25 x 16-bit) replaces four multiply-by-repeated-addition loops: compute_cursor_addr (runs on EVERY idle poll for the cursor blink — up to ~370 cycles, thousands of times per second), pop_row_ptr, pop_color_ptr, and render_cursor_row. - The render inner loop no longer saves/restores Y around advance_buf (it only touches A) — removes 8 cycles per rendered character. Disk utility: - Cursor movement inside the window repaints only the old and new selection rows via dsk_redraw_two instead of all 20 rows (~35-40K cycles per keypress) — cursor movement is now instant. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AEpe9vgM7nhKqiYP4Qw68n
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #3 — fixes the remaining data-corruption bugs from the full code review. All changes verified against a complete cc65 build (editor, all ten modules, .d64 image).
BASIC round-trip corruption (every BASIC save/load)
DATAstatement contents are now copied literally until:(outside quotes) or end of line, matching real CRUNCH. Previously keywords inside DATA items were tokenized —DATA MONDAYLISTed correctly butREADreturned garbage token bytes.?now tokenizes to PRINT ($99) as in stock BASIC instead of becoming a runtime SYNTAX ERROR."toggles, resets per line). String contents are raw PETSCII where shifted letters/graphics are ≥$80; without the flag,PRINT"<shift-A>"detokenized asPRINT"ATN"…and corrupted the text..assertguards that code+data end below the staging buffer. This caught a real overlap immediately (the modtok changes grew past the old$C400staging start; staging moved to$C480).Script tokenizer (modsct)
INCLUDEstops tokenization with anINCLUDE UNSUPPORTEDstatus instead of feeding the unfinished include pipeline (which loaded files over the running editor's code and never spliced the include pool into the deployed script). The script-runner manual documents the keyword as reserved.?fixes.Script runner / handler (modscr, modscrh, modules.asm)
$38/$39— BASIC's MEMSIZ hi byte and CURLIN lo byte — so the script handoff's CLR copied a garbage string-heap top into FRETOP and the first string operation failed. The stash moved to$0228/$0229, and modscr also establishesMEMSIZ = $A000before the handoff CLR.verify_scriptsanity-clamps the metadata length to$0002–$97FFbefore it drives a DMA over$0801+(stale REU contents can pass the 2-byte magic check; an REU length of 0 transfers 64 KB).deploy_scriptguarantees the$00at$0800required by theTXTPTR = TXTTAB-1convention.ONERRresume now uses the GOTO convention (TXTPTR = line_start − 1,JMP NEWSTT). NEWSTT does not CHRGET; the old+3convention resumed 4 bytes inside the statement for target lines < 256 and SYNTAX-ERROR-looped for the rest. The resume also resets the stack to BASIC's$FAbaseline like the stock error handler, fixing a stack leak on every caught error.SEC, so legal empty statements (trailing:,::) no longer raise spurious SYNTAX ERRORs.hnd_read_str_argis bounded tohnd_cmd_buf's 64 bytes; a 66+ character string literal used to overwrite the dispatch code that follows the buffer.🤖 Generated with Claude Code
https://claude.ai/code/session_01AEpe9vgM7nhKqiYP4Qw68n
Generated by Claude Code