v0.9.8-07

Toka v0.9.8-07

This release is a 1.0-freeze checkpoint focused on PAL safety, syntax conformance, and toolchain stability.

Highlights

• Froze the PAL core model as Path-Anchored Ledger.
• Added call-site implicit borrow checking, so payload arguments are now checked as simultaneous temporary borrows.
• Rejected unsafe call aliases such as mutable/mutable, mutable/read, and cede/read overlaps.
• Split PAL mutation operation classes so ordinary payload writes, exclusive mutations, and invalidating operations are tracked more precisely.
• Improved local CFG state merging for if, guard, match, loops, break, continue, and labeled control flow.
• Enforced explicit return dependency declarations for borrowed results such as -> str <- source.
• Locked execution-boundary capture safety for escaping closures and thread-like boundaries.
• Froze additional syntax rules for dyn traits, where bounds, handle hash placement, path visibility, kebab path boundaries, public unsafe API naming, match pattern safety, and common mistake diagnostics.
• Added optional compiler audit instrumentation for assignment and handle-surface classification.
• Hardened TKI replay/cache behavior and test-lib object reuse.
• Fixed tokafmt to comply with explicit PAL return dependencies.
• Kept optional stats headers buildable in the Emscripten playground path.
• Expanded pass/fail/warn coverage, including warning verification and several syntax conformance matrices.

Breaking / Tightened Diagnostics

This release intentionally tightens several diagnostics as part of the 1.0 freeze work.

Code returning borrowed data from parameters or fields may now need explicit dependency annotations:

fn view(source: str) -> str <- source

Function calls are also checked as simultaneous temporary borrow groups. For example, passing the same payload as both mutable and readable/mutable arguments is rejected, while repeated read-only payload arguments remain valid.

Current Limits

• dyn @Trait remains single-facet for now.
• Associated type binding on dyn @Trait is not part of this release.
• Async/thread semantics remain conservative; deeper async and concurrency consumers are post-freeze work.
• Raw pointer alias proof remains outside the safe automatic PAL subset.

Verification

• Full local rebuild completed successfully with:

./rebuild.sh

• The rebuild produced tokac, toka, tokafmt, tokalsp, and forge.
• GitHub Actions should validate the full platform matrix for this overwritten v0.9.8-07 tag.

Full Changelog

v0.9.8-06...v0.9.8-07

v0.9.8-06

Toka v0.9.8-06

This release is a broad stabilization and tooling release since v0.9.8-05, focused on the build system, TKI interface reuse, parameter/resource semantics, trait syntax, cross-platform packaging, and public documentation.

Highlights

• Introduced and stabilized the JSON dependency manifest protocol used by tokac --dump-dependencies.
• Added the incremental build driver and hardened toka build integration.
• Fixed several incremental-build ownership, cache fallback, path identity, and stale-interface issues.
• Added validated .tki metadata handling, including compiler/interface version checks, target checks, source hash checks, and source fallback.
• Added archive-backed cached library testing and expanded cached-lib coverage.
• Stabilized TKI replay for core/string, generic containers, morphology, effects, closures, postfix operators, and cached module identity.
• Added pass-suite timing and cached-lib validation to the test workflow.
• Tightened parameter morphology rules, including rejection of type-side reference parameters and clearer binding-side morphology.
• Enforced cede parameter consumption contracts.
• Added where impl constraints, trait bound set syntax, required @ markers on trait declarations, and associated type / per type syntax.
• Added the standard Iterator trait.
• Refactored and hardened member access lowering, including morphic member identity access and stricter prefix checks.
• Added WASI/WebAssembly cross-compilation support and playground self-tests.
• Fixed release packaging/linking issues on macOS x64, Windows, and zstd discovery through CMake.
• Refreshed README, syntax documentation, website content, install guidance, and project positioning.

Build System and TKI

This release includes a major step forward for Toka's build infrastructure:

• JSON dependency manifests are now versioned and machine-readable.
• .tki interface files are validated using explicit compiler, format, target, and source metadata.
• The resolver can fall back from stale or invalid interfaces to source files.
• Interface path behavior, dotted output directories, multi-root output conflicts, and source-less interface use cases are covered by tests.
• Cached library mode is now part of the pass-suite workflow.
• Cached-lib coverage now reaches 53 cached objects while preserving dependency-closed archive safety.

Language and Semantics

This release tightens several core language rules:

• Function parameters use binding-side morphology more consistently.
• Type-side borrow/reference parameter forms are rejected.
• cede in a function signature is now treated as an executable contract: the value must actually be consumed or forwarded.
• Trait declarations now require the @Trait form.
• Trait bound sets use the canonical @{TraitA, TraitB} form.
• where: constraints are supported for impl prerequisites.
• Associated types and per type associated types are introduced.
• Standard iterator support is added.

Compiler and Runtime Fixes

• Member access codegen was refactored and hardened.
• Pointer lowering and captured-resource handling were stabilized.
• Nonblocking socket ABI handling was fixed for variadic fcntl.
• Windows LLVM DLL linking and macOS x64 release linker setup were fixed.
• CMake now finds zstd more robustly across Homebrew layouts.

WebAssembly, Playground, and IDE

• Added WASI 32-bit cross-compilation support.
• Added playground runtime fixes, stdin/stdout handling, compiler-rt math builtin support, and self-tests.
• Improved VSCode extension packaging and LSP path resolution.

Documentation

• Rewrote and refreshed the README and Chinese README.
• Added an English syntax guide and upgraded the Chinese syntax guide for public use.
• Refreshed website introduction, installation, CLI/tooling, and project-structure pages.
• Clarified Toka's design positioning around explicit resource semantics, safety, performance, and simplicity.
• Updated citation/version metadata for 0.9.8-06.

Validation

The final tagged commit passes the full local validation suite:

• PASS: 287 / 287
• ODR path and interface behavior tests passed
• TKI cache validation tests passed
• Incremental build tests passed
• Cached-lib path validated with 53 cached objects

v0.9.8-05

This release introduces critical compiler bug fixes, enhanced syntax validation, ABI optimizations, and documentation updates.

🚀 Features & Enhancements

• Linux ARM64 Native Release Package: Added native compilation and packaging support for Linux ARM64 (AArch64). Release binaries are now natively built and validated under ARM64 GitHub Actions runner environments, ensuring complete platform stability.
• Cross-Compilation & Target Override: Introduced experimental support for overriding the target triple via the TOKA_TARGET_TRIPLE environment variable, enabling developers to easily generate relocatable target files (e.g., targeting aarch64-unknown-linux-gnu from standard x86 host machines).
• Relaxed Same-Directory Circular Imports: Supported same-physical-directory circular imports via a two-stage declareGlobals Sema pass, solving AST resolution recursion limits.
• Strict Pointer Morphology Checks: Enforced syntax checks to prevent pointer morphology sigils (&, *, ^, ~) from prefixing type names inside struct definitions (e.g. source_file: &SourceFile is now a parse error, suggesting &source_file: SourceFile instead).
• Explicit Morphology in Reference Bindings: Enforced checks requiring explicit pointer morphology (like &) on variables when initializing them from reference members.
• Standard Library Cleanup: Fixed field sigil positions in standard library Entry shapes inside std/hashmap.tk to strictly follow Toka pointer rules.

🐛 Bug Fixes

• Windows Path Normalization: Resolved a critical bug where backslashes \ in Windows file paths caused standard library directory checks to fail (wrongly auto-injecting core/prelude) and led to module resolution mismatches in Sema. Integrated a foolproof path-normalizer converting all backslashes to generic slashes across Driver, SourceManager, Parser, and Sema.
• Memory Safety in Coroutines: Resolved memory safety issues during LLVM coroutine execution termination and destruction paths.
• CodeGen Stability: Fixed stack buffer overflow and type mismatch bugs in conditional statements (IfExpr and GuardExpr codegen).
• Compiler Crash on Circular Imports: Standardized file paths during circular dependency checks to prevent compiler segmentation faults.

⚡ Performance Optimizations

• ABI Optimization: Redesigned pointer return conventions to completely bypass the LLVM sret (Struct Return) overhead, generating tighter machine code.
• Intrinsics Refactoring: Migrated raw_ptr to compiler-level intrinsics to optimize memory copying (copy_memory removal).

📝 Documentation & Ecosystem

• Windows Build Environment Guide: Updated documentation to properly guide MSYS2 UCRT64 terminal dependency setup and pacman commands.
• Academic Citation: Strongly linked and synchronized professional citations with the official Toka arXiv research paper.

v0.9.8-04

[pass 266/266] test: optimize bash test suites and introduce native p…

v0.9.8-03: Default @arc, Unified Diagnostics & Parallel Incremental Build

Toka v0.9.8-03 Release Notes

Toka v0.9.8-03 is a milestone release that delivers critical advancements in static safety guarantees, standard library architecture, parallel build tooling, and multi-threaded runtime safety. This release hardens the compilation boundary, aligns standard library containers with physical hardware models, introduces a parallel incremental build engine, and establishes default atomic reference counting with optimal memory orderings.

🚀 Key Achievements Since v0.9.8-02

1. Multi-Threaded Runtime Safety: Default Atomic Reference Counting

• Default @arc Semantics: Promoted all reference counting operations for the shared pointer ~ to be atomic by default. This eliminates critical memory safety race conditions under multi-threaded environments.
• Optimal Happens-Before Memory Orderings:
  • Inc (Retain): Utilizes Monotonic (Relaxed) ordering to minimize runtime overhead. It avoids expensive Data Memory Barrier (DMB) instructions on AArch64 (Apple Silicon) platforms while remaining extremely lightweight on x86_64.
  • Dec (Release): Utilizes AcquireRelease ordering. This establishes a hardware-enforced barrier that synchronizes all historical writes before triggering the @encap/drop destructor, preventing Use-After-Free (UAF) and race conditions during deallocation.
• Future-Proof Memory Policies: Silently reserved backend code generation slots for @policy suffixes (such as @local and @weak) in the Type and CodeGen systems, avoiding breaking changes for Toka 2.0.

2. Standard Library Redesign: CPU Contiguous Memory & Power-of-Two RingCore

• Physical Hardware Alignment: Deleted the classic doubly LinkedList from the standard library (std). This aligns Toka with modern contiguous CPU physical memory hierarchies to maximize cache locality.
• Zero-Division Power-of-Two RingCore: Refactored standard library queue and buffer containers with a power-of-two RingCore, substituting expensive modulo division with fast bitwise operations.
• True O(1) FIFO Channels: Upgraded standard library channel queues to leverage RingCore, providing zero-cost, high-throughput asynchronous communication.

3. Safety Hardening: Generational Slab & TCB Escaping Analysis

• Generational Slab Allocator: Implemented the standard generational Slab allocator and SlabID with strict encapsulation.
• Zero-Copy Option Borrows: Refactored Slab::get and Slab::get_mut to return safe, zero-copy Option borrows and strictly enforced pointer morphology sigils.
• TCB Boundary Reinterpret Cast Guard: Hardened collectDeps and returnedDeps algorithms in the semantic analyzer to proactively detect and block unsafe pointer reinterpret-cast escapes across the Trusted Computing Base (TCB).

4. Developer Tooling: Forge Parallel Incremental Build Engine

• Parallel Build Orchestrator: Developed forge, Toka's native parallel incremental build engine. It compiles dirty targets concurrently utilizing standard channels and waker mechanics.
• Instant Cache Resolution: Embedded a persistent cache store (.forge_cache) that tracks compile-time metadata, file sizes, and modification timestamps to achieve O(1) clean target resolution.
• Self-Contained FFI Interop: Leveraged native libc_system and thread-local memory allocation systems to manage task lifecycle dynamically.

5. Code Quality & Unified Diagnostics

• 100% Strongly-Typed Error Engine: Unified and migrated all hand-written compiler error diagnostics into a single source of truth (spec/diagnostic.map.json). It automatically generates type-safe templates, expanding the diagnostics library to 395 entries.
• Prohibiting Pointer Sigils on Method self: Upgraded HasPointer checks to IsRawPointer and defined ERR_SELF_POINTER_SIGIL to block unsafe morphology symbols on method self parameters at the parser level.
• Zero-Trigraph Compiler Build: Eliminated -Wtrigraphs preprocessor warnings during C++ compiler bootstrap by escaping legacy query strings inside the error definitions.

---

🧪 Verification & Regression Testing

This release has passed comprehensive regression test suites:

• Positive Test Suite (test_pass.sh): 263 out of 263 tests passed (100.00% success rate), including multi-threaded stress tests (atomic_stress.tk and sync_mutex.tk).
• Negative Test Suite (test_fail.sh): 113 out of 113 diagnostic failure tests passed (100.00% success rate).
• LSP & Tooling Warnings: Cleaned up all package manager and language server warnings.

---

📦 Commit & Tag Information

• Tag: v0.9.8-03
• Baseline Hash: b190895

Toka v0.9.8-02: UTF-8 Refactoring & Diagnostics 2.0

Toka v0.9.8-02 Release Notes

We are proud to announce the release of Toka v0.9.8-02. This release marks a historical milestone in Toka's development, delivering the complete decoupling of the 2.0 Diagnostic Layer, introducing 5 advanced industrial-grade behavior lints, and executing a massive physical refactoring of the core string/byte system under the strict Toka 1.0 Unified Text & Byte-Stream Constitution.

---

🏛️ 1. Diagnostic System 2.0 & FSM Decoupling

Toka v0.9.8-02 formally introduces the Compiler Dual-Layer Diagnostic Protocol, separating diagnostic facts from IDE/LSP semantics to ensure zero compiler bloat and permanent version-lock protection.

• External Diagnostic Specification (spec/diagnostic.map.json): Offloaded warning and error explanations, domains, severities, and auto-fix metadata to a detached spec file. This specification is protected by a strict compiler_compat version-lock gate.
• Robust Semantic Node Mapping (semantic_id): The compiler now emits high-fidelity semantic facts. Every diagnostic report is bound to a three-dimensional semantic_id structure:
  semantic_id = { file_id, node_serial, expansion_context }
  This ensures real-time, error-free diagnostics tracking inside IDEs and LSP toolings.
• Error Count Double-Increment Resolution: Eliminated a critical bug where ErrorCount was incremented twice when JSON output was disabled. This resolver ensures compiler error thresholds are hit exactly, preventing truncated compilation logs and snapshot mismatches.

---

⚙️ 2. Behavioral Guidance & High-Aesthetic Structural Lints

Five major program sanity and structural lints have been built into the Sema pipeline to guarantee code hygiene and catch logic bugs before codegen:

• W0402 (Unused Variable): Detects defined but unreferenced local variables and arguments. Automatically respects _ prefixes and self to prevent false positives.
• W0403 (Unused Import): Performs multi-to-one dependency tracking to identify redundant standard library or local imports, preserving dependency DAG cleanness.
• W0406 (Unused Result): Catches standalone statements discarding valuable Result or Option payloads, eliminating unhandled logical vulnerabilities.
• S0401 (Unreachable Code): Warns when statements are placed after unconditional terminations (ReturnStmt).
• S0402 (Potential Non-Progress Loop): Employs a zero-overhead compile-time heuristic to scan loop cond {} blocks, raising structural alerts if conditional variables are not mutated inside the loop.
• Magenta Caret Visual Hierarchy: Structural warnings (S class) are now rendered in bold 洋红色 (Magenta Bold) on CLI outputs, creating a distinct visual hierarchy alongside yellow warnings and red errors.

---

🧹 3. Standard Library Surgical Clean & FFI Corrections

To set a standard for clean, warning-free system development, the entire Toka Standard Library has been surgically cleaned:

• Zero-Warning Standard: Cleaned all unused imports, unused variables, and mutability warnings in lib/std, lib/stdx, and lib/sys.
• Obsolete File Deprecation: Physically purged the deprecated /lib/std/memory.tk to eliminate dead code and private extern definitions.
• Process & Net ABI Alignments: Refactored lib/std/process.tk command arguments mapping, resolving reference life issues during parameter loops using standard Option::Some(&val) borrowing with clone bounds.

---

❄️ 4. The Core String & Byte Stream Physical Refactoring

In accordance with the Toka 1.0 Final Spec, the text and byte system has been physically redesigned into a Stream View Model, separating textual observation from physical binary manipulation.

• Txt & Byte Dual-Track Alignment:
  • str (16 bytes): Strictly UTF-8 text view. Supports $O(1)$ byte_count(), $O(N)$ logical count(), character-based slice(), at(), and zero-cost as_bytes() downgrade.
  • bytes (16 bytes): Strictly binary view. Supports $O(1)$ size(), boundary-checked $O(1)$ slice_bytes(), and FFI-safe UTF-8 promotion try_to_str().
• Abolition of Reverse Character Scanning: Strictly removed all backward character scans from standard libraries. Backtracking is now explicitly managed by the Cursor stream decoder, featuring a bounded $O(4)$ safe rewind(k) FFI alignment check.
• FFI Memory Equivalence (memcmp Equality):
  • Rewrote equals(), starts_with(), ends_with(), and index_of_str() on str views using direct raw_ptr() comparisons. This ensures sub-views and sliced strings evaluate with bitwise accuracy under raw pointers, completely solving reference slice address boundaries.
• Import Symbol Pollution Prevention: Removed private fn panic definitions from str.tk and string.tk. Bound checks now call __toka_panic_handler directly, avoiding global namespace collisions during wildcard ::* imports.

---

📊 Verification Suite Results

• Compiler: make -C build -j8 ➔ 100% Passed
• Sema Regression Suite: ./tools/scripts/test_pass.sh ➔ [PASS 262/262] (0 Failures)
• Negative Suite: ./tools/scripts/test_fail.sh ➔ [FAIL 110/110] (100% Match)

v0.9.8-01

Toka v0.9.8-01: Standard Library Slimming & 10-Year Anti-Rot Architec…

v0.9.8

Toka v0.9.8: The Canonical Lowercase string, Zero-Import Prelude, & Z…

v0.9.7-04

Toka v0.9.7-04: The Production-Ready, Clean Ecosystem & Cross-Platfor…

v0.9.7-03

v0.9.7-03: Add native Span/SourceMap, expand LLVM, and enhance Arena.…