v1.0.0

Summary

• Introduce schema DSL + validation API for LLM-safe outputs.
• Add streaming IO helpers and CTON-B binary envelope.
• Refresh README for LLM-first usage and CLI upgrades.

Performance

• Faster decoder scalar scans and reusable encoder buffer.

CLI

• New flags: --schema, --stream, --to-binary, --from-binary.

Tests

• Added schema, streaming, binary, and CLI coverage.

v0.4.0

Added

• Comment Support: CTON now supports single-line comments using # syntax. Comments are ignored during parsing, allowing for annotated data files.

  • Decoder skips comments (from # to end of line) during parsing
  • Encoder can emit comments via new comments: option: Cton.dump(data, comments: { "key" => "description" })

• Validation API: New methods to validate CTON syntax without full parsing:

  • Cton.valid?(string) returns true or false
  • Cton.validate(string) returns a ValidationResult object with detailed error information
  • ValidationResult includes valid?, errors, and to_s methods
  • ValidationError includes message, line, column, and source_excerpt

• Token Statistics API: Analyze and compare CTON vs JSON token efficiency:

  • Cton.stats(data) returns a Stats object with comprehensive metrics
  • Cton.stats_hash(data) returns stats as a Hash
  • Stats includes json_chars, cton_chars, savings_percent, estimated_json_tokens, estimated_cton_tokens
  • Stats.compare(data) compares multiple format variants (CTON, CTON inline, CTON pretty, JSON, JSON pretty)

• Custom Type Registry: Register custom serializers for domain objects:

  • Cton.register_type(klass, as: :object) { |value| ... } registers a type handler
  • Cton.unregister_type(klass) removes a handler
  • Cton.clear_type_registry! clears all handlers
  • Supports :object, :array, and :scalar modes

• Enhanced CLI: New command-line options:

  • --stats / -s: Show token savings statistics comparing JSON vs CTON
  • --validate: Validate CTON syntax without conversion
  • --minify / -m: Output CTON without separators (fully inline)
  • Improved error messages with line/column information and colored output

• Enhanced Error Reporting: ParseError now includes structured location information:

  • line and column attributes for precise error location
  • source_excerpt showing context around the error
  • suggestions array for helpful hints
  • to_h method for programmatic error handling

v0.3.0

Added

• Performance tunables: Cton.dump now accepts decimal_mode: :fast | :precise, allowing callers to trade float-format determinism for lower allocation pressure. Specs cover both modes.
• Benchmark harness: New bench/encode_decode_bench.rb script (wired into the README/Development docs) exercises encode/decode hot paths and prints comparative JSON vs CTON timings. On Ruby 3.1.4/macOS the fast encoder completes 1,000 iterations in ~0.63s and the new inline decoder stress test wraps 400 concatenated documents in ~4.14s.
• Regression tests: Added specs for streaming documents without separators plus validation around the new decimal mode toggle.

Changed

• Encoder: Memoizes table schemas per array instance, adds a fast-path for homogeneous scalar lists, and reduces float/BigDecimal copying by favoring Ruby's native float formatting before falling back to BigDecimal. Unsupported decimal_mode values now raise immediately.
• Decoder: Replaces high-allocation StringScanner tokenization with raw string slicing, improves key-boundary detection for inline payloads, and keeps symbolization logic untouched. Boundary heuristics now prefer alphabetic key starts to avoid splitting numeric payloads.
• Documentation: README now calls out the tuning flags, inline caveats, and benchmark instructions; Development workflow highlights how to rerun the perf suite.

Fixed

• Inline parsing: Eliminated the runaway allocations and incorrect key splits when processing long documents with separator: "".
• Float normalization: Restored canonical 9.2-style output in fast mode while keeping the new perf optimizations.

v0.2.0

Added

• CLI Tool: New bin/cton executable for converting between JSON and CTON from the command line. Supports auto-detection, pretty printing, and file I/O.
• Streaming IO: Cton.dump now accepts an IO object as the second argument (or via io: keyword), allowing direct writing to files or sockets without intermediate string allocation.
• Pretty Printing: Added pretty: true option to Cton.dump to format output with indentation and newlines for better readability.
• Extended Types: Native support for Time, Date (ISO8601), Set (as Array), and OpenStruct (as Object).
• Enhanced Error Reporting: ParseError now includes line and column numbers to help locate syntax errors in large documents.

Changed

• Ruby 3 Compatibility: Improved argument handling in Cton.dump to robustly support Ruby 3 keyword arguments when passing hashes.

v0.1.1

Changed

• Performance: Refactored Encoder to use StringIO and Decoder to use StringScanner for significantly improved performance and memory usage.
• Architecture: Split Cton module into dedicated Cton::Encoder and Cton::Decoder classes for better maintainability.

Fixed

• Parsing: Fixed an issue where unterminated strings were not correctly detected.
• Whitespace: Improved whitespace handling in the decoder, specifically fixing issues with whitespace between keys and structure markers.

Added

• Type Safety: Added comprehensive RBS signatures (sig/cton.rbs) for better IDE support and static analysis.
• Tests: Expanded test coverage for validation, complex tables, mixed arrays, unicode values, and error cases.

v0.1.0

Added

• Initial Release: First public version of the cton gem.
• CTON Encoder: Cton.dump (aliased as Cton.generate) to convert a Ruby Hash into a CTON string.
  • Encodes objects, arrays, strings, numbers, booleans, and nil.
  • Automatic table detection for arrays of uniform hashes, creating a highly compact representation: key[N]{h1,h2}=v1,v2;....
  • Smart string quoting: only quotes strings containing special characters, whitespace, or those that could be misinterpreted as numbers, booleans, or null.
  • Number normalization: canonicalizes floats and BigDecimal to a clean, exponent-free format. NaN and Infinity are converted to null for safety.
  • Configurable separator (\n by default) between top-level key-value pairs.
• CTON Decoder: Cton.load (aliased as Cton.parse) to parse a CTON string back into a Ruby object.
  • Handles all CTON structures: objects (), arrays [], and compact tables []{}.
  • Supports an optional symbolize_names: true argument to convert all hash keys to symbols.
  • Robust parsing of scalars, including quoted and unquoted strings.
• Error Handling: Cton::EncodeError for unsupported Ruby types during encoding and Cton::ParseError for malformed CTON input.
• Documentation: README.md with a format overview, usage examples, and rationale.
• Testing: Comprehensive RSpec suite ensuring round-trip integrity and correct handling of edge cases.