Skip to content

Wide scanners#1418

Closed
jnorthrup wants to merge 13 commits intohsutter:mainfrom
jnorthrup:wide-scanners
Closed

Wide scanners#1418
jnorthrup wants to merge 13 commits intohsutter:mainfrom
jnorthrup:wide-scanners

Conversation

@jnorthrup
Copy link

No description provided.

…ization

- Document auto-vectorization approach for SIMD acceleration
- Include vectorization-friendly code patterns
- Cover cross-platform compatibility through compiler optimization
- Outline phased implementation roadmap
- Emphasize maintainability and future compatibility
- Propose 5 new SIMD-accelerated parser combinators for cpp2 scanner
- WideTokenParser: SIMD token recognition with boundary detection
- WideWhitespaceSkipper: Bulk whitespace processing
- WideDelimiterMatcher: Multi-character delimiter detection
- WidePatternMatcher: Bridge with cpp2regex.h SIMD patterns
- WideBoundaryDetector: Parallel token boundary analysis
- Addresses cpp2regex.h TODO for vectorization (line 478)
- 10-40x performance improvement potential
- Phased implementation roadmap for integration
- Complete fusion of cpp2 scanner + cursive SIMD + wide combinators
- 40x throughput improvement potential through intelligent mode switching
- Zero breaking changes with progressive adoption via feature flags
- Addresses cpp2regex.h TODO (line 478) through WidePatternMatcher bridge
- Production-ready 8-week implementation roadmap
- Cross-platform SIMD detection (AVX2/NEON/SSE2/generic)
- Modular architecture enabling future optimizations

Fusion Components:
├── Component 1: Cpp2 Scanner (battle-tested foundation)
├── Component 2: Cursive SIMD (BBCursive/RBCursive engines)
└── Component 3: Wide Combinators (5 new SIMD parsers)

Performance Targets:
├── Throughput: 65 MB/s → 2.6 GB/s (40x improvement)
├── Latency: 15ns → 3ns (5x improvement)
└── Compatibility: 100% backward compatibility

Ready for PR and implementation phases.
- Complete set of metaclass examples from Herb Sutter's work
- Covers all built-in metafunctions: @interface, @value, @enum, @flag_enum, @union, @polymorphic_base, @struct, @regex
- Includes concrete implementations and test functions
- Based on P0707 paper and cppfront regression tests
- Ready for metaclass functionality validation
… isomorphic attributes, and command whitelist system
- Implement @coroutine metaclass for suspend function/coroutine equivalent
- Add @channel metaclass for CSP-style concurrency (Channel<T>)
- Implement @flow metaclass for reactive streams (Flow<T>)
- Add @async metaclass for async/await pattern
- Implement @mutex metaclass for synchronization
- Update Makefile to build cppfront executable and handle conversion errors gracefully
- Generate comprehensive test files in target/ directory
- Fix syntax errors in regression tests
@jnorthrup jnorthrup marked this pull request as draft September 2, 2025 21:47
@jnorthrup
Copy link
Author

delete me! I am awful at the github-PR version of simon says! i was tricked into coming here!

@jnorthrup jnorthrup closed this Sep 2, 2025
@MaxSagebaum
Copy link
Contributor

Why do you even open up the pull requests? Usually you have to do several clicks to open one. Are you using some automation?

@gregmarr
Copy link
Contributor

gregmarr commented Sep 3, 2025

When you are in a fork and are opening a PR from a branch in that fork, the default base is the default branch in the upstream repo, not the default branch of the fork. You have to remember to change the repo every time. I wish there was a way to change that default.

@jnorthrup jnorthrup deleted the wide-scanners branch September 20, 2025 01:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants