Skip to content

feat: add register_statement_class API for extending PPI grammar#360

Draft
toddr-bot wants to merge 2 commits into
masterfrom
koan.toddr.bot/fix-issue-270
Draft

feat: add register_statement_class API for extending PPI grammar#360
toddr-bot wants to merge 2 commits into
masterfrom
koan.toddr.bot/fix-issue-270

Conversation

@toddr-bot

@toddr-bot toddr-bot commented Apr 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a public API (PPI::Lexer->register_statement_class()) that allows external code to register custom keywords (like async, method, around) that should be mapped to specific statement classes during lexing. This addresses the long-standing request to make PPI's grammar extensible without monkey-patching internals.

Fixes #270

Changes

  • PPI::Lexer: Add register_statement_class($keyword, $class) and unregister_statement_class($keyword) class methods that modify the internal %STATEMENT_CLASSES hash
  • PPI::Statement::Sub::name(): Handle prefix keywords before sub — when the expected name position holds sub (e.g., async sub hello {}), skip past it to find the actual name
  • PPI::Token::Whitespace: Relax prototype detection to recognize prototypes after any word preceding sub name (, not just my/our/state
  • Tests: 8 new tests covering the API, prefix+sub patterns, sub-like keywords without sub, anonymous prefix subs, and cleanup verification

Example usage

use PPI;
use PPI::Lexer;

# Register 'async' as a sub-like keyword
PPI::Lexer->register_statement_class('async', 'PPI::Statement::Sub');

my $doc = PPI::Document->new(\"async sub hello { ... }");
my $sub = $doc->child(0);
# $sub isa PPI::Statement::Sub
# $sub->name returns 'hello'
# $sub->block returns the PPI::Structure::Block
# $sub->prototype works correctly

# Also works for sub-like keywords without 'sub':
PPI::Lexer->register_statement_class('method', 'PPI::Statement::Sub');
my $doc2 = PPI::Document->new(\"method hello { ... }");
# $doc2->child(0) isa PPI::Statement::Sub, name() returns 'hello'

# Clean up when done
PPI::Lexer->unregister_statement_class('async');

Test plan

  • All 8 new tests pass (API, async sub with block/forward/prototype/attributes, method keyword, anonymous async sub, cleanup)
  • Full test suite: 53089 tests across 69 files, all passing, zero regressions
  • Verified round-trip safety preserved

Generated by Kōan /fix


Quality Report

Changes: 4 files changed, 155 insertions(+), 8 deletions(-)

Code scan: clean

Tests: skipped

Branch hygiene: clean

Generated by Kōan post-mission quality pipeline

toddr-bot and others added 2 commits April 26, 2026 19:08
Add failing tests (marked $TODO) for a new register_statement_class()
API on PPI::Lexer that will allow external code to register custom
keywords (like 'async') that map to statement classes (like
PPI::Statement::Sub).

Tests cover:
- API existence (register/unregister methods)
- prefix+sub patterns: async sub hello {}, async sub hello;,
  async sub hello ($) {}, async sub hello : method {}
- sub-like keywords without 'sub': method hello {}
- anonymous prefix subs: async sub {}
- cleanup: unregistered keywords don't affect parsing

Refs: #270

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add PPI::Lexer->register_statement_class() and unregister_statement_class()
methods that allow external code to register custom keywords that map to
specific statement classes during lexing. This enables modules introducing
new keywords (async, method, around, etc.) to have them parsed into the
correct statement types.

Changes:
- PPI::Lexer: add register/unregister_statement_class class methods that
  modify the internal %STATEMENT_CLASSES hash
- PPI::Statement::Sub::name(): handle prefix keywords before 'sub'
  (e.g. "async sub hello {}" now correctly returns 'hello')
- PPI::Token::Whitespace: relax prototype detection to recognize
  prototypes after any word preceding 'sub name (' (not just my/our/state)
- Tests: remove $TODO markers, all extension API tests now pass

Example usage:
  PPI::Lexer->register_statement_class('async', 'PPI::Statement::Sub');
  my $doc = PPI::Document->new(\"async sub hello {}");
  # $doc->child(0) is now a PPI::Statement::Sub
  # $doc->child(0)->name returns 'hello'

Fixes #270

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Extending PPI

1 participant