Tokenizer is a wrapper around the token_get_all
function. It's built up of three classes:
Token
- Each token returned fromtoken_get_all
is wrapped in this.TokenListIterator
- ImplementsSeekableIterator
andCountable
with extra methods to allow easy moving through tokens.Tokenizer
- Callstoken_get_all
on the source, turns each result into aToken
representation and returns aTokenListIterator
object.
You can install Tokenizer by adding hippophp/tokenizer
to your composer.json
file.
HippoPHP Tokenizer is licensed under The MIT License (MIT)