Skip to content

0.0.4: Fix parsing issues (OSK-7) (#8)

Choose a tag to compare

@BlankDev117 BlankDev117 released this 23 Jun 00:57
· 2 commits to main since this release
60fe6b1
Motivation
----
Some issues arose when the token reader was used to parse text and
delimeter file tokens

Modifications
----
* Updated the GenericTokenStateHandler/FileTokenReader to better handle
state with text file tokens
* Updated DefaultTokenStateHandler to account for
comments/delimeters/etc. as expected

Result
----
Text, delimeter, and other token isses should be fixed with
DefaultTokenStateHandler

Fixes #7