[BUG] Token misread when there is comment in model config file in preview version

[AsakusaRinne]

The environment

In branch preview, windows 11, .NET 6

The description

In the preview branch, when adding comments to config files with "#", the tokens will be misread if the comment contains a comma.

For instance, we add a comment to the rbac_model.conf in the examples of unit tests, as shown below.

[request_definition]
r = sub, obj, act

[policy_definition]
p = sub, obj, act   #This is a comment, splited by a comma

[role_definition]
g = _, _

[policy_effect]
e = some(where (p.eft == allow))

[matchers]
m = g(r.sub, p.sub) && r.obj == p.obj && r.act == p.act

Then the token count of policy definitions will be 4, which is one more than what is expected. The tokens are read as sub, obj, act #This is a comment, splited by a comma.

I think it's an unexpected behavior. If it is, I would like to solve it soon.

[casbin-bot]

@sagilio @xcaptain @huazhikui

[hsluoyz]

@sagilio

[github-actions]

🎉 This issue has been resolved in version 2.0.0-preview.5 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[github-actions]

🎉 This issue has been resolved in version 2.0.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

[github-actions]

🎉 This issue has been resolved in version 2.0.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀