Skip to content

feat: add BracketAwareCsvParser to parse CSV file better#494

Merged
hsluoyz merged 6 commits intoapache:masterfrom
HashCookie:improve_policy_line
Nov 22, 2024
Merged

feat: add BracketAwareCsvParser to parse CSV file better#494
hsluoyz merged 6 commits intoapache:masterfrom
HashCookie:improve_policy_line

Conversation

@HashCookie
Copy link
Copy Markdown
Contributor

@HashCookie HashCookie commented Nov 19, 2024

Part of: apache/casbin-editor#164

Original Issue
Limitations of CSV parser:

  • The csv-parse library is mainly used for processing standard CSV format.
  • Unable to handle complex policy expressions correctly
  • Improper handling of commas and parentheses within quotation marks.
  • Does not support nested expressions

Improvement plan
Custom parser:

const tokens: string[] = [];
let currentToken = '';
let inQuotes = false;
let bracketCount = 0;
  • Can handle commas inside quotation marks
  • Support nested parentheses
  • Correctly handle escape characters
image

@HashCookie
Copy link
Copy Markdown
Contributor Author

HashCookie commented Nov 20, 2024

Continue using csv-parse,improves the HelperloadPolicyLine method by modifying the parsing logic to address two significant scenarios that the previous implementation couldn’t handle effectively:

  1. Support for Nested Brackets in Policy Lines
  • In policy lines containing nested structures (e.g., keyMatch("IT", r.sub.org) or regexMatch("(abc|def)", attr)), the previous implementation would incorrectly split tokens due to its reliance on CSV parsing alone.
  • The updated logic now ensures that nested brackets are tracked and only complete expressions are added as tokens.
  1. Proper Handling of Quote Escaping
  • Policy lines with escaped quotes (e.g., keyMatch(""IT"", r.sub.org)) are now correctly processed. Escaped quotes ("") are replaced with a single quote ("), ensuring accurate policy values.

@hsluoyz hsluoyz changed the title fix: online editor gives different results to pyCasbin Enforcer.enforce feat: add BracketAwareCsvParser to parse CSV file better Nov 22, 2024
@hsluoyz hsluoyz merged commit 4c73883 into apache:master Nov 22, 2024
github-actions Bot pushed a commit that referenced this pull request Nov 22, 2024
# [5.36.0](v5.35.0...v5.36.0) (2024-11-22)

### Features

* add BracketAwareCsvParser to parse CSV file better ([#494](#494)) ([4c73883](4c73883))
@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version 5.36.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants