Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting a root fs.FS in a parser. #393

Merged
merged 12 commits into from
Sep 8, 2022

Conversation

anuraaga
Copy link
Contributor

@anuraaga anuraaga commented Sep 2, 2022

fs.FS is a great abstraction that allows operating on files with the same code anywhere from the filesystem (except I had to implement io.OSFS, not sure why standard library doesn't provide that... found out why and added note), embed.FS, zip.Reader, and anything a user may implement, maybe even a remote cloud storage.

By allowing setting a fs.FS, we can support loading config from any of these locations. Notably, we are currently embedding rules into the Envoy filter due to current lack of support to access the filesystem, and while currently we take CRS and preprocess them to remove Include and FromFile statements, with this we should be able to remove most if not all such processing when embedding.

@codecov-commenter
Copy link

codecov-commenter commented Sep 2, 2022

Codecov Report

Base: 76.78% // Head: 76.75% // Decreases project coverage by -0.03% ⚠️

Coverage data is based on head (3170f92) compared to base (4c9dc4d).
Patch coverage: 66.66% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           v3/dev     #393      +/-   ##
==========================================
- Coverage   76.78%   76.75%   -0.04%     
==========================================
  Files         136      136              
  Lines        5957     5966       +9     
==========================================
+ Hits         4574     4579       +5     
- Misses       1113     1118       +5     
+ Partials      270      269       -1     
Flag Coverage Δ
no-tinygo 76.87% <66.66%> (-0.04%) ⬇️
overall 76.75% <66.66%> (-0.04%) ⬇️
tinygo ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
rule.go 94.28% <ø> (ø)
internal/io/file.go 8.69% <25.00%> (+8.69%) ⬆️
seclang/parser.go 92.30% <85.71%> (+0.13%) ⬆️
operators/from_file.go 86.95% <100.00%> (+6.95%) ⬆️
operators/ip_match_from_file.go 88.00% <100.00%> (ø)
operators/pm_from_file.go 72.97% <100.00%> (ø)
seclang/rule_parser.go 82.40% <100.00%> (+0.04%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@jcchavezs jcchavezs requested review from piyushroshan and jptosso and removed request for piyushroshan September 2, 2022 09:22
@jcchavezs
Copy link
Member

LGTM

@jptosso
Copy link
Member

jptosso commented Sep 2, 2022

I've read about this issue, caddyserver does the same for s3 mounts or similar in 2.6. I will take a look of the code but looks nice

internal/io/file.go Outdated Show resolved Hide resolved
@jcchavezs jcchavezs requested a review from fzipi September 3, 2022 11:34
@anuraaga
Copy link
Contributor Author

anuraaga commented Sep 5, 2022

BTW I could confirm it works well here

https://github.com/anuraaga/coraza-wasm-filter/pull/18

Include ftw-config.conf\nInclude coraza.conf-recommended\nInclude crs-setup.conf.example\nInclude crs/*.conf

Glob works fine too (added a commit :) ).

seclang/parser.go Outdated Show resolved Hide resolved
@@ -142,6 +145,7 @@ func (p *Parser) evaluate(data string) error {
p.options.Config.Set("last_profile_line", p.currentLine)
p.options.Config.Set("parser_config_file", p.currentFile)
p.options.Config.Set("parser_config_dir", p.currentDir)
p.options.Config.Set("parser_root", p.root)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry don't quite follow the link, do you mean remove p.options.Config.Set/Get("working_dir") and have p.root default to working dir if user doesn't call SetRoot?

In that case, this won't work even if the user doesn't call SetRoot

coraza/waf.exe
config/rules.conf

Since .. wouldn't work. While we are discussing being OK with ../ not working with an explicit call to SetRoot, I'm not sure about when it hasn't been called at all, I guess it should be allowed though.

@anuraaga
Copy link
Contributor Author

anuraaga commented Sep 6, 2022

I have gone ahead and removed SetCurrentDir in this PR.

Note that this PR is very important for the proxy-wasm plugin so I'm finding myself merging this branch locally to any other bugfixes like #399 while debugging ftw tests. Still no rush, but a sooner than later merge would be highly appreciated :D

@anuraaga
Copy link
Contributor Author

anuraaga commented Sep 7, 2022

@jptosso Can you take a look at this? It's blocking a lot of work on the wasm filter. Thanks!

@jptosso jptosso merged commit 812ed46 into corazawaf:v3/dev Sep 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants