-
Notifications
You must be signed in to change notification settings - Fork 44
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix URL-encoded filesystem entries in the Velociraptor loader #700
base: main
Are you sure you want to change the base?
Conversation
This occurs with some Velociraptor collections. It is currently unknown why this issue occurs.
The names of the files and directories collected by Velociraptor are URL-encoded, which are decoded by the ZIP loader in order to prevent errors when executing plugins. This mainly impacts Unix targets because files like `%2Ebash_history` -> `.bash_history` are used by the plugins. Loading a directory is removed, because using the ZIP as a target is the preferred and enforced method.
Wouldn't this also be an issue on non-zip (directory) Velociraptor targets? |
@Schamper yes it will, I would like to enforce the usage of a ZIP file as target, because extracting a ZIP file causes all kinds of errors. For example, the only option to extract a ZIP file in an automated manner is with Should this explicitly be documented in the comments of the loader or should the loader also support a non-zip directory Velociraptor target? |
We don't use Velociraptor so I can't really speak for it, but I do think I've heard people say they sometimes work with unzipped collections. If there are no other reasons to drop support for it, I'd suggest to let it in. I'm not a huge fan of the explicit URL decoding under a generic |
After a short discussion we decided it may be best to tweak the I've created #707 and #708 to implement this in the respective filesystems. Would you be up for making these changes? |
Yes but I don't understand how to fix the issues. I will add comments to the issues. |
Would be nice to keep the support for a non-zipped folder since we noticed a huge hit on performance when working with zip archives rather than unzipped directories. @Zawadidone wondering if you do not have the same problem with zipped vs unzipped |
To add to the need for this. Example, the following path is not picked up by
This totally makes sense since the Can we include the fix not only for the zip loader, but also for the folder loader? |
@Zawadidone Under which circumstances do you run into files that are overwritten? |
Yes we have the same performance hit when working with a collection stored in a ZIP file, but only when executing the MFT plugin. The other plugins are 'fast' but that's only because they return less records compared to the MFT plugin. I haven't had the time for it, but I want to look into solving the following issues:
|
Yes this PR will include a fix for both the ZIP as well as the DIR loader. |
On Linux systems that use symlinks see #699, but this could occur on all kind of systems that use symlinks. |
Will be fixed by fox-it#793
…target into fix/velociraptor_zip
Closes #707 and #708