Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use StrictCollectionPath for uploading collection snapshot #3991

Merged
merged 1 commit into from
Apr 9, 2024

Conversation

generall
Copy link
Member

@generall generall commented Apr 8, 2024

  • Prevent security risk of uploading snapshots into collection with malformed name

Copy link
Contributor

@mac-chaffee mac-chaffee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=92d0e7af8fbf49b5b39326c6c7bbdafa

Although to me, it feels like a half-measure, since a slight change to the allowed collection names would bring the vulnerability back (are we 100% certain we'd never approve a future PR that says "allow / in collection names"?). Like trying to solve SQL injection with more sanitization rather than just using prepared statements.

This change is fine for now, but a more thorough fix might be adding validation like this right before doing any filesystem operations:

let snapshot_path = Path::new(&self.storage_config.snapshots_path);
let absolute_snapshot_path = snapshot_path.canonicalize()?;
let absolute_upload_path = upload_path.canonicalize()?;

assert!(absolute_upload_path.starts_with(absolute_snapshot_path))

(this is probably buggy code, just used for illustration purposes)

@generall generall merged commit 15479a4 into dev Apr 9, 2024
17 checks passed
@generall generall deleted the fix-collection-name-validation branch April 9, 2024 07:43
@timvisee
Copy link
Member

timvisee commented Apr 10, 2024

Like trying to solve SQL injection with more sanitization rather than just using prepared statements.

There is no such thing as prepared statements in here.

We may be able to solve this with using cap-std internally, but it proved extremely hard to implement in a way that makes sense. It also is not compatible with our S3 interface for snapshots.

@mac-chaffee
Copy link
Contributor

There is no such thing as prepared statements in here.

Just an example of solving the problem at the lowest layer (right before execution) rather than the highest layer (sanitizing inputs).

We are very lucky that we can say "Qdrant should never be operating on paths that are outside of snapshots_path or storage_path". But none of our functions that perform filesystem operations enforce this assumption. In the cloud, we enforce this assumption with readOnlyRootFilesystem.

Adding something like assert!(absolute_upload_path.starts_with(absolute_snapshot_path)) to a few known places seems simpler than chasing down every location where user-provided input is used to construct file paths, for all current and future code :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants