Skip to content

Commit

Permalink
handle column references that involve variables (#74)
Browse files Browse the repository at this point in the history
To support root column references we need to handle column references that involve variables. Currently we're using the built-in variable `$$ROOT` to reference the root collection (which is wrong, but we'll fix that in another PR, and the correct solution will also require a variable.) Before this PR our column references don't work with variable references like `$$ROOT` due to the two different kinds of expressions that are found in MongoDB aggregation pipelines. Specifically there are,

- match queries, where the reference is a key in the document used in a `$match` aggregation stage
- aggregation expressions which appear in a number of contexts

The code we had would create a match stage like this:

```json
{
  "$match": {
    "$$ROOT.field": { "$eq": 1 }
  }
}
```

That doesn't work because `"$$ROOT.field"` is in a match query context which does not allow using `$` to reference field names or variables. But there is a match query operator, `$expr`, that switches to an aggregation expression context. So the correct solution looks like this:

```json
{
  "$match": {
    "$expr": {
      "$eq": ["$$ROOT.field", 1]
    }
  }
}
```

This PR updates the code for producing `$match` stages to use switch to aggregation expression context to handle cases like this. Specifically I introduced an enum, `ColumnRef`, which signals cases where the reference needs to be in an aggregation expression context.

Since we're now supporting both expression contexts it's now possible to handle binary and unary comparisons on field names that contain `$` or `.` without erroring out. This PR does that by replacing uses of `safe_name` (which can throw errors) in `$match` stage building with `ColumnRef::from_comparison_target` (which does not throw errors).

Supporting aggregation expression contexts was also a blocker for column-to-column binary comparisons. So I added that support to this PR. But those comparisons don't support relationship paths for the time being.

[MDB-154](https://hasurahq.atlassian.net/browse/MDB-154)
  • Loading branch information
hallettj committed Jun 12, 2024
1 parent bab5c93 commit 3b58fb8
Show file tree
Hide file tree
Showing 12 changed files with 569 additions and 135 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -6,52 +6,42 @@ data:
movies:
- title: Blacksmith Scene
imdb:
rating:
$numberDouble: "6.2"
rating: 6.2
votes: 1189
- title: The Great Train Robbery
imdb:
rating:
$numberDouble: "7.4"
rating: 7.4
votes: 9847
- title: The Land Beyond the Sunset
imdb:
rating:
$numberDouble: "7.1"
rating: 7.1
votes: 448
- title: A Corner in Wheat
imdb:
rating:
$numberDouble: "6.6"
rating: 6.6
votes: 1375
- title: "Winsor McCay, the Famous Cartoonist of the N.Y. Herald and His Moving Comics"
imdb:
rating:
$numberDouble: "7.3"
rating: 7.3
votes: 1034
- title: Traffic in Souls
imdb:
rating:
$numberInt: "6"
rating: 6
votes: 371
- title: Gertie the Dinosaur
imdb:
rating:
$numberDouble: "7.3"
rating: 7.3
votes: 1837
- title: In the Land of the Head Hunters
imdb:
rating:
$numberDouble: "5.8"
rating: 5.8
votes: 223
- title: The Perils of Pauline
imdb:
rating:
$numberDouble: "7.6"
rating: 7.6
votes: 744
- title: The Birth of a Nation
imdb:
rating:
$numberDouble: "6.8"
rating: 6.8
votes: 15715
errors: ~
22 changes: 20 additions & 2 deletions crates/mongodb-agent-common/src/comparison_function.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ impl ComparisonFunction {
.ok_or(QueryPlanError::UnknownComparisonOperator(s.to_owned()))
}

/// Produce a MongoDB expression that applies this function to the given operands.
pub fn mongodb_expression(
/// Produce a MongoDB expression for use in a match query that applies this function to the given operands.
pub fn mongodb_match_query(
self,
column_ref: impl Into<String>,
comparison_value: Bson,
Expand All @@ -70,4 +70,22 @@ impl ComparisonFunction {
_ => doc! { column_ref: { self.mongodb_name(): comparison_value } },
}
}

/// Produce a MongoDB expression for use in an aggregation expression that applies this
/// function to the given operands.
pub fn mongodb_aggregation_expression(
self,
column_ref: impl Into<Bson>,
comparison_value: impl Into<Bson>,
) -> Document {
match self {
C::Regex => {
doc! { "$regexMatch": { "input": column_ref, "regex": comparison_value } }
}
C::IRegex => {
doc! { "$regexMatch": { "input": column_ref, "regex": comparison_value, "options": "i" } }
}
_ => doc! { self.mongodb_name(): [column_ref, comparison_value] },
}
}
}
10 changes: 9 additions & 1 deletion crates/mongodb-agent-common/src/mongodb/sanitize.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ use crate::interface_types::MongoAgentError;

/// Produces a MongoDB expression that references a field by name in a way that is safe from code
/// injection.
///
/// TODO: equivalent to ColumnRef::Expression
pub fn get_field(name: &str) -> Document {
doc! { "$getField": { "$literal": name } }
}
Expand All @@ -33,10 +35,16 @@ pub fn variable(name: &str) -> Result<String, MongoAgentError> {
}
}

/// Returns false if the name contains characters that MongoDB will interpret specially, such as an
/// initial dollar sign, or dots.
pub fn is_name_safe(name: &str) -> bool {
!(name.starts_with('$') || name.contains('.'))
}

/// Given a collection or field name, returns Ok if the name is safe, or Err if it contains
/// characters that MongoDB will interpret specially.
///
/// TODO: Can we handle names with dots or dollar signs safely instead of throwing an error?
/// TODO: MDB-159, MBD-160 remove this function in favor of ColumnRef which is infallible
pub fn safe_name(name: &str) -> Result<Cow<str>, MongoAgentError> {
if name.starts_with('$') || name.contains('.') {
Err(MongoAgentError::BadQuery(anyhow!("cannot execute query that includes the name, \"{name}\", because it includes characters that MongoDB interperets specially")))
Expand Down
Loading

0 comments on commit 3b58fb8

Please sign in to comment.