Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a new experimental feature: QueryGraphs #56

Merged
merged 3 commits into from
Apr 15, 2020
Merged

Add a new experimental feature: QueryGraphs #56

merged 3 commits into from
Apr 15, 2020

Commits on Apr 14, 2020

  1. Add a new experimental feature: QueryGraphs 

    This is the first step to QueryGraphs on production. There are no precise
measurements yet, but it gives us power to execute queries that would be
completely impossible to optimize before. For example:

```
ursadb> select into iterator {1? 2? 3? 4? 5? 6? 7? 8?};
[2020-04-14 23:17:11.177] [info] {
    "result": {
        "file_count": 5,
        "iterator": "1c36d313",
        "mode": "iterator"
    },
    "type": "select"
}
```

There are only 5 files (out of 50k) in the result, a reduction factor of
1/10k. Not bad. The performance is... better than expected (there are
basically no optimisations yet), but needs testing on bigger datasets.

The most important caveat is that this is not stable yet. It's relatively
easy to kill the DB using a query like:

```
select {?? ?? ?? ??};
```

We need a way to prune query graphs during expansion, but I'll leave it
for future PRs (this one is big enough already).

To test, configure cmake with EXPERIMENTAL_QUERY_GRAPHS:

```
cmake -DEXPERIMENTAL_QUERY_GRAPHS=ON ..
make
```
    @msm-code
    msm-code committed Apr 14, 2020
    Configuration menu
    Copy the full SHA
    9f5bd4f View commit details
    Browse the repository at this point in the history

  2. Forgotten include (works on my machine!)

    @msm-code
    msm-code committed Apr 14, 2020
    Configuration menu
    Copy the full SHA
    eaab080 View commit details
    Browse the repository at this point in the history

  3. Fix formatting this time...

    @msm-code
    msm-code committed Apr 14, 2020
    Configuration menu
    Copy the full SHA
    f515585 View commit details
    Browse the repository at this point in the history