Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Covering text index scans return tokens, not requested full text strings #1017

Closed
nschiefer opened this issue Aug 18, 2020 · 0 comments · Fixed by #1018
Closed

Covering text index scans return tokens, not requested full text strings #1017

nschiefer opened this issue Aug 18, 2020 · 0 comments · Fixed by #1018
Assignees
@nschiefer
Labels
bug Something isn't working

Comments

@nschiefer
Copy link
Contributor

Currently, the planner thinks that full text index entries contain the whole text, when they only contain individual tokens. Therefore, a query that requires the text fields often results in a covering index scan with the "text" being just a single token.
@nschiefer nschiefer self-assigned this Aug 18, 2020
@nschiefer nschiefer added the bug Something isn't working label Aug 18, 2020
@nschiefer nschiefer mentioned this issue Aug 18, 2020
Merged
nschiefer added a commit to nschiefer/fdb-record-layer that referenced this issue Aug 27, 2020
@nschiefer
Resolves FoundationDB#1016: Covering filters on legs of unions/inters…
7657afb 
…ections.

In FoundationDB#1001, we added an optimization that allows filters to be executed
on partial records produced by covering index scans. In FoundationDB#992, we also
added the ability to perform union/intersection/primary key distinct
operations before fetching the base records by primary key.

This refactors the work from FoundationDB#1001 into a substitution visitor that can
work in concert with the union/intersection/distinct visitors. It also
corrects some (untriggered) bugs in the existing visitors.

This also fixes FoundationDB#1017; full text fields now require looking at the
primary records and are not "satisfied" by the tokens in the full text
index entries.
MMcM added a commit that referenced this issue Aug 28, 2020
@nschiefer @MMcM
Resolves #1016: Covering filters on legs of unions/intersections. (#1018
770409d 
)

* Resolves #1016: Covering filters on legs of unions/intersections.

In #1001, we added an optimization that allows filters to be executed
on partial records produced by covering index scans. In #992, we also
added the ability to perform union/intersection/primary key distinct
operations before fetching the base records by primary key.

This refactors the work from #1001 into a substitution visitor that can
work in concert with the union/intersection/distinct visitors. It also
corrects some (untriggered) bugs in the existing visitors.

This also fixes #1017; full text fields now require looking at the
primary records and are not "satisfied" by the tokens in the full text
index entries.

* Respond to review comments.

Co-authored-by: Mike McMahon <mmcm@comcast.net>

Co-authored-by: Mike McMahon <mmcm@comcast.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nschiefer nschiefer

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Resolves #1016: Covering filters on legs of unions/intersections. nschiefer/fdb-record-layer
1 participant
@nschiefer