Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support simple query pushdown for Postgres #896

Closed
seeforschauer opened this issue Jul 15, 2022 · 1 comment · Fixed by #1207
Closed

Support simple query pushdown for Postgres #896

seeforschauer opened this issue Jul 15, 2022 · 1 comment · Fixed by #1207
Assignees
Labels
code/feature Some user-visible feature is not implemented yet
Milestone

Comments

@seeforschauer
Copy link
Contributor

seeforschauer commented Jul 15, 2022

Support simple query pushdown

The goal of this task is to enable simple query pushdown for querier that look like {_id: <ObjectID>}.
Only that field name (_id) is supported, and only that value type (ObjectID).

In the future, we will add support for other fields (starting with simple scalar fields),
other values (starting with strings, numbers, and other simple scalar values),
other conditions like {_id: <ObjectID>, field: value} (so only the first part is pushed down).

Postgres

If a field name is different, value type is not ObjectID, or some other condition is present,
we should use the previous version of the code that SELECTs the whole table without any WHERE condition.

If the value type is ObjectID but the data in the value is somehow corrupted, raise an error (fjson unmarshal).

If those conditions are met, we send a SELECT query with a WHERE condition.

Proof of concept for a {_id: <ObjectID>} pushdown query, PostgreSQL:

GIN index is not used by PostgreSQL in any queries below.
So let's use the first one just because it is first.

-- this will not use these indexes:
-- CREATE INDEX values_id_idx ON public."values" USING gin ((((_jsonb -> '_id'::text) -> '$o'::text)))
-- CREATE INDEX values_id_idx ON values (((_jsonb ->> '_id')::text));
select * from values where (_jsonb->'_id')::jsonb->>'$o' = '507f1f77bcf86cd799439011'; -- no, seq scan will it use index? not that indexes

Test

Integration tests: Provide a new test flag that enables a query pushdown.
Run with and without flag: results must be the same.
When explain feature will be ready, compare the query plan.

Links

Analysis task: #847

@seeforschauer seeforschauer added the code/feature Some user-visible feature is not implemented yet label Jul 15, 2022
@FranckPachot
Copy link

GIN index is not used by PostgreSQL in any queries below.
So let's use the first one just because it is first.

I'm not sure I understand this but, just in case, the following index works in PostgreSQL and YugabyteDB:

yugabyte=# create table values (_jsonb jsonb);
CREATE TABLE
yugabyte=# CREATE INDEX values_id_idx ON values (((_jsonb -> '_id'::text) ->> '$o'::text));
CREATE INDEX
yugabyte=# explain
yugabyte-# select * from values where (_jsonb->'_id')::jsonb->>'$o' = '507f1f77bcf86cd799439011';
                                         QUERY PLAN
---------------------------------------------------------------------------------------------
 Index Scan using values_id_idx on "values"  (cost=0.00..5.27 rows=10 width=32)
   Index Cond: (((_jsonb -> '_id'::text) ->> '$o'::text) = '507f1f77bcf86cd799439011'::text)
(2 rows)

It is not a GIN index because it is a single value. But the indexed expression must match the one used in the equality predicate, like here.

@AlekSi AlekSi self-assigned this Aug 22, 2022
@AlekSi AlekSi added this to the v0.6.0 Alpha milestone Aug 22, 2022
@AlekSi AlekSi modified the milestones: v0.5.3, v0.5.4 Sep 8, 2022
@AlekSi AlekSi modified the milestones: v0.5.4, v0.6.0 Alpha Sep 22, 2022
@AlekSi AlekSi assigned w84thesun and unassigned AlekSi Oct 3, 2022
w84thesun pushed a commit that referenced this issue Oct 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code/feature Some user-visible feature is not implemented yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants