Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pushdown simplest sorting for find command #2506

Merged
merged 16 commits into from Apr 28, 2023

Conversation

noisersup
Copy link
Member

@noisersup noisersup commented Apr 24, 2023

Description

Closes #2438.

The pushdown for the find command. The aggregation will come soon in other PR, just need to clarify some concerns.

The PR also changes explain command to use internal data to specify if pushdown occured, rather than using Index cond/filter fields from postgres.

Readiness checklist

  • I added/updated unit tests.
  • I added/updated integration/compatibility tests.
  • I added/updated comments and checked rendering.
  • I made spot refactorings.
  • I updated user documentation.
  • I ran task all, and it passed.
  • I ensured that PR title is good enough for the changelog.
  • (for maintainers only) I set Reviewers (@FerretDB/core), Labels, Project and project's Sprint fields.
  • I marked all done items in this checklist.

@codecov
Copy link

codecov bot commented Apr 24, 2023

Codecov Report

Merging #2506 (bc504ad) into main (203467c) will increase coverage by 0.01%.
The diff coverage is 73.94%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2506      +/-   ##
==========================================
+ Coverage   64.34%   64.35%   +0.01%     
==========================================
  Files         403      403              
  Lines       19956    20029      +73     
==========================================
+ Hits        12840    12890      +50     
- Misses       6187     6204      +17     
- Partials      929      935       +6     
Impacted Files Coverage Δ
internal/handlers/pg/pg.go 79.10% <ø> (ø)
internal/handlers/pg/pgdb/query.go 79.53% <69.51%> (-1.99%) ⬇️
internal/handlers/pg/msg_explain.go 67.96% <75.00%> (-0.79%) ⬇️
internal/handlers/pg/msg_find.go 71.96% <83.33%> (-1.26%) ⬇️
internal/handlers/common/sort.go 84.21% <100.00%> (ø)
internal/handlers/pg/msg_aggregate.go 83.18% <100.00%> (ø)
internal/handlers/pg/msg_delete.go 68.33% <100.00%> (ø)
internal/handlers/pg/pgdb/collections.go 67.85% <100.00%> (ø)
internal/handlers/pg/pgdb/database_metadata.go 82.50% <100.00%> (ø)
internal/handlers/registry/registry.go 70.96% <100.00%> (+0.96%) ⬆️

... and 1 file with indirect coverage changes

Flag Coverage Δ
integration 57.17% <47.89%> (-0.12%) ⬇️
mongodb 5.05% <0.00%> (-0.02%) ⬇️
pg 57.10% <47.89%> (-0.12%) ⬇️
unit 25.58% <45.37%> (+0.17%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@mergify
Copy link
Contributor

mergify bot commented Apr 27, 2023

@noisersup this pull request has merge conflicts.

@mergify mergify bot added the conflict PRs that have merge conflicts label Apr 27, 2023
@mergify mergify bot removed the conflict PRs that have merge conflicts label Apr 28, 2023
@noisersup noisersup added the code/chore Code maintenance improvements label Apr 28, 2023
@noisersup noisersup changed the title Pushdown simplest sorting with ORDER BY Pushdown simplest sorting for find command Apr 28, 2023
@noisersup noisersup marked this pull request as ready for review April 28, 2023 10:29
@noisersup noisersup requested a review from a team as a code owner April 28, 2023 10:29
@noisersup noisersup enabled auto-merge (squash) April 28, 2023 10:29
@noisersup noisersup requested review from a team and w84thesun April 28, 2023 10:35
w84thesun
w84thesun previously approved these changes Apr 28, 2023
Copy link
Contributor

@w84thesun w84thesun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

rumyantseva
rumyantseva previously approved these changes Apr 28, 2023
Copy link
Member

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea and implementation looks good to me. To unblock you for aggregation implementation, I approve this PR.

To add in the second PR (apart from aggregation):

  • a (unit) test for iter, queryRes, err := QueryDocuments(ctxGet, tx, qp) that queryRes is returned correctly
  • check code cov if branches for sort pushdown are covered

@rumyantseva
Copy link
Member

(Please fix the failing tests)

@noisersup noisersup dismissed stale reviews from rumyantseva and w84thesun via d34e056 April 28, 2023 14:15
@noisersup
Copy link
Member Author

It seems that integration is not panicing right now, I've also added unit tests.

rumyantseva
rumyantseva previously approved these changes Apr 28, 2023
Copy link
Member

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally, looks good to me.
One tiny suggestion - as it costs nothing to include the info about sort pushdown to explain, let's add it.

internal/handlers/pg/msg_explain.go Outdated Show resolved Hide resolved
w84thesun
w84thesun previously approved these changes Apr 28, 2023
@noisersup noisersup dismissed stale reviews from w84thesun and rumyantseva via bc504ad April 28, 2023 15:39
Copy link
Member

@rumyantseva rumyantseva left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

I tried it with and without the flag, works as expected!

test> db.foo.find().sort("hello").explain()
{
  queryPlanner: {
    Plan: {
      'Node Type': 'Sort',
      'Parallel Aware': false,
      'Startup Cost': 97.78,
      'Plan Width': 64,
      Plans: [
        {
          'Parent Relationship': 'Outer',
          'Parallel Aware': false,
          'Async Capable': false,
          Schema: 'test',
          Alias: 'foo_a9f37ed7',
          Output: [ '_jsonb', "(_jsonb -> 'hello'::text)" ],
          'Node Type': 'Seq Scan',
          'Startup Cost': 0,
          'Total Cost': 27,
          'Plan Rows': 1360,
          'Plan Width': 64,
          'Relation Name': 'foo_a9f37ed7'
        }
      ],
      'Async Capable': false,
      'Total Cost': 101.18,
      'Plan Rows': 1360,
      Output: [ '_jsonb', "((_jsonb -> 'hello'::text))" ],
      'Sort Key': [ "((foo_a9f37ed7._jsonb -> 'hello'::text))" ]
    }
  },
  explainVersion: '1',
  command: { find: 'foo', filter: {}, sort: { hello: 1 }, '$db': 'test' },
  pushdown: false,
  sortingPushdown: true,
  serverInfo: {
    host: 'ferretdb',
    version: '6.0.42',
    gitVersion: 'bc504ad73fbe30deb7096cc82d76f6ba8370cbaa',
    ferretdbVersion: 'v1.0.0-95-gbc504ad7-dirty'
  },
  ok: 1
}

@noisersup noisersup added no ci and removed no ci labels Apr 28, 2023
@ptrfarkas ptrfarkas removed the request for review from chilagrow April 28, 2023 16:51
@ptrfarkas ptrfarkas disabled auto-merge April 28, 2023 16:51
@ptrfarkas ptrfarkas merged commit 9b05371 into FerretDB:main Apr 28, 2023
38 of 45 checks passed
@noisersup noisersup deleted the pushdown-sort-2438 branch April 28, 2023 16:52
@AlekSi AlekSi added this to the v1.1.0 milestone May 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code/chore Code maintenance improvements
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Pushdown simplest sorting with ORDER BY
5 participants