Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YSQL] [PERF] - DocDB should seek and send requested data partially in rowsets of prefetch-limit size #2171

Open
nocaway opened this issue Aug 27, 2019 · 1 comment
Open

[YSQL] [PERF] - DocDB should seek and send requested data partially in rowsets of prefetch-limit size #2171

nocaway opened this issue Aug 27, 2019 · 1 comment
Assignees
@sushantrmishra
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
YSQL

Comments

@nocaway
Copy link
Contributor

nocaway commented Aug 27, 2019 •
edited by yugabyte-ci

Jira Link: DB-4835
Currently, DocDB in each TServer would load all requested data from disk and then send response to proxy server or client. For large set of data, reading from disk could take a long time, so the proxy-server & client would be idle for a long time and therefore hurt our performance.

To fix this performance issue, we could try the following.

  • Read 1024 rows from disk (The default value for GFLAG ysql_prefetch_limit is 1024 rows)
  • Send 1024 rows to client.
  • Immediately read the next 1024 rows from disk while data is being sent over the network and while proxy server & client are processing the previous 1024 rows.
@nocaway nocaway added the kind/enhancement This is an enhancement of an existing feature label Aug 27, 2019
@nocaway nocaway self-assigned this Aug 27, 2019
@nocaway nocaway added this to To do in YSQL via automation Aug 27, 2019
@nocaway nocaway moved this from To do to In progress in YSQL Aug 27, 2019
@nocaway nocaway changed the title [YSQL] Performance issue - DocDB should send data while loading from disk [YSQL] [PERF] - DocDB should send data while loading from disk Aug 28, 2019
@nocaway
Copy link
Contributor Author

nocaway commented Aug 28, 2019

'cc @m-iancu @mbautin

@nocaway nocaway changed the title [YSQL] [PERF] - DocDB should send data while loading from disk [YSQL] [PERF] - DocDB should seek and send requested data partially in smaller sets Aug 29, 2019
@nocaway nocaway changed the title [YSQL] [PERF] - DocDB should seek and send requested data partially in smaller sets [YSQL] [PERF] - DocDB should seek and send requested data partially in rowsets of prefetch-limit size Aug 29, 2019
@rthallamko3 rthallamko3 added the area/ysql Yugabyte SQL (YSQL) label Jan 3, 2023
@yugabyte-ci yugabyte-ci added the priority/medium Medium priority issue label Jan 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sushantrmishra sushantrmishra

Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
YQL-beta
Status: No status
YSQL
  
In progress
Milestone
No milestone
Development

No branches or pull requests
4 participants
@sushantrmishra @rthallamko3 @yugabyte-ci @nocaway