Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[YSQL] Connection scaling #10452

Open
m-iancu opened this issue Nov 1, 2021 · 0 comments
Open

[YSQL] Connection scaling #10452

m-iancu opened this issue Nov 1, 2021 · 0 comments
Assignees
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue roadmap-tracking-issue This issue tracks a major roadmap item, and usually appears in the roadmap list.
Projects
Milestone

Comments

@m-iancu
Copy link
Contributor

m-iancu commented Nov 1, 2021

Jira Link: DB-978
In YSQL, like in vanilla Postgres, each backend connection has some overhead, including to load and maintain the metadata cache for the system catalog tables.
The relative overhead is increased on low-resource machines and geo-distributed cluster when the
This issue tracks tasks to minimize the per-connection overhead for both initializing and maintaining backend connections.

Staging into commits (Release target: v2.16)

Status Feature Github Issue Comments
Functional Spec and Design Doc TBD -- migrate to github
Load pg_depend and pg_shdepend on demand #10433 Reduce RPC requests on connection init time
Improve batching for catalog cache (re)load #10518 Reduce number of per-connection RPCs
Remove YBClient from YSQL/pggate #9936 Add custom PgClient and PgClientService instead
Add tserver-side cache for syscatalog #10821 Connections can (re) load their cache with mostly local requests
🕓 Reduce cache-refresh per-connection memory utilization #14666 Connection cache-refresh over-uses memory as analyzed in YSQL connection memory utilization breakdown
🕓 Pre-load all common catalog tables into metadata cache #14667 We need to pre-load all commonly used catalog tables to avoid ad-hoc metadata loading during query planning/execution
@m-iancu m-iancu added the area/ysql Yugabyte SQL (YSQL) label Nov 1, 2021
@m-iancu m-iancu added this to the 2.13 milestone Nov 1, 2021
@m-iancu m-iancu added this to Backlog in YSQL via automation Nov 1, 2021
@m-iancu m-iancu added the roadmap-tracking-issue This issue tracks a major roadmap item, and usually appears in the roadmap list. label Nov 4, 2021
@bmatican bmatican added the area/docdb YugabyteDB core features label Feb 12, 2022
@yugabyte-ci yugabyte-ci added kind/bug This issue is a bug priority/medium Medium priority issue labels Jun 8, 2022
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature and removed kind/bug This issue is a bug labels Jul 29, 2022
@yugabyte-ci yugabyte-ci removed the area/docdb YugabyteDB core features label Oct 12, 2022
@m-iancu m-iancu modified the milestones: 2.13, 2.16 Oct 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue roadmap-tracking-issue This issue tracks a major roadmap item, and usually appears in the roadmap list.
Projects
Status: No status
YSQL
  
Backlog
Development

No branches or pull requests

5 participants