-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sql: PlaceholderInfo should use slices indexed by placeholder name instead of maps #30086
Labels
A-sql-pgwire
pgwire protocol issues.
C-performance
Perf of queries or internals. Solution not expected to change functional behavior.
Comments
nvanbenschoten
added
C-performance
Perf of queries or internals. Solution not expected to change functional behavior.
A-sql-pgwire
pgwire protocol issues.
labels
Sep 11, 2018
Hah, to add insult to injury, these lines are showing up in heap profiles: cockroach/pkg/sql/conn_executor_prepare.go Lines 306 to 307 in 26bbb77
and cockroach/pkg/sql/conn_executor_prepare.go Line 334 in 26bbb77
|
This actually is also a crashing bug in Cockroach:
|
RaduBerinde
added a commit
to RaduBerinde/cockroach
that referenced
this issue
Jan 4, 2019
This change modifies `tree.Placeholder` to store an integer ID (between 1 and 65535) instead of a string. This saves silly allocations in pgwire, and in a future change we can switch the PlaceholderInfo maps to slices. Informs cockroachdb#30086. Release note: None
RaduBerinde
added a commit
to RaduBerinde/cockroach
that referenced
this issue
Jan 4, 2019
This change modifies `tree.Placeholder` to store an integer ID (between 1 and 65535) instead of a string. This saves silly allocations in pgwire, and in a future change we can switch the PlaceholderInfo maps to slices. Informs cockroachdb#30086. Release note: None
RaduBerinde
added a commit
to RaduBerinde/cockroach
that referenced
this issue
Jan 5, 2019
This change modifies `tree.Placeholder` to store an integer ID (between 0 and 65535) instead of a string. This saves silly allocations in pgwire, and in a future change we can switch the PlaceholderInfo et al maps to slices. Informs cockroachdb#30086. Release note: None
RaduBerinde
added a commit
to RaduBerinde/cockroach
that referenced
this issue
Jan 5, 2019
This change modifies `tree.Placeholder` to store an integer ID (between 0 and 65535) instead of a string. This saves silly allocations in pgwire, and in a future change we can switch the PlaceholderInfo et al maps to slices. Informs cockroachdb#30086. Release note: None
RaduBerinde
added a commit
to RaduBerinde/cockroach
that referenced
this issue
Jan 6, 2019
This change modifies `tree.Placeholder` to store an integer ID (between 0 and 65535) instead of a string. This saves silly allocations in pgwire, and in a future change we can switch the PlaceholderInfo et al maps to slices. Informs cockroachdb#30086. Release note: None
craig bot
pushed a commit
that referenced
this issue
Jan 6, 2019
33515: sql: use integer placeholder IDs r=RaduBerinde a=RaduBerinde This change modifies `tree.Placeholder` to store an integer ID (between 1 and 65535) instead of a string. This saves silly allocations in pgwire, and in a future change we can switch the PlaceholderInfo maps to slices. Informs #30086. Release note: None Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
RaduBerinde
added a commit
to RaduBerinde/cockroach
that referenced
this issue
Jan 15, 2019
We switch `PlaceholderTypes` to a slice instead of a map. We also fix the handling of cases where some placeholders are unused (e.g. `SELECT $2:::int`) which now error out (before they would crash during execution). Note that PG also errors out in this case. Fixes cockroachdb#30086. Release note (bug fix): Preparing queries with missing placeholders (e.g. `SELECT $2::int`) now results in an error.
craig bot
pushed a commit
that referenced
this issue
Jan 16, 2019
33716: sql: switch QueryArguments, PlaceholderTypes to slices r=RaduBerinde a=RaduBerinde #### sql: change QueryArguments map to a slice Release note: None #### sql: switch PlaceholderTypes to a slice We switch `PlaceholderTypes` to a slice instead of a map. We also fix the handling of cases where some placeholders are unused (e.g. `SELECT $2:::int`) which now error out (before they would crash during execution). Note that PG also errors out in this case. Fixes #30086. Release note (bug fix): Preparing queries with missing placeholders (e.g. `SELECT $2::int`) now results in an error. Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
A-sql-pgwire
pgwire protocol issues.
C-performance
Perf of queries or internals. Solution not expected to change functional behavior.
CockroachDB and PostgreSQL both mandate that all placeholder names are parsable as uint64 values. CockroachDB enforces that this value fits within an int64. Postgres doesn't allow more than 65535 parameters because it uses 16-bit integers for the parameter count in its wire protocol. Furthermore, Postgres does not allow sparse placeholder names:
We should adopt this behavior as well.
This strongly implies that we should be storing placeholder names as
int16
values instead ofstring
values. More importantly, it implies thatPlaceholderInfo
should replace itsmap[string]T
maps with slices that can be indexed directly by placeholder name.This has real performance implications. While running
workload init tpcc --warehouses=1000
, which performs a large number of batched inserts, I saw that placeholder manipulation was responsible for about 7% of CPU utilization. Most of this time was spent dealing withPlaceholderInfo
maps. We should be able to eliminate this overhead almost entirely with a few small changes to how we store placeholders.Assigning @jordanlewis for triage.
The text was updated successfully, but these errors were encountered: