Skip to content

perf: Optimize SQL string construction#13

Merged
emicklei merged 20 commits into
mainfrom
perf/optimize-sql-building
Aug 21, 2025
Merged

perf: Optimize SQL string construction#13
emicklei merged 20 commits into
mainfrom
perf/optimize-sql-building

Conversation

@emicklei
Copy link
Copy Markdown
Owner

The SQL() function in sql_writing.go was inefficiently building the SQL string. It used an intermediate formatted string created with a bytes.Buffer, and then performed multiple strings.ReplaceAll calls to clean it up.

This change introduces a custom io.Writer implementation, onelineWriter, that builds the final, single-line SQL string in a single pass. It replaces tabs and newlines with spaces and collapses multiple spaces into one on the fly.

This new approach reduces memory allocations and CPU usage, improving the performance of every query built by the library.

The `SQL()` function in `sql_writing.go` was inefficiently building the
SQL string. It used an intermediate formatted string created with a
`bytes.Buffer`, and then performed multiple `strings.ReplaceAll` calls
to clean it up.

This change introduces a custom `io.Writer` implementation,
`onelineWriter`, that builds the final, single-line SQL string in a
single pass. It replaces tabs and newlines with spaces and collapses
multiple spaces into one on the fly.

This new approach reduces memory allocations and CPU usage, improving
the performance of every query built by the library.
@emicklei emicklei requested a review from Copilot August 20, 2025 08:25
@emicklei emicklei marked this pull request as ready for review August 20, 2025 08:25

This comment was marked as outdated.

This commit introduces two significant performance improvements to the pgtalk library.

1.  **Optimized SQL String Construction:**
    The `SQL()` function in `sql_writing.go` was inefficiently building the SQL string using multiple `strings.ReplaceAll` calls. This has been replaced with a custom `io.Writer` implementation (`onelineWriter`) that builds the final, single-line SQL string in a single pass, reducing memory allocations and CPU usage.

2.  **Optimized Result Set Iteration:**
    The `resultIterator.Next()` method was inefficiently matching result columns to selectors using a nested loop for every row. The iteration process has been optimized by pre-ordering the selectors to match the result set's field order. This matching is now done only once, before the iteration begins, significantly improving the performance of row scanning.
@emicklei emicklei requested a review from Copilot August 20, 2025 08:35

This comment was marked as outdated.

google-labs-jules Bot and others added 4 commits August 20, 2025 08:40
This commit introduces two significant performance improvements to the pgtalk library.

1.  **Optimized SQL String Construction:**
    The `SQL()` function in `sql_writing.go` was inefficiently building the SQL string using multiple `strings.ReplaceAll` calls. This has been replaced with a custom `io.Writer` implementation (`onelineWriter`) that builds the final, single-line SQL string in a single pass, reducing memory allocations and CPU usage.

2.  **Optimized Result Set Iteration:**
    The `resultIterator.Next()` method was inefficiently matching result columns to selectors using a nested loop for every row. The iteration process has been optimized by pre-ordering the selectors to match the result set's field order. This matching is now done only once, before the iteration begins, significantly improving the performance of row scanning.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@emicklei emicklei requested a review from Copilot August 20, 2025 08:44

This comment was marked as outdated.

emicklei and others added 3 commits August 20, 2025 10:46
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@emicklei emicklei requested a review from Copilot August 20, 2025 08:48

This comment was marked as outdated.

emicklei and others added 4 commits August 20, 2025 10:51
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@emicklei emicklei requested a review from Copilot August 20, 2025 13:38
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR optimizes SQL string construction by replacing inefficient string manipulation with a custom writer implementation. The optimization improves performance by reducing memory allocations and CPU usage when building SQL queries.

Key changes:

  • Introduces onelineWriter that formats SQL strings in a single pass instead of multiple string replacements
  • Optimizes result iteration by pre-ordering column selectors to avoid nested loops
  • Refactors iterator logic to use ordered selectors for improved performance

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
sql_writing.go Implements onelineWriter and refactors SQL() function for single-pass formatting
queryset.go Pre-orders selectors using a map lookup to optimize result iteration
mutationset.go Applies similar selector ordering optimization for mutation results
iterator.go Updates to use pre-ordered selectors, eliminating nested loops in Next()

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread mutationset.go Outdated
Comment thread mutationset.go Outdated
@emicklei emicklei merged commit c30fb53 into main Aug 21, 2025
1 check passed
@emicklei emicklei deleted the perf/optimize-sql-building branch August 21, 2025 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants