2.3.3

Merged PRs

dolt

• 11757: go/store/nbs: byte_sink.go: Fix an error in non-archive push and GC paths which could result in table files with missing indexes.
  Failure to properly handle a boundary condition in BufferedFileByteSink meant that a table file writer that was producing a large table file could end up writing it out with a broken index. All chunk data made it to the file, but the file itself was unusable without manual recovery because the index portion of the file was wrong.
• 11746: Added DESC indexes and NULL ordering
  This implements DESC indexes and NULL ordering (nulls can be set to be first or last). NULL ordering is primarily used by Doltgres. This is a feature version bump for Dolt because, now that we support DESC indexes, old databases trying to read a DESC index would read it incorrectly and return incorrect results.
  Related PRs:
  • dolthub/vitess#491
  • dolthub/go-mysql-server#3820
  • dolthub/doltgresql#3285
• 11745: Coalesce git remote read fetches, answer absent keys from the cache.
  A cache-missing Get or Exists ran its own git fetch of the whole data ref, so a burst of concurrent readers opened a connection each and every probe for a key that does not exist fetched again. Concurrent read-path fetches now share one fetch, and a key missing from a cache already merged from a commit is reported absent without fetching. A fetch that finds the head unchanged now also refreshes the dedup window, which previously never renewed against a quiet remote.
• 11744: go: git remotes: kill the process group on cancel, bound the wait on inherited pipes.
  A cancelled DOLT_PUSH/DOLT_FETCH wedged sql-server indefinitely (#11741): exec.CommandContext's default Cancel kills only the direct git child, so the transport it spawned survives holding the write end of the pipe Run reads from, and cmd.Wait() waits on an EOF nobody will send. Set cmd.Cancel to kill the whole group -- CmdSetsid already makes git the group leader, and Windows shells out to taskkill /T -- and set cmd.WaitDelay as a backstop for a holder that left the group, such as an ssh ControlMaster. Cancellation and expired-wait errors now report ctx.Err() rather than a bare kill signal, and skip the credential hints NormalizeError appends.
• 11743: integration-tests/mysql-client-tests/Dockerfile: Bump java_clients_build off EOL Debian bullseye to maven:3.9-eclipse-temurin-17-noble.
• 11742: go/store/nbs: Clone table file sources which are already open on Conjoin, instead of always opening them anew.
• 11710: Fix misleading paths in multiple .doltcfg error message
  Fix #10688
• 11697: Report branch update, force push, and tag status on dolt push
  Dolt now prints standard status messages when pushing branches and tags to a remote database. Update display the commit transition range old...new, and when forced are explicitly marked (forced update).
  • Inspects the remote reference state before transferring chunks in actions.Push. to capture OldHash, NewHash, and existence.
  • Add PushRefResult to distinguish new branches, fast-forward, updates, forced updates, deletions, and tags.
  • Add formatPushSuccess to format push success messages.
    Fix #11352
    Close #11590
• 11693: Name the conflicting ref in ExistingRefError
  Include the full conflicting ref in ExistingRefError messages, and give tag and workspace creation their own already-exists errors.
  Fixes #11434
  Close #11708
• 11692: build(deps): bump google.golang.org/grpc from 1.82.1 to 1.83.1 in /go
  Bumps google.golang.org/grpc from 1.82.1 to 1.83.1.
  Release notes

  Sourced from google.golang.org/grpc's releases.

  Release 1.83.1

  Security

  • xds/rbac: Fix a bug where nested Principal or Permission rules with :scheme or grpc- prefixed header matchers were not rejected, which could cause DENY rules to fail open. (#9258)
    • Special Thanks: @​nvxbug
  • xds/rbac: Fix a bug where the host header matcher was not being replaced with :authority in nested Principal or Permission rules. (#9258)
    • Special Thanks: @​nvxbug
  • xds/rbac: Fix a bug where a header matcher whose name was not lowercase, such as X-Role, matched no header, which could cause DENY rules to fail open. (#9332)
    • Special Thanks: @​alimony
  • xds/rbac: Fix a bug where a :scheme or grpc- prefixed header matcher was accepted when its name was not lowercase. (#9332)
    • Special Thanks: @​alimony
  • xds/rbac: Fix a bug where a Host header matcher was not replaced with :authority. (#9332)
    • Special Thanks: @​alimony

  Performance

  • transport: Restrict memory overhead of buffering small data frames. (#9331)

  Release 1.83.0

  Security

  • server: Stop reading from connections when flooded by HTTP/2 frames to mitigate resource exhaustion. The default value for this limit is 100 frames, excluding DATA and HEADERS, and may be changed by setting environment variable GRPC_GO_EXPERIMENTAL_CONTROL_BUFFER_THROTTLE_LIMIT.
  • xds/rbac: Support Metadata and RequestedServerName permissions matcher fields. If present in a DENY rule, previously these would be ignored and fail-open.
  • xds/rbac: Fix panic when parsing unsupported fields in NotRule/NotId permissions.
  • xds/rbac: Support the deprecated source_ip principal identifier by treating it as equivalent to direct_remote_ip.
  • xds: Fix panic when parsing route header matchers configured with empty exact_match, prefix_match, or suffix_match strings. (#9223)

  New Features

  • xds/googlec2p: Enable DirectPath over Interconnect support for on-premises clients via the force-xds target URI query parameter. (#9133)
  • xds: Enable xDS configuration to control which fields get propagated from ORCA backend metric reports to LRS load reports. (#9145)
  • authz: Add OnPolicyUpdate callback to FileWatcherOptions to notify when an authz policy is loaded or updated. (#9142)
    • Special Thanks: @​hnefatl
  • xds: Add support for the GCP Authentication HTTP Filter, which automatically fetches and attaches GCP Service Account Identity JWT tokens to outgoing RPCs.
    • This feature can be enabled by setting environment variable GRPC_EXPERIMENTAL_XDS_GCP_AUTHENTICATION_FILTER=true. (#9119)
  • xds: Add support for xDS-based HTTP CONNECT proxies.
    • This feature can be enabled by setting environment variable GRPC_EXPERIMENTAL_XDS_HTTP_CONNECT=true. (#9151)
  • xds: Add support for contains_match in route header matchers. (#9223)

  Bug Fixes

  • credentials/alts: Fix panic when processing malformed frames by validating that the message frame length exceeds the message type field size. (#9197)
  • grpc: Fix compilation on Plan 9 targets (GOOS=plan9), broken since v1.81.0. (#9255)
    • Special Thanks: @​Yusufihsangorgel

  Release 1.82.2

  Security

  • server: Reject requests missing both :authority and Host headers with HTTP 400 and status Internal. (grpc/grpc-go#9365)

2.3.2

Merged PRs

dolt

• 11665: go: sqle/resolve: Have SearchPath() parse the search path in a way which is more compliant with postgres.
  Correctly handle quoted identifiers and ToLower any unquoted ones.
• 11659: build(deps): bump google.golang.org/grpc from 1.82.1 to 1.83.1 in /integration-tests/go-sql-server-driver
  Bumps google.golang.org/grpc from 1.82.1 to 1.83.1.
  Release notes

  Sourced from google.golang.org/grpc's releases.

  Release 1.83.1

  Security

  • xds/rbac: Fix a bug where nested Principal or Permission rules with :scheme or grpc- prefixed header matchers were not rejected, which could cause DENY rules to fail open. (#9258)
    • Special Thanks: @​nvxbug
  • xds/rbac: Fix a bug where the host header matcher was not being replaced with :authority in nested Principal or Permission rules. (#9258)
    • Special Thanks: @​nvxbug
  • xds/rbac: Fix a bug where a header matcher whose name was not lowercase, such as X-Role, matched no header, which could cause DENY rules to fail open. (#9332)
    • Special Thanks: @​alimony
  • xds/rbac: Fix a bug where a :scheme or grpc- prefixed header matcher was accepted when its name was not lowercase. (#9332)
    • Special Thanks: @​alimony
  • xds/rbac: Fix a bug where a Host header matcher was not replaced with :authority. (#9332)
    • Special Thanks: @​alimony

  Performance

  • transport: Restrict memory overhead of buffering small data frames. (#9331)

  Release 1.83.0

  Security

  • server: Stop reading from connections when flooded by HTTP/2 frames to mitigate resource exhaustion. The default value for this limit is 100 frames, excluding DATA and HEADERS, and may be changed by setting environment variable GRPC_GO_EXPERIMENTAL_CONTROL_BUFFER_THROTTLE_LIMIT.
  • xds/rbac: Support Metadata and RequestedServerName permissions matcher fields. If present in a DENY rule, previously these would be ignored and fail-open.
  • xds/rbac: Fix panic when parsing unsupported fields in NotRule/NotId permissions.
  • xds/rbac: Support the deprecated source_ip principal identifier by treating it as equivalent to direct_remote_ip.
  • xds: Fix panic when parsing route header matchers configured with empty exact_match, prefix_match, or suffix_match strings. (#9223)

  New Features

  • xds/googlec2p: Enable DirectPath over Interconnect support for on-premises clients via the force-xds target URI query parameter. (#9133)
  • xds: Enable xDS configuration to control which fields get propagated from ORCA backend metric reports to LRS load reports. (#9145)
  • authz: Add OnPolicyUpdate callback to FileWatcherOptions to notify when an authz policy is loaded or updated. (#9142)
    • Special Thanks: @​hnefatl
  • xds: Add support for the GCP Authentication HTTP Filter, which automatically fetches and attaches GCP Service Account Identity JWT tokens to outgoing RPCs.
    • This feature can be enabled by setting environment variable GRPC_EXPERIMENTAL_XDS_GCP_AUTHENTICATION_FILTER=true. (#9119)
  • xds: Add support for xDS-based HTTP CONNECT proxies.
    • This feature can be enabled by setting environment variable GRPC_EXPERIMENTAL_XDS_HTTP_CONNECT=true. (#9151)
  • xds: Add support for contains_match in route header matchers. (#9223)

  Bug Fixes

  • credentials/alts: Fix panic when processing malformed frames by validating that the message frame length exceeds the message type field size. (#9197)
  • grpc: Fix compilation on Plan 9 targets (GOOS=plan9), broken since v1.81.0. (#9255)
    • Special Thanks: @​Yusufihsangorgel

  Release 1.82.2

  Security

  • server: Reject requests missing both :authority and Host headers with HTTP 400 and status Internal. (grpc/grpc-go#9365)

  ... (truncated)

  Commits

  • 1550d9e Change version to 1.83.1 (#9336)
  • ebba6f3 Cherry-pick #9258 and #9332 into v1.83.x (#9335)
  • 8cfeca0 Cherry-pick #9331 to v1.83.x (#9333)
  • dec6951 Change version to 1.83.1-dev (#9229)
  • 4c226da Change version to 1.83.0 (#9228)
  • c198988 Cherrypick 9223 into v1.83.x (#9279)
  • 8ce3ebf Cherrypick PR 9255 into v1.83.x (#9263)
  • e393849 Cherry-pick recent changes from master (#9240)
  • 2a112a8 authz: add onPolicyUpdate callback to authz file watcher (#9142)
  • 1a80fca vet: adds a check to disallow usage of regex.Compile in xDS code (#9216)
  • Additional commits viewable in compare view
  
  [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
  Dependabot commands and options

  
  You can trigger Dependa...

2.3.1

Merged PRs

dolt

• 11568: build(deps): bump undici to 6.28.0

  Summary

  • bump the transitive undici dependency from 6.24.0 to 6.28.0 in the SES email action lockfile
  • resolve Dependabot alerts #137, #138, #139, #140, #171, #172, and #173

  Validation

  • npm ci --ignore-scripts
  • npm run build
  • npm audit --omit=dev (the undici advisories are resolved; the remaining fast-xml-builder finding is covered by #11022)
    [no-release-notes]
• 11567: /.github/workflows: pin add-and-commit
• 11556: archives: parallelize/coalesce chunk fetches
  There are a fair number of unit tests are added, but I've also had this verified by the user that raised the issue initially. Local testing with some added trace further convinced me this works as expected.
• 11537: backup: stop writing the working set on sync
  dolt_backup no longer commits the calling session's open transaction before it copies.
  • Syncing an idle database leaves the backup unchanged.
  • ROLLBACK after sync now works correctly.
    Fix #11488
• 11335: Bump google.golang.org/grpc from 1.79.3 to 1.82.1 in /go
  Bumps google.golang.org/grpc from 1.79.3 to 1.82.1.
  Release notes

  Sourced from google.golang.org/grpc's releases.

  Release 1.82.1

  Security

  • server: Stop reading from the connection when flooded by HTTP/2 frames. The default value for this limit is 100 frames, excluding DATA and HEADERS, and may be changed by setting environment variable GRPC_GO_EXPERIMENTAL_CONTROL_BUFFER_THROTTLE_LIMIT.
  • xds/rbac: Support Metadata and RequestedServerName permissions matcher fields. If present in a DENY rule, previously these would be ignored and fail-open.
  • xds/rbac: Fix panic when parsing unsupported fields in NotRule/NotId permissions.
  • xds/rbac: Support the deprecated source_ip principal identifier by treating it as equivalent to direct_remote_ip.

  Release 1.82.0

  Behavior Changes

  • server: Remove support for GRPC_GO_EXPERIMENTAL_DISABLE_STRICT_PATH_CHECKING environment varibale. Strict incoming RPC path validation (which has been the default since v1.79.3) can no longer be disabled. (#9112)
  • transport: Add environment variable to change the default max header list size from 16MB to 8KB. This may be enabled by setting GRPC_GO_EXPERIMENTAL_ENABLE_8KB_DEFAULT_HEADER_LIST_SIZE=true. This will be enabled by default in a subsequent release. (#9019)
  • balancer: Load Balancing policy registry is now case-sensitive. Set GRPC_GO_EXPERIMENTAL_CASE_SENSITIVE_BALANCER_REGISTRIES=false (and file an issue) to revert to case-insensitive behavior. (#9017)

  New Features

  • experimental/stats: Expose a new API, NewContextWithLabelCallback, to register a callback that is invoked when telemetry labels are added. (#8877)
    • Special Thanks: @​seth-epps
  • client: Return a portion of the response body in the error message, when the client receives an unexpected non-gRPC HTTP response, to make debugging easier. (#8929)
    • Special Thanks: @​chengxilo
  • server: Add environment variable GRPC_GO_SERVER_GOROUTINE_LABELS that controls setting runtime/pprof.Labels on goroutines spawned by the server. Set GRPC_GO_SERVER_GOROUTINE_LABELS=grpc.method=true to add the grpc.method label on goroutines spawned to handle incoming requests. (#9082)
    • Special Thanks: @​dfinkel

  Bug Fixes

  • xds/server: Fix a memory leak of HTTP filter instances occurring when route configurations are updated in-place during a Route Discovery Service (RDS) update. (#9138)
  • grpc: In the deprecated gzip Compressor (used via the deprecated WithCompressor dial option), enforce the MaxRecvMsgSize limit on the decompressed message buffer, preventing excessive memory allocation from highly compressed payloads. (#9114)
    • Special Thanks: @​evilgensec
  • stats/opentelemetry: Record retry attempts, grpc.previous-rpc-attempts, at the call level and not the attempt level. (#8923)
  • encoding: Ensure Close() is always called on readers returned from Compressor.Decompress if possible. (#9135)
  • channelz: Fix the LastMessageSentTimestamp and LastMessageReceivedTimestamp fields in SocketMetrics to ensure they contain correct timestamp values. (#9109)

  Release 1.81.1

  Security

  • xds/rbac: Fix a potential authorization bypass caused by incorrectly falling through URI/DNS SANs to Subject Distinguished Name (DN) when matching the authenticated principal name. With this fix, only the first non-empty identity source will be used, as per gRFC A41. (#9111)
    • Special Thanks: @​al4an444

  Bug Fixes

  • otel: Segregate client and server RPC information used for metrics and traces, to avoid one overwriting the other. (#9081)

  Release 1.81.0

  Behavior Changes

  • balancer/rls: Switch gauge metrics to asynchronous emission (once per collection cycle) to reduce telemetry noise and align with other gRPC language implementations. (#8808)

  Dependencies

  ... (truncated)

  Commits

  • ebd8f06 Change version to 1.82.1 (#9238)
  • 4ea465d Cherry-pick commits (#9236)
  • 9494a2c Change version to 1.82.1-dev (#9171)
  • bd23985 Change version to 1.82.0 (#9170)
  • 0f3086d Fix minor issues not covered by PR #9137 (#9147)
  • fef07fb internal: Split v3procservicepb import into pb and grpc for extproc (#9163)
  • 91dd64f transport: surface subsequent data when receiving non-gRPC header (#8929)
  • adc97de test/kokoro: add config for regional-td test (#9158)
  • 57c9ff1 xds: ensure full-string matching for RBAC Filter rules (#9148)
  • b58f32d server: Set a pprof label on new stream goroutines (

1.88.2

This is not the latest Dolt release. This is a patch release against 1.88.1 for users that want to stay on 1.x.

2.3.0

Merged PRs

dolt

• 11534: go: remotesrv: http: Authenticate incoming requests against the correct remote peer address.
  For authentication, this previously defaulted to localhost, while still applying authorization on the true remote peer address.
• 11481: go: store/nbs,dprocedures_backup: Add dolt backup sync{,-url} --prune-with-grace-period.
  Backups into a destination store land new table files there before they update the manifest of the destination store to refer to the files. If the backup fails partially through, it can leave behind temporary files and unreferenced table files.
  --prune-with-grace-period is a new flag which can be used when calling backup sync and backup sync-url. It will cause all unreferenced table files in the destination store to be removed, before the sync starts, if no files in the destination store have an mtime more recent than the provided grace period.
  In general, --prune-with-grace-period should be larger than the amount of time it takes to land the backup, but smaller than the period of time between backup attempts.
  Fixes #11361

Closed Issues

• 11361: dolt backup sync: an interrupted sync leaves a complete, unreferenced archive in the destination forever — the archive is published before the manifest and there is no prune

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	17.01	2.39	0.14
groupby_scan	134.9	61.08	0.45
index_join	3.43	1.93	0.56
index_join_scan	4.25	1.32	0.31
index_scan	344.08	196.89	0.57
oltp_point_select	0.19	0.25	1.32
oltp_read_only	3.62	5.0	1.38
select_random_points	0.36	0.51	1.42
select_random_ranges	0.38	0.64	1.68
table_scan	344.08	200.47	0.58
types_table_scan	746.32	458.96	0.61
reads_mean_multiplier			0.82

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	7.7	6.21	0.81
oltp_insert	4.1	3.13	0.76
oltp_read_write	8.74	11.24	1.29
oltp_update_index	4.33	3.3	0.76
oltp_update_non_index	4.1	3.02	0.74
oltp_write_only	5.18	6.21	1.2
types_delete_insert	8.28	6.67	0.81
writes_mean_multiplier			0.91

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	95.46	52.64	1.81
tpcc_tps_multiplier			1.81

Overall Mean Multiple	1.18

2.2.4

Merged PRs

dolt

• 11528: branch: fix -d when the upstream does not resolve
  Deleting a branch with dolt branch -d could fail with branch not found when deleted from a remote. Dolt now compares against the local snapshot of the upstream, not the remote repository, and falls back to the current branch when its copy is gone.
  • Add env.UpstreamRef to resolve a branch's upstream local ref from BranchConfig.
  • Add upstreamOrHead to decide when to use the upstream branch or switch to current working branch.
  • Upstream is searched by the configured merge ref in the BranchConfig to support different names.
  • Merged predicate reads the remote-tracking ref instead of the live remote state.
  • RemoteDbProvider parameter dropped because it goes unused.
    Fix #11450
• 11520: Look up table names directly
  Change table name lookups to try directly looking up the table name, instead of always iterating over all table names in the database.
• 11515: Use sql.ErrDuplicateCheckName
  We should use the same error in GMS in Dolt.
  Depends on: dolthub/go-mysql-server#3691
• 11486: Bug fixes for binlog replication with virtual columns
  Adds support for correctly handling virtual columns and functional indexes in binlog replication, both when Dolt is acting as the replication primary and when Dolt is acting as a replica.
  Fixes: #11475
• 11443: Fix dump of tables with a generated column
  Fixes a panic in dolt dump on tables with a generated column which affected sql, csv, and parquet output. The generated INSERT statements also were updated to omit generated columns, addressing STORED generated columns non-restorable dumps.
  • mvdata/engine_table_reader.go: NewSqlEngineReader took its rows from a SELECT statement but its schema from a separate SHOW CREATE TABLE plan that was built but never ran. When a table has a generated column, the planbuilder replaces it with plan.VirtualColumnTable that does not implement sql.PrimaryKeyTable, the interface that plan uses to fill its PrimaryKeySchema. The schema now comes from the same query as the rows to avoid a zero value.
  • NewSqlEngineTableReaderWithEngine became same as the above and now delegates.
  • sqlfmt/row_fmt.go: InsertStatementPrefix and SqlRowAsTupleString both skip generated columns, keeping the column list and the value tuple the same length.
  • SqlRowAsTupleString returns an error on a row and schema width mismatch.
    Fix #11439
• 11430: go: store/nbs: reject a file manifest update that references a missing table file
  A bare local store takes no cross-process lock while a table file is landing, so between the moment a writer opens or renames a table file and the moment it commits a manifest naming it, another process can unlink it. The writer does not notice, since its own file descriptor keeps working, and it publishes a manifest that is broken for everyone else.
  Add a sanity check where we Stat the newly added files under the manifest LOCK. This allows other Dolt processes which want to safely unlink unreferenced files from the directory to do so under the same LOCK. While a write may transiently fail if an about-to-be-referenced file is deleted out from under it, it won't cause database corruption by writing a manifest which references a non-existant table file.
  This PR adds the add-files sanity check. It does not add the machinery to safely remove any files yet.
  Journaled stores are unaffected: ChunkJournal goes through journalManifest, not fileManifest.
• 11406: bug fix for nonlocal tables to work in Doltgres
• 11400: go: remotesrv: Move away from deprecated http2/h2c package. Use SetUnencryptedHTTP2 instead.
  Also adds "h2" as a supported ALPN next protocol to the remotesrv server listener when configuring with TLS.
  Fixes #11386
• 11385: /go/store/{datas,nbs}: cleanup byte sink files on error
  Currently there are some paths where temporary byte sink files may persist on disk despite an error. This PR aims to ensure these are always deleted in such cases.
• 11377: Enable signed.bats with isolated GNUPGHOME
• 11374: bug fixes for schema names in table statistics
  This is necessary to get doltgres to use table statistics.
  Also a bug fix for the declared column type of the dolt_statistics table, to match what GMS actually returns.
• 11368: Bug fix for lookup joins with extended types
  Regression tests included in: dolthub/doltgresql#2988
• 11337: Refactor AutoIncrementTracker into generitc type with interfaces
  The purpose of this PR is to enable Doltgres to create a global state for Sequences that behaves similarly to Dolt's global state for auto increment columns. In order to allow for code reuse, we have to do the following:
  • Create a generic type containing the behavior common to both auto-increment and sequences.
  • Create new interfaces that can be implemented to describe the behavior unique to each feature.
    (Bikeshedding note: I use the term "Relation" to describe a top-level object in the database, and encompasses both tables and root objects. I'm open to alternative names.)
    The new interfaces can be found in go/libraries/doltcore/sqle/globalstate/sequences/state.go
  • SequenceState is an interface for a state machine that can be incremented. Each time it is incremented, it produces a new SQL value. SequenceStates can also be compared to each other to determine which one is "further along" and "merged" to create a single state that is further along in its sequence than any of the input states. The only implementation in Dolt is AutoIncrementState, which is just a uint64. Doltgres sequences will have a more complicated implementation, since they have more parameters that affect how the state is incremented.
  • SequencedRelation is a relation that holds a SequenceState. The only implementation in Dolt is doltdb.Table, which can hold an auto-increment value. Doltgres Sequences will also implement this interface, and possibly Doltgres tables with a SERIAL column.
    The common logic is in the SequenceTracker type, which has an interface in go/libraries/doltcore/sqle/globalstate/sequence_tracker.go and an implementation in go/libraries/doltcore/sqle/dsess/sequence_tracker.go
    AutoIncrementTracker is now a type alias for a specialization of SequenceTracker.
    Finally, while GlobalState previously only held an AutoIncrementTracker, it now contains a map from arbitrary keys to SequenceTrackers. This will allow Doltgres to add an additional SequenceTracker for global tracking of Doltgres sequences.
    There are a couple places in this PR that are TODOs, because they are corner cases that aren't reachable for AutoIncrement but are reachable for Doltgres sequences. They mostly concern how to handle when branches contain incompatible Sequences. I haven't decided exactly how this should be handled, and this PR will not be merged until we know what to do in that case, but it shouldn't affect the review of the rest of the PR.
• 11329: Reject branch names that differ only by case
  Branch names are matched case-insensitively causing branches that only differ by case to resolve to the same data (e.g., main and Main).
  • Refuse to create branch whose names differs from existing one by case on checkout, branch, and other variants

  dolt branch Main
  dolt checkout -b Main
  dolt branch -c main Main
  dolt branch -f Main

  • Allow case-only rename to remove ambiguity of conflicting branch names
    Fix #11270
• 11279: Remove no-op AutoIncrement statement
  A simple cleanup: this line always errors, but the error isn't being checked. It effectively does nothing and isn't needed because:
  • the auto increment state is automatically updated by the AutoIncrement node in the plan tree
    -as long as the auto_increment as long as w.aiSet is true, the correct auto_increment value will be read from the AutoIncrementTracker.

go-mysql-server

• 3693: fix panic for GeometryFromWKT and various GeoSpatial bugs
  We added support for "pure empty geometry forms", which MySQL doesn't support, except for "GEOMETRYCOLLECTION EMPTY".
  Changes:
  • fix panics for pure empty geometry forms
  • fix panics for invalid axis-order options
  • fix ordering for axis-order options
  • expose st_latitude function
  • fix geospatial srid for st_x() and st_y() functions
    Fixes: #11492
• 3691: block certain statements from PREPARE
  This PR blocks CREATE EVENT and CREATE PROCEDURE from getting created.
  Additionally, adds tests for a variety of DDL statements like DELETE, CREATE, and ALTER.
  Fixes:
  #11417
  #11451
• 3690: Check for null before casting LAST_INSERT_ID input
  fixes #11454
• 3689: Refactor window function tests into ScriptTests
  Converts existing window function tests to use ScriptTest and mov...

2.2.3

Merged PRs

dolt

• 11366: Include full table scan in index statistics
  Have StatsController also create an "empty" statistic entry to indicate no index for costing.
• 11365: Add dolt_squash_history procedure
  This new procedure will take the content of HEAD, and squash history before it. Default behavior is to squash every commit from height 2 (commit right after initial commit) to HEAD. It is impossible to rewrite the initial commit.
  This will be much faster than rebasing all history.
• 11358: Fix left outer merge join wrong results with residual filters
  Tests added in dolthub/go-mysql-server#3655
  Fixes #11350
  Based on #11351
• 11354: add missing execution call for sqlparser.Execute statements
  We were missing calls to iter.Next() and iter.Close(), when executing prepared statements through dolt sql -q "...",
  some tests in: dolthub/go-mysql-server#3653
  fixes: #11345
• 11351: kvexec: fix left outer merge join wrong results with residual filters
  Fixes #11350 — wrong results from mergeJoinKvIter for left outer merge joins whose key runs contain duplicates and whose join condition carries a residual (non-merge-key) filter.
  Two one-line-class defects, both in the match: loop:
  1. l.matchedLeft = ok overwrote earlier successes. A left row that matched mid-run was re-flagged unmatched by a later failed pairing in the same run, so the run transition emitted a spurious null-extended row for it. Changed to l.matchedLeft = l.matchedLeft || ok; the run-transition reset remains the sole reset point.
  2. io.EOF during the left advance dropped the final owed null-extension. The compare: loop handles left-EOF via oldLeftKey; the match: loop's advance branch returned the error immediately, losing the null-extended row for a final unmatched left row. Now it emits that row (when isLeftJoin && !matchedLeft) before surfacing EOF.
     Minimal repro (returns fr:app|NULL instead of nl:app|NULL before this patch):

  create table b2 (cat varchar(50) not null, code varchar(16) not null, lang varchar(20) not null,
  primary key(cat, code, lang), key(code));
  create table t2 (id varchar(36) primary key, code varchar(16), lang varchar(16), key(code, lang));
  insert into b2 values ('cat0','P1','de:app'),('cat1','P1','fr:app'),('cat2','P1','nl:app');
  insert into t2 values ('t1','P1','de'),('t2','P1','es'),('t3','P1','fr');
  select /*+ MERGE_JOIN(p,w) */ p.lang, w.lang
  from b2 p left join t2 w on w.code = p.code and w.lang = substring_index(p.lang, ':', 1);

  The regression test registers the kvexec ExecBuilder on the test engine (as the CLI/server engines do) so the kvexec iterator — not the GMS fallback, which was already correct — is what's exercised; the new test fails 7 assertions without the fix and passes with it. kvexec suite and enginetest TestJoinQueries/TestJoinOps pass. Also verified against a 15k-row synthetic (anti-join now agrees exactly with NOT EXISTS: 5,000) and the production database where this was found (197-row orphan audit no longer inflates to ~15k when the planner picks the merge join path).
  🤖 Generated with Claude Code
  https://claude.ai/code/session_01VpTxMfzCXHoM8ojJn5aQiv
• 11344: Fail amend commits when branch head has moved
  When a SQL transaction ran CALL DOLT_COMMIT('--amend') after another connection had added a commit or merged a branch, the amend silently replaced the transaction HEAD commit up to the new HEAD commit. The data from the erased commits survived, but the commit itself, including merge records we're gone.
  • datas.CommitOptions.Amend is replaced with AmendedCommit (hash.Hash), the address of the commit an amend replaces.
  • datas.BuildNewCommit rejects an amend unless AmendedCommit is still the dataset HEAD, relying on the CAS in CommitWithWorkingSet` for atomic comparison.
  • dsess.doCommit runs validateAmendedHead before the working set merge to specify the head conflict over a row conflict.
  • Amend is refused mid merge, cherry-pick, in parity with git and revert, a Dolt-specific addition because it follows the same merge state.
  • The admin createchunk command's HEAD check bypass moves from the Amend flag to a new explicit CommitOptions.Force field.
    Fix #9072
• 11343: Add retries to branch creation on ErrOptimisticLockFailed errors
  Adds a retry loop to NewBranchAtCommit so branch creation can recover from a concurrent optimistic-lock conflict on the working set (e.g. from a background auto-GC cycle). This matches the existing retry behavior of other write paths (e.g. SQL transaction commit).
• 11333: Bug fixes for secondary indexes using virtual columns
  When a virtual generated column is used in a secondary index, rebuilding the index can generate incorrect index data in some situations. For example, dropping a column in the table will trigger a table rewrite as well a rebuilding the secondary indexes. In this case, the virtual generated column expressions were not properly being evaluated to store the correct data in the index.

go-mysql-server

• 3657: check stats prov for empty stat
  Instead of creating the (potentially same) empty stat every time, we should check if it has already been made in the StatsProvider.

• 3656: Bug fix for CASE expressions with ExtendedType instances
  Related to: dolthub/doltgresql#2980
  Doltgres fix: dolthub/doltgresql#2987

• 3653: adding prepare tests for insert and update

• 3652: Allow aliasing column names in a table function

  Depends on: dolthub/vitess#477

• 3644: add ExtendedTableFunction interface to support table function schema …
  …on OUT parameters

• 3643: Use buildScalar to get GetField expression for ON UPDATE columns
  fixes #11346
  Building the GetField expression for a column using its index in a table was causing indexing issues when there was also a CTE. Instead, we need to build the the GetField using buildScalar to properly resolve the column within the scope.

• 3641: Bug fix for a panic when calling a stored procedure without a database selected

• 3639: have coster consider not picking secondary indexes
  This PR adds a "no index" option to the coster and some additional heuristics for index costing.
  Before, we would always pick any available index, which would sometimes be sub-optimal, especially if the index is a non-covering secondary index.
  Additionally, the normal_dist and exp_dist tables are now seeded to reduce random variability between test runs.
  Benchmarks:
  #11336 (comment)

• 3638: Bug fixes for secondary indexes using virtual columns
  When updating a row in a secondary index, any virtual generated columns need their generation expression evaluated to get the correct data stored in the secondary index. There were a few edge case with virtual columns and secondary indexes where this evaluation wasn't happening. This PR closes those gaps.
  The first case is when a table rewrite is performed and secondary indexes are rebuilt (e.g. when dropping a column on a table). Not properly evaluating a virtual column's expression caused incorrect data to be stored in the index when it was rebuilt.
  A second case is for referential actions on a foreign key that update a table and its secondary index. Not properly evaluating a virtual column's expression here can also cause incorrect data to be written to the secondary index.
  Fix for failing Dolt CI integration test in: #11333

• 3634: Add support for multiple expressions in functional indexes

• 3623: memory: use stable sort for secondary index ordering
  memory.TableData.sortSecondaryIndexes sorted secondary index storage with sort.Slice, which uses Go's unstable pdqsort algorithm. Rows that tie on the indexed columns (same key, different primary key) could be reordered relative to each other on every sort, so repeated index rebuilds of the same data could produce different physical row orders.
  This PR switches to sort.SliceStable, which preserves the relative (insertion) order of tied rows, making secondary index storage ordering deterministic across rebuilds.
  Fixes #2877

• 3618: Error on SELECT @@SESSION., matching MySQL
  GMS silently returned the global value when a GLOBAL-only system variable was read with an explicitly-qualified SESSION/LOCAL scope (e.g. SELECT @@SESSION.innodb_autoinc_lock_mode), instead of raising MySQL's ERROR 1238 (ErrSystemVariableGlobalOnly).
  buildSysVar's case for SetScope_None/SetScope_Session already carries specifiedScope, which is empty for a bare @@foo reference and non-empty ("session"/"local") only when the scope was expli...

2.2.2

Merged PRs

dolt

• 11332: /go/cmd/dolt/cli: only emit backspace characters in a tty
  The CLI's progress spinners (dolt push, pull, fetch, assist, and import/sql progress output) redraw themselves in place by writing backspace control characters. This was done unconditionally, with no check for whether output was going to a terminal.
  When stdout/stderr is captured to a file or pipe — a script, a CI job, dolt push > push.log 2>&1 — those backspaces become literal bytes in the log instead of cursor movements. Rendering the log then "erases" the surrounding text. The worst case: an error is printed while the spinner is active, and the spinner's trailing backspaces overwrite the error text, so the captured log looks empty exactly when something went wrong.
  This PR fixes this so that TTY is detected and redraw is gated on it.
• 11327: Add support for functional indexes with multiple expressions
  Picks up support for functional indexes with multiple expressions from GMS (dolthub/go-mysql-server#3634) and adds additional tests for Dolt.
  Depends on: dolthub/go-mysql-server#3634
• 11326: unsafe adaptive value
• 11325: gitblobstore: fix premature flush and prune of pending write that was not referenced in current manifest
  CheckAndPut("manifest") flushed all queued pendingWrites into the commit, regardless of whether the manifest being written referenced them. That commits a table file into the git tree as an unreferenced entry, which a later manifest flush then legitimately prunes and evicts from the in-memory cache — while the file is still live (about to be added, or being re-read by addTableFiles' refCheckAllSources retry loop). The result is Blob not found. Stale pendingWrites from earlier failed pushes in the same process keep the condition sticky, which is why a restart clears it.
  This PR only flushes the pending writes the manifest actually references. Unreferenced writes stay pending until a manifest that references them is written, so they're committed and referenced atomically. After this change a live table file is always either pending and cached (not in the tree, so unprunable) or committed and referenced (never pruned) — an unreferenced tree entry can no longer coexist with being live.
• 11320: go: sql_server_driver: Fix a race on *SqlServer.Cmd access. Start the command before we spawn its wait goroutine.
• 11316: go: sqle/cluster: Improve dolt_cluster_ack_writes_timeout_secs circuit breaker behavior.
  Previously, the circuit breaker only had an open and a closed state. It would only go from open to closed when the replication thread fully quiesced. This PR adds a half-open state, where it lets a single commit probe for replication success within the configured timeout. If that wait succeeds, the circuit breaker closes.
  This lets high write throughput use cases reenable their configured replication waits even when writes arrive more requently than their typical replication delay.
• 11314: go: sqle/cluster: controller.go: Make DROP DATABASE replication participate in dolt_cluster_ack_writes_timeout_secs.
• 11313: go: Fix some data races surfaced by running go-sql-server-driver tests under -race.
• 11312: go/store/nbs: Fix a race between the journal persister and PruneTableFiles.
  When PruneTableFiles runs, it is willing to delete the journal file if there is no longer a reference to it in the manifest. This can race with bootstrapJournal making a new journal file. The journal file can be on disk, and seen by PruneTableFiles and thus selected for removal, before the journal has been added to the chunk store proper.
  This change makes the journal writer participate in the retained files machinery, same as the table file persister, so that the journal is correctly considered not-safe-for-delete-by-PruneTableFiles before it is ever created on disk and potentially selected for deletion by PruneTableFiles.
• 11305: fix stats functions to include schema
  We lookup stats with empty string for schema. Since this is not always the case, we sometimes miss stats.
• 11290: Bump golang.org/x/net v0.54.0 => v0.55.0 for Go 1.27 compatibility
  Go 1.27 change:
  • https://github.com/golang/go/discussions/60746
  • golang/go#67810
    Problem found in
  • Homebrew/homebrew-core#288803
    Build error:

  $ go1.27rc1 build -v ./cmd/dolt
  google.golang.org/grpc/internal/transport
  # google.golang.org/grpc/internal/transport
  ../../../../go/pkg/mod/google.golang.org/grpc@v1.79.3/internal/transport/handler_server.go:271:18: undefined: http2.TrailerPrefix
  

  golang.org/x/net upgrade

  $ go get golang.org/x/net@v0.55.0 && go mod tidy
  go: upgraded golang.org/x/net v0.54.0 => v0.55.0
  

  Build successful:

  $ go1.27rc1 build -v ./cmd/dolt
  

go-mysql-server

• 3634: Add support for multiple expressions in functional indexes
• 3627: fix stats functions to include schema
  We include the schema in the stats key, but only sometimes fill in the field.
• 3626: Prevent index offset when comparing rows in topRowsIter
  Fixes #11300
  Appending the row order number to the end of a sql.Row during a Top-N Heap Sort in topRowsIter was causing an index offset when evaluating SortFields that were subqueries, thus resulting in incorrect result.
  This PR
  • modifies topRowsHeap to instead take a rowWithOrder struct that separates out the sql.Row from the order number while still taking the order number into account when sorting the heap.
  • refactoring Sorter.LesserRow logic into a new CompareRows function to allow checking for row equality
  • moves top row(s) iterators to its own file as part of an effort to make our iterators more organized (#3620). This file also includes topRowsHeap since it is only ever used by topRowIter (see dolthub/go-mysql-server#3622 (comment) for next steps)
  • removes ValueRowSortersince it's actually never used anywhere and doesn't even fully implement sort.Interface
• 3617: special case on extended type for getting compare type

Closed Issues

• 11265: Dolt INSERT … SELECT with CTE + JSON_TABLE disconnects
• 3622: Clean up: move sorters in sql/expression/sort.go out of expression package

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	17.32	2.3	0.13
groupby_scan	134.9	142.39	1.06
index_join	3.43	1.89	0.55
index_join_scan	4.18	1.32	0.32
index_scan	350.33	215.44	0.61
oltp_point_select	0.19	0.25	1.32
oltp_read_only	3.62	4.91	1.36
select_random_points	0.35	0.51	1.46
select_random_ranges	0.39	0.64	1.64
table_scan	350.33	200.47	0.57
types_table_scan	746.32	467.3	0.63
reads_mean_multiplier			0.88

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	7.7	6.21	0.81
oltp_insert	4.03	3.13	0.78
oltp_read_write	8.74	11.24	1.29
oltp_update_index	4.33	3.3	0.76
oltp_update_non_index	4.1	3.02	0.74
oltp_write_only	5.18	6.21	1.2
types_delete_insert	8.28	6.67	0.81
writes_mean_multiplier			0.91

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	95.13	51.54	1.85
tpcc_tps_multiplier			1.85

Overall Mean Multiple	1.21

2.2.1

Merged PRs

dolt

• 11310: cluster: wait for outstanding DROP DATABASE replication during graceful transition to standby
  dolt_cluster_transition_to_standby determines whether a standby replica is fully caught up by waiting on three replication subsystems: per-database commit hooks, users/grants, and branch control. It never accounted for outstanding DROP DATABASE replications, which are tracked separately in Controller.outstandingDropDatabases and driven by independent fire-and-forget goroutines.
  As a result, the transition could report a standby as fully caught up and succeed while a DROP DATABASE had not yet replicated to it. This PR makes it so that dolt_cluster_transition_to_standby also waits for outstanding DROP DATABASE statements to replicate.
• 11308: go.mod: bump eventsapi_schema to latest revision

go-mysql-server

• 3626: Prevent index offset when comparing rows in topRowsIter
  Fixes #11300
  Appending the row order number to the end of a sql.Row during a Top-N Heap Sort in topRowsIter was causing an index offset when evaluating SortFields that were subqueries, thus resulting in incorrect result.
  This PR
  • modifies topRowsHeap to instead take a rowWithOrder struct that separates out the sql.Row from the order number while still taking the order number into account when sorting the heap.
  • refactoring Sorter.LesserRow logic into a new CompareRows function to allow checking for row equality
  • moves top row(s) iterators to its own file as part of an effort to make our iterators more organized (#3620). This file also includes topRowsHeap since it is only ever used by topRowIter (see dolthub/go-mysql-server#3622 (comment) for next steps)
  • removes ValueRowSortersince it's actually never used anywhere and doesn't even fully implement sort.Interface
• 3621: bug fix: honor named window reference
  An existing bug in GMS was not properly applying a named window reference. We had enginetests for named window references, but they weren't sufficient to catch this because the aggregate function they used produced identical values in both cases. New enginetest cases are added to prevent a regression.
  These two issues were identified by Ito automated review, in dolthub/doltgresql#2913
• 3619: sql: Fix ALTER USER to allow the Identity field of the User record to be updated.
  CREATE USER ... IDENTITY WITH <plugin> AS '<identity>' correctly parsed and persisted the Identity field. ALTER USER correctly parsed the Identity field but failed to persist the changes. The end result is that GMS had a bug where attempt to alter the identity field on an existing user seemed to succeed but was not reflected in the data going forward.
  Fix the bug so that updates to Identity are reflected and persisted going forward.

Closed Issues

• 11300: ORDER BY subquery loses effect as soon as LIMIT and OFFSET become involved

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	18.61	2.3	0.12
groupby_scan	134.9	144.97	1.07
index_join	3.49	1.93	0.55
index_join_scan	4.25	1.3	0.31
index_scan	350.33	219.36	0.63
oltp_point_select	0.19	0.25	1.32
oltp_read_only	3.62	4.91	1.36
select_random_points	0.35	0.51	1.46
select_random_ranges	0.38	0.64	1.68
table_scan	350.33	196.89	0.56
types_table_scan	759.88	442.73	0.58
reads_mean_multiplier			0.88

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	7.7	6.21	0.81
oltp_insert	4.1	3.19	0.78
oltp_read_write	8.74	11.24	1.29
oltp_update_index	4.41	3.3	0.75
oltp_update_non_index	4.1	3.02	0.74
oltp_write_only	5.18	6.21	1.2
types_delete_insert	8.43	6.79	0.81
writes_mean_multiplier			0.91

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	96.05	52.22	1.84
tpcc_tps_multiplier			1.84

Overall Mean Multiple	1.21

2.2.0

This release is nearly identical to 2.1.11. There are no novel merged PRs in this release.

This release is a minor version bump because the authentication of tokens with auth plugin authentication_dolt_jwt changed in 2.1.11. This could potentially effect deployments which are currently minting JWTs tokens for authentication to Dolt, but it is not expected to effect any existing deployment. See #11296 for more details.

Performance

Read Tests	MySQL	Dolt	Multiple
covering_index_scan	17.63	2.3	0.13
groupby_scan	134.9	144.97	1.07
index_join	3.55	1.93	0.54
index_join_scan	4.25	1.32	0.31
index_scan	350.33	219.36	0.63
oltp_point_select	0.19	0.25	1.32
oltp_read_only	3.68	5.0	1.36
select_random_points	0.36	0.52	1.44
select_random_ranges	0.39	0.65	1.67
table_scan	350.33	200.47	0.57
types_table_scan	746.32	450.77	0.6
reads_mean_multiplier			0.88

Write Tests	MySQL	Dolt	Multiple
oltp_delete_insert	7.7	6.21	0.81
oltp_insert	4.03	3.19	0.79
oltp_read_write	8.9	11.24	1.26
oltp_update_index	4.33	3.3	0.76
oltp_update_non_index	4.1	3.02	0.74
oltp_write_only	5.18	6.32	1.22
types_delete_insert	8.28	6.79	0.82
writes_mean_multiplier			0.91

TPC-C TPS Tests	MySQL	Dolt	Multiple
tpcc-scale-factor-1	95.75	52.65	1.82
tpcc_tps_multiplier			1.82

Overall Mean Multiple	1.20