Skip to content

DotRocks 1.4.2

Latest

Choose a tag to compare

@kidoz kidoz released this 01 Aug 11:17
· 1 commit to main since this release

A patch release of DotRocks — a native .NET driver, Entity Framework Core provider, and Roslyn analyzer suite built specifically for StarRocks. It implements its own managed StarRocks client protocol and takes no runtime dependency on any MySQL driver. This release fixes a StarRocks session leak in the experimental DotRocks.FlightSql transport — every RPC re-authenticated, so each query left sleeping frontend connections behind until the per-user connection limit was exhausted — and corrects several ADO.NET contracts in that transport. The core driver gains the standard large-value accessors and a cheaper result-row loop.

Added

  • A "Reading large results" guide documents how result sets stream, what bounds memory and time while reading, how to choose between the MySQL protocol and Arrow Flight SQL, and the capabilities DotRocks deliberately does not expose because StarRocks does not provide them — there is no fetch size or server-side cursor (StarRocks does not implement COM_STMT_FETCH) and no parallel Flight endpoint fan-out (StarRocks returns a single endpoint per query).
  • DotRocksDataReader implements GetStream and GetTextReader, so code written against the standard ADO.NET large-value accessors works without falling back to the base implementations. Both read from the already-materialized field value (a NULL yields an empty stream/reader); the reader's memory guarantee remains per row, not per field, and that is now stated on the type along with the fact that CommandBehavior.SequentialAccess is accepted but imposes no restrictions and yields no additional memory benefit.
  • The read path's per-row cost is now guarded by the performance budget. A server-free MaterializeRows benchmark drives rows through the typed DotRocksDataReader accessors (measured at ~478 bytes per row for a three-column row) with a tight allocation ceiling, and the budgeted benchmark suite runs in CI, so a regression in per-row buffering or boxing fails the build instead of going unnoticed. The live large-result test now also bounds allocation per row across the whole 100k-row drain and asserts the reader retains nothing afterwards.
  • DotRocksFlightSqlDataSource.CreateConnection hands out ADO.NET connections that share the data source's channels and authenticated sessions, so short-lived connections no longer each build a private channel and session. The data source also implements IAsyncDisposable, which is the preferred way to dispose it because releasing server sessions is a network operation.
  • DotRocksFlightSqlTransaction.IsCompleted reports whether the server confirmed completion.

Changed

  • The result-row loop reads each row into a buffer rented from ArrayPool and returns it once the row is decoded, instead of allocating a fresh array per row. Measured on the budgeted benchmark, per-row allocation drops from about 478 to 425 bytes for a narrow three-column row, and the saving grows with row width. This is safe because every decoded value copies out of the payload; a regression test poisons a recycled buffer to prove no materialized value aliases it. A row spanning continuation packets (a value larger than one 16 MB packet) still uses an exact-size array, since its length is not known until reassembly completes.
  • GetOrdinal uses a lookup built once per result set rather than scanning the column list on every call, so reading columns by name inside a row loop is no longer O(columns) per access. The typed accessors (GetInt32, GetInt64, GetDouble, and friends) now test the boxed value directly before falling back to Convert.
  • The Flight SQL benchmarks establish their connections once in setup rather than per iteration, and the direct record-batch benchmark now projects and consumes the same columns as the row benchmarks, so the comparison measures row materialization rather than a smaller projection plus per-iteration connection setup.

Fixed

  • Arrow Flight SQL no longer leaks a StarRocks session per RPC. The transport sent Basic credentials on every call, and StarRocks creates a frontend session for each authenticated call, so a single query left two sleeping sessions behind and a benchmark run reached the 1024-connection user limit (ResourceExhausted). The credentials are now exchanged once during the Flight handshake for the session bearer token that every later call reuses, and disposal releases the session with the Flight SQL CloseSession action. Measured against StarRocks 4.0.7: five queries created eleven sessions before the fix and none after it. Servers that do not implement the handshake keep the previous per-call behavior, and servers without the CloseSession action (StarRocks 3.5) hold one session per data source until it expires instead of one per call.
  • A Flight SQL transaction is marked completed only after the server confirms completion, so a failed CommitAsync or RollbackAsync leaves it active and recoverable instead of stranding an open server transaction that the client can neither retry nor roll back. A failed rollback during disposal now also releases the connection, which previously stayed bound to a transaction that could no longer be completed.
  • Flight SQL decimals report the type they actually materialize. Columns declared with more precision than System.Decimal can hold (DECIMAL(38, s) is routine in StarRocks) are typed as DotRocksDecimal, and GetDecimal/GetFieldValue<decimal> convert such values when they are representable instead of failing with an InvalidCastException from Convert.ChangeType.
  • Cancelling a Flight SQL command raises OperationCanceledException rather than a raw RpcException, so consumers can tell cancellation from a transport failure. Cancel() no longer races with the end of execution, and no longer misses a cancellation issued in the instant after execution starts.
  • DbDataReader.HasRows reports the real result. StarRocks does not declare a record count, and the Flight reader previously answered true for every such result, including empty ones; a command now fetches the first batch before returning the reader.
  • GetOrdinal and ordinal-based accessors on the Flight reader report unknown columns with IndexOutOfRangeException as ADO.NET specifies, instead of ArgumentOutOfRangeException.
  • Reading a large Flight SQL value in chunks through GetBytes/GetChars no longer re-materializes the whole value for every chunk; the materialized value is cached for the current row and column, so allocation is proportional to the value rather than to value size times chunk count.
  • A Flight endpoint that advertises several locations no longer fails when the first one is untrusted or unreachable: the trusted alternatives are tried in the order the server supplied them.
  • A result value larger than the reader's maximum logical packet size now reports that limit instead of "StarRocks returned malformed protocol bytes", which sent callers looking for a protocol bug when the real cause was an oversized field.

Compatibility

  • .NET 10 / C# 14. Source- and binary-compatible with 1.4.1 — the public surface only gains members (DotRocksDataReader.GetStream/GetTextReader, DotRocksFlightSqlDataSource.CreateConnection/DisposeAsync, DotRocksFlightSqlTransaction.IsCompleted). DotRocks.Data, DotRocks.EntityFrameworkCore, and the analyzers behave as in 1.4.1 apart from those additions and the internal row-buffer change.
  • Behavior changes in the experimental DotRocks.FlightSql transport worth noting: ExecuteReaderAsync now fetches the first record batch before returning, so it waits for the server to start producing and HasRows reflects the real result; cancellation surfaces as OperationCanceledException instead of RpcException; a missing column throws IndexOutOfRangeException; DECIMAL columns wider than System.Decimal materialize as DotRocksDecimal and GetDecimal throws DotRocksPrecisionLossException when a value genuinely does not fit; and a commit or rollback the server rejects leaves the transaction active rather than completed. Prefer await using over using for DotRocksFlightSqlDataSource, since releasing the server session is a network call. The package surface remains experimental.

Full Changelog: v1.4.1...v1.4.2