Skip to content

perf(cpp): cache per-device schema check for repeated tablet writes (#885) - #934

Open
kkzi wants to merge 2 commits into
apache:developfrom
kkzi:perf/cpp-writer-schema-check-cache
Open

perf(cpp): cache per-device schema check for repeated tablet writes (#885)#934
kkzi wants to merge 2 commits into
apache:developfrom
kkzi:perf/cpp-writer-schema-check-cache

Conversation

@kkzi

@kkzi kkzi commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Title:
perf(cpp): cache per-device schema check for repeated tablet writes (#885)

Body:

Problem

Closes #885.

TsFileWriter::do_check_schema / do_check_schema_aligned re-resolve every
measurement name against measurement_schema_map_ on every
write_tablet / write_tablet_aligned / write_record call, even though
the device schema is registered once and does not change during the file
lifecycle. For wide tablets written repeatedly this per-column string lookup
is a top CPU hotspot (profiling in #885).

Change

Cache the resolved chunk writers and data types per device in
MeasurementSchemaGroup (cpp/src/common/schema.h), resolved once and
reused while the tablet's measurement name sequence is unchanged:

  • Cache key is the name sequence, not the column count. Cached entries
    are reused by position, so the same count with a different name order (or
    one column swapped) would silently write values into the wrong column with
    the wrong data type. A mismatch drops the stale cache and re-resolves, so
    dynamic schemas keep the existing validation behavior.
  • Plain and aligned paths keep separate caches (a shared flag would let
    whichever path runs first lock the other out permanently).
  • Only fully-resolved results are cached. A NULL chunk writer for a
    not-yet-registered measurement must not be pinned, or the column would
    stay masked even after it is registered later.
  • Cached entries are the same non-owning pointers the uncached path
    returns: chunk_writer_ is only freed in destroy(), flush only resets
    it, and register_timeseries returns E_ALREADY_EXIST for an existing
    name instead of replacing the object, so a cached pointer cannot dangle.
  • The written file is byte-identical to the uncached path.

do_check_schema_table (table model) is unchanged; this targets the tree
model paths named in #885.

Tests

New suite SchemaCheckCacheTest (cpp/test/writer/tsfile_writer_schema_cache_test.cc):

  1. RepeatedSameSchemaRoundTrip — hit path, incl. after a flush seals and
    resets the chunk writers; asserts every row and column (no dropped rows).
  2. SameCountDifferentNamesAndOrder — same column count with a swapped /
    reordered name set re-resolves; values land in the column their name says.
  3. ColumnRegisteredAfterFirstWriteIsNotMasked — the fully-resolved-only
    guard: a column unregistered at first write is written once registered.
  4. AlignedRepeatedAndReorder — aligned path: own cache, hit + reorder.
  5. MultiDeviceCachesIndependent — two devices with identical measurement
    names, interleaved writes, no cross-wiring.

Full suite: 830 tests, 827 passed, 3 skipped (environment-only skips:
external-index and cross-language fixture generators), 0 failures.

Benchmark

A/B microbenchmark (not committed — happy to add it to the tree if wanted):
one device, 200 INT32/PLAIN columns, 2000 tablet writes × 1 row each (the
few-rows-wide-tablet shape from the issue), parallel write disabled, best of
5 rounds, MSVC 2019 x64:

build uncached (develop) cached (this PR) speedup saved ns/col
Release 33.4 µs/write 19.0 µs/write 1.76× ~72
Debug 246.0 µs/write 150.0 µs/write 1.64× ~480

The schema check was ~43% of the total write_tablet time in this shape.
With more rows per tablet the end-to-end percentage shrinks (the saving is
per write, the encode cost is per point), but the per-column lookup remains
the dominant fixed cost per call — which is what #885 profiles.

Notes

gx added 2 commits September 2, 2026 09:06
Repeated tablet/record writes with a fixed device schema re-resolve every
measurement name against measurement_schema_map_ on each write, which is a
CPU hotspot for wide schemas (apache#885).

Cache the resolved chunk writers and data types per device in
MeasurementSchemaGroup, keyed by the measurement NAME SEQUENCE (column
count alone is not a safe key: entries are reused by position, so a
same-count tablet with a different name order would write values into the
wrong column with the wrong data type). A mismatch drops the stale cache
and re-resolves. The plain and aligned paths keep separate caches.

Only fully-resolved results are cached: a NULL chunk writer for a
not-yet-registered measurement must not be pinned, or the column would
stay masked even after it is registered.
Five cases pinning the behaviors the cache must preserve:
- repeated same-schema writes round-trip every row (hit path, incl. after
  a flush seals and resets the chunk writers);
- a same-column-count tablet with different names/order re-resolves and
  writes each value into the column its name says;
- a column unregistered at first write is not masked by a cached NULL
  after it is registered (only fully-resolved results are cached);
- the aligned path keeps its own cache with the same guarantees;
- per-device caches never cross-wire two devices.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimize TsFileWriter::do_check_schema CPU overhead for repeated wide-tablet writes

1 participant