perf(ebean-dao): move SharedSchemaCache pre-warm to constructor#618
Merged
Conversation
PR linkedin#615 introduced SharedSchemaCache with pre-warm calls inside EbeanLocalAccess.ensureSchemaUpToDate(). In deploys where schema migrations run as a separate job (LinkedIn's prod pattern, and likely others), ensureSchemaUpToDate() is gated off, so pre-warm never runs and the first request after JVM startup pays the inline information_schema query cost (the "Inline schema cache miss" log line). Moves the pre-warm calls to the EbeanLocalAccess constructor so they always run, regardless of whether schema evolution is enabled. refreshTable() already has its own try/catch, so a transient DB hiccup during construction logs a warning rather than failing boot. Also drops the unused tableColumns field (dead code, replaced by the shared cache in linkedin#615). Two supporting changes: 1. SharedSchemaCache.clearRegistry() now also calls clearCaches() on each held instance before wiping the REGISTRY map. Previously clearing only the static map left stale entries readable through any reference (e.g. EbeanLocalAccess.validator) that callers were already holding -- a footgun that the @VisibleForTesting name did not suggest. 2. EbeanLocalDAOTest.addIndex() helper now calls clearRegistry() after its dynamic ALTER TABLE ADD COLUMN. With pre-warm now running in the constructor, the cache snapshots the schema before addIndex modifies it; the explicit invalidation keeps subsequent columnExists()/indexExists() lookups in sync with the actual table state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
a036656 to
2023f92
Compare
ybz1013
approved these changes
May 6, 2026
Contributor
ybz1013
left a comment
There was a problem hiding this comment.
LGTM! Thanks for the fix!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
PR #615 introduced
SharedSchemaCacheand added pre-warm calls insideEbeanLocalAccess.ensureSchemaUpToDate(). In deploys where schemamigrations are run as a separate job (LinkedIn's prod pattern, and
likely others),
ensureSchemaUpToDate()is gated off — so pre-warmnever runs and the first request after JVM startup pays the inline
information_schemaquery cost.In production we observed:
getWithContextcorrelated withInline schema cache miss: loading columns for table 'metadata_entity_<entity>'log lines firing inside the request path.
registeredTableswas empty, so the 9-minute refresh logged
Background schema cache refresh: 0 tablesand accomplished nothing.This PR moves
validator.registerAndPreWarm(...)fromensureSchemaUpToDate()to theEbeanLocalAccessconstructor, whereit always runs.
EbeanLocalAccessis only instantiated forNEW_SCHEMA_ONLYandDUAL_SCHEMADAOs (the ones that actually useSharedSchemaCache), so the change is naturally scoped.refreshTable()already has its own try/catch — a transient DBhiccup during construction logs a warning rather than failing boot,
so this is safe to run unconditionally at construction time.
Also drops the unused
tableColumnsfield (dead code, replaced bythe shared cache in #615).
Two supporting changes
SharedSchemaCache.clearRegistry()now also callsclearCaches()on each held instance before wiping the
REGISTRYmap.Previously, clearing only the static map left stale entries
readable through any reference (e.g.
EbeanLocalAccess.validator)that callers were already holding — a footgun the
@VisibleForTestingname didn't suggest.EbeanLocalDAOTest.addIndex()helper now callsclearRegistry()after its dynamicALTER TABLE ADD COLUMN.With pre-warm running in the constructor, the cache snapshots the
schema before
addIndexmodifies it; the explicit invalidationkeeps subsequent
columnExists()/indexExists()lookups in syncwith the actual table state. (The previous behavior worked by
accident, because lazy initialization populated the cache only on
first read — which happened to be after
addIndex.)Testing Done
Tested on
metadata-graph-assets(LinkedIn's MGA service) incorp-lva1.Setup
datahub-gma:0.6.185and published to localmaven:
metadata-models'sproduct-spec.jsonto consumedatahub-gma:0.6.185and ranmint build && mint snapshot && mint release,producing
metadata-models:277.0.15-SNAPSHOT.product-spec.jsonto consumemetadata-models:277.0.15-SNAPSHOTand deployed to a local QEIinstance via
mint debug.Result with the fix (snapshot deploy)
Background schema cache refresh: <N> tablesfires for everySharedSchemaCachesingleton (one perEbeanServer/dbUrl). Eachnon-zero count is direct evidence that
registerAndPreWarm()populated
registeredTablesat construction time:Total: 78 entity tables registered across 4 databases (14 + 10 + 52 + 2).
After redeploy with the fix,
Inline schema cache missno longerappears post-boot for the request path.
Result on master (without the fix)
After re-deploying master MGA (using
metadata-models:277.0.12→datahub-gma:0.6.183, which hasSharedSchemaCachebut pre-warmgated behind
ensureSchemaUpToDate()):The A/B difference confirms the constructor pre-warm is the change
that actually populates the cache in production deploys where
ensureSchemaUpToDate()is disabled.Unit test verification
Locally ran the previously-failing
EbeanLocalDAOTestcases thatexposed the pre-warm interaction with dynamic ALTER TABLE: all pass
with the test helper change.
Checklist
cc @yanbZhao (original author of #615)