GigaMap: compile-time index metamodel (annotation processor) with Lucene & JVector wiring - #699
Merged
Conversation
Adds `gigamap-codegen`, an annotation processor that emits a sibling `<Entity>_` metamodel for each entity carrying @Index / @unique / @Identity / @SpatialIndex / @indexed: typed `public static final` indexer constants plus an idempotent, reload-safe `registerIndices`. This gives compile-time index names, IDE autocomplete and refactor safety on top of the runtime IndexerGenerator (plan section D). - Mirrors IndexerGenerator.createIndexer over javax.lang.model: String/char/numeric/time/UUID/enum/Comparable/multi-value/custom, plus BINARY, BIT_SLICED, unique-implies-binary and spatial. - Reflection-free accessor selection: direct field when reachable, otherwise an accessible getter; records via component accessors. - Generated source uses imported simple names, not FQNs. - Configurable metamodel suffix via -Agigamap.metamodel.suffix (default "_"). - Custom @Index(creator=...) and full-text/vector annotations are out of scope and skipped with a note; those stay on the runtime path. New `gigamap-codegen-test` (test-only module) runs the processor over representative entities and a record, verifying queries through the generated constants, spatial near(), idempotent reload, and indexer-flavor parity with the runtime generator.
…ring Extends the index metamodel processor to cover the integration-module annotations. For an entity with @fulltext / @vector, the generated <Entity>_ now also emits: - a reflection-free DocumentPopulator subclass (Lucene) reading each @fulltext member through its accessor, and a Vectorizer subclass (JVector, embedded mode) reading the @vector float[]; - registration in registerIndices(): LuceneIndex.Category(LuceneContext) for full-text, and register-or-get VectorIndices + idempotent ensure() with the config from @vector for the vector index. The processor stays dependency-light: it depends only on gigamap-core and reads @FullText/@vector via the javax.lang.model mirror API by FQN, so bitmap-only users don't pull lucene-core/jvector onto their annotation-processor path; the generated code references those types, which are on the user's classpath whenever the annotations are used. Unlike bitmap indices there is no typed query constant (Lucene/vector indices are per-map runtime objects). Lucene directory/analyzer and @vector(onDisk=true) are not expressible in annotations: defaults are in-graph/in-memory, and onDisk=true is skipped with a note, staying on the runtime VectorAnnotationHandler. Tests (codegen-test, lucene/jvector added test-scope only): full-text query, vector search, a mixed entity wiring all three families, reload safety, the onDisk note, and parity with the runtime Lucene/Vector annotation handlers.
…rocessor) Add a "Compile-time metamodel (annotation processor)" section to the annotation-based indexing page covering gigamap-codegen: what it generates (<Entity>_ typed constants + idempotent registerIndices), the maven-compiler-plugin annotationProcessorPaths wiring, querying through the generated constants, @FullText/@vector wiring via the runtime LuceneIndex/VectorIndices handles, the metamodel suffix option, and the creator/onDisk limitations. Cross-link it from the bitmap "Defining Indices" retrieval section as a compile-time alternative to keeping the runtime GeneratedIndices handle.
Custom-creator indices were previously skipped with a note; the generated <Entity>_ now wires them up like any other index. For each @Index(creator = C.class) the processor emits a typed Indexer<Entity, K> constant backed by a private static helper: - a plain creator is instantiated reflection-free, `new C().create()`; - a Creator.MemberAware is handed the resolved index name and the reflective member (Entity.class.getDeclaredField/Method + trySetAccessible + initialize) before create(), mirroring IndexerGenerator.instantiateCreator. K is taken from the creator's Creator<E,K> binding when concrete (else a wildcard), so the constant supports equality queries (Entity_.code.is(value)); the runtime indexer flavor is opaque, so flavor-specific methods are not exposed. The constant joins the normal constants list with its @unique / @Identity flags, so registerIndices wires it (ensureAll / addUniqueConstraints / setIdentityIndices) unchanged. Graceful fallback: a creator that is not referenceable from the entity's package (private, non-static nested, or without an accessible no-arg ctor) is skipped with a note and left to the runtime generator, keeping the generated source compilable. Tests: CreatorBean (member-aware upper-casing, plain, and a @unique binary creator) + PrivateCreatorBean (inaccessible-creator fallback), covering query, unique-constraint enforcement, and parity with the runtime generator. Docs: annotations.adoc updated to describe creator support.
- gigamap-codegen compiles with -proc:none, so a clean build does not try to load the not-yet-compiled processor it registers via META-INF/services. - gigamap-codegen-test drops the JDK-21-only -proc:full flag; the explicit annotationProcessors + annotationProcessorPaths already enable processing.
Contributor
There was a problem hiding this comment.
Pull request overview
Introduces a new gigamap-codegen annotation processor that generates compile-time, reflection-free-ish GigaMap index metamodels (<Entity>_) with typed bitmap/spatial index constants and an idempotent registerIndices(GigaMap) helper, plus a gigamap-codegen-test module validating parity with runtime generators/handlers and Lucene/JVector wiring.
Changes:
- Add
gigamap-codegenmodule implementingIndexMetamodelProcessor+ supporting emit/import utilities. - Add
gigamap-codegen-testmodule that compiles against the processor and tests bitmap/spatial/full-text/vector + creator index behavior and reload/idempotency. - Extend docs to describe and reference the compile-time metamodel processor.
Reviewed changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| gigamap/pom.xml | Adds codegen and codegen-test modules to the GigaMap reactor build. |
| gigamap/codegen/src/main/resources/META-INF/services/javax.annotation.processing.Processor | Registers IndexMetamodelProcessor for non-JPMS discovery. |
| gigamap/codegen/src/main/java/org/eclipse/store/gigamap/codegen/IndexMetamodelProcessor.java | Annotation processor that discovers entities/members, emits <Entity>_ source, and generates registerIndices wiring (bitmap/spatial/Lucene/JVector/creator). |
| gigamap/codegen/src/main/java/org/eclipse/store/gigamap/codegen/IndexerEmitter.java | Generates source for typed bitmap/spatial indexer constants and Lucene/JVector helper nested classes. |
| gigamap/codegen/src/main/java/org/eclipse/store/gigamap/codegen/Imports.java | Utility to manage imports and resolve type references for generated source. |
| gigamap/codegen/src/main/java/module-info.java | Declares JPMS module and provides the processor via provides ... with .... |
| gigamap/codegen/pom.xml | New processor module POM; disables annotation processing while compiling the processor itself. |
| gigamap/codegen/LICENSE | Adds EPL-2.0 license file for the new module. |
| gigamap/codegen-test/pom.xml | New test-only module POM that runs the processor during testCompile and brings Lucene/JVector deps in test scope. |
| gigamap/codegen-test/LICENSE | Adds EPL-2.0 license file for the new test module. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/Article.java | Record-based entity fixture for metamodel generation tests. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/City.java | Spatial index fixture entity for metamodel generation tests. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/Color.java | Enum fixture used by bitmap index tests. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/CreatorBean.java | Fixture entity with custom @Index(creator=...) variants including MemberAware. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/CreatorMetamodelTest.java | Tests creator-backed constants, unique constraint wiring, runtime parity, and inaccessible creator fallback. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/Doc.java | Lucene @FullText fixture entity for generated metamodel wiring tests. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/GeneratedLuceneVectorTest.java | Tests generated Lucene/JVector registrations, reload safety, and parity with runtime handlers. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/GeneratedMetamodelTest.java | Tests bitmap/spatial constants, idempotent registration, reload safety, and runtime flavor parity. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/MixedDoc.java | Fixture entity combining bitmap + Lucene + vector annotations. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/OnDiskItem.java | Fixture entity validating @Vector(onDisk=true) skip/fallback behavior. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/Person.java | Main bitmap-index fixture covering index kinds (unique/identity/bit-sliced/enum/multi-value/getter annotation). |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/PrivateCreatorBean.java | Fixture entity validating inaccessible creator skip/fallback behavior. |
| gigamap/codegen-test/src/test/java/org/eclipse/store/gigamap/codegen/test/VectorItem.java | JVector fixture entity with private embedding read via getter. |
| docs/modules/gigamap/pages/indexing/bitmap/defining.adoc | Adds a TIP pointing readers to the compile-time metamodel processor. |
| docs/modules/gigamap/pages/indexing/annotations.adoc | Adds a new section documenting the compile-time metamodel annotation processor and usage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Sanitize generated constant identifiers: an index name that is not a valid Java identifier (e.g. @SpatialIndex(name = "geo-index")) is now mapped to a legal identifier (geo_index) instead of producing uncompilable source. - Only read members the generated metamodel can actually reach: it lives in the entity's package and is not a subclass, so a new isAccessibleFrom check (public, or declared in the entity's package) gates direct field access, getter calls, and the private-field getter fallback. Index-annotated members inherited from a different-package superclass with protected/package-private visibility now resolve through an accessible getter or report an error, rather than emitting code that does not compile. Tests: ReviewFixesTest - a spatial index named "geo-index" (sanitized constant) and an entity inheriting index-annotated members from a different-package BaseEntity (public field read directly, private field via its public getter).
The compile-time metamodel note claimed the generated code never uses reflection, which stopped being true once Creator.MemberAware creators were supported (their member is resolved via getDeclaredField/getDeclaredMethod). Scope the note to the bitmap/spatial/full-text/vector reads and call out the MemberAware exception, clarifying that its reflection targets the entity's own module and so still needs no JPMS opens.
…sing type A nested entity's metamodel was named only after its simple name and placed in the entity's package, so an inner class Customer produced Customer_ and silently shadowed the metamodel of a same-named top-level Customer in the same package (confusing compile errors). Qualify nested entities with their enclosing type names (Outer.Customer -> Outer_Customer_); top-level entities are unchanged.
zdenek-jonas
approved these changes
Jun 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Introduces
gigamap-codegen— an annotation processor that generates a compile-time index metamodel for GigaMap entities. For each annotated entityPerson, it emits a siblingPerson_with typedpublic static finalindexer constants and an idempotentregisterIndices(GigaMap)helper:It's the reflection-free, compile-time counterpart to the runtime
IndexerGenerator— same indices, but with compile-time names, IDE autocomplete, and refactor-safety. Opt in by adding the processor tomaven-compiler-pluginannotationProcessorPaths(compile-time only).Coverage
@Index/@Unique/@Identity/@SpatialIndexComparable, multi-value, binary, bit-sliced, spatial) + registration@FullText(Lucene)DocumentPopulator+ index registration@Vector(JVector)Vectorizer+ index registration@Index(creator = …)Indexer<E,K>from the creator (plain orMemberAware) + registrationThe processor depends only on the GigaMap core and reads the Lucene/JVector annotations by name, so bitmap-only projects don't pull those libraries onto the processor path.
Notes
Entity_.x.is(value)for creators with a concrete key type.@Vector(onDisk=true)and creators not reachable from the entity's package are skipped with a note and left to the runtime path — the metamodel complementsIndexerGenerator, both resolve indices by name and interoperate.-Agigamap.metamodel.suffix=(default_); builds clean on JDK 17–21+.Layout & tests
gigamap-codegen(the processor) +gigamap-codegen-test(test-only; Lucene/JVector test-scope).mvn -pl gigamap/codegen install && mvn -pl gigamap/codegen-test clean test→ 17 tests green, covering query, reload, parity with the runtime generators/handlers, and the skip-fallbacks.