Introduce JavaTypeFactory and extend JavaSourceSet with classpath fast paths#7528
Merged
knutwannheden merged 1 commit intomainfrom Apr 30, 2026
Merged
Introduce JavaTypeFactory and extend JavaSourceSet with classpath fast paths#7528knutwannheden merged 1 commit intomainfrom
JavaTypeFactory and extend JavaSourceSet with classpath fast paths#7528knutwannheden merged 1 commit intomainfrom
Conversation
Surface area for parser-time type construction is now centralized on JavaTypeFactory and threaded through every TypeMapping (Java 8/11/17/21/25, Groovy, Kotlin/KotlinIr, Scala, JavaReflection). Each compute* method returns the cached instance keyed by signature, or allocates a stub, registers it, and runs the supplied initializer. Registering the stub before the initializer runs is what makes recursive resolution safe — a recursive lookup for the same signature finds the stub instead of looping. The previous two-phase create*/populate* surface is gone. DefaultJavaTypeFactory implements compute* directly. Initializers call unsafeSet on the stub to populate fields, replacing the earlier typeFactory.populateXxx indirection. JavaTypeFactory.Provider is the parser-builder hand-off: parsers (JavaParser, Java*Parser, GroovyParser, KotlinParser, ScalaParser, GradleParser) and JavaTemplateParser all consume a Provider rather than a JavaTypeCache directly. JavaTypeCache itself is unchanged but the cache() configuration on parser builders is deprecated in favor of typeFactory(). RecipeScheduler exposes a rootCursorProvider so a recipe-scoped JavaTypeFactory can flow into JavaTemplate parses, keeping splice-time type identity consistent with the surrounding LST. JavaSourceSet gains two fast-path accessors for recipes that index by classpath: - findClasspathType(fqn) — sorted-bsearch on a per-source-set ClasspathIndex SPI, falls back to linear scan for legacy markers. - classpathTypesInPackage(pkg) — sorted-prefix range scan. Both go through ClasspathIndex.Subset, an SPI marker producers (serializer V4 lazy backings) implement to short-circuit. ImportLayoutStyle uses the per-package fast path. removeTypesMatching / removeTypesForGav also dispatch through ClasspathIndex#withGavsRemoved when the marker backing supports it, preserving lazy classpath views across recipe edits. JavaSourceSetCompat retains the legacy materialization path under @toBeRemoved(after = "2026-06-30") for recipes still calling JavaSourceSet.build(...) directly. RecipeClassLoader parent-loads JavaTypeFactory so recipe-side and parser-side factory instances unify. TypeTable now writes per-artifact JARs (instead of classes-directories) so javac uses ArchiveContainer with a cached central directory rather than DirectoryContainer, which calls openat() per list() during template parsing. Wall-mode profiling on AssertJ recipe / spring-data-commons: __open worker samples 921 → 157 (-83%); recipe wall time 1m23s → 1m16s. TypeTable annotations are now structured: AnnotationDeserializer is a public class, TypeSignature parses ASM signatures, and TypeTableSink is the abstract sink interface so writers don't go through string round-trip.
JavaTypeFactory and extend JavaSourceSet with classpath fast paths
Member
|
Nice 👍 |
steve-aom-elliott
added a commit
that referenced
this pull request
May 1, 2026
After #7528 TypeTable materializes artifacts as jar files inside the version directory rather than as a classes directory. The .tt branch in gavFromPath used fixed offsets that worked for the legacy directory layout but mis-sliced GAV components for the new jar layout. Compute versionIndex/artifactIndex off the tail and shift left by one when the trailing path component ends with .jar, so the slice points at the right components in either layout.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Parser-time type construction is independently re-implemented in every TypeMapping (Java 8/11/17/21/25, Groovy, Kotlin, KotlinIr, Scala, JavaReflection). Each one caches differently, uses a different idiom for breaking recursive resolution, and accesses
JavaTypeCachedirectly. Recipes that want to influence parser type identity — most importantly, sharing types across aJavaTemplatesplice and the surrounding LST — have no abstraction to hook into.JavaSourceSetexposes a flatList<JavaType.FullyQualified>for the classpath. Recipes that look types up by FQN or by package have been doing linear scans, which doesn't scale on classpaths with thousands of types. There is no SPI for marker producers to plug in a sorted/indexed backing.TypeTablewrites per-artifact classes-directories. When those land onJavaTemplate's parser classpath, javac usesDirectoryContainer, which callsopenatperlist(). On parsing-heavy recipe runs the syscall cost can dominate wall time.Summary
JavaTypeFactoryorg.openrewrite.java.internal.JavaTypeFactoryinterface with acompute*family covering the seven parser-constructedJavaTypekinds:computeClass,computeMethod,computeVariable,computeArray,computeIntersection,computeGenericTypeVariable,computeParameterized.compute*returns the cached instance keyed by signature, or allocates a stub, registers it in the cache before invoking the supplied initializer, then runs the initializer to populate the stub. Registering up front is what makes recursive resolution safe — a recursive lookup for the same signature finds the stub instead of looping.DefaultJavaTypeFactoryimplementscompute*directly. Initializers populate the stub by callingunsafeSeton it.ReloadableJava{8,11,17,21,25}TypeMapping,GroovyTypeMapping,KotlinTypeMapping,KotlinIrTypeMapping,ScalaTypeMapping,JavaReflectionTypeMapping.JavaTypeFactory.ProviderJavaParser,Java*Parser,GroovyParser,KotlinParser,ScalaParser,GradleParser) accept aProvider. The legacytypeCache(JavaTypeCache)builder is@Deprecatedin favor oftypeFactory(JavaTypeFactory).JavaTemplateParserconsumes the sameProviderfrom the recipe-scoped root cursor (viaTYPE_FACTORY_PROVIDER_KEY).RecipeSchedulerexposesrootCursorProvider(Supplier<Cursor>)so a recipe-scopedJavaTypeFactorycan flow intoJavaTemplateparses, keeping splice-time type identity consistent with the surrounding LST.RecipeClassLoaderparent-loadsJavaTypeFactoryso recipe-side and parser-side factory instances unify.JavaSourceSetclasspath fast pathsJavaSourceSet.ClasspathIndexSPI: marker producers can plug in a sorted/indexed backing for the classpath list.findClasspathType(fqn)— sorted-bsearch viaClasspathIndex.findFullyQualified, falls back to linear scan for legacy markers.classpathTypesInPackage(pkg)— sorted-prefix range scan viaClasspathIndex.typesInPackage, same fallback.removeTypesMatching/removeTypesForGavroute throughClasspathIndex.withGavsRemovedwhen the marker backing supports it, so an indexed classpath view is preserved across recipe edits instead of being collapsed into a flat list.ImportLayoutStyleuses the per-package fast path.JavaSourceSetCompatretains the legacy materialization path under@ToBeRemoved(after = "2026-06-30")for recipes still callingJavaSourceSet.build(...)directly.TypeTablewrites per-artifact JARs(groupId, artifactId, version), written via temp file + atomic move.ArchiveContainerfor these (central directory cached once) instead ofDirectoryContainer. On a parsing-heavy recipe run (AssertJ migration onspring-data-commons), worker__opensamples drop 921 → 157 (-83%) on async-profiler wall mode; recipe wall time drops ~9%.AnnotationDeserializer,TypeSignature, andTypeTableSinkare extracted as public types so writers can produce structured annotation data without string round-tripping.Test plan
gw checkclean across all modules (53m, 162 tasks, 0 failures)JavaTemplateParserProviderTestcoversProviderplumbing throughJavaTemplateTypeTableTestupdated to assert JAR entries instead of directory contentsJavaSourceSet.build(...)path covered byJavaSourceSetCompatand remaining recipe tests