Getting class bytes of other classes in a transformer

[LunNova]

Why do transformers depend on bytecode of other classes?

• Finding superclasses/interfaces
• • Stackmap generation (getCommonSuperClass()) - required when adding new control flow
• • SpongePowered/Mixin - ClassInfo
• Find existing Methods and fields
• • javassist - needs symbols in other classes to compile source code against them
• • SpongePowered/Mixin - ClassInfo

This list is probably incomplete.

Current situation - dangerous and slow

Existing mods which depend on previous class bytes do this in one of three ways:

• Load the class they need info on, causing that class to be fully transformed
• • "Reentrant transformer"
• • Causes classloading order to change
• • infinite recursion if two transformers do this on classes and both depend on the class bytes of each other's class
    can't be used in general on all classes due to infinite recursion risk.
    always acceptable if used only for superclass/interface loading as calling defineClass will cause these classes to get loaded anyway
• Get class bytes from disk, run through earlier transformers again
• • Slow
• • Runs transformers multiple times
• • Changes classloading order with reentrant transformer, may fail completely
• • Depends on reflection into LaunchClassLoader
• Get class bytes from disk, run only through some hardcoded non-reentrant transformers
• • Slow
• • Wrong info - if a transformer before you which isn't one of your hardcoded transformers changes super classes/adds fields you won't know about this
• • Depends on reflection into LaunchClassLoader

Current known reentrant/retransforming transformers

• net.minecraftforge.fml.common.asm.transformers.EventSubscriptionTransformer - reentrant
• net.minecraftforge.fml.common.asm.transformers.TerminalTransformer - reentrant
• org.minimallycorrect.modpatcher.api.ModPatcherTransformer$ClassTransformer - retransforms using hardcoded transformer list only
• org.spongepowered.asm.mixin.transformer.Proxy - retransforms using non-reentrant transformers only (hardcoded/additionally runtime detected)

This list is probably incomplete.

getCurrentClassBytes(name) - simple, less broken solution

The simplest solution is that when requesting the current class bytes by name inside a transformer, we run that class through all transformers until it reaches the current transformer, then store this result, then return it.

When it later comes to time to transform this class normally, the cached partial transformation will be reused, and we won't re-run the earlier transformers.

A problem occurs when you need to access a class which has already been fully transformed or has been partially transformed further than the current transformer.
Do you run it through all the earlier transformers again? Do you give it the already fully transformed result? Do we store all intermediate transformation results?

The first option causes poor performance, the second is wrong and could cause subtle bugs (different result depending on classloading order), the third causes awful memory usage.

Multi pass transformation before minecraft launch - big change from the current way, more correct?

An alternative would be to change transformation to run in passes and on all classes.
We would run all classes through transformer A. All classes through transformer B, and so on. The results would be stored in memory until the final class, then stored to disk as binpatches (or the final classes? Is there a copyright issue here? Don't think so as it's only on disk on the user's machine).
This process would happen only on the first launch, or when a new transformer is added (new mod with transformer), or a transformer signals that the cache should be invalidated. (Similar to #3)

Existing transformers which don't depend on classes existing already could continue to work. Transformers which are reentrant or retransforming would need updated.

[cpw]

This is a complicated issue. We're not going to frontload every possible class - that is not going to happen - it's slow and unecessary for the most part. Getting metadata about classes is definitely something that is planned for. I have ideas for a metadata service that should be able to help with a lot of what you're discussing here.

[LunNova]

Do you have a rough outline of how your metadata service would avoid loops?

[jredfox]

what if and here me out. That you always made sure super classes loaded first fully before transformer sub classes? there still might be an issue if superclasses load sub classes. Maybe instead of parsing the class of sub classes from a super class you could get it's bytes without launching your own asm system preventing recursion loops. I don't see any issues with this accept for access transformations which can be done using the access transformers. Also after the superclass is loaded you can ignore it on the transformers as already loaded.

Hope you guys find a solution cannot edit anything extending EntityLivingBase in forge 1.12.2. Not even a simple line edit like editing EntityPlayerMP#canUseCommand() to not hard code the seed. Also hope a solution will get into 1.12.2, 1.13 last time I checked was really laggy for me

[jredfox]

This will help you guys. It's simple clean and can be adapted using ob/deob map and redistribution is allowed just have to copy the comments that also contains the license according to their github
https://github.com/llbit/ow2-asm/blob/master/test/conform/org/objectweb/asm/ClassWriterComputeFramesTest.java#L132

Already got it converted and patched for mc 1.12.2. https://github.com/jredfox/evilnotchlib/blob/master/src/main/java/com/evilnotch/lib/asm/util/ComputeClassWriter.java