Introducing: Yatoclip

[ishland]

A faster jardiff system

How it builds yatoclip:

• Compares server artifact with member-mapped vanilla minecraft jar.
• Creates diffs under class level
• Pack it

How it patches jar:

• Fetches BuildData and vanilla jar
• Apply spigot mappings to member-mapped level
• Apply generated patches under class level
• Copy other files which is untouched by patching
• Pack it

FAQ

Q: Why not use final-mapped jars
A: Final-mapped jar outputs are different each time

[TheDGOfficial]

This so much slower than Paperclip in my testing, and prints too much verbose messages. I don't exactly know what paperclip does but it takes little time to do its thing, and it works. But this more like Spigot's BuildTools, which takes time.

I was running git-Yatopia-17 on my test server and found out 18 and 19 is released, so decided to try them. (I'm not using Yatopia, but I have a starting script which auto downloads paper by default for selected game version, and configurably, yatopia)

Also, the new version has different version string:
[20:18:41 INFO]: This server is running Yatopia version git-Yatopia-ver/1.16.5-19 (MC: 1.16.5) (Implementing API version 1.16.5-R0.1-SNAPSHOT)

This was just git-Yatopia-17 before. Including full branch name in the message is good when user uses a non-standard branch (i.e a PR one) but is it also necessary for main branch?

The new version also prints this warning in Java 15:

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.

(This message is not printed on Paperclip and when on git-Yatopia-17)

I don't know why and where getCallerClass is used, but, can't you can just use new Throwable().getStackTrace() or Thread.currentThread().getStackTrace()?

[duplexsystem]

Slower in what regard?

[toinouH]

I have this warning on java 14 too when I launch the server. For the rest we should wait Ishland :)

[HookWoods]

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
This warning appears after java 9 since the sun package has been deprecated in Java 9 and been removed in Java 10 or java 11 IIRC. So no, this is not a problem. In terms of performance, how many seconds it takes to launch with yatoclip?

[TheDGOfficial]

WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
This warning appears after java 9 since the sun package has been deprecated in Java 9 and been removed in Java 10 or java 11 IIRC. So no, this is not a problem. In terms of performance, how many seconds it takes to launch with yatoclip?

How is that not a problem? It uses an internal unsupported Java API which is forbidden in modular Java apps, and will be completely removed in Java 16 (JEP 396: Strongly Encapsulate JDK Internals by Default) for my understanding.

Slower in what regard?

Start-up, both first and other runs.

First run:

java version "15.0.2" 2021-01-19
Java(TM) SE Runtime Environment (build 15.0.2+7-27)
Java HotSpot(TM) 64-Bit Server VM (build 15.0.2+7-27, mixed mode, sharing)
Downloading version.json...
Downloading BuildData...
Downloading vanilla jar...
Setting up SpecialSource: cache\1.16.5\BuildData\bin\SpecialSource-2.jar
Applying class mapping...
Invoking SpecialSource with arguments: [map, --only, ., --only, net/minecraft, --auto-lvt, BASIC, --auto-member, SYNTHETIC, -i, <server folder>\cache\1.16.5\Minecraft\1.16.5.jar, -m, <server folder>\cache\1.16.5\BuildData\mappings\bukkit-1.16.5-cl.csrg, -o, <server folder>\cache\1.16.5\Minecraft\1.16.5-cl.jar]

Applying member mapping...
Invoking SpecialSource with arguments: [map, --only, ., --only, net/minecraft, --auto-member, LOGGER, --auto-member, TOKENS, -i, <server folder>\cache\1.16.5\Minecraft\1.16.5-cl.jar, -m, <server folder>\cache\1.16.5\BuildData\mappings\bukkit-1.16.5-members.csrg, -o, <server folder>\cache\1.16.5\Minecraft\1.16.5-m.jar]

Releasing SpecialSource
Applying patches...
Yatoclip server setup completed in 205063.68ms
System Info: Java 15 (Java HotSpot(TM) 64-Bit Server VM 15.0.2+7-27) Host: Windows 10 10.0 (amd64)
Loading libraries, please wait...
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.

(205 seconds/3.4 minutes)

Other runs:

Yatoclip server setup completed in 13117.53ms

(13 seconds)

The normal server start-up performance is not affected. (The Done! in X ms message) Not tested the runtime performance.

My machine is not that fast/high-end but it was like maximum 10 seconds even for the first run on Paperclip or git-Yatopia-17.

[duplexsystem]

How is that not a problem? It uses an internal unsupported Java API which is forbidden in modular Java apps, and will be completely removed in Java 16 (JEP 396: Strongly Encapsulate JDK Internals by Default) for my understanding.

This is only necessary to get it to run on Java 8. Paperclip does a similar thing with different modules. Once 1.17 is out we will be dropping Java 8 for Java 11. So we can remove this hack.

The slow downs with patching are expected. This is a trade off we made in order to get smaller jar files and orders of magnitude fast build times. This should be mitigated by the patched jar cache.

[HookWoods]

How is that not a problem? It uses an internal unsupported Java API which is forbidden in modular Java apps and will be completely removed in Java 16 (JEP 396: Strongly Encapsulate JDK Internals by Default) for my understanding.
Sun package has been removed since java 10 or 11 as I said, so if the package isn't present, it can use it and this system not work on other versions > 1.8 :)