Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: migrate to gestaltv7 #4593

Closed
wants to merge 29 commits into from
Closed

Conversation

pollend
Copy link
Member

@pollend pollend commented Mar 21, 2021

Here is the initial migration to gestalt-v7.

Changes

  • update gestalt package
    • org.terasology.gestalt.assets.management.AssetManager
    • org.terasology.gestalt.naming.Name
    • org.terasology.gestalt.assets.AssetType;
    • org.terasology.gestalt.assets.management.AssetManager;
    • org.terasology.gestalt.assets.module.ModuleAwareAssetTypeManager;
    • org.terasology.gestalt.assets.module.ModuleAwareAssetTypeManagerImpl;
    • org.terasology.gestalt.module.ModuleEnvironment;
    • org.terasology.gestalt.module.ModuleRegistry;
    • org.terasology.gestalt.module.dependencyresolution.DependencyResolver;
    • org.terasology.gestalt.module.dependencyresolution.ResolutionResult;

Issues

  • something jmh-related preventing running from IntelliJ?
  • tidy use of google() repository
  • chore: remove DebugPropertySystem #4610 Crash on F11 (DebugControlSystem failure to get MouseDevice)
  • migrate the modules: many have branches under the name chore/gestalt-v7-migration

Module Changes

@pollend
Copy link
Member Author

pollend commented Mar 21, 2021

any clue why this isn't working

Terasology/engine/src/main/java/org/terasology/engine/core/module/ModuleManagerImpl.java:105:17
java: cannot access com.github.zafarkhaja.semver.Version
  class file for com.github.zafarkhaja.semver.Version not found

Comment on lines +15 to +19
// umm need for android dependencies
repositories {
google()
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have these littered around in a few places any clue how to correct these imports. looks like we are dependencies on some android dependencies ?!

@keturn
Copy link
Member

keturn commented Mar 21, 2021

Adding it to terasology-repositories in build-logic would normally be sufficient for adding a repository, but since it's a dependency of gestalt and build-logic uses gestalt itself, I think it'll also need to be in buildscript { repository { } } sections. That includes the root build.gradle as well as the templates/build.gradle for modules.

(Yeah, I'm starting to doubt the wisdom of making build-logic depend on gestalt. Or wonder if there's some way for build-logic to be responsible for its own repositories so it doesn't leak out to the other buildscript sections.)

@keturn
Copy link
Member

keturn commented Mar 22, 2021

I was just going to go look at ModuleManager and discovered that it's an interface again, undoing #4539 (which undid #1665).

did you discover a reason for that to be an interface that we missed when I submitted #4539?

engine/build.gradle Outdated Show resolved Hide resolved
@pollend
Copy link
Member Author

pollend commented Mar 22, 2021

I was just going to go look at ModuleManager and discovered that it's an interface again, undoing #4539 (which undid #1665).

did you discover a reason for that to be an interface that we missed when I submitted #4539?

I was referencing @DarkWeird older PR. I'll undo that change.

@pollend
Copy link
Member Author

pollend commented Mar 22, 2021

I'm having some strange rendering problems where you only see the last dropped item. not sure what is causing it and how to resolve the problem.

@DarkWeird
Copy link
Contributor

DarkWeird commented Mar 22, 2021

I'm having some strange rendering problems where you only see the last dropped item. not sure what is causing it and how to resolve the problem.

Dropped items has same LocationComponent because using default CopyStrategy because nui-reflect haven't reflections.cache (and Reflections didn't fetch nui-reflect at runtime, because it not part of org.terasology.nui package)

Probably it can be resolved with java 8 (reflections have several glitches at java 11... classloadersss...)
Or provide reflections.cache from nui-reflect`

@keturn
Copy link
Member

keturn commented Mar 22, 2021

If this is using MovingBlocks/TeraNUI#46, does that mean we don't need to do the change to the NUI namespaces in MovingBlocks/TeraNUI#27 at the same time?

edit: oh, wait, that's what DW's comment just prior to this one was about, things failing because a nui package is not part of org.terasology.nui.

@DarkWeird
Copy link
Contributor

If this is using MovingBlocks/TeraNUI#46, does that mean we don't need to do the change to the NUI namespaces in MovingBlocks/TeraNUI#27 at the same time?

edit: oh, wait, that's what DW's comment just prior to this one was about, things failing because a nui package is not part of org.terasology.nui.

I think use BSA's PR firsts, then provide support in nui-gestalv7 module is better then syntetic module

@keturn keturn added this to In progress in gestalt v7 migration (module & asset) via automation Mar 24, 2021
@keturn keturn added this to the v4.4.0 milestone Mar 24, 2021
@keturn
Copy link
Member

keturn commented Mar 24, 2021

Sounds like you've been learning about what works and what doesn't.

We are going to end up with a PR that changes a zillion import statements; with changing the packages of things like Name, ResourceUrn, and API, that's unavoidable. Are there any pieces of this we can pull out and do in more easy-to-review PRs?

The thing with engine/src/jmh seems like it's probably one, since it moves a bunch of stuff but I'm not really sure what that's about.

Are there other fixes that would also apply to the current code even without the gestalt change? e.g. the camera position?

@keturn
Copy link
Member

keturn commented Mar 29, 2021

I haven't tried it out yet, but I strongly suspect the thing you described with jmh being pulled in to the reflections is from this:

task cacheReflections {
description = 'Caches reflection output to make regular startup faster. May go stale and need cleanup at times.'
inputs.files sourceSets.main.output.classesDirs,
// getClassesDir from all sourceSets (for any jvm (seems) language)
configurations."${sourceSets.main.runtimeClasspathConfigurationName}"

note how the comment says "all sourceSets." Which might be useful if we had parts of engine written in Kotlin or Scala, but is probably not helping us as it is.

And yes, I'm still hoping to roll back the jmh part of this PR. Not just for size; also because I think in general we want to move in the direction of "move things from engine-test/src/test to engine/src/test" instead of "move things from engine to engine-test." Admittedly I am really not at all clear on where the jmh sources fall in that or how they are used.

@pollend
Copy link
Member Author

pollend commented Mar 29, 2021

I haven't tried it out yet, but I strongly suspect the thing you described with jmh being pulled in to the reflections is from this:

task cacheReflections {
description = 'Caches reflection output to make regular startup faster. May go stale and need cleanup at times.'
inputs.files sourceSets.main.output.classesDirs,
// getClassesDir from all sourceSets (for any jvm (seems) language)
configurations."${sourceSets.main.runtimeClasspathConfigurationName}"

note how the comment says "all sourceSets." Which might be useful if we had parts of engine written in Kotlin or Scala, but is probably not helping us as it is.

And yes, I'm still hoping to roll back the jmh part of this PR. Not just for size; also because I think in general we want to move in the direction of "move things from engine-test/src/test to engine/src/test" instead of "move things from engine to engine-test." Admittedly I am really not at all clear on where the jmh sources fall in that or how they are used.

yep, go ahead and revert the jmh part of this PR.

@keturn
Copy link
Member

keturn commented Apr 2, 2021

Getting two NPEs after I save a screen shot. One of them is coming from NUISkinEditorScreen, which is very puzzling given that I don't have that screen open nor have I used it this run at all:

[main] ERROR o.t.engine.core.TerasologyEngine - Uncaught exception, attempting clean game shutdown
com.google.common.base.VerifyException: Module "assets" not found in current module environment.
	at com.google.common.base.Verify.verify(Verify.java:124)
	at com.google.common.base.Verify.verifyNotNull(Verify.java:500)
	at org.terasology.engine.rendering.nui.editor.layers.AbstractEditorScreen.getPath(AbstractEditorScreen.java:336)
	at org.terasology.engine.rendering.nui.editor.layers.NUISkinEditorScreen.setSelectedAssetPath(NUISkinEditorScreen.java:510)
	at org.terasology.engine.rendering.nui.editor.layers.AbstractEditorScreen.loadAutosave(AbstractEditorScreen.java:299)
	at org.terasology.engine.rendering.nui.editor.layers.AbstractEditorScreen.onDraw(AbstractEditorScreen.java:157)
	at org.terasology.nui.canvas.CanvasImpl.drawStyledWidget(CanvasImpl.java:442)
	at org.terasology.nui.canvas.CanvasImpl.drawWidget(CanvasImpl.java:428)
	at org.terasology.engine.rendering.nui.internal.NUIManagerInternal.render(NUIManagerInternal.java:570)
	at org.terasology.engine.core.modes.StateIngame.renderUserInterface(StateIngame.java:231)
	at org.terasology.engine.core.modes.StateIngame.render(StateIngame.java:215)
	at org.terasology.engine.core.subsystem.lwjgl.LwjglGraphics.postUpdate(LwjglGraphics.java:89)
	at org.terasology.engine.core.TerasologyEngine.tick(TerasologyEngine.java:497)

@keturn
Copy link
Member

keturn commented Apr 2, 2021

The other is from DebugControlSystem:

[main] ERROR o.t.e.e.e.internal.EventSystemImpl - Failed to invoke event
java.lang.NullPointerException: null
	at org.terasology.engine.logic.players.DebugControlSystem.onKeyDown(DebugControlSystem.java:133)
	at org.terasology.engine.logic.players.DebugControlSystemMethodAccess.invoke(Unknown Source)
	at org.terasology.engine.entitySystem.event.internal.EventSystemImpl$ByteCodeEventHandlerInfo.invoke(EventSystemImpl.java:389)
	at org.terasology.engine.entitySystem.event.internal.EventSystemImpl.sendConsumableEvent(EventSystemImpl.java:268)
	at org.terasology.engine.entitySystem.event.internal.EventSystemImpl.send(EventSystemImpl.java:247)
	at org.terasology.engine.core.bootstrap.eventSystem.AbstractEventSystemDecorator.send(AbstractEventSystemDecorator.java:67)
	at org.terasology.engine.network.NetworkEventSystemDecorator.send(NetworkEventSystemDecorator.java:54)
	at org.terasology.engine.core.bootstrap.eventSystem.AbstractEventSystemDecorator.send(AbstractEventSystemDecorator.java:67)
	at org.terasology.engine.recording.RecordingEventSystemDecorator.send(RecordingEventSystemDecorator.java:34)
	at org.terasology.engine.entitySystem.entity.internal.BaseEntityRef.send(BaseEntityRef.java:205)
	at org.terasology.engine.input.InputSystem.send(InputSystem.java:525)
	at org.terasology.engine.input.InputSystem.sendKeyEvent(InputSystem.java:454)
	at org.terasology.engine.input.InputSystem.processKeyboardInput(InputSystem.java:383)
	at org.terasology.engine.input.InputSystem.update(InputSystem.java:131)
	at org.terasology.engine.core.modes.StateIngame.handleInput(StateIngame.java:192)
	at org.terasology.engine.core.subsystem.lwjgl.LwjglInput.postUpdate(LwjglInput.java:39)
	at org.terasology.engine.core.TerasologyEngine.tick(TerasologyEngine.java:497)

[I really wish java or github or something in this toolchain would inline sources in tracebacks.]

We're not on Java 14 yet so we don't have useful NPE messages, but that line is

so I guess it's probably mouseDevice that is null???

@keturn
Copy link
Member

keturn commented Apr 2, 2021

  • ERROR o.t.engine.registry.InjectionHelper - BlockCommands wanted WorldProviderCoreImpl injected but CoreRegistry has none.
  • ERROR o.t.engine.registry.InjectionHelper - DebugControlSystem wanted MouseDevice injected but CoreRegistry has none.

Copy link
Member

@keturn keturn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think like we have some things to clarify about the fundamental the goals of subsystem extraction.

Comment on lines 113 to 114
private Module loadEngineModule(List<Class<?>> classesOnClasspathsToAddToEngine) {
// FIXME: is `classes…toAddToEngine` gone? Did we ever use it in the first place?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you check for classesOnClasspathsToAddToEngine usage? I know it's not usually used, so maybe this is a non-issue, but I had a vague memory about it being used somewhere...

Comment on lines 245 to 246
// private void enrichReflectionsWithSubsystems(Module engineModule) {
// Serializer serializer = new XmlSerializer();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why doesn't the gestalt-v7 branch need an enrichReflectionsWithSubsystems method? v7 hasn't done away with Reflections yet…

Comment on lines +94 to +99
private void createDependency(Module parent, Module child) {
DependencyInfo dependencyInfo = new DependencyInfo();
dependencyInfo.setId(child.getId());
dependencyInfo.setMinVersion(child.getVersion());
dependencyInfo.setMaxVersion(child.getVersion().getNextPatchVersion());
parent.getMetadata().getDependencies().add(dependencyInfo);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 makes sense to have a function for this.

TODO: refactor ensureModulesDependOnEngine method (just below this one) to use this.

FIXME: note 20546f1 — getNextPatchVersion may break for this purpose when base version is a SNAPSHOT.


registry.add(nui);
registry.add(engineModule);
registry.add(discordSubsystem);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hrmmm. Not sure about adding hardcoded references to subsystems in engine's ModuleManager.

The point of extracting subsystems—as I understand it—is to make it so the engine doesn't always need to depend on them. i.e. a facade should be able to start an engine that doesn't have a DiscordRPC subsystem at all.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, at the moment its a hack to get the game to run. System is registered but it can't resolve autoconfig when it tries to scan with environment.

ensureModulesDependOnEngine();

setupSandbox();
loadEnvironment(Sets.newHashSet(engineModule), true);
loadEnvironment(Sets.newHashSet(engineModule, nui, discordSubsystem), true);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: We should switch this to the new resolveAndLoadEnvironment method. It'll do dependency resolution so we don't have to manually keep these lists in sync with every change.

@keturn
Copy link
Member

keturn commented Apr 4, 2021

Build #25 (20546f1) had 2 failing tests: http://jenkins.terasology.io/teraorg/job/Terasology/job/engine/job/PR-4593/25/

it has now regressed to 43 failures.

and PlayerConfig-to-AutoConfig was inculded in build 25, so that wasn't the problem.

@pollend
Copy link
Member Author

pollend commented Apr 4, 2021

I decided to move the discord package since trying to register it to the environment is very difficult and harder to maintain if we introduce more subsystems. @keturn

@keturn
Copy link
Member

keturn commented Apr 10, 2021

I went back a few steps to see if I could reproduce the troubles you were having with DiscordRPC. and indeed I did! so I'm not sure how I had things working as well as I remember; maybe they worked until I merged in the AutoConfig changes from develop, and I didn't re-check after that? That's my best guess.

I also discovered that we seem to have the modules using the JsonSerializer for Reflections but engine and subsystems are still on XML, which is sure to be a source of confusion.

@keturn
Copy link
Member

keturn commented Apr 10, 2021

I plan to experiment a little to see if we can work things without sacrificing the engine/subsystem separation.

The plan is something like:

  1. Make sure engine's Reflections are actually being loaded.
  2. try re-instituting enrichReflectionsWithSubsystems
  3. and probably switch all the Reflections caches to the JSON serializer for consistency, even if it's not strictly required by the above

That branch can be found at #4622

@keturn
Copy link
Member

keturn commented Apr 14, 2021

@pollend Which thing was it that called for the engine-test/…/module.txt file to move?

@pollend
Copy link
Member Author

pollend commented Apr 14, 2021

uuuh, I don't remember why

@pollend Which thing was it that called for the engine-test/…/module.txt file to move?

@jdrueckert
Copy link
Member

@keturn
Copy link
Member

keturn commented May 2, 2021

superseded by #4622

@keturn keturn closed this May 2, 2021
gestalt v7 migration (module & asset) automation moved this from In progress to Done May 2, 2021
@Cervator Cervator deleted the feature/migrate-gestalt-v7 branch October 4, 2021 02:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants