refactor: Make game data pluggable #691

Rhuagh · May 1, 2018

Remove Dispatcher from Application, and add a pluggable generic type.
Add a direct replacement for the old game data, featuring only a Dispatcher.
Make the states take a new StateData type, containing references to World and pluggable game data.
Move responsibility for doing system dispatch to the states.

Opening this early to get feedback.

Resolves #576

This change is

I'll need a more complex example to really get the feel of it.
Maybe an example with:
-Global systems
renderer,input,ui,assets
-StateLoading
waits for assets to be all loaded, shows a ui specific to this state (loading ui), which get removed automatically when changing state (because when you switch you lose this World)
-StateGame
simple ui showing the loaded asset

The prototype looks good ;)

jojolepro · May 1, 2018

I'll need a more complex example to really get the feel of it.
Maybe an example with:
-Global systems
renderer,input,ui,assets
-StateLoading
waits for assets to be all loaded, shows a ui specific to this state (loading ui), which get removed automatically when changing state (because when you switch you lose this World)
-StateGame
simple ui showing the loaded asset

The prototype looks good ;)

The World is not replaced, it's just the Dispatcher that's been replaced with a generic type.

Rhuagh · May 2, 2018

The World is not replaced, it's just the Dispatcher that's been replaced with a generic type.

I was thinking of maybe adding a loading state to the renderable example, since some of those assets take a while to load in debug mode.

Rhuagh · May 2, 2018

I was thinking of maybe adding a loading state to the renderable example, since some of those assets take a while to load in debug mode.

This is basically my stab at #576.

Because we use World for a lot of things internally in Application and a lot of the core, my proposal is basically to keep the core World in place, and simply abstract away the Dispatcher, and any "extra" data that the user might want to keep centrally. The naming of the different types could use improvements, but the core concept is solid I believe.

Rhuagh · May 2, 2018

This is basically my stab at #576.

Because we use World for a lot of things internally in Application and a lot of the core, my proposal is basically to keep the core World in place, and simply abstract away the Dispatcher, and any "extra" data that the user might want to keep centrally. The naming of the different types could use improvements, but the core concept is solid I believe.

Xaeroxe · May 3, 2018

Xaeroxe approved these changes May 3, 2018

View changes

I was pleasantly surprised with how this turned out.

LGTM!

I think it might be a good idea to add a chapter to the book called Advanced Game Architecture that shows how to use multiple dispatchers in a custom GameData structure.

Xaeroxe · May 3, 2018

I think it might be a good idea to add a chapter to the book called Advanced Game Architecture that shows how to use multiple dispatchers in a custom GameData structure.

I was planning on creating an example atleast once I get some feedback on the basic design :)

Rhuagh · May 3, 2018

I was planning on creating an example atleast once I get some feedback on the basic design :)

I'll re-review it soon.

jojolepro · May 3, 2018

I'll re-review it soon.

Aceeri · May 4, 2018

Aceeri approved these changes May 4, 2018

View changes

torkleyy · May 4, 2018

torkleyy approved these changes May 4, 2018

View changes

I like it, thanks for your work! A chapter would really be helpful, I'm not sure I understand how all the *Data types interact exactly.

torkleyy · May 4, 2018

amethyst_animation/src/util.rs

-    controls.entry(entity).unwrap().or_insert_with(AnimationControlSet::default)
+    controls
+        .entry(entity)
+        .unwrap()


This should not unwrap but return the error.

That's a known (and reported issue), yes, will fix that in a separate PR as it changes the API.

torkleyy · May 4, 2018

src/game_data.rs

@@ -0,0 +1,240 @@
+use std::sync::Arc;


Missing top-level docs

jojolepro · May 4, 2018

jojolepro requested changes May 4, 2018

View changes

That looks good, but since this is a really big breaking change, I have a few questions.

Main one:
Is it possible to have a GameData that has:
global_dispatch
state_specific_dispatch
where the state_specific_dispatch is created during a State start() method?

Basically, sometimes you want to have systems run only for some states (like in this example), but you want the state to conditionally set the systems. Also, creating it in the start (or similar) method makes it easy to find the relation between State and state_specific_dispatch.

I don't want to have
GameData
-global_dispatch
-state1_dispatch
-state2_dispatch
-state3_dispatch
-state4_dispatch
Especially since sometimes creating a System might need data loaded by a previous state. For example, a System specific to GameMode1State would require a Handle loaded by LoadingState.

jojolepro · May 4, 2018

amethyst_ui/src/layout.rs

-        (entities, mut locals, parents, anchors, stretches, screen_dim, hierarchy): Self::SystemData,
-){
+    fn run(&mut self, data: Self::SystemData) {
+        let (entities, mut locals, parents, anchors, stretches, screen_dim, hierarchy) = data;


Any reason for this?

Formatting. The previous code complained about too long row every time I ran rustfmt, and I got tired of it.

jojolepro · May 4, 2018

examples/game_data/README.md

@@ -0,0 +1,5 @@
+# Renderable Example


s/Renderable/Game Data/

jojolepro · May 4, 2018

examples/game_data/game_data.rs

+use amethyst::{DataInit, Error, Result};
+use rayon::ThreadPool;
+
+pub struct GameData<'a, 'b> {


I got really really confused, because this custom GameData has the same name has amethyst::GameData. I couldn't figure out why this one had two dispatchers and the other one had one.

Please rename it to CustomGameData.

jojolepro · May 4, 2018

examples/game_data/main.rs

@@ -0,0 +1,583 @@
+//! Demonstrates how to load renderable objects, along with several lighting


I doubt it!

jojolepro · May 4, 2018

examples/game_data/main.rs

+//! Demonstrates how to load renderable objects, along with several lighting
+//! methods.
+//!
+//! TODO: Rewrite for new renderer.


hmmmmmmmm :P

jojolepro · May 4, 2018

examples/game_data/main.rs

+
+mod game_data;
+
+struct DemoState {


I think having all the lighting stuff makes the example way more complicated than how it needs to be. Its harder to really spot what GameData does when there is so much unrelated code. A simple rotating cube with a fixed camera is all you would need I think.

The renderable example is the only one where loading is slow enough to actually warrant a Loading state, that's why I chose it.

You could keep the .obj loading, which is the slow part, but remove the lights. If not then its fine.

jojolepro · May 4, 2018

examples/game_data/main.rs

+        data.data.update(&data.world, true);
+        match self.progress.as_ref().unwrap().complete() {
+            Completion::Failed => {
+                println!("Failed loading assets");


printerr! or error!

Didn't see a need to pull in an extra crate for an example.

printerr! then?

printerr? Not a part of core rust from what I know ?

Ya sorry its "eprintln!" actually :P

jojolepro · May 4, 2018

examples/game_data/main.rs

+            .with_pass(DrawShaded::<PosNormTex>::new())
+            .with_pass(DrawUi::new()),
+    );
+    let game_data = GameDataBuilder::default()


Games don't only have a base and a running state. There should be a way to provide multiple systems that are specific to each of the states. Those systems should be added during the state start function, to allow for better logic locality.

Ofc not, but I'm not about to write a whole game in a single example, and this example is for demonstrating a custom game data implementation, not teach people how to structure a game code wise.

jojolepro · May 4, 2018

examples/game_data/resources/display_config.ron

+(
+  fullscreen: false,
+  multisampling: 0,
+  title: "Renderable Example",


Renderable again

It's a generic type without bounds, you can put anything you like in it. Data would be in the World though, so there's no need to "share" that. Personally, I would put state specific dispatchers in the state, and not in a central location.

Rhuagh · May 4, 2018

It's a generic type without bounds, you can put anything you like in it. Data would be in the World though, so there's no need to "share" that. Personally, I would put state specific dispatchers in the state, and not in a central location.

So GameData is global? If so I'm not sure what you can do with it that you couldn't do using Resource.

jojolepro · May 4, 2018

So GameData is global? If so I'm not sure what you can do with it that you couldn't do using Resource.

@jojolepro The main thrust of the PR is to permit people to access data in a way that isn't via specs. So this would be for people who don't even want a Resource.

Xaeroxe · May 4, 2018

@jojolepro The main thrust of the PR is to permit people to access data in a way that isn't via specs. So this would be for people who don't even want a Resource.

What would be the use case for that?
Is it worth the added complexity?

jojolepro · May 4, 2018

What would be the use case for that?
Is it worth the added complexity?

@Rhuagh mentioned #576 in a comment on this PR (Maybe we should mention it in the opening PR description)

If we go to that issue some helpful context is present. Specifically this quote:

As more use cases for Amethyst begin to explore the virtues and limitations of
the current data architecture some hurdles are beginning to present themselves for
both optimization and flexibility. The current ECS design forces a specific way of
thinking that works for a lot of designs but requires use of anti-pattern solutions
in others. When examining how one would implement a dense voxel map idiomatically in
Amethyst general consensus is that each voxel should be an entity within
the system. While very simple this is not an optimal structure for chunked densely
packed Voxel data accessed by range query. Thus Voxel information would be
relegated to an outside resource with array data and a second ecs world containing
Voxel definitions. The access pattern of this second ecs is random and differs
greatly from the sequential access pattern of the primary ecs iterator.

Xaeroxe · May 4, 2018

@Rhuagh mentioned #576 in a comment on this PR (Maybe we should mention it in the opening PR description)

If we go to that issue some helpful context is present. Specifically this quote:

As more use cases for Amethyst begin to explore the virtues and limitations of
the current data architecture some hurdles are beginning to present themselves for
both optimization and flexibility. The current ECS design forces a specific way of
thinking that works for a lot of designs but requires use of anti-pattern solutions
in others. When examining how one would implement a dense voxel map idiomatically in
Amethyst general consensus is that each voxel should be an entity within
the system. While very simple this is not an optimal structure for chunked densely
packed Voxel data accessed by range query. Thus Voxel information would be
relegated to an outside resource with array data and a second ecs world containing
Voxel definitions. The access pattern of this second ecs is random and differs
greatly from the sequential access pattern of the primary ecs iterator.

And how does Resource not solve this? My current plan for a voxel world involves having it as a resource and having Systems use that resource to create entities (optimised) that will end up getting rendered.

For me, this change is replicating what specs Resources is doing already.

jojolepro · May 4, 2018

And how does Resource not solve this? My current plan for a voxel world involves having it as a resource and having Systems use that resource to create entities (optimised) that will end up getting rendered.

For me, this change is replicating what specs Resources is doing already.

Yeah a Resource could be used for this. That means that to me there's one remaining major reason to merge this PR, and that is to facilitate types that aren't Any + Send + Sync.

Xaeroxe · May 4, 2018

Yeah a Resource could be used for this. That means that to me there's one remaining major reason to merge this PR, and that is to facilitate types that aren't Any + Send + Sync.

You can't store systems in a resource.

Rhuagh · May 4, 2018

You can't store systems in a resource.

@Rhuagh is correct due to this: https://docs.rs/shred/0.7.0-alpha5/shred/struct.Dispatcher.html#impl-Send

Xaeroxe · May 4, 2018

@Rhuagh is correct due to this: https://docs.rs/shred/0.7.0-alpha5/shred/struct.Dispatcher.html#impl-Send

That is a good argument. ;)
Okay then.

jojolepro · May 4, 2018

That is a good argument. ;)
Okay then.

Re-review if you can please.

Rhuagh · May 7, 2018

Re-review if you can please.

jojolepro · May 7, 2018

jojolepro approved these changes May 7, 2018

View changes

Still not a fan of all the lighting stuff, but that's fine ;)

Thanks!

When examining how one would implement a dense voxel map idiomatically in
Amethyst general consensus is that each voxel should be an entity within
the system.

Didn't we have that discussion before?

torkleyy · May 8, 2018

When examining how one would implement a dense voxel map idiomatically in
Amethyst general consensus is that each voxel should be an entity within
the system.

Didn't we have that discussion before?

Reviewed 3 of 8 files at r1, 5 of 10 files at r2, 19 of 21 files at r3, 3 of 3 files at r4, 1 of 2 files at r6, 13 of 20 files at r7.
Review status: all files reviewed at latest revision, 10 unresolved discussions.

Comments from Reviewable

torkleyy · May 8, 2018

Reviewed 3 of 8 files at r1, 5 of 10 files at r2, 19 of 21 files at r3, 3 of 3 files at r4, 1 of 2 files at r6, 13 of 20 files at r7.
Review status: all files reviewed at latest revision, 10 unresolved discussions.

Comments from Reviewable

torkleyy · May 8, 2018

torkleyy approved these changes May 8, 2018

View changes

LGTM

I thought cfg!(target_os = "ios") didn't work?
anyway
Waiting for label change + build checks to merge. @Rhuagh

jojolepro · May 9, 2018

I thought cfg!(target_os = "ios") didn't work?
anyway
Waiting for label change + build checks to merge. @Rhuagh

I still need to add a chapter to the book, and I'm currently focusing on getting specs out.

Rhuagh · May 9, 2018

I still need to add a chapter to the book, and I'm currently focusing on getting specs out.

Rebased, squashed, book chapter added. @Xaeroxe feel free to take over.

Rhuagh · May 9, 2018

Rebased, squashed, book chapter added. @Xaeroxe feel free to take over.

Ready for final review @jojolepro @torkleyy @Xaeroxe

Rhuagh · May 10, 2018

Ready for final review @jojolepro @torkleyy @Xaeroxe

outside most of the day, I'll review tonight if I can, or tomorow

jojolepro · May 10, 2018

outside most of the day, I'll review tonight if I can, or tomorow

Xaeroxe · May 14, 2018

Xaeroxe approved these changes May 14, 2018

View changes

LGTM! bors r+

err actually nevermind, I think this might fail on CI due to some new examples.

Xaeroxe · May 14, 2018

err actually nevermind, I think this might fail on CI due to some new examples.

bors r=Xaeroxe,torkleyy,jojolepro

Rhuagh · May 14, 2018

bors r=Xaeroxe,torkleyy,jojolepro

Build failed

continuous-integration/appveyor/branch

bors · May 14, 2018

Build failed

continuous-integration/appveyor/branch

bors r=Xaeroxe,torkleyy,jojolepro

Rhuagh · May 14, 2018

bors r=Xaeroxe,torkleyy,jojolepro

Build succeeded

bors · May 14, 2018

Build succeeded

Rhuagh added type: improvement diff: hard pri: important status: working project: main crate labels May 1, 2018

Rhuagh requested review from jojolepro, Xaeroxe and torkleyy May 2, 2018

Rhuagh removed diff: hard pri: important labels May 7, 2018

Rhuagh changed the title from [WIP] refactor: Make game data pluggable to refactor: Make game data pluggable May 7, 2018

Rhuagh added status: ready status: working and removed status: working status: ready labels May 7, 2018

Rhuagh added status: ready and removed status: working labels May 9, 2018

Rhuagh deleted the Rhuagh:refactor/game-data branch May 14, 2018

Xaeroxe referenced this pull request May 15, 2018
Closed
Make main functions return Results #712

amethyst/amethyst

Join GitHub today

refactor: Make game data pluggable #691

Conversation

Rhuagh commented May 1, 2018 • edited by Xaeroxe Edited 4 times Xaeroxe edited May 4, 2018 (most recent) torkleyy edited May 1, 2018 Rhuagh edited May 1, 2018 torkleyy edited May 1, 2018

Rhuagh added type: improvement diff: hard pri: important status: working project: main crate labels May 1, 2018

This comment has been minimized.

jojolepro commented May 1, 2018

This comment has been minimized.

Rhuagh commented May 2, 2018

This comment has been minimized.

Rhuagh commented May 2, 2018

This comment has been minimized.

Rhuagh commented May 2, 2018

Rhuagh requested review from jojolepro, Xaeroxe and torkleyy May 2, 2018

Xaeroxe approved these changes May 3, 2018 View changes

This comment has been minimized.

Xaeroxe commented May 3, 2018 • edited Edited 1 time Xaeroxe edited May 3, 2018 (most recent)

This comment has been minimized.

Rhuagh commented May 3, 2018

This comment has been minimized.

jojolepro commented May 3, 2018

Aceeri approved these changes May 4, 2018 View changes

torkleyy approved these changes May 4, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

jojolepro requested changes May 4, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Rhuagh commented May 4, 2018

This comment has been minimized.

jojolepro commented May 4, 2018

This comment has been minimized.

Xaeroxe commented May 4, 2018

This comment has been minimized.

jojolepro commented May 4, 2018

This comment has been minimized.

Xaeroxe commented May 4, 2018

This comment has been minimized.

jojolepro commented May 4, 2018 • edited Edited 1 time jojolepro edited May 4, 2018 (most recent)

This comment has been minimized.

Xaeroxe commented May 4, 2018

This comment has been minimized.

Rhuagh commented May 4, 2018

This comment has been minimized.

Xaeroxe commented May 4, 2018

This comment has been minimized.

jojolepro commented May 4, 2018

Rhuagh removed diff: hard pri: important labels May 7, 2018

Rhuagh changed the title from [WIP] refactor: Make game data pluggable to refactor: Make game data pluggable May 7, 2018

Rhuagh added status: ready status: working and removed status: working status: ready labels May 7, 2018

This comment has been minimized.

Rhuagh commented May 7, 2018

jojolepro approved these changes May 7, 2018 View changes

This comment has been minimized.

torkleyy commented May 8, 2018

This comment has been minimized.

torkleyy commented May 8, 2018

torkleyy approved these changes May 8, 2018 View changes

This comment has been minimized.

jojolepro commented May 9, 2018 • edited Edited 1 time jojolepro edited May 9, 2018 (most recent)

This comment has been minimized.

Rhuagh commented May 1, 2018 •

edited by Xaeroxe

Edited 4 times

Xaeroxe edited May 4, 2018 (most recent)

torkleyy edited May 1, 2018

Rhuagh edited May 1, 2018

torkleyy edited May 1, 2018

Xaeroxe approved these changes May 3, 2018

View changes

Xaeroxe commented May 3, 2018 •

edited

Edited 1 time

Xaeroxe edited May 3, 2018 (most recent)

Aceeri approved these changes May 4, 2018

View changes

torkleyy approved these changes May 4, 2018

View changes

jojolepro requested changes May 4, 2018

View changes

jojolepro commented May 4, 2018 •

edited

Edited 1 time

jojolepro edited May 4, 2018 (most recent)

jojolepro approved these changes May 7, 2018

View changes

torkleyy approved these changes May 8, 2018

View changes

jojolepro commented May 9, 2018 •

edited

Edited 1 time

jojolepro edited May 9, 2018 (most recent)