Asset hot reloading

[torkleyy]

I've figured most parts out already, however I'm stuck at the last one. After a file has changed, it's been read, parsed and turned into an asset, how should we replace the old assets?

Most likely assets live as components in the world, but still, how should we track them? Or is limiting it to the World a good idea at all?

I'm not sure whether or not we need this feature before 0.5 release, probably that needs discussion, too.

[Xaeroxe]

In order to prevent memory corruption issues we'll have to pause execution while the replacement occurs. My first reaction is to put assets in an Arc<RwLock> so we can track them and still hand out references to them. We could keep some kind of Boolean "dirty" bit (Per application, not per asset) representing that assets are ready to be replaced, and check that every frame. If they are ready to be replaced then pop the new assets from a queue and replace each one until the queue is empty.

EDIT: This also gives us an easy answer to "what happens if an asset is unloaded that's still attached to an entity?" Well the asset won't unload until the entity in question is removed from the game. In my opinion that's the best answer we could give to that question as it means the code won't spontaneously have assets disappear until it is done with them.

[zakarumych]

Global pauses are unnecessary. There are data structures which permit update of value while it is read.
We should use one that works for one writer and many readers.

[Xaeroxe]

@omni-viral I don't know of any such data structures. Can you provide one for me to read about?

[zakarumych]

@Xaeroxe I'll find link to the paper to post it here.
Our case has quiescent points so we can search in that direction.

[zakarumych]

Maybe there is no need of fancy data structures either if we can update assets between systems run

[torkleyy]

Still, how would we trace the assets?

[Xaeroxe]

We would need some kind of central AssetManager structure that keeps a HashMap<String, Arc<RwLock<T>>> so it can be referred to when the time to replace the asset comes.

[Xaeroxe]

(Said structure would also require a way to remove references so that the memory the assets consume can be freed)

[zakarumych]

Put all updates in a queue. Replace the queue atomically in maintenance phase. Flush updates into the world.

[zakarumych]

@Xaeroxe no locks please)

[Xaeroxe]

Then the update will have to occur in unsafe code, which is fine but we'll have to be much more careful about memory corruption.

[zakarumych]

Well. We should wrap unsafe code into nice and safe type and test it heavily

[torkleyy]

Not sure if the result is something we want, but what you're asking for sounds like shred's `TrustCell`.

[Xaeroxe]

Ok here's a nice approach that gives us minimal pausing: Arc<Cell<Box<T>>>

Rather than locking the value while it is replaced you just atomically call "set" on the cell and replace the Box value. No unsafe code, no locking.

[Xaeroxe]

Except that doesn't work because Box doesn't implement Copy. Yeah we'll have to use unsafe code, maybe TrustCell?

[torkleyy]

The std `Cell` only works for `Copy` types IIRC.

[zakarumych]

@torkleyy I can't find any docs on TrustCell
upd: Found it in source

[Xaeroxe]

@omni-viral It's not part of the official API, look here

[zakarumych]

@torkleyy TrustCell would work find as asset holder inside World for async read acces

[zakarumych]

One more question. How many asset producers will be there? Single or multiple?

[Xaeroxe]

@omni-viral Current code permits multiple, which is nice for asynchronous loading from multiple systems but we may require some kind of unification to implement hot reloading.

[zakarumych]

I'm suggesting following algorithm.
Have Arc<Mutex<Vec<AssetUpdate>>> inside Engine
Store all updates in this queue. Blocking of updating threads is OK I think.
At maintenance phase (between systems run) swap queue with empty one. And flush all updates into TrustCells

AssetUpdate = Box<FnOnce(&mut World) + Send> is a closure with new asset and id to find TrustCell in world

[zakarumych]

Mutex in Arc<Mutex<Vec<AssetUpdate>>> could be replaced with something that will allow not to block main thread.

[Xaeroxe]

I disagree, that's not performance critical code and in my opinion it doesn't warrant the risk of unsafety to replace Mutex in that type with a non-locking unsafe variant.

[Xaeroxe]

Alternatively if blocking is really that undesirable, we could use a channel to send the AssetUpdates

[zakarumych]

@Xaeroxe mpsc channel is the data structure with internal unsafe you were opposed to one comment earlier 😄

[Xaeroxe]

Yeah I just thought you meant some kind of TrustCell for the queue, which seemed excessive to me. Channel works much better though :)

[torkleyy]

The thing is, I'm not sure how deeply hot reloading should be integrated in the design, since it is a development feature, not something that's necessary for the actual game.

[Xaeroxe]

That is a good point. In order to make the feature work we need some kind of tracking and continuous updating though. This seems to be the easiest way to accomplish that. Perhaps we could use #[cfg(build = "debug")] to prevent hot reloading code from being in the final game.

[zakarumych]

#[cfg(build = "debug")] isn't best here. You probably want to run your game with release build during development cause it may run too slow in debug.
Instead there should be distinct feature.

[torkleyy]

That's what we planned to use `amethyst_release` for.

[Rhuagh]

If hot reloading is a pure development feature, some risk might be worth it, for a simpler solution?

[Xaeroxe]

Ok, what are you proposing? Mpsc channel seems to be a best of all worlds approach. It's a non-locking std library type with internal unsafe and no need for us to write unsafe sections of code.

[Rhuagh]

Oh, I don't know what would be best, my suggestion is just to do the simplest solution that fits the bill if it's only gonna be used for dev, and not for released product :)

[torkleyy]

So essentially that problem we discussed about (updating the asset) disappeared. This was part of my second assets rework, which introduced asset storages and handles, allowing us to control reads and writes in a central place.

[erlend-sh]

Fair amount of discussion about this in the Rust community recently thanks to @phaazon's warmy.

Seems @icefoxen is already using it successfully in ggez.

[torkleyy]

@erlend-sh I mostly like what we have currently, it fits nicely into ECS and works in parallel. warmy has dependencies, which is something that's still missing.

But at a first glance we couldn't even use warmy because it's not thread-safe.

[hadronized]

warmy is not thread-safe yet. :)