Background (threaded) import of resources to allow using the editor while during import

[akien-mga]

Operating system or device, Godot version, GPU Model and driver (if graphics related):
All systems, current master branch.

Issue description:
Importing and re-import assets for the new 3.0 workflow can take quite some time, and the project can't be edited while the import process is running.

Moving the import logic to a thread could alleviate this inconvenience. This should likely be implemented for 3.0 Beta, so that users can test thew new features with a better workflow.

[Zylann]

Could we have a progress bar / spinner with text somewhere in a corner so that we know the editor does some background work?

Also I remember ProceduralSky was also moved to a thread when it gets modified, is that also eligible for such "background jobs"?

[ghost]

+1 this.. the only thing that irritates me to no end in 3.0 is editing a .png and saving it then alt-tabbing to Godot and having to wait for that 100-500ms

It's fast now because i'm editing tiny little 5kb images, however when I edit an atlas or whatnot I'm sure it'll be even more annoying. I liked how it was before i have no idea why i have .import files all over my damn directories as well lol

Thought they'd all stay in .import and i don't have to deal with them but now there all over >_<

[Zylann]

@Dillybob92 ever worked with Unity? ^^
If these import files aren't there, they should be somewhere else, because they can't be stored within the asset itself. You have to deal with them because they represent how you decide to import each asset, and how to export them as well depending on platform. They are useful to be versionned too, and must be considered as well if you move the asset somewhere else or give it to a team mate.

Also I thought about the following:
let's say I put loads of assets in Godot, or a few big ones, it will start import all of them automatically. It could take a lot of time in background, but then, what if I want to change how they get imported WHILE they are importing? Because, maybe I don't want Godot to do all this work for no reason^^

[ghost]

@Zylann I see now thank you --

I guess it's similar to the old .flags file we had in 2.xx. I just found out that when deleting images in the Godot FileSystem tab, it also deletes the corresponding import files/etc, which is nice.

I originally thought we had to manually delete all corresponding .import files. Looks like the new 3.0 is now moving us towards to managing files inside the FileSystem tab for max efficiency.

Guess I just need to get used to it :D

[toger5]

Could we maie those files hidden ? .filename.jpeg.import? Or would that ibroduce other issues?

[Zylann]

@toger5 might help browsing files, but again, you still need to remember they are here and required.
In Unity I think they have an option to put meta files in the import folder, but it makes versionning a bit harder.

[reduz]

Background import in itself is not worth it, as ensuring thread safety of the import process is to much work. What we need to do is for textures

…

On Sep 10, 2017 11:32 AM, "Marc" ***@***.***> wrote: @toger5 <https://github.com/toger5> might help browsing files, but again, you still need to remember they are here and required. In Unity I think they have an option to put meta files in the import folder, but it makes versionning a bit harder. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#10583 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AF-Z27YqCDdbndoVzQP77bskdW_1lJBtks5sg_LlgaJpZM4O_416> .

[Zylann]

I'm curious to know why this import is so slow. I understand there is a compression optimization thingy at work, but the speed of it is currenly ridiculous, is it really supposed to be that slow? Isn't there something wrong going on?

[Zireael07]

Etc2 is making it slow AFAIK

[Zylann]

Well, yeah, my question is why^^

[akien-mga]

Kicking to 3.1, the import system has already been made more forgiving and a bit faster; multithreading can be done later on.

[Calinou]

Bugsquad note: This issue is still relevant as of commit d87307d.

[Calinou]

To make reimporting feel faster (if you can spare the increased CPU usage), we could have an option to reimport files continuously even if the editor isn't focused. This would require listening for changes on the filesystem to avoid polling.

Someone should also try hiding the EditorProgress dialog and check if importing is any faster, as it's known to slow down the processes it's involved in. (Of course, we can't remove it for long operations, but hiding it for short operations might be an option.)

Edit: Proposal opened: godotengine/godot-proposals#3252

[Calinou]

In 4.0, resource importing is now multi-threaded and ETC2 importing was sped up significantly. This should make import speeds in 3D projects much more bearable overall, closing. See #10583 (comment) as for why background importing is not viable for Godot's use cases.