Add options to control if ".uid" files are generated

[Daylily-Zeleen]

The main purpose of this pr is to allow not generate ".uid" files for "res://addons/", by setting "editor/resource/generate_uid_files_for_addons" to false in project settings.

Sometimes plugins are not included in version control system( for example, using gd-plug to reference plugins by single script), generate ".uid" for plugin's scripts and reference them by using uid are not reasonable.

Additionally, I add "editor/resource/generate_uid_files" to allow not generate ".uid" for the project, but default is true.
Generate ".uid" for each script make the file resource manager become ugly, and we should manipulate 2 files now.
There are some personal preferences here, I believe some people also have this feeling.
For old users, they should used drag files in editor's file system dock, it should not breaks the reference relationship by using this way, and uid is unreadable, we don't use them for coding directly.
In other words, ".uid" files are not necessary for some users.

• Bugsquad edit, closes: Add an option to disable generation of UID files godot-proposals#11571

[JoNax97]

I think Godot should commit one way or the other. Leaving this as an optional feature greatly limits it's potential in the future. All future features that build up from the uid system (or the metadata-companion-file concept in general) will be impossible to implement fully if some users don't have uids available.

[Daylily-Zeleen]

@JoNax97 What features? You should provide specific situation to make your viewpoint convincing and solve the problem that I mentioned about (".uid" in "res://addons").

If you want to talk about future, to make plugins' detail files outside the version control system is a good future, too.
To be honest, plugin should be referenced like nuget in C# instead of including its detail files, right?

[JoNax97]

I agree with you that add-ons should be handled via some package management system. But until then, we cannot guarantee that a plugin does or does not rely on uids and so it's not safe to disable them for all addons.

My main concern is not with add-ons specifically but in general. Fundamental mechanisms that the engine uses to manage its internals should not be up for disabling IMO.

[Daylily-Zeleen]

I think I grab the point that you focus about.
Maybe we should just disable generate ".uid" in "res://addons/" for compatibility and enabled it for other directories.

We should make a decision before 4.4 release.

[AThousandShips]

I don't think making UID files optional makes sense, it'd just be disabling an important and helpful feature and solves issues that people have been frustrated at for years, having them be optional just resets that and creates additional sources of bugs and frustration

Now there should be a proposal discussing how to handle UID files in addons, but regardless this PR needs a proposal, this shouldn't be discussed here as per the workflow guidelines, PRs are for discussing the implementation, proposals for discussing the idea

[Daylily-Zeleen]

This pr can be superseded by #100994 now.

---

Here just talk about make the generate behavior become optional.

There have many remonstrances in #97352, even though it is merged, because not all people have spare time to focus each proposal and pr, they just pull and compile by themselves, and discovery the problem at this time.
We not disagree using uid to handle reference, we just hate it make the filesystem become ugly.
For old users, we can avoid break reference by using editor's file system dock, and we don't want when we actually need manage files outside editor, files become mess.

I very agree this comment.
And I prefer to intergrate all ".uid" into one file and add it to vcs, although it is even less reliable.

Anyway, I know this pr can't solve all problems and satisfy everyone, just provide a possible option to make godot reach to a good future.

[elenakrittik]

This pr can be superseded by #100994 now.

I don't think that PR is a sufficient replacement for this PR. It only contains the "disable for addons" part of this PR, and leaves out the "disable for project" part, which is core to the proposal this PR aims to address.

[Daylily-Zeleen]

This pr can be superseded by #100994 now.

I don't think that PR is a sufficient replacement for this PR. It only contains the "disable for addons" part of this PR, and leaves out the "disable for project" part, which is core to the proposal this PR aims to address.

Although the change of “disable for project” part seems more significant, the main purpose of this pr is to temporary avoid compatibility issue of plugins by disabling generate ".uid" files in "res://addons/".

[elenakrittik]

Although the change of “disable for project” part seems more significant, the main purpose of this pr is to temporary avoid compatibility issue of plugins by disabling generate ".uid" files in "res://addons/".

I believe that's the purpose of the PR you nominated as a replacement, #100994. This PR is linked to godotengine/godot-proposals#11571, which says "Add an option to disable generation of UID files" (i.e. disable it in general, not just for addons).

[pietru2004]

• Is there any chance that editor/resource/generate_uid_files could have popup when upgrading from lower godot version asking should it be enabled/disabled?
• Is there any chance that editor/resource/generate_uid_files could be switchable as an option when creating new project?
• Also is there chance this could be part of 4.4 before final release?

Asking cause I want to know what to expect for 4.4 regarding UID files. Basically will I be able to switch UID files off for my project.

[pietru2004]

Would somebody be able to tell how long review of this PR might take?
I mean only way I can realistically prevent uid files in my git repo is by adding them to git ignore of each of my projects.

[AThousandShips]

This is a new feature and we are in feature freeze until after 4.4 is released, so this can't be merged until then, but this idea itself is being discussed, see the comments on that for alternative solutions

[pietru2004]

Is this rl a new feature? To be able to disable another feature?
Can't a setting be added during feature freeze so specified behaviour can be disabled?

[pietru2004]

It might just me being stupid, but I believe Option for switching off UID files could be added to 4.4
Cause it would be something to control UID files creation.

[AThousandShips]

One could argue it would be part of the same feature, like the UID upgrade tool added recently, but this is not something that's clearly decided to be wanted or clear that it is a good solution

I don't know what the production team thinks in this case, but I'd say it'd at least require a consensus before this can be considered for 4.4, and there isn't a strong consensus if you check the reactions on this PR (15 positive, and 10 negative)

But the production team can take a look and see what they think

[pietru2004]

How could production team be asked to check this topic?
By this topic I mean option to disable UID file generation.

I am worried that UID files will be trashing my filesystem - I have a lot of script and txt type files (txt for things such as dialogues and items) Also I had experienced a situation in the past where godot would self change uids when saving open resource.

[AThousandShips]

Also I had experienced a situation in the past where godot would self change uids when saving open resource.

That's something to report as a bug, not something that should be solved by not using .uid files IMO

[pietru2004]

Also I had experienced a situation in the past where godot would self change uids when saving open resource.

That's something to report as a bug, not something that should be solved by not using .uid files IMO

I seen it already beeing reported, I added that as just a way of saying uids never been working for me, also I belive path is easier to read than uid

[elenakrittik]

I see there's a lot of activity and expectations around this PR, or in general having the possibility to disable .uid files in Godot 4.4, so let me clarify the project's current position.

UID support was made for a reason (see this blog post), and we didn't decide to add a new type of file on a whim. We considered and debated various options a lot and came to the conclusion that this is the lesser evil. So while we understand that some users are bothered by the new files added to the filesystem, we are not keen on adding fragmentation in the ecosystem by making it opt-out at this stage. Making UIDs optional means that everything that UIDs are meant to guarantee can no longer be achieved, and this would seriously limit what we can do in editor tooling thanks to UIDs.

"Somebody else's problem is solved by this. We think that that problem is more important than the problems this solution introduced."

We just released Godot 4.4, and we want to give it time to get proper feedback from the community, see what usability issues we can improve further with the UID support, and evaluate after some time if we need to change course, or add more flexibility.

"Besides, 4.4 just released and we're trying to hot fix as much issues that we introduced as possible without actually changing course. We first want to see if there's enough complaining to warrant rethinking our approach."

(This would normally be an okay strategy. However, in this case the feedback discussion thread alone has TONS of feedback already given to you, most of which is not addressed by all the band-aids the project implemented to try and alleviate pain. It's already day clear what this solution fixes well and what problems it causes.)

If your main annoyance with the .uid files are just that there are new files that you need to check in your VCS, I would suggest that you give it a try and see if this is really that terrible in the long run. I wasn't fond myself of .import files when we introduced this in 3.0, and I grew used to it and have absolutely no qualms with this anymore. I expect the same will happen with .uid files for most users. Note that there are also more usability improvements that can be done to ease the management of .uid files, both in Godot and in external IDE integrations.

In the meantime, just try using it. I'm sure at some point you will have been coping with the issues for so long that you will begin to accept this as a normal part of your workflow, just like with .import files back in the day. (The whole argument is "It felt bad, then i no longer cared enough". What else can i say.)

We also have some work going on to try to fix the aforementioned issues. This work so far didn't focus on the core issue, but should help ease some of the coping. (This is essentially a re-hash of the above paragraph.)

If you have other concerns such as actual problems caused by UID files, be sure to open proposals or bug reports about it so that it can be evaluated.

If you have "actual" (..sorry, what?) problems with .uid files, feel free to file even more issues and proposals than there already are.

[Daylily-Zeleen]

I just gona say UIDs are the thing that always changes when you re-save resource... or at least that be a case in my projects...

It souds like bug, you can try this in a new project and create a minimum reproduce project to submit issue.

[adamscott]

I think the Godot team should hear the voice of the user instead of forcing them to accept a bunch of dedicated uid files to mess up their file system.

As @akien-mga said, we didn't just implement this system on a whim. We were in fact listening to users complaining about numerous issues linked the lack of file tracking, especially when moving files outside of Godot (which can happen a lot, even if you're cautious about it).

We recognize that it sucks, yes, but I reject the ideas we don't listen to users, which is you're suggesting.

For example, a centralized file option for personal users, and embeded UID or dedicated UID files for the users who have collaborative needs.

A centralized file defeats the purpose of tracking files, as Godot cannot recognize that a file moved outside. Hashing the file doesn't work, because multiple files can share the same hash (when they are identical).

Embedding the UID was considered, but it's super difficult to implement on text files, as adding it can mess up the parsing of that file. It also changes the hash of the file.

[pietru2004]

I just gona say UIDs are the thing that always changes when you re-save resource... or at least that be a case in my projects...

It souds like bug, you can try this in a new project and create a minimum reproduce project to submit issue.

I think it was happening in 4.0 dev or beta... so some time ago, I didn't look did it happen since

[pietru2004]

Also I have spoken to few people that I don't think will look into this thread even after I gave them, link, but they also have issues with UID files and prefer to stay 4.3

[949886]

I think the Godot team should hear the voice of the user instead of forcing them to accept a bunch of dedicated uid files to mess up their file system.

As @akien-mga said, we didn't just implement this system on a whim. We were in fact listening to users complaining about numerous issues linked the lack of file tracking, especially when moving files outside of Godot (which can happen a lot, even if you're cautious about it).

We recognize that it sucks, yes, but I reject the ideas we don't listen to users, which is you're suggesting.

For example, a centralized file option for personal users, and embeded UID or dedicated UID files for the users who have collaborative needs.

A centralized file defeats the purpose of tracking files, as Godot cannot recognize that a file moved outside. Hashing the file doesn't work, because multiple files can share the same hash (when they are identical).

Embedding the UID was considered, but it's super difficult to implement on text files, as adding it can mess up the parsing of that file. It also changes the hash of the file.

It is good news to hear that the Godot team is hearing users' voice and recognized it sucks. I just hope this issue can be solved as soon as possible since it's so confusing.

---

I have another question about hash or checksum.

As long as the content of the script is the same, they will work the same way ideally. It doesn't matter to refer to which one of them, and they could be seen as the same script as well. It is ok to refer to the first one or the newest one on behalf of those duplicated scripts, isn't it? When one of these duplicated scripts is removed or changed, another file will replace it since they have the same hash or checksum, and the reference doesn't change either. Why is the checksum still the wrong tool, as the Godot team mentioned in the blog:

Unfortunately, checksums are the wrong tool for the job to keep track of unique file references. If you have two files with the exact same content, they would have the same checksum, but these files are still separately referenced by other scenes or resources in the project (with the expectation that these remain separate). If we relied on the file’s checksum to determine file references, then the references would break once the file contents change.

Or in the case that the hash or checksum can not identify some files, then generate UID files to separate them. Is that possible?

[pietru2004]

What about if there is warning near disabling UID files, like THIS IS NOT RECOMENDED BY THE GODOT TEAM
Can we add UID file disable switch with that and consider it as a compromise?

If not why is there need for enable alt space menu, something like auto accept quit or Main Loop type, low processor mode and many other project settings one could consider unneeded.

What I want to say by this is, some people do not want to have UID files and consider them unnecessary as it is just a file with 1 line of text and this method just disables generation of NEW UID files and the ones already generated do work.

So please just let us select which way we want to go in our projects. Whenever we want to use UID files or whenever we want to disable their generation within our projects. Let us have this choice like you already did for many other things via Project Settings.

[elenakrittik]

What about if there is warning near disabling UID files, like THIS IS NOT RECOMENDED BY THE GODOT TEAM Can we add UID file disable switch with that and consider it as a compromise?

"This is not recommended but still supported" is a strange position to take, i think.

If not why is there need for enable alt space menu, something like auto accept quit or Main Loop type, low processor mode and many other project settings one could consider unneeded.

Besides extensibility, they force the code to work within reasonable constraints by explicitly defining an API. So, it has benefits for the codebase, even if it doesn't end up being used much in the wild. I remember a good blog post about this but sadly can't find it anymore :(

[pietru2004]

@elenakrittik that all meant to just say please let us do this...

Also please remember you can disable part of godot when making stuff, I have in mind Manage Editor Feature Profiles panel.

In it you can remove most if not all editor features, even the stuff that could be considered core and needed for making stuff.

[MarianoGnu]

If you have other concerns such as actual problems caused by UID files, be sure to open proposals or bug reports about it so that it can be evaluated.

@akien-mga i personally agree with you and the rest of the devs, however there's a usecase i am not sure is already considered.
When migrating code from one project to another, uids should be regenerated in the new project instead of using the old ones, because introducing foregin .uid files to an existing project makes an unlikely but reasonable chance to create duplicated uids for different files.

This proposal was trying to prevent that by disabling generation of uid files in the addons folder, because that is the main folder that is shared using the AssetLib (still not the only one)

But i believe a better solution (that i dont know if it is already implemented) is when adding files from the asset lib, to skip the uid files from the library that you are importing, and the addon maker should simply know to not rely in uids.

This can still have consequences, because tscn (and scn, tres/res) files will reference sub resources using uid, they will fix themselves on first save, but in the unlikely case that the uid already existed in the project, this file will now try to point to an unrelated file in the project.

I myself, can live with this, i want the .uid files, i think they are for the better

[pietru2004]

I wonder could disabling files UID allow for file paths like ./script.gd in resources?
I mean paths relative to directory an asset is in...

I find myself duplicating full directory via godot and having files from another directory be used cause UID points to it.
Like when I want to make new object in game like Machine, NPC, etc.
Let's say I have base_machine class and I am duplicating furnace directory and want to make assembler, it end's up having furnace's script instead of script that is in it's directory. In case UID is a problem since I can have edited it for scene to have relative and not absolute from project res:// path.

[MarianoGnu]

I wonder could disabling files UID allow for file paths like ./script.gd in resources? I mean paths relative to directory an asset is in...

I find myself duplicating full directory via godot and having files from another directory be used cause UID points to it. Like when I want to make new object in game like Machine, NPC, etc. Let's say I have base_machine class and I am duplicating furnace directory and want to make assembler, it end's up having furnace's script instead of script that is in it's directory. In case UID is a problem since I can have edited it for scene to have relative and not absolute from project res:// path.

when you duplicate a tscn file, internally subresources are mapped using first uid, then file path as fallback. the proces of duplicating he scene may not understand you also want to duplicate the subresource (because it doesn't see what other files are in the queue of files to duplicate) so it simply keeps using the old one. This is normal and was happening before, the only difference is now it happens also for scripts and not only for other resources like textures and sounds.
To mee it sounds like a different issue to fix in the fylesystem, it has no correlation to the generation or not of uid files.

[pietru2004]

Can we just get this feature? The option to disable files UID generation?

Preety Please.

I realize there are reasons why it was implemented, but just please let me chose do I want them or not.

[akien-mga]

@pietru2004 Please stop spamming this PR. I already explained the current status in #100973 (comment).

There is no intention to merge this feature for the time being.

To reiterate:

Making UIDs optional means that everything that UIDs are meant to guarantee can no longer be achieved, and this would seriously limit what we can do in editor tooling thanks to UIDs.

So even if you personally accept the limitation, this will make it a problem for engine maintainers and prevent further improving the editor handling of files and dependencies.

You are welcome to merge it in your own build if it's critical for your use case.

[pietru2004]

Ok

[akien-mga]

@MarianoGnu

When migrating code from one project to another, uids should be regenerated in the new project instead of using the old ones, because introducing foregin .uid files to an existing project makes an unlikely but reasonable chance to create duplicated uids for different files.

I think collisions are pretty unlikely in this specific use case, but we can ensure that if such a collision happens, the editor can resolve it gracefully. Collisions can actually happen easily when duplicating files from outside Godot and that's already something we need to solve properly.

If we find that UID collisions with addons are actually happening, there are also options to make UIDs more deterministic based on the file paths, so addon UIDs wouldn't clash as they would be seeded from their addons/<slug>/<file> which should be unique (at least until two addons use the same slug and file names but that's a problem bigger than just UIDs).

I don't think disabling UIDs for addons specifically is a good option. Addons are one of the most relevant uses of UIDs to ensure that code can be reused independently of the filesystem structure IMO.

[MarianoGnu]

Yes, i also believe disabling uids is not the right way to go. Acceptable solutions i think of are:

• Automatically skip any .uid file when importing files from AssetLib, so the project importing it assign new ones (dont know if already done)
• Editor should print errors if detects more than 1 resource is explicitly using the same uid, this may be already happening, last time i checked the last resource was just overriding the first one

[Daylily-Zeleen]

I don't think disabling UIDs for addons specifically is a good option. Addons are one of the most relevant uses of UIDs to ensure that code can be reused independently of the filesystem structure IMO.

The purpose of disabling ".uid" in "res://addons/" is to temporary avoid UID conflict and generate different UID in different devices if reference plugins without details.

Like I said in #11464, we can remove this limitation if we can solve these issues in future.

But it's already too late, 4.4 already release, even if we solve these problems in the future, at least plugins created by 4.4 stable are have potential UID conflict risks.

I don't care anymore, I am powerless to change anything about it.

[949886]

So many days later, maybe I should take back my words that the Godot team is hearing the voice from users.
There is no good news, not even a reply to my question from the Godot team.

These days, I always get such errors after I upgraded my project to Godot 4.4.

Godot 4.4 says my resource is not found, but it actually dose exist and it's referred by the scene.

And when I click fix Godot 4.4 says there is nothing that should be fixed

If this pr won't be approved, can the Godot team just fix these errors?

[akien-mga]

@949886 Please file a bug report with steps to reproduce.

[949886]

Too tired and disappointed to rewrite an issue and care about this.
Dealing with this makes me exhausted and kills my good mood for days.
Maybe I should go back to Unity and wait for some changes.

[MarianoGnu]

@949886 your problem is not new, and doesn't seem related to uids, this happens from time to time sadly.
Godot is not good showing good errors when is unable to load a resource because a subresource is failing to load
In this case the resource that is failing is fireball.tres
if you open fireball.tres in a text editor and see line 22 it will show you the actual resource that is missing (or maybe another chain reaction? it never happened to me to be so nested) to load because seems to be missing, but the problem is not eigther firebal.tscn, nor fireball.tres. it is a dependency

[949886]

@949886 your problem is not new, and doesn't seem related to uids, this happens from time to time sadly. Godot is not good showing good errors when is unable to load a resource because a subresource is failing to load In this case the resource that is failing is fireball.tres if you open fireball.tres in a text editor and see line 22 it will show you the actual resource that is missing (or maybe another chain reaction? it never happened to me to be so nested) to load because seems to be missing, but the problem is not eigther firebal.tscn, nor fireball.tres. it is a dependency

Thank you for your reply!

I'm just a little frastuated by this problem.
And now I think I have found what the problem is. Maybe relevant to this:

Allow circular dependencies/references for resources in export var · Issue #7363 · godotengine/godot-proposals

In my case. The reference is not broken; every reference is fine.
Resource fireball.tres and scene fireball.tscn have circular dependency and cause these errors.

[MarianoGnu]

Well, i am glad you found a ray of light to, at least, try to workaround the issue. Indeed circular dependencies between resources are bad, one way to prevent it is using @export_file("*.tscn") and then load() the scene from your fireball.tres on demand, that will prevent the ciclic dependenci because the scene will be loaded on demand instead of on instantation (unless you run the load() on the _init() function, if you do that it will still try to load fireball scene while fireball scene is already trying to be loaded)

[akien-mga]

This is going in circles and not constructive, so I'm locking this discussion.

We've heard the feedback, if we decide that we want to make UIDs optional, we'll revisit this PR.

Until then it's available for people who want to make this decision for themselves, that's the power of open source.