Terrain data is not saved

[winston-yallow]

Describe the bug
When creating a terrain the data is not saved. Therefore it will not appear when running or exporting the game. Also, when reopening godot the terrain disappears. This happens only in 1.5.2 and not in 1.5.1

To Reproduce

1. Create terrain
2. Not sure if necessary for reproduction, but I did:
   • import textures
   • modelling of some mountains
   • paint textures onto terrain
3. Save scene
4. Close godot / Go back to project list
5. Reopen project. The terrain is not there anymore.

Expected behavior
Terrain data should be saved.

Environment

• OS: Linux (solus) 64bit
• Graphics card: GeForce GTX 1050
• Godot version: 3.2.3 official stable
• Plugin version: 1.5.2
• Renderer used: GLES3

[Zylann]

Again?
#190
#120
#101

This happens only in 1.5.2 and not in 1.5.1

There is nothing changing saving stuff in 1.5.2 so I really wonder what kind of black magic is going on Oo
Does this always happen when you try it? Are you sure it only happens in 1.5.2?

Reopen project. The terrain is not there anymore.

You mean there is no file at all in the data directory? Was there files before?

Did any errors occur?

And of freaking* course, I tested it following your steps, and could not reproduce your issue :|
*yes this has happened a few times as you can see... I know something's wrong given several people reported it, yet it's still impossible to figure it out.

I tried this twice:

• Once in my dev project which contains my developement version of the plugin: created a new scene, followed your steps, no issue.
• Once in a new empty project in which i downloaded the plugin from the AssetLib tab. Created a scene, added terrain, selected new data directory, sculpted a bit, saved, closed, reopened: no issue.

One of the reports hinted that the resource saver was not even invoked by Godot. So far I only suspect that Godot's resource saving system might contains a Heisenbug... This system is tied to class_name, so any issue with class_name could very well affect it too.

[winston-yallow]

Are you sure it only happens in 1.5.2?

Ok after fiddling around with this a bit more, I can reproduce this in other versions as 1.5.2 too. Though, I cannot reproduce this reliably. It sometimes works, but sometimes doesn't. I will put some print statements into all your resource handling code and log that to a file. If I do encounter this bug again, I will share the logs. Very very weird.

Was there files before?

In the data dir? I think no, but will need to check again once I am able to reproduce this again.

Did any errors occur?

No, though I might have missed them. I may be able to report this in more detail soon since I now log everything to a file.

PS: Sorry for opening yet another issue. I thought it was introduced in 1.5.2, and I saw that none of the issues created since that release mentioned this bug. So I did not search for other existing bugs. Should have used the search regardless...

[Zylann]

I will put some print statements into all your resource handling code

Some advice when doing that: always add them while the plugin is turned off. Or in doubt, do it when Godot is not running. It has a tendency to badly reload modified tool scripts, causing all sorts of errors.

[David-Ochoa]

I had the same problem, in a new project, it worked fine in other projects so I disabled new plugins I was trying (Hungry Proton's Scatter and Jake Young's Mono debugger) and now the terrain saves correctly.

[winston-yallow]

Just for reference, for me this happened with only the heightmap plugin. So it may not be caused by other plugins.

On another note: since I added some logging it did not happen again. Really really weird.

[Zylann]

Just for reference, for me this happened with only the heightmap plugin.

My plugin uses a custom ResourceFormatSaver. You mean other plugins using that system never failed to save their stuff?

On another note: since I added some logging it did not happen again. Really really weird.

Yes indeed :D I'll never say it enough but: so far the only clue I got is that sometimes, for some reason, Godot just doesn't call it (from an earlier report). And it's completely random, of course, and could never be reproduced on my end... so could not fix it so far...

It could also be that the workflow I had to go for (choosing a directory path rather than forcing you to create the file yourself or embedding the resource in the scene) follows a code path Godot was largely untested for, and somehow causes such random issues. I don't know.

So the only things I can do is make shots in the dark. For example here is one:

Did any of you have this problem with an existing terrain? Was it always with a new one?
See this function:

godot_heightmap_plugin/addons/zylann.hterrain/hterrain.gd

Line 461 in 362d78c

func _set_data_directory(dirpath: String):

This is called when you choose the data directory. It creates the resource in memory and assigns the path where it will be saved. At this point the resource does not exist yet as a file, but in theory, when you save your scene, Godot is supposed to call HTerrainResourceSaver which will produce the files at the specified path. And it does! ... but not reliably, I guess.
The file is not saved immediately because it's legit to open a new scene with some new terrain, mess around, and then decide you don't want that scene and close without saving. Godot doesnt force you to create a .tscn. No files will be created until you actually save.
Now, back to the hypothesis: if the problem always occurs with new terrains, mayyybe you could try to add this line to force the resource to be saved immediately after the data directory is chosen?

				# Create new
				var d := HTerrainData.new()
				d.resource_path = fpath
				set_data(d)
				ResourceSaver.save(fpath, d) # <-- HERE

I have no idea if that's supposed to work, and no idea if that will help working around the problem. But it's just a random idea.

[winston-yallow]

My plugin uses a custom ResourceFormatSaver. You mean other plugins using that system never failed to save their stuff?

No, I mean that no other plugin was installed when the error occured for me. Therefore I don't think that the combination of your plugin with another plugin is the error, as for me the error occures without any other plugins installed.

Did any of you have this problem with an existing terrain?

For me, no. I only had this happen with new terrains.

you could try to add this line to force the resource to be saved immediately

I personally won't add it, as that would (hopefully) prevent the error from happening. I still hope to recreate this error with my logging on... so I will not try this fix for now. Thank you for the suggestion though! I will probably put it in if I can't reproduce soon.

[tlalexander]

Hi folks! Well I was all excited to try making terrain but I am hitting this bug. It never saves for me.

I tried adding:

				var err = ResourceSaver.save(fpath, d)
				if err != OK:
				  _logger.error("Could not save '{0}', error {1}".format([fpath, Errors.get_message(err)]))

Along with the appropriate import at the top of the file for Errors:
const Errors = preload("./util/errors.gd")
and I get this when I set a new folder for the resource:

ERROR: call: hterrain.gd: Could not save 'res://terrain2/data.hterrain', error [15]: File Unrecognized error
   At: modules/gdscript/gdscript_functions.cpp:803.

If yall even could figure out how to get this workaround working that would help for me. And if there's anything I can do to debug please let me know. Thanks!

[tlalexander]

And I just double checked and I am running godot 3.2-stable, built from scratch on my Ubuntu 20.04 machine from git commit 4e7d75ccd350b98bccea517c034b367f52cf127e

[tlalexander]

I've tried three versions of this plugin from 2020 and going back to Oct 2019 and they don't save data for me.
Specifically this is the oldest commit I have tried:
6dfdef3

Here is how I reproduce:
Start Godot and open my existing game. Create a new scene. Save the scene. Add an HTerrain node to the scene. Select a data folder for that node. Save the scene. Close the scene. Re-open the scene and get the "file not found" error for the missing datafile. I also confirm in a terminal that nothing is saved in that folder with "ls -lahtr" to include hidden files.

I can compile godot if that helps with debugging at all. Thank you!

[Zylann]

godot 3.2-stable

Have you tried godot 3.2.3? Godot 3.2 is fairly older.

Start Godot and open my existing game. Create a new scene. Save the scene. Add an HTerrain node to the scene. Select a data folder for that node. Save the scene. Close the scene. Re-open the scene and get the "file not found" error for the missing datafile.

if I take instructions to the letter, so without unfocusing the editor, I do get an error, but the files did get saved. I know you didnt see files in the terminal, but I'd like clarify things:

Case 1: The error happens while there are files in the directory

When you save a terrain (for the first time OR after editing), the plugin creates or updates a bunch of PNG files in the terrain data folder. But Godot does not automatically import or re-import these files, and there is no GDScript API allowing me to force Godot to do this.
This does not mean your terrain was not saved. If you look into your terrain data folder from an OS file explorer (not Godot), you should see those files. and they should be up to date.

This also explains the fact it doesn't happen again at some point. Because Godot WILL import those textures. The workaround is to unfocus the editor and refocus it, or restart it.

I suspect this happened to many people here. Before reporting that it did not save at all, please look outside of Godot if your terrain files are present. There is a description of this in the documentation. Because this is a very different issue, and it's difficult to track down what really happened in each report so far.

Case 2: files are not present at all

I also confirm in a terminal that nothing is saved in that folder with "ls -lahtr" to include hidden files.

Now this is the situation I was completely unable to reproduce for months (years?), and this is what the present issue is about. Even with your specific steps followed to the letter, in an existing project as well, I can only get Case 1. Exactly like this video I recorded in an older version: https://zylannprods.fr/dl/godot/godot_hterrain_trying_to_repro_issue120.mp4
To reharse things, the only potential clue is that Godot is not calling HTerrainResourceSaver at all, which would require more intricate debugging, but it's extremely hard to do if we don't have a reliable repro.

One reason was due to the addons folder being named Addons with an A, which made Godot unable to locate the HTerrainResourceSaver class_name: #120 (comment)
But I believe most of you didn't do this?

@tlalexander Are you able to reproduce Case 2 every single time? If you can, you may be able to debug it further than me, and it might involve debugging Godot itself.

(And of course: if you plan to debug the GDScript part, close Godot before each attempt, because editing tool scripts has its own issues and could compromise reproduction tests.)

[tlalexander]

Oops sorry, I thought I replied.

It turns out 3.2.3 works fine for me. I had a lot of fun making terrain last weekend.

So yes in 3.2-stable I did have Case 2 every time. The folder was named addons (lowercase).

I thought I could help by debugging with modifications to the GDScript, but for some reason a search of your repo only shows "ResourceSaver" a single time, so I am not seeing where it is defined... I do see "HTerrainDataSaver" and "ResourceFormatSaver". Where is ResourceSaver defined? If there's anything I can do to debug let me know!

[tlalexander]

Also note this project's readme says:

Heightmap-based terrain for Godot 3.1 and 3.2. It supports texture painting, colouring, holes, level of detail and grass, while still targetting the Godot API.
Note: The current Godot master branch isn't supported yet. Use Godot 3.2 if you want to use this plugin.

So if 3.2.3 is needed we could update the docs.

[Zylann]

I do see "HTerrainDataSaver"

This is the one. ResourceFormatSaver is a Godot class https://docs.godotengine.org/en/stable/classes/class_resourceformatsaver.html

[CoreVamore]

I am having similar issues, and have tried the "ResourceSaver.save(fpath, d) # <-- HERE" modification as well.

Godot's Debugger is showing:

E 0:00:00.508 _load: Resource file not found: res://Scenes/Terrain/data.hterrain.
<C++ Error> Condition "!file_check->file_exists(p_path)" is true. Returned: RES()
<C++ Source> core/io/resource_loader.cpp:282 @ _load()

E 0:00:00.508 poll: res://Main.tscn:16 - Parse Error: [ext_resource] referenced nonexistent resource at: res://Scenes/Terrain/data.hterrain
<C++ Source> scene/resources/resource_format_text.cpp:440 @ poll()

Note: The scene isnt saved AND I can not hit F5 to run the game due to this either.

I am not sure what's occuring. I am in the process of comparing the setting in the demo to see if perhaps a missing setting is stopping the layer from being saved.

[Zylann]

I still haven't advanced at all on that issue. I've been blocked for almost a year. There are a few cases that are simple to figure out in case the issue comes from textures, but more people keep getting the issue about data.hterrain, yet nothing new and no reliable repro.

Note: The scene isnt saved

What does that mean? If you didn't save the scene how come it's loading Main.tscn?

[tlalexander]

@CoreVamore what version is your godot? Have you tried 3.2.3?

[CoreVamore]

@CoreVamore what version is your godot? Have you tried 3.2.3?

I was on 3.2.2, I just tried 3.2.3 with the same result. I am going to try it from an empty project and see if I get the same behavior.

[Zylann]

E 0:00:00.508 _load: Resource file not found: res://Scenes/Terrain/data.hterrain.

Note that if this happens while the data.hterrain does not actually exist, then testing the launch of the game through different versions won't change anything. The problem to reproduce is the fact data.hterrain isnt getting saved. When you say you get the same result, what steps are you actually following?

[CoreVamore]

Give me a sec, im reading through the usage doc's, I may have missed a step. I'll get back to you shortly.

1. The file saves on a blank/new project....

OK, I think ive worked out the issue, though I need to retest it. Firstly, I got the plugin working on my existing project by disabling the plugin, removing the HTerrains etc related to it. Removed the "addons/zylann.hterrain" folder from within Godots FileSystem Tool, then closed Godot. Made sure there was nothing left in the addons file system using the normal linux system file browser.

Then using the linux file manager copied a fresh new copy of the "zylann.hterrain" plugin folder back under the "addons" folder within the Godot project; Opened the project and then activated the plugin, added a HTerrain instance in the project, told that instance where the Data Directory was, made some changes on the height map, and hit F5 - and it worked/file created!

Now, I THINK I may know what caused the original issue. I think the first time around I IMPORTED the pluggin directory, then copied across the files that didnt import manually using the linux file system tool. Im trying it again now to see if I can replicate it
Hmmmm, I cant reproduce what caused the issue, but my gut feel at this stage is that either the full pluggin files were not copied across to the "addon" directory properly, or were perhaps corrupted if imported into Godot's "res:// FileSystem"

Hope that helps - at least there is a fix for it now ;)

[Zylann]

Looks like the same things I suspect again:

• Godot not being able to import the plugin on first load requiring a restart of the editor (that one being caused by the import system of Godot being too limited to figure out the dependencies)
• or Godot not detecting the ResourceSaver's class_name which prevents the save from working (today yet another user mentionned an error where the save would not work, despite the class being present in project.godot, but once again it seems to have "fixed" itself and could never be reproduced reliably)

[CoreVamore]

Perhaps the scene tree is loading before the plugin finishes loading and doesnt call the classes for the plugin? (Just a wild guess - I am very much a beginner in all of this)

[Zylann]

No I think the plugin loads before that, and the resource loader loads even before plugins. The terrain classes should work without the editor plugin as well (so even if you turn it off the node will still run).

[silasb]

I ran into this issue and ended up doing a couple things that made this work, but I'm not sure why....

I preloaded the logger into both the HTerrainDataSaver and HTerrainDataLoader. I also instanced a logger in both and used the logger in both of these classes. AFAIK, that is the only change that I did to make any difference.

My process was to first start with HTerrainDataSaver and after preloading the logger I'd notice the files would actually save, but wouldn't load and crash with ERROR: No loader found for resource: res://terrain/map01/data.hterrain.. After adding preloading the logger into the Loader class it just worked. 🤷 Uploaded an example of the changes I made:

While doing this, I noticed that the icon of my data.hterrain had a red X in it, but after preloading the logger in the Loader class, it doesn't anymore.

[Zylann]

Unfortunately there isn't much I can do about it, even with that example. It just keeps showing Godot sometimes randomly fails to use the saver and loader, and that ability may be affected by anything that touches the script, like editing it.

[krazyjakee]

Not sure if you want feedback on the godot4 branch but this issue is present there. After reloading Godot, the terrain is visually gone. The configuration is there but no data.hterrain is saved and no terrain is visually present.

[Zylann]

I lost motivation in fixing this... it's been so long and I just have no idea what causes it at this point. The plugin isn't doing black magic here, it modifies a resource using Godot's UndoRedo feature (granted, using a dummy function because it's so inadapted to terrain painting, but as far as Godot is concerned, a function is called to change the resource). The test scene I made in Godot 4 didn't have this problem. I just made another test scene right now, painted on it, saved, and it did save. Then I restarted Godot, reopened the scene, sculpted some more, then saved, and it did save. Sometimes it happens, sometimes it doesn't... I just need a way to reproduce this 100%, and so far nobody has found anything.

[krazyjakee]

Man I feel you. I read through all the issues and tried to reproduce a way to fix it or work around it but I haven't got it to save once haha... typical.

Anyway, I feel like this is some good closure until someone can find a reproduction case.

[Janders1800]

In my case the "Data Directory" field is never saved. So the plugin never knows were the data directory is.

[Zylann]

@Janders1800 any errors in the console? That looks like a different issue.
Otherwise I need reproduction steps, as I have never seen this happen. I opened existing terrains just now with Godot 4.1.1, and created a new one, and had no saving problem.

[Janders1800]

Error:
No loader found for resource: res://TerrainData/data.hterrain.

The reproduction steps are just the bare minimum.

• Add a HTerrain node
• Set data directory
• Save
• Reload project

The Data Directory field is empty and so is the data folder.

[Zylann]

No loader found for resource: res://TerrainData/data.hterrain

The loader is right here: https://github.com/Zylann/godot_heightmap_plugin/blob/master/addons/zylann.hterrain/hterrain_resource_loader.gd

For some reason Godot is not detecting it. That's the issue. But the problem is HOW does that issue happen. I can't force Godot to find it.
I have no idea why Godot is behaving like it does not exist sometimes... worst of all, I am still unable to reproduce this issue. I have done exactly the steps you described in my previous comment, to no avail.

The Data Directory field is empty and so is the data folder.

But then if that's true, how come is Godot printing No loader found for resource: res://TerrainData/data.hterrain, instead of "file not found"? When did the error print in the first place? Was it when you saved? Or when you reopened the project? Was it the only error in the entire log?
The file should have been created as soon as you saved...
I tried yet another time just now and it worked...

Video of retrying from scratch, unedited:
https://zylannprods.fr/dl/godot/hterrain/232_not_happening_2023_09_05.mp4

Something particular must be happening somewhere that's different from all of my attempts... some specific condition, timing... I'm out of ideas

[Janders1800]

Just deleted the plugin and installed it again and now it works OK. ¯\_(ツ)_/¯

Edit: Had to nuke the project cache tho.

[Zylann]

More reports of this in #425

I'm quite fed up of this issue so I decided to attempt a different behavior to workaround it.

I added this logic: f95447d
Download the plugin from the repo if you want to get it.

Now if you assign a new data directory, if that directory doesn't contain data.hterrain, it will save terrain files immediately into it, without using ResourceSaver. i.e it won't wait until you save the scene to do this. Later saves will still only occur when you save the scene. Hopefully that wouldnt cause extra problems? Let me know if saving problems still occur after this.
If the issue persists, I assume it would manifest in the terrain's shape not changing in game after having sculpted it in the editor and saved the scene. But we'll see... whatever is causing the original issue is still there and unsolved.

Note: remember Godot also has to import terrain textures when they are saved, so sometimes the terrain might not always look changed when you play the game (in particular color, normals or textures; heights dont go through import at the moment). This is different. If that happens, make sure Godot has imported the files. You can do that by just tabbing in/out of Godot.

[A13ks4]

This seems to have fixed it for me. It used not to load the terrain every other time when I ran the scene, now it works fine.
Thanks for all the hard work!