Error trying to use 2.6.149 (Error extracting file)

[beelee1]

Error extracting file: /tmp/triplea6560939776671296847/Global_40_House_Rules.zip, /tmp/triplea-unzip2861211619708353438/global_40_house_rules-master, {} java.nio.file.DirectoryNotEmptyException: /tmp/triplea-unzip2861211619708353438/global_40_house_rules-master If this problem happens frequently and is something you cannot fix, please report it to TripleA

Trying to replicate this error #9250 but i can't dl map. I get a bunch of errors on startup trying to unzip all the maps in dl maps. Which maxed out my "root", which i had to spend a couple hours figuring out what and how to purge so it'd work again.

So basically, for me, This is unusable

Edit
I removed the map and tried to dl it and it'd dl but not showup. I then manually unzipped the map and put the folder directly in my dl maps and that didn't work either.

[DanVanAtta]

Do you wind up with a folder named something like "Global_40_House_Rules" in your downloaded maps folder?

[beelee1]

So, I tried this again and no folder showed up. I removed the zip and no folder showed up. When I choose it in DL maps, it throws the above error. It will then not show up as a DL option even though it's not in my downloaded maps folder.

I close DL maps and then it'll show up as a DL option again. I have a map I'm working on that is in a folder, no zip and it shows up and works correctly.

I just tried unzipping and loading the map manually and it shows up now. Idk y it didn't earlier. I must've screwed something up when I first tried it.

[DanVanAtta]

Extraction from zip to folder occurs at startup.
If there is no zip, there is nothing to extract to a folder.

DL listing is based on the folders that are present and specifically what is listed in the 'map.yml' files in those folders. There could be an issue here where at present maps do not have a 'map.yml' file, that file is generated on startup, so you could run into an inconsistent state after downloading a map and it then not having a map.yml file. If you re-open the downloader, it might not think you have the map downloaded anymore as it can't find the corresponding map.yml file (even though you have the have the map extracted), at which point you run into the above error as it'll download a zip and then try to incorrectly overwrite the previous.

To help this from a game engine perspective, we can do the following:

1. We need to finally get map.yml files uploaded to all maps
2. The engine should be checking if the map folder already exists and give a better error message or just go ahead and overwrite it. Probably overwrite it as we want the downloader to update maps and not just install maps.

[beelee1]

right on. Just to clarify, when I first fired up 2.6.149, I had all my existing maps from 2.5 in "downloadedMaps". The biggest issue was "root" getting maxed out. Took me a while to figure that out, so trying to save others from the same experience.

[DanVanAtta]

That would be worthwhile for us to investigate as well, notably if we can ensure that folders are unzipped one at a time with the zip removed after each one and second that we can better fail if there is insufficient disk space.

[sumpfralle]

I think the main issue of the original submitter of this bug report was not the missing cleanup of partially downloaded files (fillung up root), but the inability to download a map.
At least this is happening for me (with the same exception as the one described in the initial submission of this issue):

23:59:04.769 [Thread-14] INFO  g.s.e.f.m.f.s.l.ZippedMapsExtractor - Extracting map zip: /tmp/triplea1721834735953102823/age_of_tribes.zip -> /home/lars/triplea/downloadedMaps/age_of_tribes
23:59:05.874 [Thread-14] WARN  g.s.e.f.m.f.s.l.ZippedMapsExtractor - Error extracting file: /tmp/triplea1721834735953102823/age_of_tribes.zip, /tmp/triplea-unzip16717882356039247212/age_of_tribes-master, {}
java.nio.file.DirectoryNotEmptyException: /tmp/triplea-unzip16717882356039247212/age_of_tribes-master
        at java.base/sun.nio.fs.UnixCopyFile.ensureEmptyDir(UnixCopyFile.java:384)
        at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:484)
        at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)
        at java.base/java.nio.file.Files.move(Files.java:1422)
        at games.strategy.engine.framework.map.file.system.loader.ZippedMapsExtractor.unzipMapThrowing(ZippedMapsExtractor.java:137)
        at games.strategy.engine.framework.map.file.system.loader.ZippedMapsExtractor.unzipMap(ZippedMapsExtractor.java:99)
        at games.strategy.engine.framework.map.download.DownloadFile.lambda$startAsyncDownload$3(DownloadFile.java:83)
        at java.base/java.lang.Thread.run(Thread.java:829)

Such an exception occurs after trying to download any map.
Thus at the moment there are no maps available to me locally, since I cannot download any.

The temporary directory mentioned in the stack trace contains the full content of the map's zip file after the error dialog was displayed. Thus everything was extracted properly.
I could imagine some kind of concurrency issue (the files being extracted before another code path expects these). But this is just a wild guess.

I am running Debian Bullseye on amd64 with the current snapshot (2.6.339).

Thank you for your time!

[DanVanAtta]

This could potentially be fixed now in the very latest. The map update logic was not deleting existing maps before updating.
@beelee1 would you mind checking if you are still seeing this problem?

[beelee1]

@DanVanAtta I'll try and check out in a bit.

[emmanuel-perlow]

Just checked it, the behaviour reported by @sumpfralle still happens on 2.6.379 (a11fe86) on Linux.

In case a workaround is needed:
You can manually move the map folder in /tmp/triplea-unzipXXXXX/ to ${HOME}/triplea/downloadedMaps/ (You may need to close the "Select Game"/"Download Maps" dialog and reopen it, to see the new map).

[beelee1]

just tried this in 2.6.574 pre and it still happens

[DanVanAtta]

What are the exact sequence of actions you take after launching the game to then observe this error @beelee1 ? Are you simply launching the game?

What is the exact text message of the error?

Could you please check the log file generated by triple for a stack trace and attach that log file to this issue?

[beelee1]

@DanVanAtta yea each individual map shows up as separate errors. I'll fire it up again and post a few. back in a few minutes

Edit
@DanVanAtta yea so it only threw 1 map error this time.
/home/greg/Pictures/Screenshot from 2022-02-17 19-07-02.png

oops this is map error
/home/greg/Pictures/Screenshot from 2022-02-17 19-07-37.png

So it's only throwing this one and quits doing it now. I had 3 or 4 when i first fired it up and then shut it down as I'm about maxed out again on space. Anyway, I then got the Feign error which is known when I tried to DL maps
/home/greg/Pictures/Screenshot from 2022-02-17 19-09-21.png

This the only map that showed up by default. It threw a missing image error, which probably map related, so didn't actually get it to fire up
/home/greg/Pictures/Screenshot from 2022-02-17 19-10-25.png

[beelee1]

@DanVanAtta I'll clear some space and try this again. Maybe it only does it the first time with the consecutive map errors and if you close it and restart then it does this. I'll try again in a bit

Edit
Yea I can't get in to run to get to the "Debug" in the console. I'll try and a map manually and see if that works

[beelee1]

@DanVanAtta OK I found the triplea.log see if I can upload it
triplea.log

[DanVanAtta]

@beelee1 , the attached log file is empty. Recommend you just move all your maps into some other folder & delete the contents of /tmp, then try again.

[RoiEXLab]

@beelee1 Looks like this issue has been taken care of, closing this...

[asvitkine]

@beelee1 mentioned that this is still not working for them on: https://forums.triplea-game.org/topic/2998/2-6-testing-report-2-6-problems-here/54?page=3

@beelee1 - seems you tried to post some screenshots on Feb 17, but it just looks like some file paths and not the actual images. Do you mind trying to post them again?

If this is preventing you from trying/using 2.6, I'd like us to figure out the problem and fix it.

[beelee1]

Here's another attempt at the log
triplea log.zip
looks like it worked this time. I'll post the error pics later

[asvitkine]

Thanks!

So looks like we're getting an error from ensureDirectoryEmpty() in Unix FS code, which does:

        if (sourceAttrs.isDirectory()) {
            ensureEmptyDir(source);
            copyDirectory(source, sourceAttrs, target, flags);

So seems it requires the source directory to be empty before moving? That seems... surprising to me.

The stack trace:

java.nio.file.DirectoryNotEmptyException: /tmp/triplea-unzip3447058957441722647/global_40_expansion-master
	at java.base/sun.nio.fs.UnixCopyFile.ensureEmptyDir(UnixCopyFile.java:384)
	at java.base/sun.nio.fs.UnixCopyFile.move(UnixCopyFile.java:484)
	at java.base/sun.nio.fs.UnixFileSystemProvider.move(UnixFileSystemProvider.java:267)
	at java.base/java.nio.file.Files.move(Files.java:1422)
	at org.triplea.io.FileUtils$FileMoveOperation.move(FileUtils.java:265)
	at org.triplea.io.FileUtils.replaceFolder(FileUtils.java:276)
	at org.triplea.io.FileUtils.replaceFolder(FileUtils.java:259)
	at games.strategy.engine.framework.map.file.system.loader.ZippedMapsExtractor.unzipMapThrowing(ZippedMapsExtractor.java:144)
	at games.strategy.engine.framework.map.file.system.loader.ZippedMapsExtractor.unzipMap(ZippedMapsExtractor.java:111)
	at games.strategy.engine.framework.map.file.system.loader.ZippedMapsExtractor.lambda$unzipMapFiles$2(ZippedMapsExtractor.java:55)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)

[asvitkine]

Documentation for Files.move() which we're using says:

This method may be invoked to move an empty directory. In some implementations a directory has entries for special files or links that are created when the directory is created. In such implementations a directory is considered empty when only the special entries exist. When invoked to move a directory that is not empty then the directory is moved if it does not require moving the entries in the directory. For example, renaming a directory on the same FileStore will usually not require moving the entries in the directory. When moving a directory requires that its entries be moved then this method fails (by throwing an IOException). To move a file tree may involve copying rather than moving directories and this can be done using the copy method in conjunction with the Files.walkFileTree utility method.

[asvitkine]

@beelee1 Do you know if your downloadedMaps and /tmp directories are on different partitions?

That could explain the error.
Perhaps we can make the temp dir in a subdir of downloadedMaps.

[asvitkine]

I sent #10512 - I don't have a way to test, but I suspect it should resolve this problem. Please confirm once that PR is merged into a prerelease!

[beelee1]

@asvitkine right on. I'll give it a shot Thanks

[beelee1]

@asvitkine Sweet ! Works like a charm : ) Knew it had been a while but didn't realize it was a year. Be nice to use the prerelease again :)

One thing I noticed, is it unzips everything now, so the first time you run it, it'll take a minute if you have a lot of maps. From then on, it'll fire right up. Some people might not be aware of that.

Once again BIG Thank You !!!

[beelee1]

Yea so the game behavior works correctly with #10749 fix with 2.6.838 but the updated notes don't show up. The fix was in the units folder. The xml changes don't show up though. Which were Game Version/Notes only.

So Idk if that means 2.6 can't see any xml changes or not. I'm gonna do some xml updates to G 40 Expansion and then try with 2.5 and latest stable to see if they work.

Here's a save
Harbor 12 works but Notes don't.zip
Should be 2.72 in the Notes

@asvitkine gonna reopen

Also, I tested version 2.71 and confirmed the error with issue #10729

[beelee1]

yea so the Notes are all jacked up on 2.6.838 Here's 2.5

Here's 2.6838

Notifications worked correctly and the xml change other than the Notes worked, but I'm going to run another test, an obvious one : ) , for that to make sure.

I wonder if the Notes change has anything to do with it. Not only do you have to scroll the other way and make an additional click, it blocks your other windows. You can't click on anything, you have to close Notes first. Quite annoying.

At any rate, I'll test some more.

[beelee1]

I changed Infantry from "Attack 1" to "Attack 5" and it worked correctly on 2.5 as well as 2.6.838 so "Game Notes" and "Map Version" are what are not showing up correctly.

The map version doesn't show before game start in the pre as it does in the stable either.
Stable

Pre

Idk if any of that is related but ... Maybe the fact that the new downloaded map isn't starting as azip or at least it doesn't seem too, has something to do with it ?

Anyway, let me know if you want me to test something else.

[beelee1]

I'm gonna close this and open a new one on the "Notes". I still think the two are related but w/e will close for now