installing apps to wii u seems very slow compared to switch counterpart

[thedig123]

I am on hb appstore 2.2 and I am installing retroarch. I know it mentions it is a large download I did manage to download it pretty quickly but the install seems to just be crawling, Is this normal for the Wii U? I mean retroarch is only 27 MB surely it shouldnt take that long (currently not even half way after 30 minutes) granted I am extremely new to Wii U modding but it seems like it is chugging slowing, I love the app but would it be easier to FTP into the wii U and copy the files fore the other parts. My SD card is a samsung evo plus U3, maybe that is the issue. sorry for making an issue about this

[Masamune3210]

Its zipped and has a lot of small files, it might just be easier to download and extract it on a computer and FTP the files over

[vgmoose]

As you noted, the download is faster, but then the install is where it actually goes and pulls each file out of the downloaded zip. Retroarch has 740 total files, and each file seems like it has a lot of overhead. It's like this on Switch as well (1655 files) but not nearly as slow as described here.

There is a print for every extracted file, would you be able to try this fix that silences that print: artifacts-7.zip (built from gitlab)

If it's not the print slowing it down, then likely there's some other wiiu-specific slow down with the zip extraction logic that becomes an issue when it's called repeatedly. (Kind of like an n+1 query, the overhead for each file within the zip being extracted being too great compared to just copying in the file like in FTP)

[thedig123]

Its zipped and has a lot of small files, it might just be easier to download and extract it on a computer and FTP the files over

yeah i did that with more ra cores and the arcade cores

[vgmoose]

This may be a result of reading-from-SD (from the newly downloaded zip) and then writing-to-SD (for the extracted file) too rapidly without a large enough buffer in between.

hb-appstore uses libget's ZipUtil for extraction:
https://github.com/fortheusers/libget/blob/master/src/ZipUtil.cpp#L297-L298

With a relatively small fixed buffer size for going in between extract and write. The problem is then exaggerated by doing this for each file in the zip as well.

Some options: load entire / larger chunks of the zip into memory as a whole, or increase buffer size within the Extract function in ZipUtil, and see how it affects performance

[thedig123]

This may be a result of reading-from-SD (from the newly downloaded zip) and then writing-to-SD (for the extracted file) too rapidly without a large enough buffer in between.

hb-appstore uses libget's ZipUtil for extraction: https://github.com/fortheusers/libget/blob/master/src/ZipUtil.cpp#L297-L298

With a relatively small fixed buffer size for going in between extract and write. The problem is then exaggerated by doing this for each file in the zip as well.

Some options: load entire / larger chunks of the zip into memory as a whole, or increase buffer size within the Extract function in ZipUtil, and see how it affects performance

apologies for the late response I didn't see the message and truthfully forgot about this issue, I will check out those options how can I do those options sorry I am a bit new to this thank you so much for the response

[vgmoose]

I've been experimenting with a larger buffer size, and still the extracting is painfully slow. Since we have a fair amount of memory on Wii U available, going to try a change that loads the entire zip into ram first before extracting.

However, if the issue is related to just the cost of file creation on wiiu, this may not be fixable with the current state of the toolchain. For reference, downloading Opensupaplex (~100 files) has the following extraction times on different platforms:

1. PC: immediately extracts
2. Switch: around 20 seconds
3. Wii U: around 5 minutes

[vgmoose]

Should be resolved in 2.3 with unzip changes, see #89 (comment) for an explanation.

[vgmoose]

Fixed in 2.3.1