Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[VFS] Add support for loading ZArchive files #165

Merged
merged 2 commits into from Sep 3, 2023

Conversation

rtissera
Copy link

This PR adds support for loading ZAR (ZArchive files) of a decrypted dump of XBOX 360 game.
The process is basically to dump your disc, extract files using your favorite tool, and create a ZAR file from it using ZArchive tool.
ZArchive is a format created by @Exzap for Cemu to store Wii U dumps originally (WUA files). It is very lightweight and uses modern Zstd compression.

This PR implements ZArchive supports through zstd and zarchive dependencies as third-party libraries, premake build support and full C++ VFS implementation for this specific kind of device.

I have personally tested it on around 60 games without noticing any behavior difference or performance hit between a decrypted folder and a ZAR file.

Comments, enhancements, and possible mistakes are welcome.

@AhayriSG
Copy link

There is no support for XBLA digital titles atm, right?
image

@rtissera
Copy link
Author

rtissera commented May 22, 2023 via email

@Z3USI0TA
Copy link

Right, disc based games only so far but I can have a look on how to support XBLA games if possible. Le lun. 22 mai 2023 à 12:23, AhayriSG @.> a écrit :

There is no support for XBLA digital titles atm, right? [image: image] https://user-images.githubusercontent.com/59465544/239880383-1798b8ce-c4ab-4328-a310-b6a8a820e2d9.png — Reply to this email directly, view it on GitHub <#165 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAW4WBMTOAPDQ6I6QMGCZ4DXHM5DRANCNFSM6AAAAAAYJSXXHY . You are receiving this because you authored the thread.Message ID: @.
>

Nice work!!! Thanks!!!
Working with XBLA games but use before wxPirs.exe and after ZArchive.exe

@AhayriSG
Copy link

Right, disc based games only so far but I can have a look on how to support XBLA games if possible. Le lun. 22 mai 2023 à 12:23, AhayriSG @.> a écrit :

There is no support for XBLA digital titles atm, right? [image: image] https://user-images.githubusercontent.com/59465544/239880383-1798b8ce-c4ab-4328-a310-b6a8a820e2d9.png — Reply to this email directly, view it on GitHub <#165 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAW4WBMTOAPDQ6I6QMGCZ4DXHM5DRANCNFSM6AAAAAAYJSXXHY . You are receiving this because you authored the thread.Message ID: _
@**
.**_>

Nice work!!! Thanks!!! Working with XBLA games but use before wxPirs.exe and after ZArchive.exe

Yup, works for me as well.
image

@mirh
Copy link

mirh commented Sep 2, 2023

Any compressed disk format is nice I guess, but what are the upsides and downsides compared to (say) chd, xz or aaru?

@rtissera
Copy link
Author

rtissera commented Sep 2, 2023

ZArchive is based on zstd which is faster (both compression and decompression-wise) than LZMA / LZMA2 (xz).
CHD is based on LZMA(1) and slow, eats quite a bit of CPU.
I have never tested aaru so far.

ZArchive sounds like a very good middleground as we benefit from compressed disc-images (I have not tested nor implemented XBLA yet) while provide similar performance to non-compressed images.

@Rex000

This comment was marked as off-topic.

@rtissera
Copy link
Author

rtissera commented Sep 2, 2023

@Rex000 the PR being not merged yet, this is the expected behavior.

Are you running a custom build including this PR ?

@Rex000

This comment was marked as spam.

@rtissera
Copy link
Author

rtissera commented Sep 2, 2023

@Rex000 this is not custom builds with this PR built in.

@Rex000

This comment was marked as spam.

@rtissera
Copy link
Author

rtissera commented Sep 2, 2023

Ask @AhayriSG he seems to have a build.

@Rex000

This comment was marked as spam.

@Rex000
Copy link

Rex000 commented Sep 2, 2023

Why are my comments being marked as off-topic and spam?

@Margen67
Copy link
Member

Margen67 commented Sep 2, 2023

Why are my comments being marked as off-topic and spam?

Tech support doesn't belong on GitHub. Use the Xenia Discord server for that.

@Gliniak Gliniak merged commit 18daf4a into xenia-canary:canary_experimental Sep 3, 2023
2 checks passed
@rtissera
Copy link
Author

rtissera commented Sep 3, 2023

Thanks for merging !

@Zorklis
Copy link

Zorklis commented Sep 3, 2023

Thanks for merging !

Thanks for making this :)

@remote009
Copy link

One thing the zar format does in Cemu is allow for base game, update, and DLC to be packed into a single zar file. Then, the updates and DLC are automatically applied when loading, bypassing the install step. As far as I can tell, this is not implemented here--is that correct?

@rtissera
Copy link
Author

@remote009 correct

kencinder pushed a commit to SteamGridDB/steam-rom-manager that referenced this pull request Dec 8, 2023
ZAR is the new compressed format adopted by Xenia-Canary.

More info here :xenia-canary/xenia-canary#165

It's also way easier to launch XBLA games converted to .zar
@ElektroStudios
Copy link

ElektroStudios commented Dec 31, 2023

Excuse me but can I ask why this awesome feature is still not implemented in the stable release (I mean in the master branch / the emulator that the end user downloads from https://xenia.jp/download/ ) since May 21 of 2023?.

It's really very unlikely that one can find out that this brilliant feature exists in Xenia, and even more unlikely that one figures out that it is only implemented in the xenia-canary experimental release... (or at least that is what I assume since loading a ZAR file in the master branch's release it crashes Xenia)

Brief history about how I got here: I discovered this PR because it was mentioned at the very end of a long article about compression formats for emulators, and no body reads that, only few freaks like me that has enough time to investigate and to read bibles about compression optimizations for collecting game disc images...

I'm nobody to say what the developers of Xenia should do about anything, let no one misunderstand me, but I love this feature and I humbly think this should be available for all end-users through the normal/stable/latest release of Xenia, and also that Xenia developers could pump visibility and adoption for this awesome zar feature!.

I wrote this request because I previously kept my XBox 360 discs extracted and compressed in RAR5 format (which has better compression than ZAR, but it has the disadvantage that I need to extract the RAR file to be able to load the game in Xenia), and I discovered this feature of ZAR format support randomly, so I would like other people to be able to enjoy it directly from the stable version of the master branch.

@Zorklis
Copy link

Zorklis commented Dec 31, 2023

I discovered this PR because it was mentioned at the very end of a long article about compression formats for emulators, and no body reads that.

No people do read that wiki. You can also find out about these sorta PR changes from Emulator subreddit on Reddit.

why this awesome feature is still not implemented in the stable release (I mean in the master branch).

I don't know, the "stable" Xenia (Master) should basically be seen as base Xenia, while yes Xenia Canary is "experimental" it's still stable, both are work in progress and unfinished which will take years if not a decade to play every game, even then awesome people will keep showing up and improving on the previous work (see other emulators like PCSX2).
Xenia is pretty slow on making changes, I see Canary as more of a Xenia Advanced and everyone really just use Canary anyways.
tl:dr check out the PR on their side.

@ElektroStudios
Copy link

ElektroStudios commented Dec 31, 2023

everyone really just use Canary anyways

Maybe experienced or advanced users like you, but Xenia (master) is the one that is provided through the official website, so it is the one that common or casuals users like me finds, downloads, and ends using it. There is no mention to Xenia-canary on the Download section of the Xenia's website, just a hyperlink to Xenia's (Master) repo on Github. Then I think probably many people don't even know about Xenia-canary existance. EDIT: I found a mention on the QuickStart guide.

Thanks for your participation!

@Zorklis
Copy link

Zorklis commented Dec 31, 2023

Maybe experienced or advanced users like you, but Xenia (master) is the one that is provided through the official website, so it is the one that common or casuals users like me finds, downloads, and ends using it I think probably many people don't even know about Xenia-canary existance.

Reddit, Twitter, YouTube, TikTok exists. on PCGamingWiki I've made quite clear that Canary is what everyone on those platforms and discord recommends. Anyways this is off topic

@Darknior
Copy link

Darknior commented Feb 18, 2024

One thing the zar format does in Cemu is allow for base game, update, and DLC to be packed into a single zar file. Then, the updates and DLC are automatically applied when loading, bypassing the install step. As far as I can tell, this is not implemented here--is that correct?

It is what i search and dream for x360 :)
A file format like WUA on CEMU or XCI on SWITCH to compress and include the game and his updates + DLC in it.
Really it will change the game ... will it be possible and planned here on Xenia please ?

Thanks @rtissera for adding this new feature :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet