Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NTFS Support for OPL #110

Closed
dirsors opened this issue Jul 23, 2018 · 24 comments
Closed

NTFS Support for OPL #110

dirsors opened this issue Jul 23, 2018 · 24 comments

Comments

@dirsors
Copy link

dirsors commented Jul 23, 2018

USB flash drive with FAT32 system can only hang up to 4GB file size. DVD game images can easily achieve beyond this limit. And for external HDD drives, the maximum partition size is 32GB (with 16 KB clusters). How about OPL support for NTFS? Very large discs and files will work!

@gingerbeardman
Copy link

AFAIK there is no official documentation for the NTFS file format

@uyjulian
Copy link
Member

Don't use Windows built in formatting tool, as it has the 32GB restriction.
There are many tools for splitting ISOs.

@gingerbeardman
Copy link

https://www.google.com/search?q=ps2+iso+split

@mirh
Copy link

mirh commented Aug 11, 2018

I'm pretty sure NTFS-3G counts almost as if it was "official documentation"

On the other hand, if we are talking about external disks for a ps2... exFAT seems waaay more fit for the role.

@uyjulian
Copy link
Member

exFAT is a bad idea for spinning hard disks because it doesn't backup the FAT table.

@mirh
Copy link

mirh commented Aug 12, 2018

I thought we were talking about usb flash drives?
And even if we are talking about external hdds.. Is there some special kind of write operation you'd been performing?

@uyjulian
Copy link
Member

uyjulian commented Aug 12, 2018

Most flash drives are bad for OPL since they are usually slow. The newer USB 3.0 flash drives are okay.
As far as I'm aware, exFAT is primarily used for flash memory devices like SD cards and USB flash drives.

@gingerbeardman
Copy link

exFAT would be a good choice, but it's also undocumented and needs to be licensed.

@ildar
Copy link

ildar commented Aug 12, 2018 via email

@mirh
Copy link

mirh commented Aug 12, 2018

Well, aside of the licensing (which anyway, for as much as talked, didn't seem so important to stop various linux distros to include it) the same also applies to NTFS too then.
I kind of wonder if you aren't mixing "open" with "documented" though...

But if you exclude these then, there's only ext or f2fs that you could use.

@uyjulian
Copy link
Member

FAT32 is the best option since it is universal, exists for many years, and has read and write support from many OSes and libraries. The different filesystems do not work with all OSes, has limited write support, and may not be documented extensively.
If you have a ISO bigger than 4GB, USBUtil is a really nice tool.

@TnA-Plastic
Copy link

Aside from any 'perspective', it is possible to implement pretty much any filesystem more easily, for USB-storage-devices and possibly internal HDDs, once BDM is ready... So 'yes', it is possible! The current OPL however doesn't have BDM and porting support for a filesystem based on the current implementation would take quite a bit more time...

So long... Just wait (or help out) for BDM to be ready and this feature(-additions) will be quite more likely to come once upon in time. :)

@motolav
Copy link

motolav commented Sep 16, 2019

https://docs.microsoft.com/en-us/windows/win32/fileio/exfat-specification
In case some haven't heard the news exFAT is opensource now and here's the spec for it so it might be worth adding support for it over EXT2/3/4 or NTFS

@uyjulian
Copy link
Member

Too bad exFAT has only one FAT. Decide to power off or reset while a file is open? Data loss. This can happen more frequently on the PS2, where there is no handler for the reset button on some models.

Microsoft didn't open source TexFAT, which has two copies of the FAT (like FAT32).

@mirh
Copy link

mirh commented Sep 17, 2019

Microsoft did not open source anything in fact.
They just gave green light about patents, now you are free to make your own driver with blackjack and hookers. The transaction-safe patent is probably still under intellectual monopoly, but that's really not stopping source code from being written.

The thing is instead, there are a thousand and one similar file systems. And if you really want to go fancy and disregard the ubiquity pro of FAT64, then why can't you just use the modern standard of F2FS?

@carl0sjt
Copy link

Soon PS3 will support exfat. PS4 already officially support it.

@Lalarian
Copy link

Yes, I backup this request. Support for a modern file system is long overdue. FAT32 is super unreliable and together with the need for using special splitting tools, it's just painful.
Any younger filesystem would be better than FAT32, no matter if it's flash optimized or nor.
Be it ext4, NTFS, f2f2 or exFAT.

@gingerbeardman
Copy link

I don't mind what FS is used if it does away with the need to defragment

@uyjulian
Copy link
Member

Defragment will probably always be needed since loading a filesystem driver takes up a part of the few IOP RAM, so storing a single absolute address of the ISO and accessing it that way will always use less RAM.

@carl0sjt
Copy link

instead of a new FS support why not a new format like the ones used on PS3 Loaders. slice the ISOs beyond 4 GB in another 4GB or less file like .iso .iso0 .iso1 that is less files on the disk and more performance.

@Lalarian
Copy link

Lalarian commented Apr 22, 2020

Because slicing involves the requirement for specific tools to use, while with a modern filesystem you could simply copy ISOs to the storage. Also FAT32 is very unreliable with bigger data amounts than the few Megabytes it was once made for, and is very prone to corruption.
For me using OPL with USB instead of SMB currently is just a proof of concept, because it takes just about 5 play sessions in average for your savegames and/or the ISOs on te USB storage to be crippled and lost. For example I was not able to play through ShadowOfTheColossus, although starting over three times, because my savegame was lost over and over.

Filesystems like even the old ext2 are much more stable and will also get rid of the USB-Util-problem (which honestly is no fun tool to use). Also ext2 has much less overhead, which would effectively increase the real data transfer rate and decrease stuttering video or audio aswell as loading times a bit.

@gingerbeardman
Copy link

gingerbeardman commented Apr 23, 2020

Your issue of corrupting ISO and SAVES seems odd, I once had this and it went away by changing the formatting of my USB to the recommended settings (default Allocation unit size) @Lalarian

@J013k
Copy link
Contributor

J013k commented Apr 24, 2020

Some games currently have problems with VMC, so no matter if you use HDD or SMB problems will still remains.
So I doubt that file-format will change anything.
Did you try real memory card?

@TnA-Plastic
Copy link

On another note: You don't need a specific tool, but can technically slice ISOs and add it to the "game list"-file via commandline or scripts!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests