Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The 2.50 version of UDF image for --media-type HD can not be created. #20

Open
fengjiannan2010 opened this issue Jun 6, 2018 · 24 comments

Comments

@fengjiannan2010
Copy link

Hello:
I use the udftools version 2.0 to make udf image can not mount

mkudffs --lvid="20180606-1" --media-type=hd --udfrev=2.50 b.udf

mmount -o loop,rw b.udf disc/
mount: /dev/loop4 readonly

i will read write mode mount

@pali
Copy link
Owner

pali commented Jun 6, 2018

mkudffs --lvid="20180606-1" --media-type=hd --udfrev=2.50 b.udf

mkudffs: Error: UDF revision above 2.01 is not currently supported for specified media type

And also UDF driver in Linux kernel does not support write operation for UDF 2.50+ filesystems (it is read-only).

UDF 2.50 for non-VAT disks requires metadata partition which mkudffs is not able to generate (yet).

@pali
Copy link
Owner

pali commented Jun 9, 2018

Also patches for generating metadata partition which is needed for UDF 2.50 support on HD are welcome!

@wdlkmpx
Copy link

wdlkmpx commented Sep 29, 2018

If the kernel doesn't support mounting udf 2.50+ rw.. how do you add files?

Maybe through a fuse driver?

There is 'mkisofs': this app can create hybrid iso9660/udf1.0 "isos".. maybe some day mkudffs will be able to create blu-ray data isos from a directory ..

@pali
Copy link
Owner

pali commented Sep 29, 2018

There is (userspace) project udfclient which can add or delete files on UDF disk, including UDF 2.50.

But adding files via mkudffs when formatting disk is not bad idea. Patches for such functionality are welcome.

@wdlkmpx
Copy link

wdlkmpx commented Oct 3, 2018 via email

@pali
Copy link
Owner

pali commented Oct 3, 2018

It could be absorbed by this project..

I do not see reason why. udfclient is separated project and works without udftools. Basically udfclient and udftools provides different utilities.

I've never used that app

It is working fine on Linux. You can really try it. E.g. Debian already has package for it.

It's really sad the linux kernel didn't add rw support for udf 2.50

That is truth. As always patches are welcome!

@Randrianasulu
Copy link

digging into old feature requests via google I found link to this project:
https://sourceforge.net/p/fudf/news/2008/07/fudf-v20-released/

description says:
"version 2.0 supports creation, reading and editing of UDF2.50 compliant filesystems." But.. not sure if anything from this project can be reused..?

@pali
Copy link
Owner

pali commented Nov 22, 2021

Well, anything which is GPLv2+ license compatible can be reused. But it means that somebody has to take foreign code and adopt it. As I wrote, patches and contributions are welcome. (Sometimes reusing foreign code for another projects means lot of work, more than rewriting it from scratch...)

@Randrianasulu
Copy link

so, I compiled udfclient 0.8.11 (with small edit in uio. h due to conflicting iovec define.. on termux/android, arm 32 bit system).

next I booted netbsd 9.2 boot iso (200mb) on qemu-system-i386 (pentium2 cpu, 320mb ram). I append fileimage to qemu as '-hda file', but this resulted in 512 byte/sector virtual hdd as seen by guest netbsd

next I formatted it with newfs_udf with '-v' and '-V' switches both set to 2.50. (on /dev/rwd0d!) then mounted with 'mount - t udf /dev/wd0d /mnt' and copied some stuff and unmounted and 'shutdown - h now' my virtual machine.

then I tried udfclient -W .. only to discover..
*** marked read-only due to read-only support for Metadata partition ***

newfs_udf from udfclient software on linux makes strange udf with version (both of them) set to 0.00. mkudffs - m bdr also makes image not yet writable by udfclient.

A lot of learning happened, but no solution was found ( (not counting qemu vm with netbsd, probably going via vndconfig on file located on virtual hdd! (for 512 vs 2048 byte/sector mismatch!). I have no such amount of free space on my tablet..

@wdlkmpx
Copy link

wdlkmpx commented Nov 23, 2021

FUDF v2, looks like it was written for Windows, RtlAnsiStringToUnicodeString()

a lot of .cpp files, but I think it doesn't use C++ specific stuff, looks like C for MSVCRT.......... but some classes are implemented CDataBaseCore

I don't think it will compile on UNIX(-like) systems, but it's interesting. There's no project to build the lib, so with some effort an autotools project for MinGW might make the thing compile

It would take a long time to convert it to a pure C project and port it to Linux, but as an open source project it's interesting ...

Ah... there's no LICENSE anywhere, the author forgot to add a license, so it must be treated as public domain ...

@Randrianasulu
Copy link

also there was mkudfiso from 2010 on sourceforge, but it only makes udf 1.02?

from my download manager:
https://phoenixnap.dl.sourceforge.net/project/blurayauthor/mkudfiso/mkudfiso-svn-checkout/mkudfiso-svn-checkout-20100208.tar.gz

also there is 'our' bdwrite, but it probably does not do real 2.50 level stuff (not tested its output yet)
https://git.cinelerra-gg.org/git/?p=goodguy/cinelerra.git;a=blob;f=cinelerra-5.1/cinelerra/bdwrite.C;h=6a2b4ca3cb4b9bcec54d7b8620a74b51cafd7395;hb=HEAD

and there is netbsd kernel commit
"First part of shrinking/growing metadata partition support:

  • extending the metadata partition"
    netbsd

@pali
Copy link
Owner

pali commented Nov 23, 2021

If you want to add support for UDF 2.50+ into mkudffs for non-VAT images, then I think it is required to implement just these two things:

  • create empty metadata partition during formatting (similarly like is created sparable or vat partition)
  • in function udf_alloc_blocks() insert blocks also into metadata partition

And if you want to implement inserting files into filesystem during formatting by mkudffs, then it is just needed to implement ICBTAG_FLAG_AD_SHORT and ICBTAG_FLAG_AD_LONG cases in insert_data() function (currently there are errors with exit(1) calls). As support for creating files and directories is there, just missing support for putting data (ADs) to (empty) files.

I think that for both things it would be much harder to copy this functionality from external 3rd UDF implementation into udftools as writing it from scratch for udftools.

When you want to add files into UDF filesystems during formatting, there is no need to implement shrinking/growing metadata partition. Just calculate size which is enough for all data you want to put.

@Randrianasulu
Copy link

Randrianasulu commented Nov 25, 2021 via email

@Randrianasulu
Copy link

some more links

[https://www.yumpu.com/en/document/view/26257714/all-approved-udf-260-dcns](udf 2.60 errata? ) - talks about BD-R requrements and how (dcn-5102) Metadata Bitmap file shall not be recorded for read-only media..

[https://stackoverflow.com/questions/8832031/udf-filesystem-reading-bluray-metadata-partition-libdvdread-with-iso-images](stackowerflow question dealing with analysis of some udf 2.5+ images)

[https://github.com/firecore/libudfread/blob/master/src/udfread.c](libudfread source) - so, this metadata partition is by itself virtual, and main three (two?) files within it really live on real partition as files with special filetype?

But how to construct those metadata files?

@pali
Copy link
Owner

pali commented Nov 25, 2021

But how to construct those metadata files?

@Randrianasulu What about to read official specification :D and not looking at random/incomplete suggestions?
Nothing wrong, just I want to point that that all important details are in official specifications, including DCNs for BD-R requirements.

@pali
Copy link
Owner

pali commented Nov 25, 2021

Just to note that udfinfo has already support for reading/parsing metadata partition and metadata file.

@Randrianasulu
Copy link

Randrianasulu commented Nov 25, 2021 via email

@Randrianasulu
Copy link

Randrianasulu commented Nov 26, 2021 via email

@pali
Copy link
Owner

pali commented Nov 26, 2021

mkudfiso project (from sourceforge.net) produce invalid and broken UDF images. I was using it during testing of udfinfo to ensure that such broken/invalid images do not cause segfaults of udfinfo and instead print error messages.

@Randrianasulu
Copy link

Randrianasulu commented Nov 26, 2021 via email

@Randrianasulu
Copy link

@pali, I found patent apparently describing udf 2.50 write process

sony patent
Examples of meta-information recorded in the metadata partition include an FSDS, an ICB (Information Control Block) and an FID (File Identifier Descriptor) created for each directory and file.
so... it seems metadata partition creation part in itself not very complex, but managing metadata files even for create-only-once case is not trivial (for outsiders like me).

On the related news udf verifier compiles on arm32/termux and reports no errors for Netbsd-created udf 2.50 filesystem image (with blocksize 512)

@Randrianasulu
Copy link

giant hack, but appears to work on image files:
https://github.com/Randrianasulu/newfs_udf-linux/tree/main

now, thing is I am not sure if mkudffs itself should hold logic for creating random dirs and extending files and working with directory trees like mkiso.. May be separate script (using dd for per-sector writing!) can be hacked together just for putting some data in.

But yes, it seems extending at least metadata files should be implemented for formatting 2.50+ udfs...

@pali
Copy link
Owner

pali commented Aug 20, 2023

Yea, generating metadata partition should not be too hard. Just take UDF 2.60 spec, fill required structures with correct data, place them into list of extends and finally write to the disk. Just somebody needs to do it.

@KingDuckZ
Copy link

What's the current state on this issue? Is it possible to create a non-empty UDF 2.50 (or 60) filesystem in 2023 on Linux? I tried newfs_udf and it's not 2.50, whereas mkudffs does seem to create a 2.60 filesystem but I found no way to fill it with data. udfclient is not able to even create an empty directory on filesystems as created by mkudffs. Any known hack/workaround to make this work?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants