No longer can create usable UDF images

[berarma]

I've used mkudffs in the past for creating UDF images. After that I loop-mounted the image to copy the data in and I burned to BDR media using growisofs.The command I used was very simple:
mkudffs -l "label" mybdr.udf

And evertything worked. Now I'm trying the same and the written BDR can't be mounted. I have to use the loop option to be able to mount it.

I've read this issue can be related to the UDF image using 512 bytes sectors and BDR having 2048 bytes sectors. So I've changed the command to:

mkudffs -m dvdr -b 2048 -l "label" mybdr.udf

I'm not using -m bdr because it uses UDF2.50 and I can't write to that image.

The new issue is that I can't longer mount the image to copy the files in: "wrong fs type, bad option, bad superblock on /dev/loop1, missing codepage or helper program, or other error."

I've thought maybe I should mount specifying the new block size. I've tried:
mount -o loop,bs=2048 mybdr.udf /mnt

And also using losetup:
losetup -b 2048 /dev/loop0 mybdr.udf
mount /dev/loop0 /mnt

It doesn't work either: "can't read superblock on /dev/loop0"

It's like I'm missing some critical piece of the inner workings of filesystem images or maybe I'm doing it all wrong. How else could I write BDR from the command line?

I'm using Debian Buster with udftools 2.1.

Thanks for this software.

[pali]

I'm using Debian Buster with udftools 2.1.

Could you please try it with the latest version?

[berarma]

Could you please try it with the latest version?

I've tried it and it's the same.

I've tried another combination of options and it works with both 2.1 and master branch:
mkudffs -b 2048 -l "label" mybdr.udf

Not using the option to select the media type seems to fix the mounting issue. Now I have to try to write to BDR media but I guess it should work.

I think the HD type is the default. Is it an issue creating a HD image and then writing it to a BDR?

I'm unsure about what I'm doing.

Thanks.

[pali]

I was looked more deeply on description on your issue and now I see where is the problem.

mkudffs's --media option specify media type, it sets various geometry setting for UDF image. And these are different for hard disks, CDs, DVDs and BD. Also they are different between -ROM, -R, -RW and -RAM variants of medias.

For example, if you choose -R (as recordable), created UDF image is for (sequential) recording via VAT. -RW is for read-write media with spartable. -ROM is just read-only without VAT and without spartable (can be used also on -R or -RW medias).

As --media specify geometry, you cannot use DVD geometry on BDs or CDs. In most cases it would not work (at least correctly). But you can try.

-R type differs from all other as it creates UDF image with VAT and it should be "burned" (or packet written) into optical disc. It is just small image which later can be incrementally modified on optical disc. It is possible that kernel would not be able to read such "images" if they are not on real optical media. Optical media provides additional meta information (last written block, table of content, etc...) which are stored "outside" of image itself. And without these meta information image itself cannot be read correctly. You can look at recent changes in git for udfinfo tool. There are new options --blocksize=, --startblock=, --lastblock= and --vatblock= which can be used to provide these metainformation if you want to read udf images of optical disc correctly (and not reading directly optical disc).

So if your purpose is to generate image and then modify, -R is fully unsuitable for you (unless you have a tool which will simulate incremental writing via VAT).

--media=bdr automatically choose UDF 2.50. Reason for this is because standards mandates that BD disc must use UDF 2.50 or higher and previous versions do not have to work. I was reported that there are more devices which really cannot read UDF 2.01 on BD disc.

If your purpose is to create -ROM data (you do not want to modify it later after burning to optical media, neither via incremental writing, multisession or real -RW), you can try to choose hard disk type and set correct block size (which is for optical discs 2048). But as I said, it does not have work with all optical drives and devices.

Normally also block size is "geometry" information and --media sets it correctly. Hard disks have in most cases 512 bytes long blocks and therefore this value is default for --media-type=hd.

I will look if there is a way how to mount generated -R disc on Linux by just "loop" module. I agree it would be nice if it works out of the box.

Also please note that Linux kernel is not able to write or modify any data on UDF if VAT or Metadata is used or if UDF rev is above 2.01. So Linux kernel would not be able to modify -R images for sure. Metadata is required part of UDF 2.50.

[berarma]

Thanks for the explanation.

I was doing it wrong by using types bdr and dvdr. The problem with using a ROM media type is that it can't be mounted read/write, so I can put any files into it. So I'm left with an empty image.

If your purpose is to create -ROM data (you do not want to modify it later after burning to optical media, neither via incremental writing, multisession or real -RW), you can try to choose hard disk type and set correct block size (which is for optical discs 2048). But as I said, it does not have work with all optical drives and devices.

This is the only way I can create an image, mount it, write files to it and burn to dics. It works but some other drive might not be able to read it, that's the downside.

It's a pretty unsatisfactory situation. BDs are great for file backups, .

[pali]

I was doing it wrong by using types bdr and dvdr. The problem with using a ROM media type is that it can't be mounted read/write, so I can put any files into it. So I'm left with an empty image.

bdr and dvdr are not ROM media type but they are Recordable write once media type, used for incremental recording via VAT. Linux kernel currently does not support "updating" UDF with VAT yet.

ROM media types are cd, dvd (and in future there could be bd).

This is the only way I can create an image, mount it, write files to it and burn to dics. It works but some other drive might not be able to read it, that's the downside.

It's a pretty unsatisfactory situation. BDs are great for file backups, .

What is missing in mkudffs is support to pre-master images, i.e. putting files into filesystem at the time when disc/image is formatting/generating. So it would not be needed to mount image and fill it externally. Patches for such support are for sure welcome.

[berarma]

bdr and dvdr are not ROM media type but they are Recordable write once media type, used for incremental recording via VAT. Linux kernel currently does not support "updating" UDF with VAT yet.

Yes, I was just recalling that neither -R nor -ROM are good.

What is missing in mkudffs is support to pre-master images, i.e. putting files into filesystem at the time when disc/image is formatting/generating. So it would not be needed to mount image and fill it externally. Patches for such support are for sure welcome.

While I may have the technnical skills, I'm totally lacking the knowledge. It would take a long time just understanding how it works. And we'd still be missing 2.50/2.60 support in the kernel so still lacking compatibility.

I just want to make backups of regular files. What would be the best option for this? Should I use ISO9660?

[pali]

And we'd still be missing 2.50/2.60 support in the kernel so still lacking compatibility.

Kernel has readonly support UDF 2.50 and because UDF 2.60 is in read mode backward compatible with UDF 2.60 it means that kernel can read also UDF 2.60 filesystems.

I just want to make backups of regular files. What would be the best option for this? Should I use ISO9660?

Depends on your needs. If you want to read them only on Linux machine with not too old kernel versions, you can use UDF 2.01 in HD mode with 2048 block size (as I written above). Kernel would read such UDF fs on BD-R discs without issues.

If you need to read them in other machines then probably ISO9660 would be better. But beware that original ISO9660 has support only for 8.3 filnames (8 chars per name, 3 per extension; all uppercase) and only for a few subdirectories. No linux permissions, etc. For this purpose and wide support you need to use both Joliet and RockRidge extensions. (And still I'm not sure for requirements of ISO9660 on BD-R)

If you really need compliance with BD-R and UDF, you need to use other SW. I know that e.g. Nero for Linux has support for UDF 2.50 and UDF 2.60 and support also BD-R discs.

[berarma]

Yes, I've read about ISO9660 with extensions for longer case-sensitive names and directory levels, that's all I'd need. I think K3B and Gnome use ISO9660 for BD-R. I'm not aware of the possible issues. Anyway, since I'll be using only Linux I guess both ISO9660 and UDF2.01 would be valid.

Thank you very much. Your responses are extremely useful to help me understand the issues.

[pali]

Anyway, since I'll be using only Linux I guess both ISO9660 and UDF2.01 would be valid.

Yes, linux kernel file system drivers are designed to work independently of media. If you find combination which does not with the latest kernel, file system developers can look at it. Inability to modify udf 2.50 fs by linux kernel is known missing feature which nobody implemented yet.

Thank you very much. Your responses are extremely useful to help me understand the issues.

OK! I'm going to close this issue as current behavior of media type described here is expected.

[rogerxxxx]

Dear God! I just ran into this (bug), spending ~12+ hours wondering why I couldn't do what I previously could do; formatting UDF filesystem, mounting/editing via loop and then writing/recording the UDF image.

HOWTO.udf
Under section "Support for UDF filesystems"

-- mkudffs udfimage
++ mkudffs --blocksize=2048 udfimage
++ NOTE: Prior to mkudffs 1.1 default value was always 2048.  (Block size 2048 is required for writing the UDF image to ROM media.)

Shrugs, here's a quick patch to the doc/HOWTO.udf doc file. (Funny, finnicky attachment doesn't accept .patch/.diff files.)
patch.file:///home/roger/tmp/HOWTO.udf.blocksize.patch
HOWTO.udf.blocksize.txt

[rogerxxxx]

Ran into this bug again!

An additional note should be put into the HOW TO and/or documentation and/or manual page, until this is fixed, stating:

Do not use "--media-type=" with a specified optical media type when mounting and copying files using the loop file system into the created UDF image file, as the resulting CD/DVD/BDR media will not be readable after recording. Work around by explicitly setting the block size using "--blocksize=2048" when the final destination will be recorded to optical media.

Took me a very long time (years, with lots of poorly written optical media discs) to get this procedure correct!

[pali]

@rogerxxxx both your two post with proposed changes are wrong. See my detailed response above for explanation: #43 (comment) Also do not hijack old already closed issues as nobody is going to monitor them.