container matching doesn't work with directory-only paths: fmt/1196 (SIARD)

[ross-spencer]

Hi Richard,

As discussed I've attached what I think to be an accurate skeleton container file for fmt/1196. What do you think? (Also noted that SF might not have this capability to identify either as of yet!)

(Unzipping should provide you a SIARD zip)

fmt-1196-container-signature-id-31020.zip

cc. @Dclipsham. I can't get it to pass in DROID either, but l am reluctant to say the problem is there given the recent proximity of the DROID release. PS. David - If there were public samples that could be shared, that would be a great help!

[Dclipsham]

Hi Ross,
My test samples for building the signature came from the SFA GitHub - https://github.com/sfa-siard/SiardCmd/tree/master/testfiles - off hand I cannot see anything 'wrong' with your file, but it isn't IDing for me either. Note that in the SFA files the '2.1' directory is empty but in yours I can see a zero-byte file - possibly related?

[ross-spencer]

@Dclipsham it sounds like there is a good chance of that! Thanks for giving the file a once-over. I created the new logic in a bit of a rush and writing a zip with the zero-byte file was the easiest way forward. I wasn't expecting it to impact the identification, but it could I guess? - Will try without.

[richardlehane]

👍 @ross-spencer - another notch on the skeleton suite's belt! I hadn't envisioned this type of signature (matching on a directory rather than file) and will need to do a bit of a re-work of my container matching. So this will likely be a bug until the next major release. 👍 @Dclipsham for that extra detail, is good to know that these directories may be empty. 👎 siard team ;), what kind of insanity is this:

[marhop]

Hi guys,

I think this is a tricky issue with the skeleton file, not with Siegfried or DROID or the SIARD specification.

The SIARD ZIP provided by Ross contains just one entry:

• 'header/siardversion/2.1/empty_skeleton_container_file'

Extracting this ZIP file creates the expected directory tree with the empty file at the bottom, yes. But still, inside the ZIP file there is only one entry with this unusual name containing slashes. That's different from the entry 'header/siardversion/2.1/' expected by the PRONOM signature. Although this directory is created when extracting the ZIP file, there is no entry of this name in the ZIP file.

Compare this to the attached ZIP file which is corretly identified as fmt/1196 (by DROID at least). It contains the following four entries, amongst them the one expected by the signature:

• 'header/'
• 'header/siardversion/'
• 'header/siardversion/2.1/'
• 'header/siardversion/2.1/empty_skeleton_container_file'

The differences between the two ZIP files can best be perceived when feeding them to the zipinfo tool or viewing them in a hex editor. (In the latter case, you may find this commented ZIP file example useful for orientation.)

Anyway, it should be noted that although the added file does not impede identification, it may lead to validation failure since the SIARD specification explicitly demands an empty directory. But I suppose we shouldn't worry about validation of skeleton files, so never mind. ;-)

PS: It stands to reason whether one of the two approaches to zipping nested directories is "more correct" than the other. I couldn't find anything specific in the ZIP format specification except that forward slashes are allowed in a file name (see section 4.4.17.1).

[richardlehane]

thanks very much for this extra detail @marhop. I tested your sample with siegfried and it fails to ID properly, so a problem there too. Also, same issue with fido from my quick check.

[marhop]

It does not? Hm, weird. I used DROID 6.4, signature file 94, container signature file 20180920 with 'open archives' option set to false. Does siegfried correctly identify a "real" SIARD 2.1 file like the one linked by David?

[PhillipTommerholt]

Hi guys,

There are many aspects in this thread that I am not sure about, so please disregard if my comment has no relevance.

First thing I notice in the ZIP is that there is no content-folder.
See P_4.2-1 requirement in SIARD2.1

Can this help?

[richardlehane]

@marhop good point - I checked the SIARD 2.1 examples referenced and they don't identify either. Is clearly an assumption I built into siegfried's container matching that I'll just need to rework. fido is also failing for these too so it might have a similar issue with its container algo

@PhillipTommerholt - thanks for this extra detail. It shouldn't matter in this context that the content folder is missing as the container signature only includes that header../ path

siard team - sorry for my little dig above, 👍 for building a very useful digipres tool!