GT should consistently have file extensions

[tboenig]

Files referenced in the METS and stored in the data directory should have file extensions to make life easier for PAGE Viewer etc.

[bertsky]

I totally agree. We have started to make this a pattern in the processors (preprocessing, OCR), but IMO core should lead by good example in Workspace.add_file – at least if no local_filename was already specified by the caller – and in the bagger (see OCR-D/core#258).

[kba]

Implementing OCR-D/core#258 will most likely fix this since the extensions are present until bagging...

[bertsky]

Yes, but processors that do not care about this and do not use local_filename when doing their Workspace.add_file currently also effectively suppress extensions. This case is still in core's responsibility.

[kba]

You're right, I just meant that for our provided GT the problem is the bagger. Workspace.add_file must be fixed too ofc.

[bertsky]

Oh, now I got it. We just keep agreeing you know!

[wrznr]

@kba Fixed?

[kba]

It's fixed for the bagger but I'm still evaluating whether

when doing their Workspace.add_file currently also effectively suppress extensions. This case is still in core's responsibility.

is still an issue in core.

[kba]

Yes, but processors that do not care about this and do not use local_filename when doing their Workspace.add_file currently also effectively suppress extensions.

Not sure whether I can follow. Can you give me an example when this might happen @bertsky ?

[bertsky]

Yes, but processors that do not care about this and do not use local_filename when doing their Workspace.add_file currently also effectively suppress extensions.

Not sure whether I can follow. Can you give me an example when this might happen @bertsky ?

I can't see it myself right now. What I do understand is that Workspace.add_file does not in itself require either the local_filename or url kwarg, especially if it does not pass a content with it. So OcrdMets.add_file will instantiate a new OcrdFile and then set local_filename=None.

So it all depends on what then happens with that file reference later-on in the processor. If content was passed to Workspace.add_file, then an exception will come up. (I have already complained about this as a documentation issue.) Otherwise, the processor might use OcrdMets.find_files to get a reference and then do things to it. Somewhere along that path local_filename will/must be set. There we have to look whether it is in core's responsibility to ensure filename extensions.

Sorry, that's all I can offer ATM.

[cneud]

It appears all file extensions are available now in assets/data, a related issue OCR-D/core#332 in core was closed - can this be closed too?

[kba]

Yes, fixed in assets and we're doing file extensions in the processors now as well.