Resources with inconsistent schema; e.g. missing filename_extension field

[huyz]

I'm trying to run the joplin-blog tool and we're running into problems with the resources Web Clipper API endpoint returning empty filename_extension fields.

I took a look at my database and either it's corrupted or it has vestiges of an old DB schema because the fields title, filename, and filename_extension are not consistent:

This is the API output for a non-working attachment:

❯ curl 'http://localhost:41184/resources/7de408d4ec98e0fa8314c6b092156956?fields=id,title,filename,file_extension&token=c8a14932909ae1...54846490'
{"id":"7de408d4ec98e0fa8314c6b092156956","title":"IMG_20130713_134357.026.jpg","filename":"IMG_20130713_134357.026.jpg","file_extension":"","type_":4}%

This is the API output for a working attachment:

❯ curl 'http://localhost:41184/resources/070b6ba4b5124ab89c1565397069e519?fields=id,title,filename,file_extension&token=c8a14932909ae1...54846490'
{"id":"070b6ba4b5124ab89c1565397069e519","title":"IMG_6992.jpeg","filename":"","file_extension":"jpeg","type_":4}%

Environment

Joplin version: 2.4.9
Platform: macOS
OS specifics: 10.15.7

Steps to reproduce

1. Start using Joplin many years ago
2. Over time, collect a Resources DB table that seems to have multiple schemas
3. Try to query the Resources Web Clipper with http://localhost:41184/resources/<id>?fields=id,title,file_extension&token=<token>
4. Ponder whether the data is corrupted or if there is a deterministic and complete algorithm to determine the filename of the attachment on the filesystem.

Describe what you expected to happen

Would like answers to:

1. is my data corrupted?
2. if corrupted, how can I fix it?
3. if not corrupted, is it that the Resources schema follows multiple schemes, the API should be backwards-compatible, but there's a bug because it's not backwards-compatible?
4. If the data is not corrupted and there is no bug in the API, is there a complete and deterministic algorithm for figuring out the filename of the attachment on the filesystem?

Logfile

[laurent22]

This is not a support forum and probably you should ask on the joplin-blog repo anyway.

[huyz]

This is not a support forum and probably you should ask on the joplin-blog repo anyway.

I will quote the great Laurent on the Joplin Support forum 😄 :

For bug reports, please post them on GitHub - https://github.com/laurent22/joplin/issues
If you are not sure if it's a bug or not, PLEASE POST ON GITHUB ANYWAY.
If you post a bug report here, we cannot track it and there's a chance it won't be looked at and fixed.

If you look closer at the content of my issue, you will see that this is potentially a bug report. So that means that Github is the appropriate venue unless you say that this is definitely not a Joplin bug (and it's not considered important how my data got corrupted by Joplin). (I still think it's—more likely than not—a Joplin bug).

And also I've already linked to the original discussion on the joplin-blog repo at rxliuli/joplin-utils#19 (comment), which points fingers at Joplin core.

[laurent22]

Ok fair enough. But no it's not a bug in the app because it doesn't affect any of the official Joplin apps, so indeed the right place is to on the joplin-utils repo.

[rxliuli]

Ok fair enough. But no it's not a bug in the app because it doesn't affect any of the official Joplin apps, so indeed the right place is to on the joplin-utils repo.

I want to know what will cause the extension to be empty. If it is not an error, is it possible for me to obtain the extension in a more accurate way? @laurent22

[laurent22]

Not all resources will have a file extension or even a name, it all depends how it got there. In general the app uses the id as the name and check the mime type if a file extension is needed.

You can check the mimeUtils file in this repo for conversion between mime type and file extension.

[huyz]

Sounds to me like either (1) the Data API could be improved to simply return the filename of a resource or (2) the Data API docs could be improved to explain the "complete and deterministic algorithm for figuring out the filename of the attachment on the filesystem", as I wrote in my initial post.

@rxliuli maybe you could figure out that algorithm and submit that as a PR to the Data API docs?