Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add WebP image support. #159

Closed
mcpierce opened this issue Feb 27, 2020 · 29 comments
Closed

Add WebP image support. #159

mcpierce opened this issue Feb 27, 2020 · 29 comments
Assignees
Labels
runtime Issues relating to the runtime requirements and behavior.
Milestone

Comments

@mcpierce
Copy link
Contributor

See this artifact for WebP codec support.

@mcpierce mcpierce added the runtime Issues relating to the runtime requirements and behavior. label Feb 27, 2020
@mcpierce mcpierce added this to the 0.6 milestone Feb 27, 2020
@mcpierce mcpierce self-assigned this Feb 29, 2020
mcpierce added a commit to mcpierce/comixed that referenced this issue Feb 29, 2020
mcpierce added a commit to mcpierce/comixed that referenced this issue Feb 29, 2020
mcpierce added a commit that referenced this issue Mar 2, 2020
* [Issue #159] Added dependency on webp-imageio-sejda@0.1.0

* [Issue #159] Modify the image loader setup to support WebP-encoded images.
@mcpierce mcpierce closed this as completed Mar 2, 2020
@bareheiny
Copy link

Doesn't seem to be working as expected.

Covers aren't returned during search for import:

WebP - no covers

Then when an import is attempted, I get this in the cmd window:

Exception in thread "Jarvis-ComiXed" java.lang.UnsatisfiedLinkError: C:\Users<user>\AppData\Local\Temp\4033803682668637191webp-imageio.dll: Can't find dependent libraries
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.load0(Unknown Source)
at java.lang.System.load(Unknown Source)
at com.luciad.imageio.webp.NativeLibraryUtils.loadFromJar(WebP.java:131)
at com.luciad.imageio.webp.WebP.loadNativeLibrary(WebP.java:31)
at com.luciad.imageio.webp.WebP.(WebP.java:38)
at com.luciad.imageio.webp.WebPDecoderOptions.(WebPDecoderOptions.java:20)
at com.luciad.imageio.webp.WebPReadParam.(WebPReadParam.java:24)
at com.luciad.imageio.webp.WebPReader.getDefaultReadParam(WebPReader.java:147)
at javax.imageio.ImageIO.read(Unknown Source)
at javax.imageio.ImageIO.read(Unknown Source)
at org.comixed.model.library.Page.getImageMetrics(Page.java:145)
at org.comixed.model.library.Page.(Page.java:120)
at org.comixed.loaders.ImageEntryLoader.loadContent(ImageEntryLoader.java:45)
at org.comixed.adaptors.archive.AbstractArchiveAdaptor.processContent(AbstractArchiveAdaptor.java:211)
at org.comixed.adaptors.archive.ZipArchiveAdaptor.loadAllFiles(ZipArchiveAdaptor.java:90)
at org.comixed.adaptors.archive.ZipArchiveAdaptor.loadAllFiles(ZipArchiveAdaptor.java:45)
at org.comixed.adaptors.archive.AbstractArchiveAdaptor.loadComic(AbstractArchiveAdaptor.java:138)
at org.comixed.task.model.ProcessComicTask.startTask(ProcessComicTask.java:60)
at org.comixed.task.runner.Worker.run(Worker.java:187)
at java.lang.Thread.run(Unknown Source)

@mcpierce
Copy link
Contributor Author

mcpierce commented Mar 2, 2020

@bareheiny Okay, there's a DLL mentioned in the stacktrack:

C:\Users<user>\AppData\Local\Temp\4033803682668637191webp-imageio.dll

(the username seems to be redacted)

Can you verify if that DLL is present on your system when CX is running? (the filename might change since it's a temporary file)

Opening the JAR file I'm seeing the following contents:

        0  05-24-2019 12:21   native/win/32/
   794112  05-24-2019 12:21   native/win/32/webp-imageio.dll
        0  05-24-2019 12:21   native/win/64/
  1100800  05-24-2019 12:21   native/win/64/webp-imageio.dll

and the native library for the Mac ran for me here.

@bareheiny
Copy link

I redacted the username as it's not an alias...which is fine for anybody how already know that bareheiny = Guy Incognito, but not something that I want all and sundry to know :P

Anyways, I can see the temp webp-imageio.dll files - the one mentioned in the log is still there, and another was created when I just started CX.

As some WebP comics were still in the processing queue, CX threw the same error, just referencing the newer copy of the dll.

@dukeofradish
Copy link

dukeofradish commented Mar 3, 2020

Same error here under Win10. The dll exists while the CX is running.
2020-03-02 21:46:57.489 DEBUG 9076 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor : Processing time: 67ms
Exception in thread "Jarvis-ComiXed" java.lang.UnsatisfiedLinkError: C:\Users<user>\AppData\Local\Temp\4769299251234480738webp-imageio.dll: Can't find dependent libraries
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.load0(Unknown Source)
at java.lang.System.load(Unknown Source)
at com.luciad.imageio.webp.NativeLibraryUtils.loadFromJar(WebP.java:131)
at com.luciad.imageio.webp.WebP.loadNativeLibrary(WebP.java:31)
at com.luciad.imageio.webp.WebP.(WebP.java:38)
at com.luciad.imageio.webp.WebPDecoderOptions.(WebPDecoderOptions.java:20)
at com.luciad.imageio.webp.WebPReadParam.(WebPReadParam.java:24)
at com.luciad.imageio.webp.WebPReader.getDefaultReadParam(WebPReader.java:147)
at javax.imageio.ImageIO.read(Unknown Source)
at javax.imageio.ImageIO.read(Unknown Source)
at org.comixed.model.library.Page.getImageMetrics(Page.java:145)
at org.comixed.model.library.Page.(Page.java:120)
at org.comixed.loaders.ImageEntryLoader.loadContent(ImageEntryLoader.java:45)
at org.comixed.adaptors.archive.AbstractArchiveAdaptor.processContent(AbstractArchiveAdaptor.java:211)
at org.comixed.adaptors.archive.ZipArchiveAdaptor.loadAllFiles(ZipArchiveAdaptor.java:90)
at org.comixed.adaptors.archive.ZipArchiveAdaptor.loadAllFiles(ZipArchiveAdaptor.java:45)
at org.comixed.adaptors.archive.AbstractArchiveAdaptor.loadComic(AbstractArchiveAdaptor.java:138)
at org.comixed.task.model.ProcessComicTask.startTask(ProcessComicTask.java:60)
at org.comixed.task.runner.Worker.run(Worker.java:187)
at java.lang.Thread.run(Unknown Source)

@bareheiny
Copy link

Maybe pointing to the issue? For sure copying the dll into the java/bin directory didn't work - unsurprising I guess.

https://stackoverflow.com/questions/6092200/how-to-fix-an-unsatisfiedlinkerror-cant-find-dependent-libraries-in-a-jni-pro#6092576

@mcpierce
Copy link
Contributor Author

mcpierce commented Mar 3, 2020

Testing this on my Windows laptop, the WebP comic page appears to have loaded:

2020-03-03 07:39:46.694 DEBUG 7196 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Loading file: name=example.webp size=5368028
2020-03-03 07:39:46.694 DEBUG 7196 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Loading entry: name=example.webp size=5368028
2020-03-03 07:39:46.805 DEBUG 7196 --- [ Jarvis-ComiXed] org.comixed.utils.FileTypeIdentifier     : Attempting to detect mime type for stream
2020-03-03 07:39:46.812 DEBUG 7196 --- [ Jarvis-ComiXed] org.comixed.utils.FileTypeIdentifier     : result=image/webp
2020-03-03 07:39:46.812 DEBUG 7196 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Content type: webp
2020-03-03 07:39:46.813 DEBUG 7196 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Loading content: filename=example.webp length=5368028
2020-03-03 07:39:46.814 DEBUG 7196 --- [ Jarvis-ComiXed] org.comixed.loaders.ImageEntryLoader     : Loading image into comic
2020-03-03 07:39:46.816 DEBUG 7196 --- [ Jarvis-ComiXed] org.comixed.model.library.Page           : Creating offset: filename=example.webp content.size=5368028
2020-03-03 07:39:46.816 DEBUG 7196 --- [ Jarvis-ComiXed] org.comixed.model.library.Page           : Generating MD5 hash
2020-03-03 07:39:46.831 DEBUG 7196 --- [nio-7171-exec-6] o.c.service.library.LibraryService       : Finding up to 101 comics updated since Tue Mar 03 07:39:45 EST 2020 for comixedadmin@localhost
2020-03-03 07:39:46.847 DEBUG 7196 --- [nio-7171-exec-6] o.c.service.library.LibraryService       : Checking comic: timestamp: 1583239185072 >= 1583239185072 id: 105 > 105
2020-03-03 07:39:46.852 DEBUG 7196 --- [nio-7171-exec-6] o.c.service.library.LibraryService       : Returning 0 updated comics
2020-03-03 07:39:46.862 DEBUG 7196 --- [ Jarvis-ComiXed] org.comixed.model.library.Page           : Returning: 37411EF12E6D61D8E051303C4A2CD5F7

And peeking at the page and at the database, all of the metrics were extracted, which could only be done if the WebP library worked.

My Windows laptop specs are:

image

@bareheiny
Copy link

I tried another Windows machine - same thing's happening.

Both machines are 64bit Win 10 Pro. Specs on both are lower than your machine, but I wouldn't expect that to be an issue.

Both are also running JRE 1.8.

@dukeofradish
Copy link

Is there value in looking back to the part where we first find the comics? It tries to create thumbnails but it doesn't create thumbnails for webp images. It does create thumbnails for jpg in this part. It doesn't crash when opening CBZs or CBRs during this part. Only when trying to use the import function.

@bareheiny
Copy link

I put the dll in the java\bin dir and tried again.

I’m not getting that error anymore - it’s been replaced by another >_<

No thumbnails generated either, and the logs say the archive doesn’t have any images...

@mcpierce
Copy link
Contributor Author

mcpierce commented Mar 4, 2020

@bareheiny Can you give me a little more detail when you say you "put the dll in the java\bin dir and tried again"? Did you remove it from the JAR, or just copy it over there? I don't think Java's going to look for DLLs in that directory unless you explicitly tell Java to do so. And can you paste in the text from the logs?

Also, please know that CX does not generate thumbnails. All it does it extract the image from the archive and return the bytes to the browser, and the browser itself renders the image from those bytes.

The flow for loading images is:

  1. before the comic has been processed during import, the backend will attempt to find the cover for the comic by loading the first archive entry that has a MIME type of "image/*", and
  2. after the comic has been processed, getting the cover just gets the first page record out of the database, finds the entry filename for the page, and then extracts that entry without considering its MIME type.

In both cases, the backend doesn't try to decode the image: it only loads the bytes and returns them to the frontend.

This WebP (and other image-processing) code only comes into play when the backend is performing the initial comic import, and only when getting the metrics of the image file. After that, it doesn't care what the bytes are.

@bareheiny
Copy link

I'll need to start CX up again to get the logs - will do that when I get some free time.

I don't muck around with the CX code these days, so I just copied the DLL from the temp directory, stripped the extra characters (so it's named webp-imageio.dll) and put it in the java\bin folder in my machines "Program Files" directory. As the bin directory is in the Windows PATH variable, I understood that Java or Windows looks there by default.

Anyway, it caused another error so I was likely barking up the wrong tree.

Cover images aren't being extracted and forwarded to the browser at any point (see the image attached to my first comment in this thread), and although I'm sure I saw text about MIME types and WebP in the logs, I definitely saw messaging saying the archive didn't contain any images.

@bareheiny
Copy link

bareheiny commented Mar 5, 2020

CX log (no webp-imageio.dll in the java\bin directory):
comiXed.log

Text from the CMD window:
CMD.txt

@mcpierce
Copy link
Contributor Author

mcpierce commented Mar 5, 2020

How many pages were in the WWDE01WebP80 (1).cbz file? I see where the comic was processed, and it loaded one page out of it:

2020-03-05 21:01:13.132 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.task.runner.Worker           : Found a persisted task to run: type=PROCESS_COMIC
2020-03-05 21:01:13.132 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.task.adaptors.TaskAdaptor    : Getting task adaptor: type=PROCESS_COMIC
2020-03-05 21:01:13.134 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.t.encoders.ProcessComicTaskEncoder   : Deleting persisted task: id=101 type=PROCESS_COMIC
2020-03-05 21:01:13.143 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.t.encoders.ProcessComicTaskEncoder   : Decoding process comic task: comic=100
2020-03-05 21:01:13.146 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.task.runner.Worker           : Starting task: Process entry task: filename=D:\Comics\WWDE01WebP80 (1).cbz
2020-03-05 21:01:13.233 DEBUG 1560 --- [ Jarvis-ComiXed] o.comixed.task.model.AbstractWorkerTask  : Processing comic: id=100
2020-03-05 21:01:13.271 DEBUG 1560 --- [ Jarvis-ComiXed] o.comixed.task.model.AbstractWorkerTask  : Getting archive adaptor
2020-03-05 21:01:13.271 DEBUG 1560 --- [ Jarvis-ComiXed] o.comixed.task.model.AbstractWorkerTask  : Loading comic
2020-03-05 21:01:13.271 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Processing arms
2020-03-05 21:01:13.587 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Loading entire comic: D:\Comics\WWDE01WebP80 (1).cbz
2020-03-05 21:01:13.587 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Processing entries for archive
2020-03-05 21:01:13.587 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Loading file: name=P00001.webp size=786986
2020-03-05 21:01:13.588 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Loading entry: name=P00001.webp size=786986
2020-03-05 21:01:13.607 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.utils.FileTypeIdentifier     : Attempting to detect mime type for stream
2020-03-05 21:01:13.608 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.utils.FileTypeIdentifier     : result=image/webp
2020-03-05 21:01:13.608 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Content type: webp
2020-03-05 21:01:13.608 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Loading content: filename=P00001.webp length=786986
2020-03-05 21:01:13.608 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.loaders.ImageEntryLoader     : Loading image into comic
2020-03-05 21:01:13.611 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.model.library.Page           : Creating offset: filename=P00001.webp content.size=786986
2020-03-05 21:01:13.611 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.model.library.Page           : Generating MD5 hash
2020-03-05 21:01:13.646 DEBUG 1560 --- [ Jarvis-ComiXed] org.comixed.model.library.Page           : Returning: 18ms
2020-03-05 21:01:15.188 DEBUG 1560 --- [ Jarvis-ComiXed] o.c.adaptors.archive.ZipArchiveAdaptor   : Closing archive: D:\Comics\WWDE01WebP80 (1).cbz

So that exception's getting thrown when the Page class is trying to get the image metrics. I'm not sure why your system isn't finding the DLL that comes with the JAR file. But this is ultimately a bug with the WebP library, not something we can fix at the CX level since it's happening outside of our code space.

I'm opening a bug with them to get them to fix it.

@bareheiny
Copy link

WWDE01WebP80 (1).cbz has 52 pages.

There's a lot of "Loading entry: name P000..." type entries in the log as well, which all occur before the first (and only) hash is generated.

I've converted my collection to WebP now - which reduced the space used significantly. While I do have un-converted backups, I'll likely be less active with regards to CX for a bit until I can get my converted library loaded.

@mcpierce
Copy link
Contributor Author

mcpierce commented Mar 5, 2020

The log's no longer available (the link is 404). Can you repost it?

@bareheiny
Copy link

I noticed the logs were missing earlier, but no clue where they went.

CMD.txt
comiXed.log

@bareheiny
Copy link

I rebuilt my surface (for unrelated reasons), and tried this again - completely different set of errors. Then I tried again, and back to the missing libraries.

Unfortunately I didn’t capture the logs. May try again later.

But it got me to wondering how you converted a test comic to WebP? I’m getting a bunch of “no image in archive” messages...so perhaps the conversion process is corrupting the image somehow.

@mcpierce
Copy link
Contributor Author

mcpierce commented Mar 6, 2020

@bareheiny Well, I never said "converted". 😉 What I did was to take some existing WebP-encoded images and put them into a zipfile with a "cbz" extension and then import it as a comic. Can you email one of those comics to me to test locally?

Ultimately, though, I don't think this is a CX issue, but a bug in the WebP library JAR. I've files a bug with them and hopefully they'll get a fix together for it. Maybe go over there and let the developer know you're intested in the fix as well?

@bareheiny
Copy link

Thinking about it, the conversion wouldn't trigger a library issue anyways (I assume!).

Here are two copies of the same file - one was converted via CR, the other via cwebp. Oddly, the file sizes are different - so I may look into why, but for now I'm more interested in whether or not you can import them (and view the images).

https://www.dropbox.com/s/v5uv4ys8ju9dshv/A52%20%28cwebp%20converted%29.cbz?dl=0
https://www.dropbox.com/s/pesgxkt0qrksg9h/A52%20%28CR%20converted%29.cbz?dl=0

Let me know once you've picked the files up.

I'm watching the issue you created, but I'll add a comment shortly :)

@dukeofradish
Copy link

dukeofradish commented Mar 14, 2020

(copied from sejda-pdf/webp-imageio#1)

@mcpierce The tool you want is Dependencies. https://github.com/lucasg/Dependencies
Running that across the webp-imageio.dll showed a missing dependency for 'ucrtbased.dll'. That DLL is installed with the VS2017 Build Tools Windows 10 SDK.

After installing that DLL webp comics import properly.

The only problem I see is when choosing a library to import, the covers do not appear correctly. They appear as broken images.
webp-dependency

@bareheiny
Copy link

@dukeofradish noice! I'd installed most of VS on my desktop, and got WebP archives importing...but using a few GB of space just to get that working wasn't viable on my Surface.

Re the cover images, I'm seeing the same thing - once a comic is fully imported, they display fine. In the CX logs I'm seeing a lot of entries indicating CX is checking every file in the archive...any chance you're seeing that?

It's almost like CX is opening the archive, checking the first file....going "nope, not a cover image", closing the archive, reopening it...loading the first file, then the second...checking it, "nope, not a cover image" and then repeating.

On larger archives, I'm seeing hundreds of lines like that repeated...

@bareheiny
Copy link

On a 64bit Win 10 Surface Pro 4, I'm seeing VCRUNTIME140D.dll as missing as well.

@dukeofradish
Copy link

Try this. https://www.microsoft.com/en-us/download/details.aspx?id=52685

@dukeofradish
Copy link

I've attached a log of importing 2 of the same comic, same meta data, etc but one is webp and the other is jpg.
jpgwebimport.txt

@bareheiny
Copy link

I grabbed the missing dll from my other machine :)

And yup, that's the stuff I'm seeing in the log. Will create a bug report about it.

@dukeofradish
Copy link

@bareheiny This is probably tangential. In the 6-prerelease is the reader supposed to be functional?
When I'm on the server machine and attempt to open a book I get an error in the upper right corner with "No such comic found: id=100".
I understand that Comixed is not intended to be a reader so I didn't know if this was WaI for the pre-release.
Screenshot (11)

@bareheiny
Copy link

I think @mcpierce doesn't intend for CX to be a reader.

I have noticed that "No such comic found" message before - I usually look at it, scratch my head because the comic info is displaying and then move on.

I really need to get better at raising bugs for things like that as I'm working through processing my library >_<

@mcpierce
Copy link
Contributor Author

@dukeofradish As @bareheiny mentioned, CX isn't going to be a reader. Granted, there's a page viewer, but it's just for viewing pages; i.e., it doesn't treat that as reading the comic, nor does it track last page or anything. It's just for doing page-related maintenance (blocking/unblocking/deleting/undeleting pages).

@mcpierce
Copy link
Contributor Author

@bareheiny That toaster popup is a side effect of the comic state not having a proper "no such comic" field in it. So when you first go to a comic's details page, before it's loaded the comic, that message comes up. I haven't added that new field yet, but am going to open a ticket now so I don't forget to do it. It's annoying, and I see it all the time too. 😄

mcpierce added a commit that referenced this issue Jun 22, 2020
* [Issue #159] Added dependency on webp-imageio-sejda@0.1.0

* [Issue #159] Modify the image loader setup to support WebP-encoded images.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
runtime Issues relating to the runtime requirements and behavior.
Projects
None yet
Development

No branches or pull requests

3 participants