Processing an ACSM file added via a plugin

[ping]

Bug description

I've been using your plugin together with DeDRM to download from the library. The library is a non-US library and cannot send directly to the Kindle so calibre + DeACSM + DeDRM has been a real life saver.

I'm working on a plugin that downloads the .acsm. The plugin doesn't do much else except call calibre to download the .acsm file and depends on DeACSM and DeDRM to do the heavy lifting.

The problem is that the epub from the plugin downloaded .acsm does not contain any meta data at all. The epub is otherwise fine and viewable from calibre. My calibre setup seems fine because dragging a downloaded .acsm into calibre works perfectly (full meta, cover, title, author, etc).

Log from processing plugin-provided .acsm

DeACSM v0.0.16: Trying to parse file TheLastMusketeer_9780062048400_564352.acsm
DeACSM v0.0.16: Try to fulfill ...
Not notifying any server since that was disabled.
DeACSM v0.0.16: Downloading book ...
DeACSM v0.0.16: Loading book from https://acs.cdn.overdrive.com/ACSStore1/0293-1/941/FDF/FF/{941FDFFF-6CFD-4E11-A53D-178C18A32631}Fmt410.epub
Download took 524 ms (HTTP 200)
That's a ZIP file -> EPUB
DeACSM v0.0.16: File successfully fulfilled ...
DeACSM v0.0.16: Executing plugin DeDRM ...
DeDRM v10.0.3: Trying to decrypt 73bhkix9.epub
DeDRM v10.0.3: Verifying zip archive integrity
DeDRM v10.0.3: 73bhkix9.epub is a secure Adobe Adept ePub for UUID 53c5d855-9583-4ff0-8d85-76fa3c47d485
DeDRM v10.0.3: Trying UUID-matched encryption key DeACSM_uuid_53c5d855-9583-4ff0-8d85-76fa3c47d485
eda6cq9i.epub is a secure Adobe Adept ePub.
DeDRM v10.0.3: Decrypted with key DeACSM_uuid_53c5d855-9583-4ff0-8d85-76fa3c47d485 after 0.2 seconds
Watermark: Successfully stripped 39 ADEPT watermark(s) from ebook.
DeDRM v10.0.3: Post-processing took 0.1 seconds
DeDRM v10.0.3: Finished after 0.3 seconds
DeACSM v0.0.16: Plugin returned path '/var/folders/kr/bxxq54x51l714b1mrzyj0t1h0000gn/C/calibre_6.21.0_tmp_jxa1dsxh/trmhsgp9.epub', updating.
DeACSM v0.0.16: Deleting existing ACSM file /var/folders/kr/bxxq54x51l714b1mrzyj0t1h0000gn/C/calibre_6.21.0_tmp_jxa1dsxh/hkl6l6sr/TheLastMusketeer_9780062048400_564352.acsm ...
Job: 0 Downloading <redacted> finished
Starting job: Downloading <redacted>

Log from processing downloaded .acsm

DeACSM v0.0.16: Trying to parse file The Last Musketeer - Stuart Gibbs.acsm
DeACSM v0.0.16: Try to fulfill ...
Not notifying any server since that was disabled.
DeACSM v0.0.16: Downloading book ...
DeACSM v0.0.16: Loading book from https://acs.cdn.overdrive.com/ACSStore1/0293-1/941/FDF/FF/{941FDFFF-6CFD-4E11-A53D-178C18A32631}Fmt410.epub
Download took 74 ms (HTTP 200)
That's a ZIP file -> EPUB
DeACSM v0.0.16: File successfully fulfilled ...
DeACSM v0.0.16: Executing plugin DeDRM ...
DeDRM v10.0.3: Trying to decrypt 3wgeii6t.epub
DeDRM v10.0.3: Verifying zip archive integrity
DeDRM v10.0.3: 3wgeii6t.epub is a secure Adobe Adept ePub for UUID 53c5d855-9583-4ff0-8d85-76fa3c47d485
DeDRM v10.0.3: Trying UUID-matched encryption key DeACSM_uuid_53c5d855-9583-4ff0-8d85-76fa3c47d485
qfk0il7s.epub is a secure Adobe Adept ePub.
DeDRM v10.0.3: Decrypted with key DeACSM_uuid_53c5d855-9583-4ff0-8d85-76fa3c47d485 after 0.1 seconds
Watermark: Successfully stripped 39 ADEPT watermark(s) from ebook.
DeDRM v10.0.3: Post-processing took 0.1 seconds
DeDRM v10.0.3: Finished after 0.3 seconds
DeACSM v0.0.16: Plugin returned path '/var/folders/kr/bxxq54x51l714b1mrzyj0t1h0000gn/C/calibre_6.21.0_tmp_jxa1dsxh/lifprm8h.epub', updating.
DeACSM v0.0.16: Deleting existing ACSM file /<redacted>/The Last Musketeer - Stuart Gibbs.acsm ...
Added The Last Musketeer to db in: 0.1
Added 1 books in 3.0 seconds

The plugin is very much in a draft state so I'm not quite ready to publish it publicly. But briefly, the plugin calls calibre's download_ebook() with a custom browser instance that sends the required auth HTTP headers.

endpoint_url, headers = self.client.get_loan_fulfilment_details(
    loan["id"], loan["cardId"], format_id
)

def create_custom_browser():
    br = browser()
    for k, v in headers.items():
        br.set_header(k, v)
    return br

self.gui.download_ebook(url=endpoint_url, create_browser=create_custom_browser)

Operating system

MacOS

Which version of Calibre are you running?

6.21.0

Which version of the ACSM Input plugin are you running?

v0.0.16

Import type

No response

Further information

No response

[Leseratte10]

Thanks for the report - that plugin sounds like a great idea. This is very likely to be a bug in Calibre where it doesn't correctly hand over the downloaded file to all the plugin(s).

There used to be similar bugs in Calibre in the past - bug 1945889 for eBooks added through the auto-add functionality and bug 1992244 for eBooks added through the web server; both of which resulted in broken metadata.

It's likely that this (adding books through Calibre's download_ebook function) has a very similar bug.

I'm going to try and reproduce this, and when I confirmed it's a Calibre bug I'll open a bug report on their bug tracker to hopefully get this fixed.

[Leseratte10]

Just tested it myself, this is most likely a Calibre bug. I'll open a bug report.

[Leseratte10]

Bug report: https://bugs.launchpad.net/calibre/+bug/2025355

[ping]

Thanks so much for your help!
This is my first attempt at writing a calibre plugin so I wasn't entirely sure if the screw up was on my end.

[ping]

The reported calibre bug has been fixed, so this can be closed as far as I am concerned.
Thanks for your help!