Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Calibre web optional #3500

Closed
wants to merge 11 commits into from
Closed

Calibre web optional #3500

wants to merge 11 commits into from

Conversation

jvonau
Copy link
Contributor

@jvonau jvonau commented Mar 20, 2023

Fixes bug:

none new feature Edit: now a refinement in the install process to make use of an available wheel.

Description of changes proposed in this pull request:

new feature to install Calibre-web's optional addons Edit: uses --system-site-packages to find the deb based python package from the system as the upstream doesn't provide a current wheel.

Smoke-tested on which OS or OS's:

U-22.04 VM This was more an demo of how to declare explicit dependencies without tossing the kitchen sink at the problem. Each package containing the needed header files was added until the install was successful.

Mention a team member @username e.g. to help with code review:

@jvonau
Copy link
Contributor Author

jvonau commented Mar 20, 2023

@OzzieIsaacs @holta

@holta
Copy link
Member

holta commented Mar 20, 2023

Interesting, I wasn't aware of:

https://github.com/janeczku/calibre-web/blob/master/optional-requirements.txt

Above headings clarify the purpose:

  • Google Drive Integration
  • Gmail
  • Goodreads
  • LDAP login
  • OAuth
  • metadata extraction
  • Comics (ComicAPI ?)
  • Kobo integration

Thank you to @OzzieIsaacs whose doc explains further:

https://github.com/janeczku/calibre-web/wiki/Dependencies-in-Calibre-Web-Linux-and-Windows

@jvonau
Copy link
Contributor Author

jvonau commented Mar 20, 2023

Note: "Failed building wheel for cchardet" cchardet-2.1.7.tar.gz was one package that needed to be compiled no python3.10 wheel

@holta
Copy link
Member

holta commented Mar 20, 2023

You should delete the not needed lines in the optional-requirements.txt before installation.

You should only install the optional components you are needing.

several incompatible updates caused crashes of Calibre-Web in the past

( FYI above 3 are from @OzzieIsaacs doc https://github.com/janeczku/calibre-web/wiki/Dependencies-in-Calibre-Web-Linux-and-Windows#optional-features-for-installation-with-manual-installation )

@jvonau
Copy link
Contributor Author

jvonau commented Mar 20, 2023

Understood, just wanted to see what was required for all the options and ferret out other python3.10 issues that might pop up. All in all just reminder if the wheel is not available then the needed compiler tool-chain is required. Just seeing where wheel support is at for the newer pythons and which packages are lagging behind offering support for them, those are the ones that need to be compiled. Bigger question in python land are what to do with cchardet and I mentioned the other PR 'netifaces'. Not having current wheels available is a red flag to something is amiss in the package maintenance.

@OzzieIsaacs
Copy link

One additional thing:
I started to provide a deb file with the latest release (for amd64 and also ARM64), including all optional dependencies. The ARM version is build on an raspberry Pi4 with Raspberry Pi OS:
https://github.com/janeczku/calibre-web/releases/tag/0.6.19

I have a build script here: https://github.com/OzzieIsaacs/calibre-web-test/blob/main/build/make_release.py (I think is only used by me, so error handling is not the best). This could be an option for you to build your own deb file (I don't know which hardware /distributions you are supporting) and provide it

@holta
Copy link
Member

holta commented Mar 21, 2023

Thanks @OzzieIsaacs for explaining.
Your dedication is truly unmatched.

How often do you plan on publishing new .deb files, out of curiosity?

Noting that what Internet-in-a-Box has works extremely well (with an intentionally strong focus on Community CI/CD of your mainline Calibre-Web branch, nurturing community feedback of everything you refine for schools/clinics/families/etc month-by-month, and week-by-week).

Whereas in hindsight we'd most definitely have chosen your apt/deb approach ~5 years ago, had it been available back then! 💯

ASIDE: I find Calibre's Command-Line tools (like calibredb add to import books) to be quite surprisingly glitchy — sometimes barely tolerable in fact! Long term I wonder if there might (maybe!) be room for improvement here...

@OzzieIsaacs
Copy link

The deb file is updated once every release, but not for the nightly updates.

ASIDE: I find Calibre's Command-Line tools (like calibredb add to import books) to be quite surprisingly glitchy — sometimes barely tolerable in fact! Long term I wonder if there might (maybe!) be room for improvement here...

I'm not the maintainer of calibre, only of calibre-web, beside the name they only share the database format, but nothing more

@holta
Copy link
Member

holta commented Mar 21, 2023

I'm not the maintainer of calibre, only of calibre-web, beside the name they only share the database format, but nothing more

Indeed we're very much aware! We stopped using Calibre and began using Calibre-Web about 5 years ago for good reason!

I mention this larger question (about the fragility of calibredb add) as it greatly affects some communities that invest themselves in Calibre-Web.

Hence affecting the entire ecosystem of grassroots libraries, E-Book pragmatism in low-income communities, ETC!

So one day there MIGHT come a time when Calibre-Web (or some associated community?) might offer a distinctly cleaner approach for mass importing books, far more easily + reliably? 🌟

@jvonau
Copy link
Contributor Author

jvonau commented Mar 21, 2023

Further refinement, using --system-site-packages and installing the python deb version provides the needed wheel without needing to be compiled. I need to circle back on a clean install to confirm optional-requirements.txt but that is not really required right now so I'm going to comment out that for future work if wanted.

@holta
Copy link
Member

holta commented Mar 21, 2023

@jvonau's suggestion, for those that haven't yet seen it:

@jvonau
Copy link
Contributor Author

jvonau commented Mar 21, 2023

Circled back around for the optional parts and found a replacement for cchardet, filed upstream PR.

@jvonau
Copy link
Contributor Author

jvonau commented Mar 22, 2023

Closing this one in favor of 3503, this one can be updated in the future if requested.

@jvonau jvonau closed this Mar 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants