debian packaging report

[okias]

First, I'd like to thank you for Foliate development, it was great experience on desktop. I really enjoy reading books on it (even over the fact I have ebook reader).

At this moment I'm looking forward to porting Foliate to Debian (and Mobian & PureOS, distributions for mobile phones).
For that purpose would be best, if Foliate would become part of Debian (+ Ubuntu and other .deb based distros).

I slightly looked into your debian directory with vision of incorporating Foliate into Debian repository.
I managed to upgrade debian standards, format & fix few issues #529 , but some warnings remains.
If you would give a hand with fixing these (mostly describing licences), that would be great and could relieve you deb packaging burden in future.

$ lintian -iIE --pedantic --show-overrides --color auto --no-tag-display-limit ../com.github.johnfactotum.foliate_2.4.2-1_amd64.changes
E: com.github.johnfactotum.foliate changes: bad-distribution-in-changes-file bionic
N: 
N:    You've specified an unknown target distribution for your upload in the
N:    debian/changelog file. It is possible that you are uploading for a
N:    different distribution than the one Lintian is checking for. In that
N:    case, passing --profile $VENDOR may fix this warning.
N:    
N:    Note that the distributions non-free and contrib are no longer valid.
N:    You'll have to use distribution unstable and Section: non-free/xxx or
N:    Section: contrib/xxx instead.
N:    
N:    Refer to Debian Policy Manual section 5.6.14 (Distribution) for details.
N:    
N:    Severity: error
N:    
N:    Check: fields/distribution
N: 

as I understood you targeting for Ubuntu, so it's fine, no need to care about it

E: com.github.johnfactotum.foliate: no-copyright-file
N: 
N:    Each binary package has to include a plain file
N:    /usr/share/doc/<pkg>/copyright
N:    
N:    Refer to Debian Policy Manual section 12.5 (Copyright information) for
N:    details.
N:    
N:    Severity: error
N:    
N:    Check: debian/copyright
N:

this should be addressed, licensing is important

E: com.github.johnfactotum.foliate: python-script-but-no-python-dep usr/share/com.github.johnfactotum.Foliate/assets/KindleUnpack/compatibility_utils.py #!python
N: 
N:    Packages with Python scripts should depend on the package python. Those
N:    with scripts that specify a specific version of Python must depend,
N:    recommend or suggest on that version of Python (exactly).
N:    
N:    For example, if a script in the package uses #!/usr/bin/python, the
N:    package needs a dependency on python. If a script uses
N:    #!/usr/bin/python2.6, the package needs a dependency on python2.6. A
N:    dependency on python (>= 2.6) is not correct, since later versions of
N:    Python may not provide the /usr/bin/python2.6 binary.
N:    
N:    If you are using debhelper, adding ${python3:Depends} or
N:    ${python:Depends} to the Depends field and ensuring dh_python2 or
N:    dh_python3 are run during the build should take care of adding the
N:    correct dependency.
N:    
N:    In some cases a weaker relationship, such as Suggests or Recommends,
N:    will be more appropriate.
N:    
N:    Severity: error
N:    
N:    Check: scripts
N: 
E: com.github.johnfactotum.foliate: python-script-but-no-python-dep usr/share/com.github.johnfactotum.Foliate/assets/KindleUnpack/mobi_split.py #!python
E: com.github.johnfactotum.foliate: python-script-but-no-python-dep usr/share/com.github.johnfactotum.Foliate/assets/KindleUnpack/mobiml2xhtml.py #!/usr/bin/python
E: com.github.johnfactotum.foliate: python-script-but-no-python-dep usr/share/com.github.johnfactotum.Foliate/assets/KindleUnpack/unipath.py #!python

python stuff, probably needs to be better specified (python3 ? ) + run dh_python3 for these files

E: com.github.johnfactotum.foliate source: source-is-missing src/assets/libarchivejs/libarchivejs-1.3.0/dist/wasm-gen/libarchive.js line length is 32549 characters (>512)
N: 
N:    The source of the following file is missing. Lintian checked a few
N:    possible paths to find the source, and did not find it.
N:    
N:    Please repack your package to include the source or add it to
N:    "debian/missing-sources" directory.
N:    
N:    If this is a false-positive, please report a bug against Lintian.
N:    
N:    Please note, that very-long-line-length-in-source-file tagged files are
N:    likely tagged source-is-missing. It is a feature not a bug.
N:    
N:    Severity: error
N:    
N:    Check: cruft
N: 
E: com.github.johnfactotum.foliate source: source-is-missing src/assets/libarchivejs/libarchivejs-1.3.0/dist/worker-bundle.js line length is 32768 characters (>512)
E: com.github.johnfactotum.foliate source: source-is-missing src/assets/libarchivejs/libarchivejs-1.3.0/src/webworker/wasm-gen/libarchive.js line length is 32549 characters (>512)
E: com.github.johnfactotum.foliate source: source-is-missing src/web/jszip.min.js

making possible to compile these js files while building would be best option I guess

W: com.github.johnfactotum.foliate: initial-upload-closes-no-bugs
N: 
N:    This package appears to be the first packaging of a new upstream
N:    software package (there is only one changelog entry and the Debian
N:    revision is 1), but it does not close any bugs. The initial upload of a
N:    new package should close the corresponding ITP bug for that package.
N:    
N:    This warning can be ignored if the package is not intended for Debian or
N:    if it is a split of an existing Debian package.
N:    
N:    Refer to Debian Developer's Reference section 5.1 (New packages) for
N:    details.
N:    
N:    Severity: warning
N:    
N:    Check: debian/changelog
N: 

I should create ITP, that's on me

W: com.github.johnfactotum.foliate source: no-debian-copyright-in-source
N: 
N:    Every package must include the file /usr/share/doc/<pkg>/copyright. A
N:    copy of this file should be in debian/copyright in the source package.
N:    
N:    Refer to Debian Policy Manual section 12.5 (Copyright information) for
N:    details.
N:    
N:    Severity: warning
N:    
N:    Check: debian/copyright
N: 

debian/copyright files needs to be created

W: com.github.johnfactotum.foliate: no-manual-page usr/bin/com.github.johnfactotum.Foliate
N: 
N:    Each binary in /usr/bin, /usr/sbin, /bin, /sbin or /usr/games should
N:    have a manual page
N:    
N:    Note that though the man program has the capability to check for several
N:    program names in the NAMES section, each of these programs should have
N:    its own manual page (a symbolic link to the appropriate manual page is
N:    sufficient) because other manual page viewers such as xman or tkman
N:    don't support this.
N:    
N:    If the name of the manual page differs from the binary by case, man may
N:    be able to find it anyway; however, it is still best practice to match
N:    the exact capitalization of the executable in the manual page.
N:    
N:    If the manual pages are provided by another package on which this
N:    package depends, Lintian may not be able to determine that manual pages
N:    are available. In this case, after confirming that all binaries do have
N:    manual pages after this package and its dependencies are installed,
N:    please add a Lintian override.
N:    
N:    Refer to Debian Policy Manual section 12.1 (Manual pages) for details.
N:    
N:    Severity: warning
N:    
N:    Check: documentation/manual
N: 

you (and users) probably don't care about man pages

I: com.github.johnfactotum.foliate: extended-description-is-probably-too-short
N: 
N:    The extended description (the lines after the first line of the
N:    "Description:" field) is only one or two lines long. The extended
N:    description should provide a user with enough information to decide
N:    whether they want to install this package, what it contains, and how it
N:    compares to similar packages. One or two lines is normally not enough to
N:    do this.
N:    
N:    Refer to Debian Developer's Reference section 6.2.1 (General guidelines
N:    for package descriptions) and Debian Developer's Reference section 6.2.3
N:    (The long description) for details.
N:    
N:    Severity: info
N:    
N:    Check: fields/description
N: 

improving description would be nice :)

I: com.github.johnfactotum.foliate: extra-license-file usr/share/com.github.johnfactotum.Foliate/assets/libarchivejs/libarchivejs-1.3.0/LICENSE
N: 
N:    All license information should be collected in the debian/copyright
N:    file. This usually makes it unnecessary for the package to install this
N:    information in other places as well.
N:    
N:    Refer to Debian Policy Manual section 12.5 (Copyright information) for
N:    details.
N:    
N:    Severity: info
N:    
N:    Check: files/licenses
N: 
I: com.github.johnfactotum.foliate: package-contains-documentation-outside-usr-share-doc usr/share/com.github.johnfactotum.Foliate/assets/libarchivejs/libarchivejs-1.3.0/LICENSE
N: 
N:    This package ships a documentation file outside /usr/share/doc
N:    Documentation files are normally installed inside /usr/share/doc.
N:    
N:    If this file doesn't describe the contents or purpose of the directory
N:    it is in, please consider moving this file to /usr/share/doc/ or maybe
N:    even removing it. If this file does describe the contents or purpose of
N:    the directory it is in, please add a lintian override.
N:    
N:    Severity: info
N:    
N:    Check: documentation
N: 

licences again....

I: com.github.johnfactotum.foliate source: testsuite-autopkgtest-missing
N: 
N:    This package does not declare a test suite.
N:    
N:    Having a test suite aids with automated quality assurance of the archive
N:    outside of your package. For example, if your package has a test suite
N:    it is possible to re-run that test suite when any of your package's
N:    dependencies have a new version and check whether that update causes
N:    problems for your package.
N:    
N:    In addition, since May 2018 these tests now influence migration from
N:    unstable to testing:
N:    
N:     https://lists.debian.org/debian-devel-announce/2018/05/msg00001.html
N:    
N:    Please add a debian/tests/control file to your package to declare a
N:    testsuite, but please make sure to only add autopkgtests if they provide
N:    meaningful coverage of your package.
N:    
N:    Refer to https://ci.debian.net/doc/ for details.
N:    
N:    Severity: info
N:    
N:    Check: testsuite
N: 

do you have some tests which can be run?

X: com.github.johnfactotum.foliate source: debian-watch-does-not-check-gpg-signature
N: 
N:    This watch file does not specify a means to verify the upstream tarball
N:    using a cryptographic signature.
N:    
N:    If upstream distributions provides such signatures, please use the
N:    pgpsigurlmangle options in this watch file's opts= to generate the URL
N:    of an upstream GPG signature. This signature is automatically downloaded
N:    and verified against a keyring stored in debian/upstream/signing-key.asc
N:    
N:    Of course, not all upstreams provide such signatures but you could
N:    request them as a way of verifying that no third party has modified the
N:    code after its release (projects such as phpmyadmin, unrealircd, and
N:    proftpd have suffered from this kind of attack).
N:    
N:    Refer to the uscan(1) manual page for details.
N:    
N:    Severity: pedantic
N:    
N:    Check: debian/watch
N:    
N:    This tag is experimental. Please file a bug report if the tag seems
N:    wrong.
N: 

would be nice, not necessary

P: com.github.johnfactotum.foliate source: source-contains-browserified-javascript src/web/epub.js code fragment:(function webpackuniversalmoduledefinition(root, factory) { ?if(typeof exports === 'obj
N: 
N:    The following file contains javascript built from browserify
N:    
N:    This file may contain javascript that is build with the help of
N:    browserify or webpack tools.
N:    
N:    You should rebuilt this file from source.
N:    
N:    Severity: pedantic
N:    
N:    Check: cruft
N: 
P: com.github.johnfactotum.foliate source: source-contains-prebuilt-javascript-object src/assets/libarchivejs/libarchivejs-1.3.0/dist/wasm-gen/libarchive.js line length is 32549 characters (>512)
N: 
N:    The source tarball contains a prebuilt (minified) JavaScript object.
N:    They are usually left by mistake when generating the tarball by not
N:    cleaning the source directory first. You may want to report this as an
N:    upstream bug, in case there is no sign that this was intended.
N:    
N:    Severity: pedantic
N:    
N:    Check: cruft
N: 
P: com.github.johnfactotum.foliate source: source-contains-prebuilt-javascript-object src/assets/libarchivejs/libarchivejs-1.3.0/dist/worker-bundle.js line length is 32768 characters (>512)
P: com.github.johnfactotum.foliate source: source-contains-prebuilt-javascript-object src/assets/libarchivejs/libarchivejs-1.3.0/src/webworker/wasm-gen/libarchive.js line length is 32549 characters (>512)
P: com.github.johnfactotum.foliate source: source-contains-prebuilt-javascript-object src/web/jszip.min.js
P: com.github.johnfactotum.foliate source: source-contains-prebuilt-wasm-binary src/assets/libarchivejs/libarchivejs-1.3.0/dist/wasm-gen/libarchive.wasm
N: 
N:    The source tarball contains a prebuilt binary wasm object. They are
N:    usually provided for the convenience of users. These files usually just
N:    take up space in the tarball and need to be rebuilt from source.
N:    
N:    Check if upstream also provides source-only tarballs that you can use as
N:    the upstream distribution instead. If not, you may want to ask upstream
N:    to provide source-only tarballs.
N:    
N:    Severity: pedantic
N:    
N:    Check: cruft
N: 
P: com.github.johnfactotum.foliate source: source-contains-prebuilt-wasm-binary src/assets/libarchivejs/libarchivejs-1.3.0/src/webworker/wasm-gen/libarchive.wasm

scripts should be build from sources

P: com.github.johnfactotum.foliate source: very-long-line-length-in-source-file src/assets/libarchivejs/libarchivejs-1.3.0/dist/wasm-gen/libarchive.js line length is 32549 characters (>512)
N: 
N:    The source file includes a line length that is well beyond the normally
N:    human made code line length.
N:    
N:    This very long line length does not allow Lintian to do correctly some
N:    source file checks.
N:    
N:    This line could also be the result of some text injected by a computer
N:    program, and thus could lead to FTBFS bugs.
N:    
N:    Last but not least, long line in source code could be used to obfuscate
N:    the source code and to hide stuff like backdoors or security problems.
N:    
N:    It could be due to jslint source comments or other build tool comments.
N:    
N:    You may report this issue upstream.
N:    
N:    Severity: pedantic
N:    
N:    Check: cruft
N: 
P: com.github.johnfactotum.foliate source: very-long-line-length-in-source-file src/assets/libarchivejs/libarchivejs-1.3.0/dist/worker-bundle.js line length is 32768 characters (>512)
P: com.github.johnfactotum.foliate source: very-long-line-length-in-source-file src/assets/libarchivejs/libarchivejs-1.3.0/src/webworker/wasm-gen/libarchive.js line length is 32549 characters (>512)

Rest of warnings are fixed as part of #529

[johnfactotum]

Thanks for working on this.

Licensing—well, everything is GPL-3.0, apart from the vendored libraries. From the README:

The following JavaScript libraries are bundled in this software:

• Epub.js, which is licensed under FreeBSD. The included file is patched with various fixes and enhancements (see git history for details).
• The minified version of JSZip, which is dual-licensed. You may use it under the MIT license or the GPLv3 license. See LICENSE.markdown
• libarchivejs, which is MIT licensed. It is a WASM port of the popular libarchive C library.
• crypto-js, which is MIT licensed. The MD5 module is used to generate identifiers for files that don't have unique identifiers.

This software also includes parts from KindleUnpack, which is licensed under GPL-3.0.

Now, the Python issue—the Python code comes from the vendored copy of KindleUnpack, which is self-contained and can be run with either Python 2.7 or Python 3.4 and later. The shebangs in those files (which I believe is why it's showing the error) have no effect as the Python scripts are run by directly spawning either python or python3.

As for tests, no, unfortunately, there aren't any at the moment (unless you count validation of desktop files, etc.). I guess we can try adding some basic tests with dogtail.

making possible to compile these js files while building would be best option I guess
scripts should be build from sources

This is currently tracked in #460.

[archisman-panigrahi]

@johnfactotum @okias Can the package (which will be submitted to Debian, not necessarily the master branch of this repository) be renamed foliate instead of com.github.johnfactotum.foliate ? The package maintainers of other distros have already done that, and it would be much more convenient for the users.

[532910]

Yep, please rename debian package, binary, and all pathes from com.github.johnfactotum.foliate to foliate.

[okias]

except python shebangs, #460, creating ITP (Intetion To Package request) and eventual rename to foliate (which would make sense) is all important stuff solved inside #529

[okias]

slowly progressing with packaging on https://salsa.debian.org/okias-guest/foliate any Debian developer who sees some space for perfection (lintian is pretty annoyed by prebuild binaries) is very welcome! :)

EDIT: also package is named foliate now, only binaries will remain complete.

[archisman-panigrahi]

@okias What are the next steps? Do we have to wait for some Debian maintainer to pick it up from the repo at salsa.debian.org?

[pymnh]

Hey there

What are the next steps? Do we have to wait for some Debian maintainer to pick it up from the repo at salsa.debian.org?

In order to get the package into Debian, you'll need to ask for a Debian Developer to sponsor the upload. See here for further information.

I maintain a few Debian packages myself but am no Debian Developer with full privileges unfortunately, but some notes on the packaging on salsa might be helpful:

1. The source tarball for foliate vendors multiple dependencies which are also partially prebuilt

• Debian does generally not include prebuilt binaries for its packages
• Debian packages should avoid vendoring, instead each library should be packaged on its own. this makes tracking of security issues easier and avoids duplication of code.
• Vendoring might be an option if only few parts of other projects are being included or if the vendored source code is modified to fullfill the needs of the project. IMHO, this would be a grey area and worth asking around on the Debian mentors mailing list
• -> Packaging the libarchivejs source on its own will also solve the problem with the prebuild binaries

2. Regarding libarchivejs: I would propose to ask Debian Javascript Team if it should be maintained under their umbrella: https://wiki.debian.org/Javascript and to also familiarize yourself with their packaging policy: https://wiki.debian.org/Javascript/Policy

3. Also the control file classifies foliate as "section: gnome". Is foliate actually a GNOME app? If yes, I would recommend getting in touch with the Debian GNOME team in order to maintain foliate under their umbrella. If foliate is not strictly a gnome app, i would suggest replacing the section with utils

[johnfactotum]

I wonder how Epiphany is packaged on Debian. It vendors a number of JavaScript libraries. And apparently they don't use NPM either — the readmes suggest that they update those dependencies by manually building and copying. This is pretty much how Foliate does things now.

[FedericoCeratto]

Foliate has packaged and uploaded in Debian. See https://packages.debian.org/source/sid/foliate and https://salsa.debian.org/debian/foliate
@johnfactotum I'm not quite sure how libarchivejs is being used. Deleting src/assets/libarchivejs does not break the ninja build and foliate starts and open epubs.

[johnfactotum]

Yes, it's optional. It's only used for non-Zip comic books. Zip-based formats are decompressed with JSZip.

I have in fact considered simply removing it, as I don't feel too comfortable using libarchivejs, either. It would probably be better to use a binding for native libarchive. And libarchive's API doesn't really seem suited for our use case, anyway. So alternatively it might be better to use a pure JS implementation. But simply removing it is an option, too, as many popular e-book readers also only support Zip, anyway.

Also libarchive has poor support for RAR files (see #417). So that's another reason to drop libarchive. It's pretty much only used to open CBR files, yet too often fails to do that.

[johnfactotum]

Update for the WIP gtk4 branch:

• It now only needs two libraries, zip.js and fflate. The files are still bundled for convenience, but they can be built from source from NPM by running npm run build.
• It no longer uses Epub.js. Instead it includes a custom renderer, foliate-js, as a git submodule. It's MIT licensed.
• It no longer bundles KindleUnpack and no longer needs Python.
• The binary has been renamed to foliate.
• The debian directory hasn't been ported yet.