Requires sufficiently new version of meson

[orblivion]

I tried with the version of meson in Debian jesse, it failed with a lexer error. I grabbed it from pip3, and it worked. May want to mention this in the README.

[orblivion]

Looks like the same goes for ninja

[legoktm]

I did most of my testing using Debian unstable (sid). jessie/stable has 0.25, sid/unstable has 0.37.

Do you know specifically what version of meson you needed? Otherwise we can just pick 0.37 as an arbitrary version and use it.

[mgautierfr]

@orblivion can you paste the output of the error. Maybe I can rewrite the meson file to handle meson 0.25.

I did testing on Fedora (but with custom meson) and Ubuntu 16.10 (with meson 0.34).

I think we can go with 0.34 as minimum version to use for now. However, I already know that it will change in the futur, I've patches on meson itself to build static binaries and cross-compile on windows.

[orblivion]

When I try jesse's version of meson:

Meson encountered an error in file meson.build, line 27, column 43:
lexer

When I try Meson 3.7 but then jesse's version of ninja:

Meson encountered an error:
Could not detect Ninja v1.6 or newer

(which is kinda weird, I would have expected pip3 to take care of getting a new ninja, but I'm still trying to figure that out)

[orblivion]

@mgautierfr It may actually be nice to get the meson file to parse under 0.25. When I try meson 0.37 (available via PyPi) it complains (as I mentioned earlier) that ninja is too old. If I try to install Ninja via PyPi it wants a newer version of cmake than what Jesse has to offer. 🤦‍♂️ Pretty soon I'll have my own hacked version of Debian Testing manually built 😛

I may instead try 0.34 (or earlier) from Meson's git repo, see if that is old enough to be okay with the version of Ninja I already got. (pypi only offers 0.37 now)

[mgautierfr]

I will write a new version to make it work with meson 0.25.
The main problem is that the find_library function not only change of "place" but also gains a new argument to search library in a specific directory and this is needed if you've installed ctpp2 in some specific directory.
As you are packaging kiwix-lib, ctpp2 should be install in standard lib dirs and it should work.

(which is kinda weird, I would have expected pip3 to take care of getting a new ninja, but I'm still trying to figure that out)

Ninja is a different (c++) project from meson. So pip cannot install or upgrade ninja. It a simple binary so if you really need it, you can just copy it somewhere and add it to your PATH. (And it's probably what I will do in kiwix-build to be sure we have the right version)

[kelson42]

@mgautierfr It's urgent to fix that problem. kiwix-lib needs to compile fine for all people on Linux. 0.34 seems to me a far too recent version. Ubuntu LTS which is only one year old has only 0.29.

We need:

• To know exactly which version of meson is needed (written in the README)
• It should work with the version of last LTS for Fedora and Ubuntu (or if really impossible - we provide an extremely clear step-by-step procedure about how to get this working).

[mgautierfr]

(Oups, auto close of github)

I've updated the meson.build script to use the old API if meson version is < 0.31.0.
I have no more syntax error with meson 0.29.0. Can you test it on your side ?

[orblivion]

I will probably try it out within the next day and let you know. Thanks!

[orblivion]

Hmm, still doesn't like something:

Meson encountered an error in file meson.build, line 30, column 15:
lexer

Right on an open square bracket. I don't suppose they didn't even have arrays back then? Again, Debian Jesse has Meson 0.21:

https://packages.debian.org/source/jessie/meson

[mgautierfr]

Hum...

I must have do some strange things with my virtualenv while testing with meson 0.29.0. The meson.version() I use doesn't exists in 0.29.0. So my previous commit 317b13b is useless and should be reverted.

The only correct (but not quick) way of doing I see seems to be to add a recent version of meson to jessie backport repository. The stretch meson package is 0.37, maybe it can be backported to jessie.

[orblivion]

Or install Meson from a release tag on github. I was going to try that next.

[orblivion]

I think pypi just doesn't offer old versions of meson for whatever reason.

[mgautierfr]

If this is for packaging kiwix-lib on debian jessie, you probably cannot install meson during your package build step.
(I suppose. What I'm sure about is that is not possible with Fedora packaging guideline. Debian guideline may differ)

[orblivion]

Oh, you're trying to do this all with normal debian jesse dependencies? Don't you already need a newer of version of libzim and libpugixml?

Also, why would you package for debian jesse? Is Debian accepting new packages into Jesse? Or you're going to make a new apt repo?

[orblivion]

Oh, I apologize. You're guessing that I'm packaging for Jesse. Not exactly. I'm packaging for sandstorm.io, which is (currently) based on Debian Jesse. I just have to start with Jesse, but I can add whatever I want. Eventually the result gets packaged into an "app" which contains all necessary dependencies (kind of like Snap). The result runs in a secure isolated container (also kind of like Snap, as I understand).

However, I want to be "proper" about this. And my understanding of "proper" is that I use "official" sources, like Debian packages, pypi, etc. And then I can use stuff I build from source. But I'd like to avoid downloading binaries that project owners have built.

(In the mean time, to make the package work within Sandstorm's chrome, I've made version of the package that just downloads Kiwix binaries; I can show you if you're curious).

[mgautierfr]

Haaa, that's ok then, you can of course install meson in a virtual env.

I don't know how the packaging for standstorm.io is working but if you have script to build/setup kiwix-* it is maybe better to add them to kiwix-build repository. This way, all scripts related to kiwix* compiling and its dependencies are at the same place. Do not hesitate to make a PR to add you script if you want to.

(Even if we don't add your script/package definition in kiwix-build, it is interesting to share this kind of script, at least to know how do you use kiwix stuff. So, yes I'm curious, please share a link :) )

[orblivion]

This is all basically to make sure that my package is, "in fact, open source". So, adding the build script to your package wouldn't be a problem from that perspective so long as it actually pulled source (as opposed to binaries, like I saw a previous version of Kiwix did). But, I think it would be mostly useful for the very specific cases of Debian Jesse, or Sandstorm. On other OSes/platforms I would guess you would want to use dependencies from whatever package manager is available.

But actually, sandstorm.io does encourage people who package others' apps to just merge the sandstorm-specific stuff into the upstream package. If you're already eager to get my change into your codebase, that would be awesome! But I bet that's a maintenance overhead, and/or it should have integration testing or something.

Here's my project (the branch where I'm working on building from source instead of binary):

https://github.com/orblivion/KiwixSandstorm/tree/kiwix-serve-from-source

If you look in .sandstorm/setup.sh and .sandstorm/build.sh you'll see the two parts of the script. Sandstorm likes to separate certain concerns between a couple different script files. (I'm not even sure I'm doing it right).

Here's what it looks like when you just start it up:

https://oasis.sandstorm.io/shared/tycxh0weZJ4E-UeMoCQZMmYR0p8zI-S3U9UzrCpQEAP

It's an ugly temporary thing and only works up to 10 megs. But it was enough to test. Here's how it looks after you upload a small-ish zim file:

https://oasis.sandstorm.io/shared/ftZlwsIgfstNopPiH3GjJ6IgSHGZyYs8QVD5TDjxURc

It would of course be better to just have the library feature but from last I checked it wasn't available for kiwix-srv. Anyway, it would be good to host any file.

[kelson42]

@orblivion Thank you very much for all your feedbacks and your packaging work on Sandstorm. My opinion is that we should not host package related files to the kiwix-lib repo. You have written it, this creates a new challenge of maintenance. If other people have arguments I'm really interested to listen to them, but for now we really try to re-organize thing in a as simple and as efficient way as possible. Maybe we should talk again about that in one year, after everything got stabilize a bit. If you still thing this is important to keep that topic open, please open a separate ticket as we have drift a bit from the original topic.

[orblivion]

Thanks @kelson42. I agree that it would be a maintenance overhead. I can keep it as a separate package (as many sandstorm packages do, I think). I actually opened this Issue initially just to report that meson should have a minimum version listed in the README, which looks like has been done now.

Though, what may be more important than Sandstorm are instructions that work for anybody who feels like building this on top of Debian Jesse for whatever reason. Would you like to be kept appraised of my progress there (in a new Issue ticket), or do you think it's not a very large target audience?

[mgautierfr]

@orblivion I think that your scripts have a place in kiwix-build repository.

So, adding the build script to your package wouldn't be a problem from that perspective so long as it actually pulled source (as opposed to binaries, like I saw a previous version of Kiwix did)

As you, I'm partisan to let our script build from source instead of download compiled version. (Or use a standard compiled version as a distribution package).

I'm for a reduction of duplication and effort and so, I think that we should put all this kind of custom build scripts in the same place and try to reduce their numbers.

For example, if we improve a bit kiwix-build.py (use system packages when available, ..), we could use it in your setup.sh and build.sh instead of your custom bash command.
Those improvements are wanted, so no reasons to refuse them.

Please open issues/PR on kiwix-build repository and we will discuss further there.
(You can also ping me on IRC, I'm starmad there)