AlpineLinux apk Package

[rvalle]

Hi!

Lots of Docker images are based on Alpine Linux, because is extremely lightweight.

Many of those images come with spaghetti code to turn docker variables into configuration files.

That is why I searched for Handlebars in C, and I found this project.

I think it would be great to make an apk package. is there support planned?

I offer help.

I will also check to integrate ansible->docker->handlebars.c->happy microservice.

I think this project could work very well for that use case.

[jbboehr]

@rvalle I don't use Alpine myself, but if you need anything done in the library or have any questions, I will help.

IIRC, the only hard runtime dependency is talloc, although you'll probably also want json-c.

This was designed mainly as a library for implementing bindings in other languages, although it is possible to execute templates via the binary.

[rvalle]

I was trying yesterday, first to make it build in AlpineLinux....

talloc does not seems to be a problem, or it seems found by the configure script.

The configure script is failing with:

configure: error: check >= 0.9.12 required

I believe this is used for testing...

I have installed check-0.11.0-r0, but still fails. I am unfamiliar with automake/autoconf.

After it builds, I will try to build the package.

Do you use vagrant? I can try to generate the package inside a box, so you can also try it out.

This was designed mainly as a library for implementing bindings in other languages, although it is possible to execute templates via the binary.

I totally understand your original intent. However, I think this use case is very powerful.

Docker allows you to run your service with minimal OS dependencies.

AlpineLinux is been a fantastic choice to build images because its "minimal root filesystem" is just 1.9 Mbytes to download.

Ruby, Nodejs, Python, etc, are really heavyweight in this world.

Your handlebars implementation seems absolutely perfect to me as it is :)

[jbboehr]

pkgconfig does need to be installed to detect check.

I believe I may remove the version check for check.

[rvalle]

pkgconfig does need to be installed to detect check.

I think it is installed.

you think it might be misinterpreting the version num?

[rvalle]

Ok, I have commented out the check check and the ./configure phase completes now ok.

[rvalle]

I had picked wrong check packages, there is a check-dev, that was required.

Now all compilation process works just fine.

I have only 1 test failing and 1 with error.

Will investigate.

[jbboehr]

@rvalle I tried compiling it in vagrant on alpine 3.6 but the tests were locking up for me. Do you have a working build process?

[rvalle]

@jbboehr try with this Dockerfile

FROM privazio/alpine-x86_64

RUN apk update \
    && apk add alpine-sdk automake autoconf libtool talloc-dev json-c-dev bison flex yaml-dev lmdb-dev \
    && apk add pcre-dev check-dev \
    && cd /srv \
    && git clone  https://github.com/jbboehr/handlebars.c.git --recursive \
    && cd /srv/handlebars.c \
    && ./bootstrap \
    && ./configure \
    && make \
    && make test

[jbboehr]

@rvalle tests pass with CFLAGS='-O0' ./configure but not CFLAGS='-O1' ./configure. I'm guessing it's an issue with my code and the newer version of GCC - I've only tested on GCC 5 up until now.

[jbboehr]

@rvalle 39e3775 should fix the failing unit test

[rvalle]

@jbboehr thanks!

I am now reviewing the packaging system for Alpine Linux.

Seems to rely on a source tarball.

[jbboehr]

@rvalle I assume the structure would be similar to this: https://github.com/alpinelinux/aports/pull/1382/files

I will tag a new release with the changes I just made.

[rvalle]

@jbboehr yes. That pretty much solves the questions I had.

Will test the package building today.

[rvalle]

@jbboehr I can see now where to get the source tarball from, and also how to deal with submodules.

I notice that submodules have more than one level of recursion, but I guess only the first level is required to build. I did a quick test, and it seems to build check out only the first level.

If that is the case, the structure of the pkgbuild would be the same.

I am reviewing the Alpine naming conventions and your work on the APT repository.

I think the build should generate 2 packages:

handlebarsc with the command line utility, matching the same of the binary.
handlebars-dev with the library for C bindings.

[jbboehr]

@rvalle Yes, you should only need the two submodules of the handlebars.c repository.

I'm not sure how it is in alpine, but, typically, in debian there would be three packages (names are examples):

• libhandlebars0 would contain the .so file
• libhandlebars-dev would contain the headers and .a file
• handlebarsc would contain the executable, with a dependency on libhandlebars0

AFAICT the PR I linked contains only a single library package that contains all installable files for libmustache

So, I'm not sure what would be appropriate here.

[rvalle]

@jbboehr yes, you are absolutely right.

I didn't notice the dynamic lib, thought handlebarsc was statically linked. Will look into it.

I saw some examples that generate more than one package. Will try today.

[rvalle]

@jbboehr got some progress with the package build process. Getting the sources, building and testing is now working, I am looking into the multi-package thing.

You can test with this dockerfile:

FROM privazio/alpine-x86_64

RUN apk update \
    && apk add alpine-sdk \
    && adduser -D privazio \
    && echo "privazio	ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers 

USER privazio

RUN cd $HOME \
    && git clone https://github.com/rvalle/aports.git \
    && sudo addgroup privazio abuild \
    && sudo mkdir -p /var/cache/distfiles \
    && sudo chgrp abuild /var/cache/distfiles \
    && sudo chmod g+w /var/cache/distfiles 

RUN abuild-keygen -a -i \
    && cd $HOME/aports \ 
    && git pull \
    && cd $HOME/aports/testing/handlebars \
    && abuild -r

[rvalle]

@jbboehr with regards to naming:

libhandlebars0 would contain the .so file

is the "0" because if the major release? I guess you are foreseeing a major 1.x that might be non-backward compatible.

[rvalle]

@jbboehr , with regars to naming:

This was designed mainly as a library for implementing bindings in other languages, although it is possible to execute templates via the binary.

Also in line with what I see in many other Alpine Linux packages, looks like the most appropriate naming would be:

• handlebars: main package, with the dynamic libs
• handlebars-dev: development files, with headers and static lib.
• handlebars-utils: the binary utility.

many other packages have drop the "lib" prefix. for example most dependencies: talloc, yaml, json-c, etc.

there are some packages that use major versioning on the name, thou they tend to start by 1.

It seems to be building the subpackages... lets see if they work.

[rvalle]

@jbboehr I think we have a candidate.

apkbuild seem to do a lot of the work by itself. Like splitting -dev subpackage, detecting dependencies, etc.

you can test with the dockerfile above.

I am also trying to test the package release on an arm target (raspberry pi 3).

You can use the same dockerfiles just change:

FROM privazio/alpine-armhf

test-suite-armhf-log.txt

[jbboehr]

@rvalle Sweet.

The 0 is standard for ubuntu packages, it appears to not be necessary for alpine.

Does it not require an ARM device?

The ARM test failure is due to my hash table implementation's text fixture using a 64bit integer, I believe.

[rvalle]

@jbboehr yes, you require a arm device, I tested on a docker on raspbian, on raspberry pi 3.

If you don't have a testing environment I can facilitate, just send me an SSH pk.

But you might not need, it also fails on x86.

You can reproduce with

FROM privazio/alpine-x86

Since x86_64 kernels can also run i386 binaries, I made a quick test and it is failing in the same way. I think.

[jbboehr]

@rvalle I've added an i386 build to Travis in #60 and fixed the failing unit test in #58.

[rvalle]

@jbboehr just make a quick pull/build/test on arm and your patch fixes also this target.

[rvalle]

@jbboehr I have tested produced packages on: x86, x86_64 and armhf and all seem to work.

I guess next steps are:

• release/tag
• make pull request to alpine/aports

Please review the APKBUILD, check if all is OK.
You can be the contributor and maintainer of the package, or if you want, I can maintain the package.

[jbboehr]

@rvalle I've just tagged v0.6.2 with the test changes

$ readelf -d /usr/lib/libhandlebars.so.5.0.1

Dynamic section at offset 0x5a3e0 contains 23 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libyaml-0.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libjson-c.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libtalloc.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libc.musl-x86_64.so.1]
 0x000000000000000e (SONAME)             Library soname: [libhandlebars.so.5]

Is lmdb compiled in statically? It's not required for your use case or for php-handlebars (php-handlebars uses only an anon shared mmap segment), so it would be ok to exclude it, but the tests and tooling might not be set up to handle it not being available at build time. Run time will of course work as long as a library doesn't use the lmdb cache backend. If it's compiled statically, I think it's ok to include it, or it may need the lmdb package.

Incidentally, I don't recommend the lmdb cache backend: readjusting all the pointers after deserialization eats into the performance saved, so I don't think it's worth the extra trouble. I've been using the mmap backend in production for a couple years, so I think it's pretty stable, but it is only usable in forking scenarios like php-handlebars with php-fpm.

I've been using handlebars.c with ./configure --disable-refcounting flag as there are some edge case incompatibilities with php-handlebars, so it might be better to use that flag. Since the runtime of handlebars templates are typically bounded, refcounting to prevent excess memory usage shouldn't generally be necessary anyway, and it's faster without it.

Otherwise it looks good to me.

[rvalle]

Is lmdb compiled in statically?

I see, most of the footprint is this library.

# du -h  /usr/lib/liblmdb.a ./libhandlebars.so.5.0.1 
408.0K	/usr/lib/liblmdb.a
648.0K	./libhandlebars.so.5.0.1

Yes, it is being included as it is present and detected during configuration.

However, if I remove it, some tests fail:

Throw with invalid jmp_buf: lmdb not available
not ok 4 - test_cache.c:LMDB Cache (GC):test_lmdb_cache_gc: Received signal 6 (Aborted)
FAIL: test_cache 4 - test_cache.c:LMDB Cache (GC):test_lmdb_cache_gc: Received signal 6 (Aborted)
not ok 5 - test_cache.c:LMDB Cache (Reset):test_lmdb_cache_reset: Received signal 6 (Aborted)
FAIL: test_cache 5 - test_cache.c:LMDB Cache (Reset):test_lmdb_cache_reset: Received signal 6 (Aborted)

I think we should remove the library, as Alpine Linux is so focused on reduced footprint.

But we need that those tests are ignored rather than failing.

there must be a define somewhere.... I guess...

[jbboehr]

Another option, which may not be ideal is, if it's possible to link lmdb dynamically in alpine, is to not mark lmdb as a runtime dependency. This would reduce the size of the library, but you would get an error if you tried to use the lmdb backend and the library was not installed.

[jbboehr]

Actually, I don't believe this would work without going through dlopen, and I don't particular want to mess with that.

[jbboehr]

In any case, I don't think there's any good reason to use the lmdb backend anyway, so let's remove (the tests for) it.

[jbboehr]

@rvalle Can you think of anything else before I tag another release?

[rvalle]

No, I think that is all...

I tested your patch with all 3 archs, and seems fine.

Please tag, and we pull request them.

[jbboehr]

@rvalle I just tagged v0.6.3

[rvalle]

@jbboehr currently git handlebars spec submodule is linked by commit hash.

@ jirutka suggests using semantic version tags instead, in order to follow Alpines package build best practice.

see alpinelinux/aports#1628

coud you? tag handlebars_spec submodule, and release a 0.6.4 with that reference?

[jbboehr]

@rvalle I believe he just wants it to use a git tag. You can use this one for that purpose: https://github.com/jbboehr/handlebars-spec/releases/tag/v4.0.5-p1

The commit linked in this repository should be functionally identical, I had just updated the submodule in my feature branches but didn't update the commit after merging them into master. I will do that today and double-check that it works.

[rvalle]

I have now tested the package and it is distributed via the edge/testing package.