AppStream 1.0 #47

probonopd · 2024-03-15T07:05:17Z

This PR shall build AppStream v1.0.x instead of what we have been using so far.

Note to self: If this doesn't work, a possibly less painful alternative might be https://github.com/probonopd/appstreamlint

probonopd · 2024-03-15T07:11:33Z

It uses meson as a build time tool. Every time meson is used, building the latest version means that meson probably needs to be updated as well (which, in practical terms, means that the whole Linux distribution needs to be updated). So annoying. I don't like moving targets.

Newst appstream needs newest meson, argh

probonopd · 2024-03-15T07:39:29Z

Newer versions of things always seem to require more dependencies. Argh!

Even with curl we get errors:

Run-time dependency libcurl found: YES 8.5.0
WARNING: Static library 'xml2' not found for dependency 'libxml-2.0', may not be statically linked
WARNING: Static library 'lzma' not found for dependency 'libxml-2.0', may not be statically linked
Run-time dependency libxml-2.0 found: YES 2.11.7
Run-time dependency yaml-0.1 found: YES 0.2.5
Did not find CMake 'cmake'
Found CMake: NO
Run-time dependency xmlb found: NO (tried pkgconfig and cmake)
Looking for a fallback subproject for the dependency xmlb
Building fallback subproject with default_library=static

Executing subproject libxmlb 

libxmlb| Project name: libxmlb
libxmlb| Project version: 0.3.16
libxmlb| C compiler for the host machine: cc (gcc 13.2.1 "cc (Alpine 13.2.1_git20231014) 13.2.1 20231014")
libxmlb| C linker for the host machine: cc ld.bfd 2.41
libxmlb| Compiler for C supports arguments -Wno-nonnull-compare: YES
libxmlb| Compiler for C supports arguments -Wno-aggregate-return: YES
libxmlb| Compiler for C supports arguments -Wunused: YES
libxmlb| Compiler for C supports arguments -Warray-bounds: YES
libxmlb| Compiler for C supports arguments -Wcast-align: YES
libxmlb| Compiler for C supports arguments -Wclobbered: YES
libxmlb| Compiler for C supports arguments -Wdeclaration-after-statement: YES
libxmlb| Compiler for C supports arguments -Wduplicated-branches: YES
libxmlb| Compiler for C supports arguments -Wduplicated-cond: YES
libxmlb| Compiler for C supports arguments -Wempty-body: YES
libxmlb| Program python3 found: YES (/usr/bin/python3)
libxmlb| subprojects/libxmlb/data/meson.build:2: WARNING: Project targets '>=0.47.0' but uses feature introduced in '0.50.0': install arg in configure_file.
libxmlb| Configuring libxmlb.test using configuration
libxmlb| Configuring xb-version.h using configuration
libxmlb| Compiler for C supports link arguments -Wl,--version-script,/appstream-1.0.2/subprojects/libxmlb/src/libxmlb.map: YES
libxmlb| subprojects/libxmlb/src/meson.build:121: WARNING: Project targets '>=0.47.0' but uses feature introduced in '0.50.0': install arg in configure_file.
libxmlb| Configuring xb-tool.1 using configuration
libxmlb| subprojects/libxmlb/src/meson.build:230: WARNING: Project targets '>=0.47.0' but uses feature introduced in '0.56.0': meson.project_source_root.
libxmlb| subprojects/libxmlb/src/meson.build:265: WARNING: Project targets '>=0.47.0' but uses feature introduced in '0.56.0': meson.project_source_root.
libxmlb| Build targets in project: 4
libxmlb| WARNING: Project specifies a minimum meson_version '>=0.47.0' but uses features which were added in newer versions:
libxmlb| * 0.50.0: {'install arg in configure_file'}
libxmlb| * 0.56.0: {'meson.project_source_root'}
libxmlb| NOTICE: Future-deprecated features used:
libxmlb| * 0.64.0: {'copy arg in configure_file'}
libxmlb| Subproject libxmlb finished.

Dependency xmlb from subproject subprojects/libxmlb found: YES 0.3.16

meson.build:169:47: ERROR: Function does not take keyword arguments.

A full log can be found at /appstream-1.0.2/build/meson-logs/meson-log.txt
Error: Process completed with exit code 1.

Meson is never new enough. Reason enough to avoid Meson altogether.
Developers always seem to assume that people building their software run the latest and greatest distributions. A never ending update cycle, a.k.a. "moving target".

probonopd · 2024-03-15T07:41:25Z

@ximion is there a way to build appstream 1.0 without curl?

Right now we are using a static build of a pre-1.0 version, and would like to go to 1.0 in the hope that the AppStream format is no longer subject to change.

ximion · 2024-03-15T10:11:30Z

is there a way to build appstream 1.0 without curl?

No, it is required for the portions of the spec that require network access.
The format will of course still change, what happened with 1.0 was the removal of deprecated features. Removal of deprecated stuff will (very likely) never happen again.

probonopd · 2024-03-25T15:06:42Z

Thanks @ximion. Do you happen to know how to produce a static build of appstreamcli on Alpine? We are using an older version of it, built I am so far not succeeding in getting 1.x built statically, since it seems to pull in additional dependencies which the earlier version did not need.

ximion · 2024-03-25T16:00:06Z

What kind of issue do you have? AppStream 1.0 doesn't have dependencies that 0.16.4 didn't also have, I think.

probonopd · 2024-03-25T16:14:28Z

Coming from 0.12.9. Currently running into meson.build:169:47: ERROR: Function does not take keyword arguments, whatever that might mean.

https://github.com/probonopd/static-tools/actions/runs/8422233832/job/23061071735#step:4:1516

probonopd · 2024-03-25T16:33:21Z

All I need is the appstreamcli command to check the validity of ("lint") AppStream MetaInfo files. However, the AppStream repo now includes much more functionality that draws in many more dependencies, e.g.,:

libxmlb is now used for caching things. I think for appstreamcli to check files, we could do without a cache ximion/appstream@a8186ed
libzstd is now used to compress (which?) things. I think for appstreamcli to check files, we could do without compression ximion/appstream@1bde5bf

Is there a way to just compile the appstreamcli command with the minimal dependencies needed to validate ("lint") files?

ximion · 2024-03-25T18:16:08Z

Coming from 0.12.9. Currently running into meson.build:169:47: ERROR: Function does not take keyword arguments, whatever that might mean.

That error indeed makes no sense, the function on that line isn't passed any kwargs. Can you easily upgrade to Meson 1.3.2, just to check if that helps?

probonopd · 2024-03-25T18:42:00Z

I don't think I can (easily) update Meson any further, as I have already updated to the latest version of Alpine as part of this PR.

ximion · 2024-03-25T19:25:42Z

I was asking because this smells a bit like a Meson bug. Do you apply any patches to AppStream? What is in the line that it complains about in your copy? Because the upstream line does not make any sense here.

probonopd · 2024-03-25T19:27:34Z

Yes, I was indeed wondering the same thing. No, I am not applying any patches, except for some ad-hoc meson.build file patches (I don't have much clue about Meson, really). My code is here:

static-tools/build.sh

Lines 90 to 115 in 0aea9a2

    
           # Build appstreamcli 
        
           trap 'cat /appstream-1.0.2/build/meson-logs/meson-log.txt' EXIT 
        
           apk add glib-static meson libxml2-dev yaml-dev yaml-static gperf curl-dev curl-static curl libxmlb-dev 
        
           # Compile liblmdb from source as Alpine only ship it as a .so 
        
           wget https://git.openldap.org/openldap/openldap/-/archive/LMDB_0.9.29/openldap-LMDB_0.9.29.tar.gz 
        
           tar xf openldap-LMDB_*.tar.gz 
        
           cd openldap-LMDB_*/libraries/liblmdb 
        
           make liblmdb.a 
        
           install -D -m 644 liblmdb.a /usr/local/lib/liblmdb.a 
        
           install -D -m 644 lmdb.h /usr/local/include/lmdb.h 
        
           cd - 
        
           wget -O appstream.tar.gz https://github.com/ximion/appstream/archive/v1.0.2.tar.gz # Keep at v1.0.x so as to not have a moving target 
        
           tar xf appstream.tar.gz 
        
           cd appstream-*/ 
        
           # Ask for static dependencies 
        
           sed -i -E -e "s|(dependency\('.*')|\1, static: true|g" meson.build 
        
           # Disable po, docs and tests 
        
           sed -i -e "s|subdir('po/')||" meson.build 
        
           sed -i -e "s|subdir('docs/')||" meson.build 
        
           sed -i -e "s|subdir('tests/')||" meson.build 
        
           # -no-pie is required to statically link to libc 
        
           CFLAGS=-no-pie LDFLAGS=-static meson setup build --buildtype=release --default-library=static --prefix="$(pwd)/prefix" --strip -Db_lto=true -Db_ndebug=if-release -Dstemming=false -Dgir=false -Dapidocs=false 
        
           # Install in a staging enviroment 
        
           meson install -C build 
        
           file prefix/bin/appstreamcli 
        
           cd -

ximion · 2024-03-25T19:40:59Z

Can you please drop the

sed -i -e "s|subdir('po/')||" meson.build
sed -i -e "s|subdir('docs/')||" meson.build
sed -i -e "s|subdir('tests/')||" meson.build

lines? That is no way to disable anything, there is a lot of interdependencies here which can cause all kinds of issues. Dropping po/ is probably harmless, but the rest isn't! Use -Dapidocs=false -Dinstall-docs=false for the same effect instead (and just remove translations after installation to a known location).

You can also drop the entire LMDB block, unless something else needs it - because AppStream does not require it anymore for a long time.

Besides all of that, sed -i -E -e "s|(dependency\('.*')|\1, static: true|g" meson.build is indeed the line that wrecks this build, it creates an invalid definition. Instead of that hack, just pass -Ddefault_library=static to Meson and you should be set.

probonopd · 2024-03-25T20:28:44Z

Thanks for your help @ximion.
Now it also needs systemd, which doesn't even exist on Alpine Linux. Argh!
I just want to check some files...

The original command asks for a password interactively, which does not work on CI

ximion · 2024-03-25T20:58:29Z

Now it also needs systemd, which doesn't even exist on Alpine Linux. Argh!
I just want to check some files...

It's not essential if you just need validation and don't need hardware information. Just pass -Dsystemd=false (device capability checking will not work then, but you won't need that for validation).

ximion · 2024-03-25T21:18:49Z

libxmlb has -Dintrospection=false -Dgtkdoc=false -Dcli=false switches that you may like ;-)

probonopd · 2024-03-25T21:19:37Z

Just was gonna elaborate... erm, complain about, that we are now firmly in Gnome land, with stuff like gobject-introspection-dev...

In an effort to fix `/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: attempted static link of dynamic object '/usr/lib/libglib-2.0.so'`

probonopd · 2024-03-25T21:34:55Z

Almost there...

[141/146] Linking target tests/as-test_xml
ninja: job failed: cc  -o tests/as-test_yaml tests/as-test_yaml.p/meson-generated_.._.._src_as-enum-types.c.o tests/as-test_yaml.p/test-yamldata.c.o tests/as-test_yaml.p/as-test-utils.c.o -flto -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -static -no-pie -Wl,--start-group src/libappstream.a /usr/lib/libglib-2.0.so /usr/lib/libintl.so /usr/lib/libgobject-2.0.so /usr/lib/libgio-2.0.so /usr/lib/libxml2.so /usr/lib/libcurl.so /usr/local/lib/libxmlb.a /usr/lib/liblzma.so /usr/lib/libzstd.so /usr/lib/libyaml.so -Wl,--end-group
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: attempted static link of dynamic object `/usr/lib/libglib-2.0.so'
collect2: error: ld returned 1 exit status
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: attempted static link of dynamic object `/usr/lib/libglib-2.0.so'
collect2: error: ld returned 1 exit status
ninja: job failed: cc  -o tests/as-test_xml tests/as-test_xml.p/meson-generated_.._.._src_as-enum-types.c.o tests/as-test_xml.p/test-xmldata.c.o tests/as-test_xml.p/as-test-utils.c.o -flto -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -static -no-pie -Wl,--start-group src/libappstream.a /usr/lib/libglib-2.0.so /usr/lib/libintl.so /usr/lib/libgobject-2.0.so /usr/lib/libgio-2.0.so /usr/lib/libxml2.so /usr/lib/libcurl.so /usr/local/lib/libxmlb.a /usr/lib/liblzma.so /usr/lib/libzstd.so /usr/lib/libyaml.so -Wl,--end-group
ninja: job failed: cc  -o tests/as-test_validate tests/as-test_validate.p/meson-generated_.._.._src_as-enum-types.c.o tests/as-test_validate.p/test-validate.c.o tests/as-test_validate.p/as-test-utils.c.o -flto -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -static -no-pie -Wl,--start-group src/libappstream.a /usr/lib/libglib-2.0.so /usr/lib/libintl.so /usr/lib/libgobject-2.0.so /usr/lib/libgio-2.0.so /usr/lib/libxml2.so /usr/lib/libcurl.so /usr/local/lib/libxmlb.a /usr/lib/liblzma.so /usr/lib/libzstd.so /usr/lib/libyaml.so -Wl,--end-group
ninja: job failed: cc  -o tests/as-test_misc tests/as-test_misc.p/meson-generated_.._.._src_as-enum-types.c.o tests/as-test_misc.p/test-misc.c.o tests/as-test_misc.p/as-test-utils.c.o -flto -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -static -no-pie -Wl,--start-group src/libappstream.a /usr/lib/libglib-2.0.so /usr/lib/libintl.so /usr/lib/libgobject-2.0.so /usr/lib/libgio-2.0.so /usr/lib/libxml2.so /usr/lib/libcurl.so /usr/local/lib/libxmlb.a /usr/lib/liblzma.so /usr/lib/libzstd.so /usr/lib/libyaml.so -Wl,--end-group
ninja: job failed: cc  -o tests/as-test_perf tests/as-test_perf.p/meson-generated_.._.._src_as-enum-types.c.o tests/as-test_perf.p/test-performance.c.o tests/as-test_perf.p/as-test-utils.c.o -flto -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -static -no-pie -Wl,--start-group src/libappstream.a /usr/lib/libglib-2.0.so /usr/lib/libintl.so /usr/lib/libgobject-2.0.so /usr/lib/libgio-2.0.so /usr/lib/libxml2.so /usr/lib/libcurl.so /usr/local/lib/libxmlb.a /usr/lib/liblzma.so /usr/lib/libzstd.so /usr/lib/libyaml.so -Wl,--end-group
ninja: job failed: cc  -o tests/as-test_pool tests/as-test_pool.p/meson-generated_.._.._src_as-enum-types.c.o tests/as-test_pool.p/test-pool.c.o tests/as-test_pool.p/as-test-utils.c.o -flto -Wl,--as-needed -Wl,--no-undefined -Wl,-O1 -static -no-pie -Wl,--start-group src/libappstream.a /usr/lib/libglib-2.0.so /usr/lib/libintl.so /usr/lib/libgobject-2.0.so /usr/lib/libgio-2.0.so /usr/lib/libxml2.so /usr/lib/libcurl.so /usr/local/lib/libxmlb.a /usr/lib/liblzma.so /usr/lib/libzstd.so /usr/lib/libyaml.so -Wl,--end-group
ninja: subcommands failed
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: as-validator.c.o (symbol from plugin): in function `as_validator_issue_tag_list':
(.text+0x0): multiple definition of `as_validator_issue_tag_list'; tests/as-test_validate.p/test-validate.c.o (symbol from plugin):(.text+0x0): first defined here
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: attempted static link of dynamic object `/usr/lib/libglib-2.0.so'
collect2: error: ld returned 1 exit status
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: attempted static link of dynamic object `/usr/lib/libglib-2.0.so'
collect2: error: ld returned 1 exit status
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: attempted static link of dynamic object `/usr/lib/libglib-2.0.so'
collect2: error: ld returned 1 exit status
/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: attempted static link of dynamic object `/usr/lib/libglib-2.0.so'
collect2: error: ld returned 1 exit status
Could not rebuild /appstream-1.0.2/build
Error: Process completed with exit code 255.

What is it this time?
Why is it trying to link .so files instead of .a?

This whole exercise is a great reminder why I like Golang so much. Writing https://github.com/probonopd/appstreamlint from scratch was a breeze compared to getting appstreamcli 1.0 built as a static binary.

ximion · 2024-03-25T22:03:27Z

If you would just use LD_LIBRARY_PATH, you could avoid the static linking annoyance completely ;-)

probonopd added 2 commits March 15, 2024 07:02

Do not hardcode appstream version

ba9bcba

v1.0.2

2dffc0e

probonopd added 4 commits March 15, 2024 07:14

Update alpine

26cd9ce

Newst appstream needs newest meson, argh

appstream now needs curl?

f8d300e

curl-dev

08ce225

--without-curl

3440f5c

probonopd added 2 commits March 25, 2024 15:59

curl-dev curl-static curl

00f737a

libxmlb-dev

5437ab6

trap 'cat ...meson-log.txt' EXIT

0aea9a2

probonopd added 3 commits March 25, 2024 20:54

Don't edit meson.build, remove liblmdb

0e56150

Remove --default-library in favor of -Ddefault_library

a181fd9

Build alpine-systemd, argh!

2387680

probonopd mentioned this pull request Mar 25, 2024

Please add libsystemd-static Artox/alpine-systemd#2

Open

useradd -m -g abuild abuild

ebbc121

The original command asks for a password interactively, which does not work on CI

probonopd mentioned this pull request Mar 25, 2024

Using the commands from the README fails on GitHub Actions Artox/alpine-systemd#3

Open

Do not use abuild user

c635bfb

abuild-keygen -a -i -n

f556cbd

probonopd added 3 commits March 25, 2024 21:59

-Dsystemd=false

698b0ef

itstool is also needed

c860f3d

libxmlb-static is missing, need to build our own

faa9c3b

probonopd added 2 commits March 25, 2024 22:20

-Dintrospection=false -Dgtkdoc=false -Dcli=false

e393f68

# ldconfig # segfaults

ca2eba1

probonopd mentioned this pull request Mar 25, 2024

Mention build arguments to decrease dependencies hughsie/libxmlb#157

Closed

probonopd added 2 commits March 25, 2024 22:29

Remove --Ddefault-library in favor of -default_library

58358b5

In an effort to fix `/usr/lib/gcc/x86_64-alpine-linux-musl/13.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: attempted static link of dynamic object '/usr/lib/libglib-2.0.so'`

Fix typo in -default-library=static

e9a78a2

probonopd added 2 commits March 25, 2024 22:36

--default-library=static with 2 dashes

ad64a71

libxml2-static

3432b93

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AppStream 1.0 #47

AppStream 1.0 #47

probonopd commented Mar 15, 2024 •

edited

Loading

probonopd commented Mar 15, 2024

probonopd commented Mar 15, 2024 •

edited

Loading

probonopd commented Mar 15, 2024 •

edited

Loading

ximion commented Mar 15, 2024

probonopd commented Mar 25, 2024 •

edited

Loading

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024

probonopd commented Mar 25, 2024 •

edited

Loading

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024 •

edited

Loading

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024 •

edited

Loading

ximion commented Mar 25, 2024

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024

probonopd commented Mar 25, 2024 •

edited

Loading

ximion commented Mar 25, 2024

AppStream 1.0 #47

Are you sure you want to change the base?

AppStream 1.0 #47

Conversation

probonopd commented Mar 15, 2024 • edited Loading

probonopd commented Mar 15, 2024

probonopd commented Mar 15, 2024 • edited Loading

probonopd commented Mar 15, 2024 • edited Loading

ximion commented Mar 15, 2024

probonopd commented Mar 25, 2024 • edited Loading

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024

probonopd commented Mar 25, 2024 • edited Loading

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024 • edited Loading

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024 • edited Loading

ximion commented Mar 25, 2024

ximion commented Mar 25, 2024

probonopd commented Mar 25, 2024

probonopd commented Mar 25, 2024 • edited Loading

ximion commented Mar 25, 2024

probonopd commented Mar 15, 2024 •

edited

Loading

probonopd commented Mar 15, 2024 •

edited

Loading

probonopd commented Mar 15, 2024 •

edited

Loading

probonopd commented Mar 25, 2024 •

edited

Loading

probonopd commented Mar 25, 2024 •

edited

Loading

probonopd commented Mar 25, 2024 •

edited

Loading

probonopd commented Mar 25, 2024 •

edited

Loading

probonopd commented Mar 25, 2024 •

edited

Loading