Critical errors on running appstreamcli refresh-cache with merged components

[mariospr]

At the moment I have a XML file in /var/cache/app-info/xmls/org.gnome.Software-eos-extra.xml.gz that I'm using from GNOME Software to extend the distro-wide collection XML so that we can customize a bit how we show certain apps in our modified version of GNOME Software: add thumbnails, relocate some apps in other categories...

However, while GNOME Software works well with what we have without using the merge="append" tag attribute described in https://www.freedesktop.org/software/appstream/docs/chap-CollectionData.html#sect-AppStream-XML, we started getting errors in our developer machines when running apt-get update, when the Post-Invoke-Success 50appstream hook installed by the appstream package tries to run appstreamcli refresh-cache.

After a while I figured out that, even if GNOME Software ignores that merge tag, the right thing would probably be to add it to each <component> node there, so that appstreamcli would not complain in our developer machines (production machines do not use APT anyway, just OSTree), so I changed the file to look like this:

<?xml version="1.0" encoding="UTF-8"?>
<components version="0.8" origin="eos-extra-app-info">
  <component type="desktop" merge="append">
    <id>brasero.desktop</id>
    <categories>
      <category>Utility</category>
    </categories>
    <metadata>
      <value key="GnomeSoftware::popular-background">https://d3lapyynmdp1i9.cloudfront.net/thumbnails/brasero/brasero-thumb.jpg</value>
    </metadata>
  </component>
  <component type="desktop" merge="append">
    <id>cheese.desktop</id>
    <categories>
      <category>Education</category>
      <category>Family</category>
    </categories>
    <metadata>
      <value key="GnomeSoftware::popular-background">https://d3lapyynmdp1i9.cloudfront.net/thumbnails/cheese/cheese-thumb.jpg</value>
    </metadata>
  </component>

   [...]
</components>

After doing that, I get the following CRITICAL errors when running appstreamcli refresh-cache:

(appstreamcli:2675): GLib-CRITICAL **: g_variant_builder_end: assertion '!GVSB(builder)->uniform_item_types || GVSB(builder)->prev_item_type != NULL || g_variant_type_is_definite (GVSB(builder)->type)' failed

(appstreamcli:2675): GLib-CRITICAL **: g_variant_new_variant: assertion 'value != NULL' failed

(appstreamcli:2675): GLib-CRITICAL **: g_variant_get_type: assertion 'value != NULL' failed

(appstreamcli:2675): GLib-CRITICAL **: g_variant_type_is_subtype_of: assertion 'g_variant_type_check (type)' failed

(appstreamcli:2675): GLib-CRITICAL **: g_variant_builder_add_value: assertion '!GVSB(builder)->expected_type || g_variant_is_of_type (value, GVSB(builder)->expected_type)' failed

(appstreamcli:2675): GLib-CRITICAL **: g_variant_builder_end: assertion 'GVSB(builder)->offset >= GVSB(builder)->min_items' failed

(appstreamcli:2675): GLib-CRITICAL **: g_variant_get_type: assertion 'value != NULL' failed

(appstreamcli:2675): GLib-CRITICAL **: g_variant_type_is_subtype_of: assertion 'g_variant_type_check (type)' failed

(appstreamcli:2675): GLib-CRITICAL **: g_variant_builder_add_value: assertion '!GVSB(builder)->expected_type || g_variant_is_of_type (value, GVSB(builder)->expected_type)' failed

I've tried this both with appstream-0.10.4 and appstream-0.10.6 drom Debian Stable and Testing and the error persists. After installing debug symbols for appstream (0.10.6) and glib (2.50.2, with some modifications from endless) I could get a backtrace for each of those criticals, which you can see attached here: appstreamcli-backtraces.txt

Just to be sure this was not an Endless-specific thing, I've setup a chroot with Ubuntu Yakkety x86_64, placed the same file inside /var/cache/app-info/xmls and got pretty much the same error right after apt finished installing the appstream package (version 0.10.1):

[...]
Setting up libappstream4:amd64 (0.10.1-1) ...
Setting up appstream (0.10.1-1) ...

(appstreamcli:31667): GLib-CRITICAL **: g_variant_builder_end: assertion '!GVSB(builder)->uniform_item_types || GVSB(builder)->prev_item_type != NULL || g_variant_type_is_definite (GVSB(builder)->type)' failed

(appstreamcli:31667): GLib-CRITICAL **: g_variant_new_variant: assertion 'value != NULL' failed

(appstreamcli:31667): GLib-CRITICAL **: g_variant_get_type: assertion 'value != NULL' failed

(appstreamcli:31667): GLib-CRITICAL **: g_variant_type_is_subtype_of: assertion 'g_variant_type_check (type)' failed

(appstreamcli:31667): GLib-CRITICAL **: g_variant_builder_add_value: assertion '!GVSB(builder)->expected_type || g_variant_is_of_type (value, GVSB(builder)->expected_type)' failed

(appstreamcli:31667): GLib-CRITICAL **: g_variant_builder_end: assertion 'GVSB(builder)->offset >= GVSB(builder)->min_items' failed

(appstreamcli:31667): GLib-CRITICAL **: g_variant_get_type: assertion 'value != NULL' failed

(appstreamcli:31667): GLib-CRITICAL **: g_variant_type_is_subtype_of: assertion 'g_variant_type_check (type)' failed

(appstreamcli:31667): GLib-CRITICAL **: g_variant_builder_add_value: assertion '!GVSB(builder)->expected_type || g_variant_is_of_type (value, GVSB(builder)->expected_type)' failed
AppStream cache update completed successfully.
Processing triggers for libc-bin (2.24-3ubuntu1) ...

This is not a show-stopper for us at all since we don't really rely on appstreamcli on production systems, but this seems wrong so it would be nice to fix it.

PS: Here is the complete file I'm testing this with: org.gnome.Software-eos-extra.xml.gz

[ximion]

Nice! Thanks for the detailed report, I hope I can resolve this rather quickly (will be a good candidate for Debian 9 too).

One thing that's already weird is that merge-type components shouldn't reach GVariant serialization at all, so something odd is going on here.

[mariospr]

Thanks for the quick response. It would be really nice to have it fixed for Debian 9 indeed!

[ximion]

Okay, I can't reproduce this, but I have a theory. Can you please run appstreamcli status on the system showing this issue and attach the output of this command?

[ximion]

Please test if that pretty trivial patch resolves the issue for you :-) (I am fairly confident about that - in this case, you only have these merge components on your system, and zero other valid components, which is likely a bug in your metadata. Still, we should handle this case more gracefully.)

If this patch doesn't resolve the issue, it's some other problem (in that case, please reopen the bug).

[mariospr]

Okay, I can't reproduce this, but I have a theory. Can you please run appstreamcli status on the system showing this issue and attach the output of this command?

$ appstreamcli status
AppStream Status:
Version: 0.10.4

Distribution metadata:
 /usr/share/app-info
  - Empty.

 /var/lib/app-info
  - Empty.

 /var/cache/app-info
  - XML:  1
  - No icons.

Metainfo files:
  - Found 1 components.
  - Found 44 components in legacy paths.

[ximion]

I guess that file in /var/cache/app-info is only the override file? In that case, the patch I wrote should work (the log clearly shows that).

WARNING: Ignored component 'org.gnome.gedit.desktop': The component is invalid. and others look weird, I guess these are the failed merges. I think a better message is useful here, but it's not a real bug, only a small annoyance in the log.

Can you please test if the patch works for you?

[mariospr]

Yes, I can confirm that patch fixed the issue. Thanks!