Descriptions are no longer displayed

[antonio-rojas]

With 0.14:

 $ appstreamcli dump org.kde.krita
<?xml version="1.0" encoding="utf-8"?>
<component type="desktop-application">
  <id>org.kde.krita</id>
  <name>Krita</name>
  <name xml:lang="es">Krita</name>
  <summary>Digital Painting, Creative Freedom</summary>
  <summary xml:lang="es">Pintura digital, libertad creativa</summary>
  <project_license>GPL-3.0-only</project_license>
  <developer_name>Krita Foundation</developer_name>
  <developer_name xml:lang="es">Fundación Krita</developer_name>
  <project_group>KDE</project_group>
  <pkgname>krita</pkgname>
  <launchable type="desktop-id">org.kde.krita.desktop</launchable>
  <icon type="cached" width="48" height="48">krita_krita.png</icon>
  <icon type="cached" width="64" height="64">krita_krita.png</icon>
  <icon type="cached" width="128" height="128">krita_krita.png</icon>
  <icon type="stock">krita</icon>
  <url type="homepage">https://www.krita.org/</url>
  <url type="bugtracker">https://docs.krita.org/en/untranslatable_pages/reporting_bugs.html</url>
  <url type="faq">https://docs.krita.org/KritaFAQ.html</url>
  <url type="help">https://docs.krita.org/</url>
  <url type="donation">https://krita.org/support-us/donations/</url>
  <categories>
    <category>Graphics</category>
    <category>2DGraphics</category>
    <category>RasterGraphics</category>
  </categories>
  <provides>
    <mediatype>application/x-krita</mediatype>
    <mediatype>image/openraster</mediatype>
    <mediatype>application/x-krita-paintoppreset</mediatype>
    <binary>krita</binary>
    <id>org.kde.krita.desktop</id>
  </provides>
  <screenshots>
    <screenshot type="default">
      <caption>Krita is a full-featured digital painting studio</caption>
      <caption xml:lang="es">Krita es un completo estudio de dibujo digital</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_001.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>The startup window now also gives you the latest news about Krita</caption>
      <caption xml:lang="es">La ventana de bienvenida también le proporciona ahora las últimas noticias sobre Krita</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_002.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>There are over ten immensely powerful brush engines</caption>
      <caption xml:lang="es">Existen unos diez inmensamente potentes motores de pinceles</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_003.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>Create and use gamut masks to give your images a coherent feel</caption>
      <caption xml:lang="es">Cree y use gamas para proporcionar a sus imágenes un aspecto coherente</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_004.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>Into animation? Krita provides everything you need for traditional, hand-drawn animation</caption>
      <caption xml:lang="es">¿Trabaja con animación? Krita proporciona todo lo necesario para la animación manual tradicional</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_005.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>If you're new to digital painting, there's an extensive, up-to-date manual</caption>
      <caption xml:lang="es">Si está empezando con el dibujo digital, dispone de un extenso y actualizado manual</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_006.png</image>
    </screenshot>
  </screenshots>
  <releases>
    <release type="stable" version="4.4.1" date="2020-10-28T00:00:00Z"/>
  </releases>
  <content_rating type="oars-1.1"/>
</component>

Notice the missing description.
With 0.13, and using the same metadata:

 $ appstreamcli dump org.kde.krita
<?xml version="1.0" encoding="utf-8"?>
<component type="desktop-application">
  <id>org.kde.krita</id>
  <name>Krita</name>
  <name xml:lang="es">Krita</name>
  <summary>Digital Painting, Creative Freedom</summary>
  <summary xml:lang="es">Pintura digital, libertad creativa</summary>
  <project_license>GPL-3.0-only</project_license>
  <project_group>KDE</project_group>
  <developer_name>Krita Foundation</developer_name>
  <developer_name xml:lang="es">Fundación Krita</developer_name>
  <description>
    <p>Krita is the full-featured digital art studio.</p>
    <p xml:lang="es">Krita es un estudio de arte digital completo</p>
    <p>It is perfect for sketching and painting, and presents an end–to–end solution for creating digital painting files from scratch by masters.</p>
    <p xml:lang="es">Es perfecto para diseñar y pintar, y ofrece una solución completa para crear desde cero archivos de pintura digital apta para profesionales.</p>
    <p>
      Krita is a great choice for creating concept art, comics, textures for rendering and matte paintings. Krita supports many colorspaces like RGB and CMYK
      at 8 and 16 bits integer channels, as well as 16 and 32 bits floating point channels.
    </p>
    <p xml:lang="es">Krita es una gran elección para crear arte conceptual, cómics, texturas para renderizar y «matte paintings». Krita permite el uso de muchos espacios de color, como, por ejemplo, RGB y CMYK, tanto en canales de enteros de 8 y 16 bits, así como en canales de coma flotante de 16 y 32 bits.</p>
    <p>Have fun painting with the advanced brush engines, amazing filters and many handy features that make Krita enormously productive.</p>
    <p xml:lang="es">Diviértase pintando con los avanzados motores de pinceles, los espectaculares filtros y muchas funcionalidades prácticas que hacen que Krita sea enormemente productivo.</p>
  </description>
  <pkgname>krita</pkgname>
  <categories>
    <category>Graphics</category>
    <category>2DGraphics</category>
    <category>RasterGraphics</category>
  </categories>
  <url type="homepage">https://www.krita.org/</url>
  <url type="bugtracker">https://docs.krita.org/en/untranslatable_pages/reporting_bugs.html</url>
  <url type="faq">https://docs.krita.org/KritaFAQ.html</url>
  <url type="help">https://docs.krita.org/</url>
  <url type="donation">https://krita.org/support-us/donations/</url>
  <icon type="cached" width="48" height="48">krita_krita.png</icon>
  <icon type="cached" width="64" height="64">krita_krita.png</icon>
  <icon type="cached" width="128" height="128">krita_krita.png</icon>
  <icon type="stock">krita</icon>
  <launchable type="desktop-id">org.kde.krita.desktop</launchable>
  <screenshots>
    <screenshot type="default">
      <caption>Krita is a full-featured digital painting studio</caption>
      <caption xml:lang="es">Krita es un completo estudio de dibujo digital</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_001.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>The startup window now also gives you the latest news about Krita</caption>
      <caption xml:lang="es">La ventana de bienvenida también le proporciona ahora las últimas noticias sobre Krita</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_002.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>There are over ten immensely powerful brush engines</caption>
      <caption xml:lang="es">Existen unos diez inmensamente potentes motores de pinceles</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_003.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>Create and use gamut masks to give your images a coherent feel</caption>
      <caption xml:lang="es">Cree y use gamas para proporcionar a sus imágenes un aspecto coherente</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_004.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>Into animation? Krita provides everything you need for traditional, hand-drawn animation</caption>
      <caption xml:lang="es">¿Trabaja con animación? Krita proporciona todo lo necesario para la animación manual tradicional</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_005.png</image>
    </screenshot>
    <screenshot type="default">
      <caption>If you're new to digital painting, there's an extensive, up-to-date manual</caption>
      <caption xml:lang="es">Si está empezando con el dibujo digital, dispone de un extenso y actualizado manual</caption>
      <image type="source">https://cdn.kde.org/screenshots/krita/2018-03-17_screenshot_006.png</image>
    </screenshot>
  </screenshots>
  <releases>
    <release type="stable" version="4.4.1" date="2020-10-28T00:00:00Z"/>
  </releases>
  <mimetypes>
    <mimetype>application/x-krita</mimetype>
    <mimetype>image/openraster</mimetype>
    <mimetype>application/x-krita-paintoppreset</mimetype>
  </mimetypes>
  <provides>
    <binary>krita</binary>
    <id>org.kde.krita.desktop</id>
  </provides>
  <content_rating type="oars-1.1"/>
</component>

[antonio-rojas]

Bisected to be026a4

[antonio-rojas]

Note that, as per the 0.13 output, the description is fully translated.

[ximion]

I haven't reproduced this yet, but this looks like a pretty critical issue. The weird thing is that at the very least the untranslated strings should be present, even if translations are dropped because not enough stuff was localized. And in this case there actually is enough translation.
The unittests do cover this case though, so I wonder what went wrong with Krita (I will have a look at it this weekend, if possible).

[antonio-rojas]

Krita was just an example. Descriptions are missing from all packages.

[ximion]

Okay, I just tested that on Debian and couldn't reproduce it - we do have all descriptions. Where is your metadata coming from? Do you get the same issue when reading the XML file directly? (For that, try appstreamcli convert --format=xml /usr/share/metainfo/org.kde.discover.appdata.xml -)

[antonio-rojas]

Reading the xml file outputs the descriptions correctly. The metadata is the distro one, extracted with appstream-generator.

[ximion]

Hmm... Which version of asgen do you use? Are the descriptions present in the source metadata (the ones in /var/{lib|cache]/app-info || /usr/share/app-info)?

[antonio-rojas]

they are in the source metadata (as I said, 0.13 works fine with the same metadata),. but not in the cache

[antonio-rojas]

Two further observations:

1. Reverting this piece of the commit fixes the issue

diff --git a/src/as-component.c b/src/as-component.c
index 7856fda8..88abe9f9 100644
--- a/src/as-component.c
+++ b/src/as-component.c
@@ -3689,6 +3677,7 @@ as_component_load_from_xml (AsComponent *cpt, AsContext *ctx, xmlNode *node, GEr
                        if (lang != NULL)
                                as_component_set_summary (cpt, content, lang);
                } else if (tag_id == AS_TAG_DESCRIPTION) {
+                       g_hash_table_remove_all (priv->description);
                        if (as_context_get_style (ctx) == AS_FORMAT_STYLE_COLLECTION) {
                                /* for collection XML, the "description" tag has a language property, so parsing it is simple */
                                if (lang != NULL) {

2. This affects only xmls of AS_FORMAT_STYLE_COLLECTION type. If I dump a random application appdata xml in /usr/share/app-info/xml the description gets indexed correctly.

[ximion]

Yeah, that g_hash_table_remove_all really didn't belong there at all. It is also quite stunning to me that this case was not ever tested in the testsuite, as it's a basic functionality - I guess that's why coverage reports are very useful.
It's fixed now, and I also added the missing test. Thank you for basically doing all debugging work! 👍