Firefox AppStream data doesn't appear in appstream, but does in appstream-glib

[Conan-Kudo]

For some reason, Firefox AppStream data doesn't render for anything that uses appstream (Plasma Discover, AppCenter), but it does for anything using appstream-glib (GNOME Software).

appstream:

[ngompa@f25-skuld-vm ~]$ appstreamcli search firefox
Identifier: kdocker.desktop [desktop-application]
Name: KDocker
Summary: Run command and dock window
Package: kdocker
Homepage: http://kdocker.com
Icon: kdocker.png
----
Identifier: icecat.desktop [desktop-application]
Name: icecat
Summary: GNU version of Firefox browser
Package: icecat
Homepage: http://www.gnu.org/software/gnuzilla/
Icon: icecat.png
----
Identifier: mozilla-lightbeam [addon]
Name: Lightbeam for Firefox
Summary: See who's tracking you online
Package: mozilla-lightbeam
Homepage: https://www.mozilla.org/lightbeam/

appstream-glib:

[ngompa@f25-skuld-vm ~]$ appstream-util search firefox
[00148] system/package/fedora/desktop/firefox.desktop/*
[00040] system/package/fedora/addon/mozilla-lightbeam/*
[00030] system/package/fedora/desktop/icecat.desktop/*
[00010] system/package/fedora/desktop/kdocker.desktop/*

This is occurring on Fedora 25.

[ximion]

Do you find firefox in the XML files in /usr/share/app-info/xml or /var/(lib|cache)/app-info/xml?
Maybe asglib reads the .desktop file data and metainfo files to obtain the data.
Do you get any error messages when running sudo appstreamcli refresh --force --verbose?

[Conan-Kudo]

I don't have /usr/share/app-info/xml. I do have /usr/share/app-info/xmls, which has xml files that look like they contain a concatenated set of appstream data for a source/repository. It also has a /usr/share/app-info/icons counterpart.

There's nothing in the /var/cache/app-info/xmls directory. (Again, no /var/cache/app-info/xml directory).

[Conan-Kudo]

I've pasted the output from appstreamcli refresh --force --verbose: https://paste.fedoraproject.org/489156/14799967/

[ximion]

That log just confirms Firefox is missing ^^
Look into the XML files in the xmls directories, does any of those contain a Firefox component? If so, can you paste it somewhere?

[Conan-Kudo]

From /usr/share/app-info/xmls/fedora.xml.gz: https://paste.fedoraproject.org/489187/00167914/

[ximion]

Strange, this looks good...
I'll try to reproduce this issue with Fedora's data when I am home.

[Conan-Kudo]

Also note that Firefox is not installed at this time.

[ximion]

Can you please check your locale and run the search command with --verbose and attach the output?
Is your AppStream compiled with stemming support?

[Conan-Kudo]

My locale, according to $LANG is en_US.utf8.

Here's the output from appstreamcli search firefox --verbose: https://paste.fedoraproject.org/489247/09577148/

AppStream in Fedora is compiled with stemming support.

[ximion]

Weird, I can not reproduce this at all here...
With the Fedora data (and no other data source) I get

appstreamcli s firefox --details    
Identifier: firefox.desktop [desktop-application]
Name: Firefox
Summary: Im Internet surfen
Package: firefox
Homepage: http://www.mozilla.org/en-US/
Description: Bringing together all kinds of awesomeness to make browsing better for you. Get 
  to your favorite sites quickly ? even if you don?t remember the URLs. Type your 
  term into the location bar (aka the Awesome Bar) and the autocomplete function 
  will include possible matches from your browsing history, bookmarked sites and open 
  tabs.
Default Screenshot URL: https://raw.githubusercontent.com/hughsie/fedora-appstream/master/screenshots-extra/firefox/a.png
License: MPL-1.1 OR GPL-2.0+ OR LGPL-2.1+
Categories: 
 - Network
 - WebBrowser
Add-ons: flash-player-properties.desktop (Firefox)
Provided Items: ?
 Mimetypes: 
   - application/vnd.mozilla.xul+xml
   - application/xhtml+xml
   - text/html
   - text/mml
   - text/xml
   - x-scheme-handler/http
   - x-scheme-handler/https

Stemming support is on, the locale doesn't matter.
What does appstreamcli status show you?

[Conan-Kudo]

[ngompa@f25-skuld-vm ~]$ appstreamcli status
AppStream Status:
Version: 0.10.1

Distribution metadata:
 /usr/share/app-info
  - XML:  10
  - Iconsets:
     fedora
     rpmfusion-free-25
     rpmfusion-nonfree-25

 /var/lib/app-info
  - Empty.

 /var/cache/app-info
  - XML:  0
  - Iconsets:

Metainfo files:
  - Found 1 components.
  - Found 95 components in legacy paths.

Summary:
We have information on 1206 software components.

[ximion]

I am out of ideas for this... Is only Firefox missing, or also other stuff?
I probably need to install a fedora VM to try to reproduce this, because so far any attempt at reproducing this issue has failed.

Can you update your AppStream to 0.10.4? 0.10.1 is rather old...

[Conan-Kudo]

@ximion I cannot tell if other stuff is missing.

If it helps, I also have the rpmfusion appstream data packages installed as well.

[ngompa@f25-skuld-vm ~]$ rpm -qa | grep appstream-data
rpmfusion-free-appstream-data-25-1.fc25.noarch
appstream-data-25-8.fc25.noarch
rpmfusion-nonfree-appstream-data-25-1.fc25.noarch

[ximion]

@Conan-Kudo The only way they could prevent Firefox from showing up is by overriding the component with ID firefox.desktop with an invalid one... So this likely isn't the issue :-/

[ximion]

Haha, the thing I suspected to happen last November is actually the case on Fedora, as we found out at the Snappy sprint :)
This is a bug in Fedora's metadata (them not using merge components), but AppStream handles this case incredibly poorly and doesn't even properly report the issue. I will make AS handle this case better in a future update, which should resolve this issue.

[Conan-Kudo]

Excellent! 👍