Data checker fails on manifest for org.gnome.Chess

[Zlopez]

I tried to use the data checker for org.gnome.Chess and got

Traceback (most recent call last):
  File "/home/zlopez/git/flatpak-external-data-checker/flatpak-external-data-checker", line 30, in <module>
    main()
  File "/home/zlopez/git/flatpak-external-data-checker/src/main.py", line 258, in main
    manifest_checker.check(filter_type)
  File "/home/zlopez/git/flatpak-external-data-checker/src/checker.py", line 196, in check
    checker.check(data)
  File "/home/zlopez/git/flatpak-external-data-checker/src/checkers/htmlchecker.py", line 81, in check
    latest_url = get_latest(external_data.checker_data, "url-pattern", html)
  File "/home/zlopez/git/flatpak-external-data-checker/src/checkers/htmlchecker.py", line 47, in get_latest
    f"{pattern_name} {pattern} does not contain exactly 1 match group"
ValueError: url-pattern re.compile('https://download.gnome.org/sources/gnome-chess/(d+.d+)/gnome-chess-(d+.d+.d+).tar.xz') does not contain exactly 1 match group

As I understand there is issue in url-pattern, because the url for gnome apps is https://download.gnome.org/sources/gnome-chess/3.36/gnome-chess-3.36.1.tar.xz, so the version is there twice, but one time without patch number and in the other with patch number. I'm not sure how to adjust the manifest to make it work. Could you give me some advice?

[wjt]

Remove the first set of parentheses:

https://download.gnome.org/sources/gnome-chess/\d+.\d+/gnome-chess-(\d+.\d+.\d+).tar.xz

We don't need to capture the first one. The second one is the full version number.

[Zlopez]

I tried the change locally and now I got rid of the error, but got:

+ 2020-09-23 15:27:51,737 WARNING src.checkers.htmlchecker: gnome-chess-3.36.1.tar.xz had no available version information
+ 2020-09-23 15:27:51,737 WARNING src.checkers.htmlchecker: gnome-chess-3.36.1.tar.xz had no available URL

But I know the 3.38.0 is out. Is there something else I have wrong in manifest?

EDIT: Here is the PR with the current state.

[wjt]

Ah…

https://download.gnome.org/sources/gnome-chess/ is not going to work because the checker only scrapes 1 page. The links to the 3.y.z tarballs are not on that page. You will have better luck with:

                    "x-checker-data": {
                      "type": "html",
                      "url": "https://download.gnome.org/sources/gnome-chess/3.38/",
                      "version-pattern": "(\d+\.\d+)",
                      "url-pattern": "https://download.gnome.org/sources/gnome-chess/\d+\.\d+/gnome-chess-(\d+\.\d+\.\d+).tar.xz"
                    }

or similar, and bump it on every GNOME release. Sorry.

[Zlopez]

Ok, so I see I need to update it when new minor version is out.

Thanks for the help. I will try something else before I close it.

[Zlopez]

@wjt Your solution didn't help I have the following:

            "sources": [
                {
                    "type": "archive",
                    "url": "https://download.gnome.org/sources/gnome-chess/3.36/gnome-chess-3.36.1.tar.xz",
                    "sha256": "b195c9f17a59d7fcc892ff55e6a6ebdd16e7329157bf37e3c2fe593b349aab98",
                    "x-checker-data": {
                      "type": "html",
                      "url": "https://download.gnome.org/sources/gnome-chess/3.38/",
                      "version-pattern": "LATEST-IS-(\d+\.\d+\.\d+)",
                      "url-pattern": "https://download.gnome.org/sources/gnome-chess/3.38/gnome-chess-(\d+\.\d+\.\d+).tar.xz"
                    }
                }
            ]

and still getting:

+ 2020-09-23 16:08:57,744 WARNING src.checkers.htmlchecker: gnome-chess-3.36.1.tar.xz had no available version information
+ 2020-09-23 16:08:57,744 WARNING src.checkers.htmlchecker: gnome-chess-3.36.1.tar.xz had no available URL

So I'm not sure if the x-checker-data is even usable for this project. :-(

[wjt]

I will try to look tomorrow

[bilelmoussaoui]

@Zlopez as I mentioned in #33 I think having a checker for gnome that would use download.gnome.org would be the way to go.

[gasinvein]

@Zlopez I think you can use json checker for this (until #92 lands).
Something like this should do:

"x-checker-data": {
    "type": "json",
    "url": "https://download.gnome.org/sources/gnome-chess/cache.json",
    "version-query": ".[2][\"gnome-chess\"] | last",
    "url-query": "\"https://download.gnome.org/sources/gnome-chess/\" + .[1][\"gnome-chess\"][$version][\"tar.xz\"]"
}

[Zlopez]

@gasinvein This is working, thank you.