Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ungoogled-chromium,chromium+llvm: use versionhistory.googleapis.com over omahaproxy.appspot.com, fix various update script issues #225281

Merged
merged 4 commits into from
Apr 30, 2023

Conversation

networkException
Copy link
Member

@networkException networkException commented Apr 8, 2023

Description of changes

This pull request updates chromium's update.py and llvms update-git.py script to use the versionhistory.googleapis.com api as a replacement for the now deprecated omahaproxy.appspot.com.

See https://groups.google.com/a/chromium.org/g/chromium-dev/c/uH-nFrOLWtE/m/PhUj_inyAQAJ for the deprecation notice and ungoogled-software/ungoogled-chromium#2260 for a discussion about the query parameters used.

Additionally, it fixes some issues related to the ungoogled-chromium packaging, namely that the update.py script would update to Windows only releases. The script will now only update to ungoogled-chromium releases matching the Linux stable channel.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.05 Release Notes (or backporting 22.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

@networkException
Copy link
Member Author

@Janik-Haag

Copy link
Member

@Janik-Haag Janik-Haag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The llvm scirpt works as expected and looks good to me, also didn't find any other references to history that weren't edited to releases.

But the chromium scirpt has a problems for me:
For downloading with prefatch-nix it creates a tmpfs which is not large-enough for the file it want's to download and causes the download to crash, because it thinks there is not enough space left.

output of df -h:

tmpfs           1,6G  1,6G     0 100% /run/user/1000

Error message:

error: writing to file: No space left on device

It then restarts the download which crashes with the same error and just ends up looping

In another PR someone should fix the deprecation warning:

~/Projects/nixos/nixpkgs/./pkgs/applications/networking/browsers/chromium/update.py:18: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives

@networkException
Copy link
Member Author

But the chromium scirpt has a problems for me:
For downloading with prefatch-nix it creates a tmpfs which is not large-enough for the file it want's to download and causes the download to crash, because it thinks there is not enough space left.

As /run/user/1000 is a persistent tempfs, I fear there's nothing the download script can really do. For my testing I adjusted the allocation limit like so (the default is 10%, you might need a higher absolute value depending on the memory available in your system)

services.logind.extraConfig = ''
  RuntimeDirectorySize=25%
'';

@Janik-Haag
Copy link
Member

Janik-Haag commented Apr 13, 2023

services.logind.extraConfig = ''
  RuntimeDirectorySize=25%
'';

This should probably be in the README.md updating part, because I assume that might be a common error people will encounter.

@networkException networkException changed the title chromium+llvm: use versionhistory.googleapis.com over omahaproxy.appspot.com {,ungoogled-}chromium+llvm: use versionhistory.googleapis.com over omahaproxy.appspot.com, fix various update script issues Apr 13, 2023
@Janik-Haag
Copy link
Member

Janik-Haag commented Apr 14, 2023

Tested both scripts, they work fine now. The latest commit fixed the problem with the update script trying to pull windows only releases.

Copy link
Member

@Janik-Haag Janik-Haag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM; for details read the previous comments.

@networkException
Copy link
Member Author

Since Google just bumped the stable channel for Linux to 112.0.5615.121 I noticed another issue:

Before the last push the script would only fetch the newest release from each channel, causing the code that filters ungoogled-chromium releases to the Linux stable channel to fail. With the newest changes, the script will now fetch all releases for all Linux channels.

Note that Google's CI has not finished publishing release tarballs for 112.0.5615.121, the stable channel currently fails to update.

Also note that the latest ungoogled-chromium release in nixpkgs is a Windows only release, meaning that running the script now would revert ungoogled-chromium back one version to .49. It should work properly once I push ungoogled-chromium release 112.0.5615.121-1.

@networkException networkException changed the title {,ungoogled-}chromium+llvm: use versionhistory.googleapis.com over omahaproxy.appspot.com, fix various update script issues ungoogled-chromium,chromium+llvm: use versionhistory.googleapis.com over omahaproxy.appspot.com, fix various update script issues Apr 14, 2023
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/2103

pkgs/applications/networking/browsers/chromium/README.md Outdated Show resolved Hide resolved
api_tag_url = 'https://api.github.com/repos/Eloston/ungoogled-chromium/tags?per_page=1'
def get_latest_ungoogled_chromium_tag(linux_stable_versions):
"""Returns the latest ungoogled-chromium tag for linux using the GitHub API."""
api_tag_url = 'https://api.github.com/repos/ungoogled-software/ungoogled-chromium/tags'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this take a github token?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean technically yes but I would assume that the package maintainers don't run the script to an extent that would require a token

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rate limiting is per IP for the entire API. If you are in a shared space then the rate limiting is often very quickly hit.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you want an optional flag that passes a github api token to the program?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be great if it just takes one from GITHUB_TOKEN env.

Version information previously available at omahaproxy.appspot.com will
have to be retrieved from versionhistory.googleapis.com in the future.

See https://groups.google.com/a/chromium.org/g/chromium-dev/c/uH-nFrOLWtE/m/PhUj_inyAQAJ
See ungoogled-software/ungoogled-chromium#2260
Version information previously available at omahaproxy.appspot.com will
have to be retrieved from versionhistory.googleapis.com in the future.

See https://groups.google.com/a/chromium.org/g/chromium-dev/c/uH-nFrOLWtE/m/PhUj_inyAQAJ
See ungoogled-software/ungoogled-chromium#2260
The main repository has moved from the Eloston user to the
ungoogled-software organisation.
ungoogled-chromium releases generally follow the chromium stable channel,
however as the patchset targets Linux, macOS and Windows, a release might
not always be intended for being packaged in nix. This patch patches the
update script to filter ungoogled-chromium releases by them being available
as stable releases for Linux.
Copy link
Member

@RaitoBezarius RaitoBezarius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On LLVM changes: sounds good; untested though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants