Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add mechanism to auto update most of packages #7977

Merged
merged 36 commits into from
Mar 29, 2022
Merged

Add mechanism to auto update most of packages #7977

merged 36 commits into from
Mar 29, 2022

Conversation

MrAdityaAlok
Copy link
Member

@MrAdityaAlok MrAdityaAlok commented Nov 13, 2021

  • Add package neovim-nightly (initially PR was for this)
  • Add functions to override auto-update script. Suggested by @thunder-coding
  • Update wiki
  • Restrict build scripts from using GITHUB_API_TOKEN. No need now since wirkflow only run for already present packages, not also for PR.

Functions:

Main function (overridable)

This should be overrided for using custom update method. For example look here.

  • termux_pkg_auto_update : determine which update script to use for auto-updating

Utility/API (available publicly)

  • termux_pkg_upgrade_version: to write new version to build.sh
  • termux_pkg_is_update_nedded: determine wheteher package needs to be updated
  • termux_pkg_get_last_updated: get last updated date for github housed packages. It is used for api cache call to lower api call rate.
  • termux_github_api_get_tag: returns latest/newest version tag
  • termux_gitlab_api_get_tag: returns latest/newest version tag
  • termux_repology_api_get_latest_version: returns latest version

Internal functions (should not be overrided)

  • termux_github_auto_update : default update algorithm for packages hosted on github.com
  • termux_gitlab_auto_update: default update algorithm for pkgs hosted on gitlab.com
  • termux_repology_auto_update: default update algorithm for pkgs hosted on repology.org

Tests

  • Test new packages for auto-update: it can be tested locally as ./update_packages <package name> (GITHUB_TOKEN should be set in env.)
  • Test existing packages (I will test and enable update for existing packages in another PR).

Copy link
Member

@thunder-coding thunder-coding left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some final changes

After this, I will experiment with these to make sure they don't cause troubles on master

scripts/updates/utility.sh Outdated Show resolved Hide resolved
scripts/updates/utility.sh Outdated Show resolved Hide resolved
@thunder-coding
Copy link
Member

Update wiki

Leave this on me, I will do this after merging

Hide GITHUB_API_TOKEN from auto-update scripts.

Glad you are looking into it :)

@MrAdityaAlok MrAdityaAlok changed the title Add package neovim-nightly Add mechanism to auto update most of packages Nov 17, 2021
@MrAdityaAlok MrAdityaAlok marked this pull request as ready for review December 17, 2021 11:00
@MrAdityaAlok MrAdityaAlok requested a review from a user December 17, 2021 11:00
@ghost ghost deleted a comment from Ladla-saith Dec 17, 2021
@MrAdityaAlok MrAdityaAlok marked this pull request as draft December 28, 2021 10:37
@MrAdityaAlok MrAdityaAlok marked this pull request as ready for review January 7, 2022 07:46
@MrAdityaAlok
Copy link
Member Author

MrAdityaAlok commented Jan 7, 2022

Please ignore ci failures, it is intended (due to lack of permission to access GH_API_KEY for PRs).
Should not fail now.

@MrAdityaAlok MrAdityaAlok marked this pull request as draft January 15, 2022 10:25
@finagolfin
Copy link
Member

Anything holding this up? We're down to only 73% of our packages being up to date, we used to be in the 90s a year or two ago.

@MrAdityaAlok
Copy link
Member Author

@buttaface I need to review it again.
Initially I didn't see @termux maintainers interest in this PR, so I moved on to work on #9333 .
But now this PR should be ready for review in 2-3 days.

@finagolfin
Copy link
Member

No hurry, but I think we should definitely expand beyond only auto-updating github-housed projects.

Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
- hook to be called when auto updating.
- it may be overridden by build.sh

Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
- compare current and latest version retrived from respective api

Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
Signed-off-by: Aditya Alok <dev.aditya.alok@gmail.com>
@MrAdityaAlok
Copy link
Member Author

Please ignore ci failures. They are due to not upto date neovim-nightly's checksum. Just ensure to run ./srcipts/bin/update-packages neovim-nightly before merging to update its checksum or ping me to do so.

@MrAdityaAlok MrAdityaAlok marked this pull request as ready for review March 28, 2022 17:45
@MrAdityaAlok
Copy link
Member Author

Ping @Grimler91

@Grimler91
Copy link
Member

@MrAdityaAlok commits and structure looks reasonable, thanks.

Please update the nightly hashsum so that we can see if the build succeeds, and so that debs can be inspected if needed

.github/workflows/package_updates.yml Outdated Show resolved Hide resolved
# Make script sourceable as well as executable.
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
declare -f termux_error_exit >/dev/null || . "$(dirname "${BASH_SOURCE[0]}")/termux_error_exit.sh"
termux_pkg_is_update_needed "$@"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have a script scripts/bin/apt-compare-versions which aims at doing pretty much the same thing as this script, with the difference that it prints the result in a human-friendly way. Please remove the other one

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I remove this whole function or only the code making it executable?
I have removed the code making it executable (though not pushed still) but did you ment also to use scripts/bin/apt-compare-versions in this function...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a more rigorous implementation than the other one, so I suggest to use this one and remove scripts/bin/apt-compare-versions. Possible improvement to this termux_pkg_is_update_needed.sh script would be to add a "human-readable" mode, which prints the versions as the other script did, but since that isn't related to auto updates or neovim it's not something that has to be added as part of this PR IMO

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still I would add that functionality in this PR only. I have exams in upcoming days, so would be busy there for few weeks...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's fine. Fix the email address and I can merge it

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added that functionality. Please review again.

@Grimler91 Grimler91 merged commit c114f86 into termux:master Mar 29, 2022
@Grimler91
Copy link
Member

Grimler91 commented Mar 29, 2022

Github's UI failed to detect the merge again Nevermind, it was just slow. Merged in c114f86.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants