Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

integrate with the tool cache somehow? #145

Open
jeremyd2019 opened this issue Jul 5, 2021 · 18 comments
Open

integrate with the tool cache somehow? #145

jeremyd2019 opened this issue Jul 5, 2021 · 18 comments
Labels
enhancement New feature or request

Comments

@jeremyd2019
Copy link
Member

Github Actions has an API for working with the tool cache:
https://github.com/actions/toolkit/tree/main/packages/tool-cache

This seems like a reasonable fit for this action. I don't know if just the installer download should be cached, or if the install should live in the tool cache, as the tool cache is preserved between runs, unlike the runner _temp directory where this stuff goes now.

@lazka
Copy link
Member

lazka commented Jul 5, 2021

Why not.

Unclear re whole install. In theory you can put your msys2 install at C:/msys64 and pass "release: false" to the actions, so it uses the "default" msys install. This way it will reuse things already.

@jeremyd2019
Copy link
Member Author

Well, it's really meant for tools that are versioned, and keeping track of different versions of tools side-by-side. Like the setup-python action does.

@lazka
Copy link
Member

lazka commented Jul 5, 2021

Yeah, re unclear I mean that it would keep state between runs. Currently it doesn't and maybe someone depends on it (??)

@jeremyd2019
Copy link
Member Author

only in self-hosted runners. github-hosted runners are apparently always fresh and clean for every run

@lazka
Copy link
Member

lazka commented Jul 5, 2021

we can try. from what I see it copies everything into the cache, so we need to check if that slows down the hosted runners.

https://github.com/actions/toolkit/blob/45647689407e7fb224e06d066dde6aefa67a365f/packages/tool-cache/src/tool-cache.ts#L434

@jeremyd2019
Copy link
Member Author

Unclear re whole install. In theory you can put your msys2 install at C:/msys64 and pass "release: false" to the actions, so it uses the "default" msys install. This way it will reuse things already.

This seems like a better idea for the whole install. I need to look at my fork and make sure that that would work with C:/msys32. I don't think I handled that case because the hosted runners didn't have that present anyway.

@jeremyd2019
Copy link
Member Author

I'm just grasping at straws to reduce the approx 1.5 to 1.75 hours it takes to get to 'Process build queue' on my raspberry pi 3B+ self-hosted runner. I suspect a large part of it is the slowness of the SD card, but I was unable to get any usb storage to work with a quick test.

@Biswa96
Copy link
Member

Biswa96 commented Jul 5, 2021

Are you booting from SD in RPi? Have you tried this https://www.worproject.ml/downloads ?

@jeremyd2019
Copy link
Member Author

I am booting from SD. I don't really have a suitable USB storage device for it, and attempting to add a usb mass storage device to the running install resulted in errors, so I don't hold out much hope that it will work. Also, I thought that booting from USB was a pi4 enhancement.

I used https://github.com/WOA-Project/WOA-Deployer-Rpi

@ZachBacon
Copy link

I am booting from SD. I don't really have a suitable USB storage device for it, and attempting to add a usb mass storage device to the running install resulted in errors, so I don't hold out much hope that it will work. Also, I thought that booting from USB was a pi4 enhancement.

I used https://github.com/WOA-Project/WOA-Deployer-Rpi

I think it has to be supported not just from the raspberry Pi itself but by the bootloader used.

@eine
Copy link
Collaborator

eine commented Jul 5, 2021

@jeremyd2019 if I don't get it wrong, the feature your are requesting is implemented already, but disabled by default. See 434d91b, #61 and #50.

@eine eine added the question Further information is requested label Jul 5, 2021
@jeremyd2019
Copy link
Member Author

jeremyd2019 commented Jul 5, 2021

Not quite. That cache is stored with github, so for a self-hosted runner still requires downloading (and uploading, which is failing for me for the packages cache). The tool cache is local to the runner and persists between runs (again, for a self-hosted runner).

@eine
Copy link
Collaborator

eine commented Jul 5, 2021

@jeremyd2019, I see. I was confused because we do have tool-cache as a dependency (https://github.com/msys2/setup-msys2/blob/master/package.json#L33), but we use it for downloading the tarball/installer only.

I think the solution would to treat this orthogonally:

  • One option is whether to cache the whole installation or the packages/tools only.
  • An independent option is whether to use the cache on GitHub or to use a local cache.

Does it make sense to you?

@jeremyd2019
Copy link
Member Author

yes. The PR I opened (#147) just caches the installer. One thing that is kind of annoying with a rolling release like msys2 is that the tool cache expects a (semantic) version, I just used the date of the installer.

@eine eine added enhancement New feature or request and removed question Further information is requested labels Jul 5, 2021
@jeremyd2019
Copy link
Member Author

Maybe the version for the install could be a (small) constant, just so there is something (since it's in the directory layout)? Like "1" or "a"? Just brainstorming

@jeremyd2019
Copy link
Member Author

jeremyd2019 commented Jul 6, 2021

Turns out that if the version doesn't match what semver wants (major.minor.patch all numeric, plus some other stuff), it fails to find the previously cached tool. It looks like a form of 0.0.1-2021-07-05 would be valid, but to be safe I am looking at 0.0.20210705.
But for an install, the shortest valid semver would probably be 0.0.1

@jeremyd2019
Copy link
Member Author

I've switched to release: false and providing a C:\msys32 install on my self-hosted runner for now.

@eine
Copy link
Collaborator

eine commented Jul 16, 2021

FTR, #145 (comment) was fixed in #148.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants