Change: [Actions] stop relying on external actions for trivial stuff, and rely on cmake tools to build, test and pack #8718
Conversation
That looks very nice :D Often I mentioned something on a single place, but I am sure you understand I mean all instances of similar change ;) |
vcpkgTriplet: '${{ matrix.arch }}-osx' | ||
path: /usr/local/share/vcpkg/installed | ||
key: ${{ runner.os }}-${{ matrix.arch }}-vcpkg-${{ steps.key.outputs.commitid }} | ||
restore-keys: ${{ runner.os }}-${{ matrix.arch }}-vcpkg- |
TrueBrain
Feb 22, 2021
Member
weird restore-key, ending with a -
?
weird restore-key, ending with a -
?
glx22
Feb 22, 2021
Author
Contributor
It matches all keys if key
is a miss, and use the most recent.
It matches all keys if key
is a miss, and use the most recent.
orudge
Feb 23, 2021
Contributor
I submitted a PR to run-vcpkg (lukka/run-vcpkg#75) which alters the caching there so that it uses the env.ImageVersion variable in the cache key, so that the cache is invalidated if the VM environment is changed. It would be worth us using this in our key (it did cause a problem on macOS before).
(I was waiting for the new release of run-vcpkg, which did happen but I then forgot about it, so never updated OpenTTD...)
I submitted a PR to run-vcpkg (lukka/run-vcpkg#75) which alters the caching there so that it uses the env.ImageVersion variable in the cache key, so that the cache is invalidated if the VM environment is changed. It would be worth us using this in our key (it did cause a problem on macOS before).
(I was waiting for the new release of run-vcpkg, which did happen but I then forgot about it, so never updated OpenTTD...)
glx22
Feb 23, 2021
Author
Contributor
Do you remember the exact issue ?
I cache less than run-vcpkg
(only vcpkg/installed
folder) and I assume vcpkg exe provided in the image works. If vcpkg in image is upgraded, it should detect if installed libs need an upgrade (at least my local vcpkg does that)
Do you remember the exact issue ?
I cache less than run-vcpkg
(only vcpkg/installed
folder) and I assume vcpkg exe provided in the image works. If vcpkg in image is upgraded, it should detect if installed libs need an upgrade (at least my local vcpkg does that)
orudge
Feb 23, 2021
Contributor
I suspect the build logs will have expired now, but I think the error was relating to missing dependencies in libstdc++.dylib or something along those lines, due to a compiler change in the VM template.
I had pinned vcpkg to a particular commit for macOS, but I think the version in the image is new enough to support our arm64 workflow, so it should hopefully be fine.
I suspect the build logs will have expired now, but I think the error was relating to missing dependencies in libstdc++.dylib or something along those lines, due to a compiler change in the VM template.
I had pinned vcpkg to a particular commit for macOS, but I think the version in the image is new enough to support our arm64 workflow, so it should hopefully be fine.
glx22
Feb 23, 2021
Author
Contributor
Well release workflow seems to work https://github.com/glx22/OpenTTD/runs/1946699269?check_suite_focus=true
And it seems run-vcpkg
always rebuilt vcpkg
executable (even for windows)
Well release workflow seems to work https://github.com/glx22/OpenTTD/runs/1946699269?check_suite_focus=true
And it seems run-vcpkg
always rebuilt vcpkg
executable (even for windows)
TrueBrain
Feb 24, 2021
Member
A closer look to how images generate their name etc, this cache-key will indeed be a bit of a problem when GitHub changes some things.
${{ runner.os }
is just macOS
for example. Not macos-latest
or macos-10.15
. This means that if they switch the base image, it is possible that the precompiled libraries no longer match the host system, resulting in compile errors, or, maybe worse, missing security fixes etc.
It seems that for cache keys the most correct approach would be to include ${{ env.ImageOs }}-${{ env.ImageVersion }}
. This changes every change of the base image, and would trigger a full recompile for us, but those events are rare. It would also mean that this cache should in theory never break our compiles .. might save a lot of issues?
What do you think @glx22 ?
A closer look to how images generate their name etc, this cache-key will indeed be a bit of a problem when GitHub changes some things.
${{ runner.os }
is just macOS
for example. Not macos-latest
or macos-10.15
. This means that if they switch the base image, it is possible that the precompiled libraries no longer match the host system, resulting in compile errors, or, maybe worse, missing security fixes etc.
It seems that for cache keys the most correct approach would be to include ${{ env.ImageOs }}-${{ env.ImageVersion }}
. This changes every change of the base image, and would trigger a full recompile for us, but those events are rare. It would also mean that this cache should in theory never break our compiles .. might save a lot of issues?
What do you think @glx22 ?
glx22
Feb 24, 2021
Author
Contributor
Hmm yes I can replace ${{ runner.os }
with ${{ env.ImageOS }}-${{ env.ImageVersion }}
.
That means cache is always rebuilt when a new image is used. It was partially the case when using the commit id in the key as it would use cached libs and upgrade them if needed, assuming binary compatibility.
Will be safer to always fully rebuild libs unconditionally.
I think I can remove commit id from the key, and also restore-keys
.
Hmm yes I can replace ${{ runner.os }
with ${{ env.ImageOS }}-${{ env.ImageVersion }}
.
That means cache is always rebuilt when a new image is used. It was partially the case when using the commit id in the key as it would use cached libs and upgrade them if needed, assuming binary compatibility.
Will be safer to always fully rebuild libs unconditionally.
I think I can remove commit id from the key, and also restore-keys
.
Will address other comments |
vcpkgTriplet: '${{ matrix.arch }}-osx' | ||
path: /usr/local/share/vcpkg/installed | ||
key: ${{ runner.os }}-${{ matrix.arch }}-vcpkg-${{ steps.key.outputs.commitid }} | ||
restore-keys: ${{ runner.os }}-${{ matrix.arch }}-vcpkg- |
glx22
Feb 22, 2021
Author
Contributor
It matches all keys if key
is a miss, and use the most recent.
It matches all keys if key
is a miss, and use the most recent.
No issues in principle |
vcpkgTriplet: '${{ matrix.arch }}-osx' | ||
path: /usr/local/share/vcpkg/installed | ||
key: ${{ runner.os }}-${{ matrix.arch }}-vcpkg-${{ steps.key.outputs.commitid }} | ||
restore-keys: ${{ runner.os }}-${{ matrix.arch }}-vcpkg- |
TrueBrain
Feb 24, 2021
Member
A closer look to how images generate their name etc, this cache-key will indeed be a bit of a problem when GitHub changes some things.
${{ runner.os }
is just macOS
for example. Not macos-latest
or macos-10.15
. This means that if they switch the base image, it is possible that the precompiled libraries no longer match the host system, resulting in compile errors, or, maybe worse, missing security fixes etc.
It seems that for cache keys the most correct approach would be to include ${{ env.ImageOs }}-${{ env.ImageVersion }}
. This changes every change of the base image, and would trigger a full recompile for us, but those events are rare. It would also mean that this cache should in theory never break our compiles .. might save a lot of issues?
What do you think @glx22 ?
A closer look to how images generate their name etc, this cache-key will indeed be a bit of a problem when GitHub changes some things.
${{ runner.os }
is just macOS
for example. Not macos-latest
or macos-10.15
. This means that if they switch the base image, it is possible that the precompiled libraries no longer match the host system, resulting in compile errors, or, maybe worse, missing security fixes etc.
It seems that for cache keys the most correct approach would be to include ${{ env.ImageOs }}-${{ env.ImageVersion }}
. This changes every change of the base image, and would trigger a full recompile for us, but those events are rare. It would also mean that this cache should in theory never break our compiles .. might save a lot of issues?
What do you think @glx22 ?
56cd3bb
to
7ac062e
Seriously nice work :) |
Motivation / Problem
We used
run-vcpkg
to managevcpkg
for windows and macos targets andrun-cmake
to build windows targets. But doing the same without these actions is quite trivial, and easier to understand in the workflow.Also we used to run
make
to build non windows targets, but it's better to letcmake
handle the calls to native build tools.Description
Implemented the above changes.
Limitations
Checklist for review
Some things are not automated, and forgotten often. This list is a reminder for the reviewers.