Added initial version of CI to rebuild dependencies package

[IvanSavenko]

Continuation of vcmi/vcmi#4461

Trying to establish flow for update of conan-built dependencies (so all other than msvc which uses vcpkg).

Currently planning following flow:

• if needed, update conanfile in vcmi/vcmi repo (in a separate branch). Right now I am using branch 'update_prebuilts' for that.
• run workflow added by this PR to generate prebuilts packages
• download artifacts & reupload them as release in this repository
• create PR in vcmi/vcmi repo with updated links for prebuilts downloads

Not sure about 1st step. We need new version of conanfile as well as few files from vcmi/vcmi repo, however it feels weird to create branch that would only be used in another repo. But haven't found better approaches.

TODO's for future:

• rebuild entire package (and not only ffmpeg, like in this PR). Both to test recipes and to replace any binaries that were built locally by devs with binaries built in clean CI environment.
• automatically create release (probably only for manually triggered workflows), to remove need of manual download & reupload of packages
• run workflow on schedule (daily? weekly?), so if upstream recipe has breaking change it will be detected by us.
• build Android package on Linux CI (and use Linux CI for android packages on vcmi/vcmi)
• add documentation on how to use this repository in place of currently empty Readme.md
• once vcmi/vcmi uses only dependencies from this repo, archive all other repositories with dependencies (other than vcpkg)

[kambala-decapitator]

isn't it possible to setup GhA to run a pipeline when specific file(s) in repo change? In such cases changes to conanfile.py (and Conan profiles) in VCMI repo should run the "build missing dependencies" pipeline.

[IvanSavenko]

Detecting changes in conanfile looks like possible but only with some assistance - third-party actions step or similar implementation by us, via git commands.

And at least one of the problems with unified workflow is that prebuilts packages are download via fixed url, e.g.
'https://github.com/vcmi/vcmi-dependencies/releases/download/1.2/dependencies-ios.tgz

So it can't be a separate pipeline (even with dependencies) - since even if we were to generate new package and create release with it, our main pipeline would still download old dependencies package since this is where URL in checked out code points to.

We also can't use URL to 'latest' release since newer package may not work with older vcmi. Like this ffmpeg update that requires to fix 1 deprecation to work with vcmi.

Currently we require following steps:

• commit new conanfile
• generate packages using new conanfile and put them into release in this repository
• update URL's to correct package in vcmi/vcmi
• build vcmi using new package

No idea how to fit all of that into a single workflow, especially one that would work with parallel development / branches.

[kambala-decapitator]

true, and that's where Artifactory would come to play... However, it should still be solvable via a parameter that will be passed to the downstream (VCMI build) job.

[kambala-decapitator]

btw XZ provides better compression, I'd keep using it instead of GZ

[IvanSavenko]

true, and that's where Artifactory would come to play... However, it should still be solvable via a parameter that will be passed to the downstream (VCMI build) job.

Maybe, but not sure how to implement such approach.

btw XZ provides better compression, I'd keep using it instead of GZ

Sure, can change. My intention was to unify compression method - mingw used gz, macos used xz. Can switch to xz everywhere.

[kambala-decapitator]

another thought on the workflow:

• we can move conanfile.py and profiles to this repo
• changing settings / bumping versions = PR to this repo (aka Deps), draft release will be created
• VCMI repo starts depending on this repo (submodule would suffice), e.g. on master branch
• testing new deps in VCMI = PR to VCMI that points to this repo's branch/commit + draft release. If all goes well, VCMI PR is closed and Deps PR is merged.
• if something goes wrong, only PR to this repo needs to be reverted
• when making a VCMI release, tag this repo as well

extra branches (e.g. develop) can be created in this repo as needed.

still, Artifactory would be the ideal solution. We should ask some other service for a free server :) JFrog seemed to ignore my request.

[IvanSavenko]

Thanks for suggestions, haven't considered submodules for CI files.

Will add docs (including these suggestions in TODO list) in a separate PR tomorrow.