Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache builds #26

Merged
merged 64 commits into from
Apr 8, 2021
Merged

Cache builds #26

merged 64 commits into from
Apr 8, 2021

Conversation

zhvrnkov
Copy link
Contributor

@zhvrnkov zhvrnkov commented Feb 23, 2021

Description

In this PR the cacher subcommand is introduced. Main features of this are simple:

  • save command. Can save folders to provided git repo. Saves by creating branch with provided name
  • get command. Will provide saved package from git repo
  • setup command. Will create "git remote storage"
  • update command. Same as save, but update
  • packages command. List all stored packages in git repo

Also notice that build-cache mechanism is embeded in spm command. To tell that spm build products should be cached, the --cache-builds arg should be passed.

Some minor/major fixes were introduced also such as:

  • Shell wrapper fix for really long outputs
  • cleanup for PackageManager architecture

Checklist

  • Follows code style
  • Has no SwiftLint warnings
  • Covered by tests
  • All tests are passed
  • Access control words were checked

zhvrnkov added 30 commits January 26, 2021 18:03
create and checkout branch softly -- don't throw an error if it exists
or already on it
handle no changes commit -- before commit check changes, if no chages,
then throw specific error
it initialize git and create empty master with single commit
this will allow using Git as git.remote.add
Logic is simillar to GitCacher but a little bit different -- need to
refactor, but idea is nice. Work only with remote repo and clone it
with --depth=1 and specified branch name, which optimize the size of
repo (don't download everythin, which it do, from the web). Also the
organization of repo is well defined right now -- repo, where each
build has unique branch. In terms of remote git repo (github etc) it
will perfectly work, and will work with cis also
The best part about git is, that it can use any url to git repo -- you
can work with any git repo as a remote repo. This feature allow to
simplify GitCacher by a lot.
When saving you should clone master branch (because your package
branch isn't created yes -- it's save), but when updating you can
directly clone your package branch (without cloning anything else) and
work with that.

In future this should be simplified from user's point of view, but
right now it is as it is.
delete cloned folder in save/update end
This method will output list of packageIDs, and in case of git repo
list of remotes. Done by 'git ls-remote' which is pretty cool since
there is no big downloads except just branches list
Previousle there was init(depofile: Depofile, ...), now its
init(packages: Packages, ...). It's required for implement caching
more easily:
- got all packages from Depofile
- some of them are cached and don't need to be builded
- so you don't pass them to Manager

It's easier to filter packages, then recreate Depofile with filtered
packages.
Also it looks more solid.
Intead pass them to Command.command (update | install | build)
Packages is flexible cause AllPackageManager can just do Packages =
Depofile. Packages should be a collection and thats flexible, but now,
for caching, build, update and install should return [Package: [URL]]
which is impossible with Packages
zhvrnkov added 21 commits February 5, 2021 15:20
In Install, Update and Build command there is a extension which adds
default `run` implementation. It constaints for ValueType ==
[Package], but this weren't like that for AllPackageManager so I fixed
that by wrapping Depofile into array
Got and freezing process when calling 'xcodebuild -list -json' for
huge project where output of this program is near 2000 lines. Tryed
the ShellOut project on github and there is no such thing when using
it. Copy their solution and redesign project's Shell
remove old Has<COMMAND>Command protocols and bunch of others
trying to replace them with single PackageManager
DepoCore is compiling with new types
Sources/Depo/Commands/Cacher.swift Show resolved Hide resolved
Sources/Depo/Commands/Cacher.swift Show resolved Hide resolved
Sources/Depo/Commands/Install.swift Show resolved Hide resolved
Sources/DepoCore/Shell/Shell.swift Outdated Show resolved Hide resolved
This url will be used to access cache if --cache-builds specified
Sources/DepoCore/Helpers/GitCacher.swift Outdated Show resolved Hide resolved
@zhvrnkov zhvrnkov merged commit 7652b54 into master Apr 8, 2021
@zhvrnkov zhvrnkov mentioned this pull request Apr 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants