Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Global cache issues #211

Closed
paulcsmith opened this issue Jun 15, 2018 · 8 comments
Closed

Global cache issues #211

paulcsmith opened this issue Jun 15, 2018 · 8 comments
Assignees
Labels
Milestone

Comments

@paulcsmith
Copy link

@paulcsmith paulcsmith commented Jun 15, 2018

I just updated to 0.25 of Crystal which uses shards 0.8 with the new global cache.

The shard cache failed to update under these circumstances:

  • Defined a shard (lucky_cli) in shard.yml using version
  • Then changed the definition to use branch: master
  • Ran shards update
  • Then made some changes to the lucky_cli
  • Ran shards update
  • Cache still used the old version of branch: master instead of updating

Temporary fix

rm -rf ~/.cache/shards and re-run `shards update.

Possible fix

  • When running shards update always update cache if using branch or tag?
@kostya
Copy link

@kostya kostya commented Jun 15, 2018

oh, i just try to find why my shard not updated, rm -rf ~/.cache/shards helps. it not update even for version not for master only. (after update crystal and shard to 0.8.0)

@paulcsmith paulcsmith changed the title 'branch: master' and global cache issues Global cache issues Jun 15, 2018
@wontruefree
Copy link

@wontruefree wontruefree commented Jun 15, 2018

It seems like if you are using a branch for the version it should always have a cache miss.
It seems like something would have to be added here https://github.com/crystal-lang/shards/blob/master/src/commands/check.cr#L41 to always return false for installed.

@kostya
Copy link

@kostya kostya commented Jun 15, 2018

dependencies:
  myhtml:
    github: kostya/myhtml
    version: ">= 0.29"
shards update
Fetching https://github.com/kostya/myhtml.git
Error resolving myhtml (>= 0.29)

but 0.29 there is, i just push it.
i think this is quite critical issue.

rm -rf ~/.cache/shards
shards update
Fetching https://github.com/kostya/myhtml.git
Installing myhtml (0.29)
Postinstall cd src/ext && make package
@ysbaddaden
Copy link
Contributor

@ysbaddaden ysbaddaden commented Jun 16, 2018

Sigh. Can someone reproduce and run shards commands with --verbose, look at the global cache structure, and start investigating?

I won't be able to look at this anytime soon.

@ysbaddaden
Copy link
Contributor

@ysbaddaden ysbaddaden commented Jun 16, 2018

It's probably not the global cache, but not mirroring the Git repository anymore. A simple --bare and fetch isn't enough to pull Git tags. Without them, Shards is incapable to resolve versions.

@ysbaddaden
Copy link
Contributor

@ysbaddaden ysbaddaden commented Jun 16, 2018

If you type git commands in the cached git folder, you probably won't see the new tags, even after a shards update.

@kostya
Copy link

@kostya kostya commented Jun 16, 2018

shards update --verbose
git ls-remote --get-url origin
Fetching https://github.com/kostya/stuffs.git
git fetch --all --quiet
git tag --list --column=never
versions: 0.1
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git show v0.1:shard.yml
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git show v0.1:shard.yml
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git show v0.1:shard.yml
Installing stuffs (0.1)
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git archive --format=tar --prefix= v0.1 | tar -x -f - -C '/Users/kostya/projects/blah5/lib/stuffs'
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git show v0.1:shard.yml

after pushing v0.2

shards update --verbose
git ls-remote --get-url origin
Fetching https://github.com/kostya/stuffs.git
git fetch --all --quiet
git tag --list --column=never
versions: 0.1
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git show v0.1:shard.yml
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git show v0.1:shard.yml
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git show v0.1:shard.yml
Using stuffs (0.1)
git ls-tree -r --full-tree --name-only v0.1 -- shard.yml
git show v0.1:shard.yml

after rm -rf ~/.cache/shards/

shards update --verbose
Fetching https://github.com/kostya/stuffs.git
git clone --bare --quiet -- 'https://github.com/kostya/stuffs.git' /Users/kostya/.cache/shards/github.com/kostya/stuffs.git
git tag --list --column=never
versions: 0.2, 0.1
git ls-remote --get-url origin
git ls-tree -r --full-tree --name-only v0.2 -- shard.yml
git show v0.2:shard.yml
git ls-tree -r --full-tree --name-only v0.2 -- shard.yml
git show v0.2:shard.yml
git ls-tree -r --full-tree --name-only v0.2 -- shard.yml
git show v0.2:shard.yml
Installing stuffs (0.2)
rm -rf '/Users/kostya/projects/blah5/lib/stuffs'
git ls-tree -r --full-tree --name-only v0.2 -- shard.yml
git archive --format=tar --prefix= v0.2 | tar -x -f - -C '/Users/kostya/projects/blah5/lib/stuffs'
git ls-tree -r --full-tree --name-only v0.2 -- shard.yml
git show v0.2:shard.yml
@kostya
Copy link

@kostya kostya commented Jun 17, 2018

i just test it, it works if replace --bare with --mirror (from https://stackoverflow.com/questions/3382679/how-do-i-update-my-bare-repo).
Git fetch not update in bare repo, but update in mirror repo.

ysbaddaden added a commit that referenced this issue Jun 17, 2018
ysbaddaden added a commit that referenced this issue Jun 17, 2018
Creating bare Git repositories doesn't configure the local
repository to fetch remote refs, such as branches and tags, which
totally broke the update command.

This patch reverts back to mirroring Git repositories, and verifies
whether a cloned repository is valid, or not before fetching new
refs.

fixes #211
@ysbaddaden ysbaddaden self-assigned this Jun 17, 2018
@ysbaddaden ysbaddaden added the bug label Jun 17, 2018
@ysbaddaden ysbaddaden added this to the v0.8.1 milestone Jun 17, 2018
ysbaddaden added a commit that referenced this issue Jun 17, 2018
Creating bare Git repositories doesn't configure the local
repository to fetch remote refs, such as branches and tags, which
totally broke the update command.

This patch reverts back to mirroring Git repositories, and verifies
whether a cloned repository is valid, or not before fetching new
refs.

fixes #211
taylor added a commit to vulk/shards that referenced this issue Aug 11, 2020
Creating bare Git repositories doesn't configure the local
repository to fetch remote refs, such as branches and tags, which
totally broke the update command.

This patch reverts back to mirroring Git repositories, and verifies
whether a cloned repository is valid, or not before fetching new
refs.

fixes crystal-lang#211
f-fr pushed a commit to f-fr/shards that referenced this issue Jan 2, 2021
Creating bare Git repositories doesn't configure the local
repository to fetch remote refs, such as branches and tags, which
totally broke the update command.

This patch reverts back to mirroring Git repositories, and verifies
whether a cloned repository is valid, or not before fetching new
refs.

fixes crystal-lang#211
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

4 participants