This repository has been archived by the owner. It is now read-only.

Using http URI's in formula dependencies causes multiple downloads #14285

Closed
staticfloat opened this Issue Aug 19, 2012 · 5 comments

Comments

Projects
None yet
3 participants
Contributor

staticfloat commented Aug 19, 2012

This is probably pretty far down on the "todo" list, but when using http URI's for formula dependencies, the depdencies are downloaded multiple times. For example, my julia formula uses two formulae that are not merged into Homebrew yet, and when installing this formula, I get output like the following:



/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/cycld/homebrew/41443150d9844e70b55ac626125a276d1cee9fcb/Library/Formula/llvm.rb -o /Library/Caches/Homebrew/Formula/llvm.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/staticfloat/homebrew/652835f810439ffdde237a1818af58140421acd1/Library/Formula/suite-sparse.rb -o /Library/Caches/Homebrew/Formula/suite-sparse.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/cycld/homebrew/41443150d9844e70b55ac626125a276d1cee9fcb/Library/Formula/llvm.rb -o /Library/Caches/Homebrew/Formula/llvm.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/staticfloat/homebrew/652835f810439ffdde237a1818af58140421acd1/Library/Formula/suite-sparse.rb -o /Library/Caches/Homebrew/Formula/suite-sparse.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/cycld/homebrew/41443150d9844e70b55ac626125a276d1cee9fcb/Library/Formula/llvm.rb -o /Library/Caches/Homebrew/Formula/llvm.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/staticfloat/homebrew/652835f810439ffdde237a1818af58140421acd1/Library/Formula/suite-sparse.rb -o /Library/Caches/Homebrew/Formula/suite-sparse.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/cycld/homebrew/41443150d9844e70b55ac626125a276d1cee9fcb/Library/Formula/llvm.rb -o /Library/Caches/Homebrew/Formula/llvm.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/staticfloat/homebrew/652835f810439ffdde237a1818af58140421acd1/Library/Formula/suite-sparse.rb -o /Library/Caches/Homebrew/Formula/suite-sparse.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/cycld/homebrew/41443150d9844e70b55ac626125a276d1cee9fcb/Library/Formula/llvm.rb -o /Library/Caches/Homebrew/Formula/llvm.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/staticfloat/homebrew/652835f810439ffdde237a1818af58140421acd1/Library/Formula/suite-sparse.rb -o /Library/Caches/Homebrew/Formula/suite-sparse.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/cycld/homebrew/41443150d9844e70b55ac626125a276d1cee9fcb/Library/Formula/llvm.rb -o /Library/Caches/Homebrew/Formula/llvm.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/staticfloat/homebrew/652835f810439ffdde237a1818af58140421acd1/Library/Formula/suite-sparse.rb -o /Library/Caches/Homebrew/Formula/suite-sparse.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/cycld/homebrew/41443150d9844e70b55ac626125a276d1cee9fcb/Library/Formula/llvm.rb -o /Library/Caches/Homebrew/Formula/llvm.rb
######################################################################## 100.0%
/usr/bin/curl -qf#LA Homebrew 0.9.2 (Ruby 1.8.7-358; Mac OS X 10.8) https://raw.github.com/staticfloat/homebrew/652835f810439ffdde237a1818af58140421acd1/Library/Formula/suite-sparse.rb -o /Library/Caches/Homebrew/Formula/suite-sparse.rb
######################################################################## 100.0%

My guess is that some kind of cache check is not working, or this is completely unimportant right now, and can wait for later. :)

Contributor

jacknagel commented Aug 19, 2012

I didn't even know that specifying deps that way was possible. Certainly should be fixed.

Contributor

jacknagel commented Aug 19, 2012

The relevant code is here: https://github.com/mxcl/homebrew/blob/master/Library/Homebrew/formula.rb#L365

Looking at it, I think specifying deps this way works mostly by coincidence. That code exists to support brew install URL, and since there is no checksum to verify in that use case, it explicitly blows away any cached formula that exists at the target path. So each time Formula.factory is called with a URL, the object at the URL is going to be downloaded.

In order to properly support depends_on 'url' we would have to intercept URLs earlier in the call chain, and add some sort of deps cache, etc. I don't know if it is worth it or not.

Contributor

adamv commented Aug 23, 2012

Wouldn't adding the const check fix this?

Contributor

jacknagel commented Aug 23, 2012

Hmm, seems like it.

Contributor

jacknagel commented Aug 23, 2012

@ghost ghost assigned jacknagel Aug 23, 2012

@jacknagel jacknagel closed this in 101c418 Aug 23, 2012

@Sharpie Sharpie pushed a commit to Sharpie/homebrew that referenced this issue Sep 11, 2012

@jacknagel jacknagel factory: always check const_defined? before requiring
Specifying dependencies with a URL works, even if by accident, but
factory is called repeatedly on this URL and this results in multiple
downloads of the same file.

Fix this by checking const_defined? here too, and DRY up the code a bit.

Fixes #14285.

Signed-off-by: Jack Nagel <jacknagel@gmail.com>
45f72e9

@snakeyroc3 snakeyroc3 pushed a commit to snakeyroc3/homebrew that referenced this issue Dec 17, 2012

@jacknagel jacknagel factory: always check const_defined? before requiring
Specifying dependencies with a URL works, even if by accident, but
factory is called repeatedly on this URL and this results in multiple
downloads of the same file.

Fix this by checking const_defined? here too, and DRY up the code a bit.

Fixes #14285.

Signed-off-by: Jack Nagel <jacknagel@gmail.com>
ed91e22

@xu-cheng xu-cheng locked and limited conversation to collaborators Feb 16, 2016

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.