Extract something like Resource from Formula.

[adamv]

I want to do something about how subformulae are currently handled.

Apktool defines a subformula:

class ApktoolExecutables < Formula
  url 'https://android-apktool.googlecode.com/files/apktool-install-macosx-r05-ibot.tar.bz2'
  sha1 'c2fb262760ccd27530e58ccc4bbef4d4a7b0ab39'
end

and installs it thus:

    ApktoolExecutables.new.brew do |f|
      libexec.install 'aapt', 'apktool'
    end

The formula machinery is used here to download, checksum, and unpack what is essentially a "resource" full of files, not a formula in itself.

Subformulae basically don't have their own def install methods, since they don't install to their own place in the Cellar.

Suggest that we extract some sort of "Resource" class from Formula for supporting subformulae.

We then enhance the formula DSL to be able to define resources, something like:

class Apktool < Formula
  homepage 'http://android-apktool.googlecode.com/'
  url 'https://android-apktool.googlecode.com/files/apktool1.5.2.tar.bz2'
  sha1 '2dd828cf79467730c7406aa918f1da1bd21aaec8'

  resource 'executables' do
    url 'https://android-apktool.googlecode.com/files/apktool-install-macosx-r05-ibot.tar.bz2'
    sha1 'c2fb262760ccd27530e58ccc4bbef4d4a7b0ab39'
  end

  def install
    ....
    resources.executables.do
      libexec.install 'aapt', 'apktool'
    end
    ....
  end
end

We can then clean subformulae from the cache since they are declared in the formula.

[MikeMcQuaid]

Seems sensible. Would be good if e.g. brew fetch and friends are all aware of resources too.

[adamv]

Seems SoftwareSpec can be reused or extended for this.

[adamv]

Trying to add a base class to Formula gives me superclass mismatch for class Formula when doing for instance brew edit.

[adamv]

Oh FFS how many times we we reopen the Formula class?

[jacknagel]

Why does Formula need a superclass :(

[samueljohn]

I like the resource idea very much. Because right now it is a PITA to write a subformula and copy stuff around. I find it hard if the main formula needs a resource to be in a subdir.

There should be a

resource('fonts').unpack

which extracts it directly in the current working dir of the main formula.

And if a separate tempdir is wanted, I'd throw the following syntax suggestion into the round:

resource('executables') do
  libexec.install 'aapt', 'apktool'
end

[adamv]

It doesn't need a superclass, I can do it as a mixin. Possibly once I get through all this it can just be a component object, but I can't see that far through the code yet.

[adamv]

Work in progress: https://github.com/adamv/homebrew/compare/sub

TODO: (A) fix the naming of downloaded resources (B) swap over formulae over (C) make other commands (fetch, etc.) resource aware

Accepting feedback now of course.

[adamv]

TODO (D) Erlang switches between stable and head of 2 different doc packages; don't want brew fetch to download all 4, just the active 2.

[adamv]

cfitsio has a conditional resource; thinking about this now due to the discussion about spec-based deps

[adamv]

To find files with subformulae:

grep -oc "< Formula" Library/Formula/* | grep -v :[01]

[adamv]

I'm unsure what naming convention to use for fetched resources to ensure proper uniqueness, but so far I have:

$ brew fetch git
==> Downloading http://git-core.googlecode.com/files/git-1.8.3.1.tar.gz
Already downloaded: /usr/local/homebrew-cache/git-1.8.3.1.tar.gz
SHA1: 32562a231fe4422bc033bf872fffa61f41ee2669
SHA256: ea2dde231b3f659da92dd0afc092a16720d63f1206cdcb271920e88f328bda83
Resource: html
==> Downloading http://git-core.googlecode.com/files/git-htmldocs-1.8.3.1.tar.gz
Already downloaded: /usr/local/homebrew-cache/git--html-1.8.3.1.tar.gz
SHA1: 94d48f6f8684aec851124e7d0b835b338a9187ad
SHA256: 0be04c5efc831bbc7b1d80707d953bf7e9d500037eb202f6fd042c0fff0538ea
Resource: man
==> Downloading http://git-core.googlecode.com/files/git-manpages-1.8.3.1.tar.gz
Already downloaded: /usr/local/homebrew-cache/git--man-1.8.3.1.tar.gz
SHA1: 0cd759579d4bd75f1cf1ba073b1ab96c49390426
SHA256: ab574826814b9469c91cdb99ecf5ad66228d0e30197e54881a4fe8ce0a17caba

thoughts?

[MikeMcQuaid]

I wouldn't print the Resource: html bit and git-html seems a bit nicer than git--html. Could always do git-html-1.8.3.1.resource.tar.gz.

[adamv]

(The Resource:... is debug output.)

[adamv]

It's slightly tough to get the .resource at the end before the extension since I'm borrowing existing download strategies which do their own naming; could work in .res or -res in the middle.

[MikeMcQuaid]

or resource- at the beginning? Cool that it's debug input.

[adamv]

This code is at the point where resources are fetched and cached.

I haven't converted all subformulae over to resources because some of them still seem to make sense as formulae, at least so far.

Accepting comments on what to tackle next here.

[jacknagel]

I wonder if it makes sense to implement SoftwareSpec in terms of Resource instead of the other way around, keeping (or moving) all the download strategy/checksum verification/caching in Resource. I say this because after this lands I'm going to work on pushing dependencies into SoftwareSpec.

[adamv]

That probably makes sense; trying to do the minimal amount here to get this working.

Between SoftwareSpec Resource and Formula some methods definitely need to move around.

[adamv]

Wondering if DownloadStrategy needs a property for the "downloaded name prefix" so that I can grab that for use in creating resource names.

[adamv]

Another thought, we probably want to be able to fetch resources in parallel with a formula install that we'll need later:

def install
  resource('something').get
  ... make install stuff that takes a while ...
  resource('something').download do ... could already be cached, or will wait until the download completes ... end
end

But of course I know nothing about concurrency in Ruby.

[adamv]

TODO - we can now audit resource URLs, so add that to brew audit.

[adamv]

Rebased on master; looking for some advice on what to do next with this.

[adamv]

Closing this in favor of the pull request.