Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Homebrew ported to Linux #3153

Closed
wants to merge 21 commits into from
@rubiojr

Hey Max, I've got a preliminary Linux port of homebrew. Tested in Ubuntu. I believe I didn't break anything on the Mac side.
Additionally, I've created a homebrew-linux-formulas repo to host the LInux tested formulas there.

Some notes on the port:

I've added a SystemCommand class in system_command.rb Abstrating every hardcoded command path. i.e.

/usr/bin/which becomes SystemCommand.which

There are system command providers for Mac and Linux currently. I've replaced every hardcoded command path with the SystemCommand equivalent.

I've enclosed every Mac specific bit using:

if SystemCommand.platorm == :mac
end

and provided sane defaults for Linux (mostly).

My current plan is to create a CentOS/RHEL rpm and include Homebrew in the upcomming release of FrameOS Linux.

I love homebrew.

Thanks!

Sergio Rubio added some commits
@mikemcquaid
Owner

The system command stuff seems like a good idea but I think it might be a better idea to have all the if/else stuff in your own long-running fork than in master as we won't be wanting to official support that code (yet at least). Also, SystemCommand.platform doesn't make much sense to me as a naming scheme, platforms don't have much to do with system commands.

@rubiojr

Right, sounds reasonable.

I'd like to work on a better approach to hide the platform specific code (global constants et al).

@mikemcquaid
Owner

Great, thanks.

I mean I'm not the authority on this but I think the goal of any merge requests should be to ease your maintenance of your own branch rather than having Homebrew support Linux natively.

@rubiojr

Sure, no problem.

I'll be working on the port for a while. Do you guys have a mailing list for homebrew? It would be nice to discuss some of this stuff and combining efforts at some point.

Thanks!

@rubiojr

My apologies. Just realized that there is homebrew@librelist.com

@mxcl
Collaborator

At this point I'm pretty interested in looking at how painful it would be to split out the platform specific portions so that the linux side can be merged into mxcl master.

I make no promises, as ultimately, I care about keeping Homebrew great first. But I'm sure there are some things we can do to make your life easier.

You'll need to adjust this to the refactor branch that is coming up though. Let me know how that works out and what we can do to assist.

@rubiojr

Porting was not very painful, half day without knowing the homebrew code base.

A platform abstraction layer and a hardware abstraction layer would be great. I could implement the providers for Linux and do the testing there if you need help. Having a config class instead of global constants would be great also.

Thank for the interest Max.

@mikemcquaid
Owner

Definitely having providers and class inheritance is the best approach and would not mess up the OSX code while allowing you to be mainlined.

Thanks for the work rubiojr!

@mxcl
Collaborator

So once the refactor goes through I suggest we look into abstracting the core, and also something like

def install
  case Homebrew.platform when Homebrew.MACOS
    #foo
  case Homebrew.LINUX
    #bar
  end
end

I'd prefer mac do … end etc. but I think the above is more useful as you can then use postfix ifs with the same logic statements.

@mxcl
Collaborator

Notably, we don't want formula for eg. libz. So we can never have these in mxcl/master.

I don't mind however having linux install steps that differ in formula that we already have in mxcl/master.

However to start with the formula diffs will stay in a separate branch/fork because we don't want to be making a mistake.

@mikemcquaid
Owner

If we don't want formula for stuff that will be needed on Linux, then I'd question some of the support.

I guess I'd see Homebrew on Linux as a way of installing a few useful things into my home directory on a machine that I don't want to use (or don't have) root access.

However, I'm against anything which will make Homebrew suck more for Mac users, something I'm concerned having different install rules might do. What might be an idea is to have a Linux subclass of the existing formula (a bit crazy I know) and just override the install rule. This would allow the subclasses to be in another directory, keep them readable for both platforms, minimise editing error and ensure that fixes from OSX (the majority platform) will go to Linux too.

@adamv
Owner

Having (slightly?) more cross-OS hooks built into Homebrew's core might also make supporting Leopard easier when Lion comes out and we want to deprecate/clean up stuff.

There's also the existing Tiger/PPC branches that might benefit, if they are still being maintained.

@mxcl
Collaborator

Mike, the point IMO of Homebrew on Linux is to complement the package manager that comes with the system. The system mnager can provide libz. Homebrew is for when you need to customise the installation of eg. taglib.

Your concern about making Homebrew suck for our core users is mine to, I wouldn't allow it.

Adam, what hooks do you anticipate us needing?

@mikemcquaid
Owner

In that case, that sounds pretty awesome and, frankly, a tool I'd use when I have to use Linux on other machines that e.g. don't supply me with Git and I don't have root.

@rubiojr

I'd like to add a couple of things to the discussion:

  1. Homebrew is quite useful for infrastructure automation 'a la Chef', but simpler to understand IMHO. Thus, it's a great tool for sysadmin/ops with ruby knowledge.
  2. Platform specific hooks don't need to be always present in a Formula. I'm sure we can implement this in a way that is painless for the Mac user. One way to do it is to have a platform tag in the formula, specifying the platforms the Formula supports. i.e.

    platform :linux, :mac, :foonix

This would be useful also to implement a simple formula manager (in the spirit of yum or apt) that only fetch/search Formulas supported in that platform.

As I stated before, I'd love to offer my help in Linux specific stuff if required.

@rubiojr

Forgot to mention a third point:

Developing Formulas is way faster than creating an RPM/Deb package usually. Learning curve is lower, which is great for ppl not interested in developing platform specific packages. Formulas could easily be cross distro also, which is GREAT.

@mxcl
Collaborator

Great. Well firstly we'll merge the refactor, and then I'll check out what needs to change in the core so that Linux support is easier. Then we'll investigate supporting platform agnostic formula.

We could in theory define the base system that Homebrew requires. Eg Homebrew depends on:

  • libz
  • libxml2
  • etc.

Then we can provide a script that will check for all this stuff in /usr (only supported location). So the user will know that all Homebrew formula will work.

This means we can avoid having to spec out dependencies that we don't already spec.

@rubiojr

Sounds great to me.

Would you like to start a thread in the mailing list and maybe a wiki page with the World Domination Plan?
I'd love to use my GMail to discuss instead of the Github BTS :).

@eregon

@mxcl: Does bringing the Linux port of Homebrew into mxcl/master make sense?

It is something to decide with great caution.

TL;DR merging now seems preemptive and might hurt Homebrew simplicity IMO

I would love too having Homebrew on Linux (whether it manages the basic dependencies itself or not), but I would hate to see complications in the formulae or in the codebase (it is actually much more readable if you do not have to think to others platforms, and Homebrew code is really easy to read, but I think it won't be that simple after).

The conditions you mentioned above just show how bad formula code will become if we don't do this properly.
When adding a new formula, most of us can just test on their own current platform, and do not want to care about other platforms.

I think a long-running fork might be better appropriate (or at least a branch).

I most of all would like to keep the current simplicity of the formulae, and I think there is no way to do that on the first try if we consider multi-platforms (except for very basic formulae).

For example,

/usr/bin/which becomes SystemCommand.which

-  if system "/usr/bin/which -s git"
+  if system "#{SystemCommand.which_s} git"

becomes totally unreadable to me. A which method might work: if which 'git'.
So, it might be as simple as include SystemCommand, but there might be many many cases which are likely harder to handle (like keeping patching clean while platform-dependent).

It seems safer to develop this in parallel, and if it happen to be great, then it would be time to merge into master, but I think this is preemptive to do this now (every project need some time before reaching a satisfying point, just look HomeBrew itself at the beginning).

The first commit is the README, you can read:

With apt, you type apt-get install wget. Now what is happening? With Homebrew
you are running a ruby script. You know what is happening. You can easily and
quickly read the source [...]

I fear the code would not be easy and quick to read, if this merge is done now.

I am just a simple contributor to Homebrew, and just wanted to say what makes Homebrew great (to me at least, but I think to many others too), is something you could hurt easily if this is not done with caution (= not merging until proven as great).

@mxcl
Collaborator

Thanks Eregon, I believe all your concerns are also felt by the collaborators. We won't spoil anything. And there is no desire to merge soon or even for sure.

@eregon

Thanks Eregon, I believe all your concerns are also felt by the collaborators. We won't spoil anything. And there is no desire to merge soon or even for sure.

Thanks for confirming my concerns are ours.
I just wanted to make it bright clear what I love in Homebrew and the impact this could have, and I probably did not understand the whole conversation.

May this not slow down Homebrew on Linux, I wish it too as said upper.

Sergio Rubio added some commits
Sergio Rubio Patches from Kirill Maksimov <https://github.com/netoneko>
* do not fix ruby binary path
b271a0e
Sergio Rubio * updated jruby formula
bbf5eb2
Sergio Rubio Patches from Kirill Maksimov <https://github.com/netoneko>
    * FreeBSD patches :)
b1a0a53
Sergio Rubio * Added BREW_FORMULA_REPOSITORY global (see https://github.com/rubioj…
ab29fc5
Sergio Rubio * backport commit 78f9af0
* Added rubinius formula
9e16849
Sergio Rubio * Add new global constants:
HOMEBREW_GIT_URL = 'https://github.com/rubiojr/homebrew';
HOMEBREW_GIT_USER = 'rubiojr'
HOMEBREW_GIT_BRANCH = 'master'

This makes easier to maintain a homebrew fork
afb48a1
@adamv
Owner

Heads up that we're going to try to merge the "refactor" branch of Homebrew into trunk, which will cause rebase madness for anyone maintaining patched copies on top of master.

@adamv
Owner

Should I link this issue from the Wiki and close it? Or...?

@mxcl
Collaborator

I'm still pretty interested in seeing the diff against refactor. Again apologies for it, but we needed it. I'm sure you'll appreciate it.

@rubiojr

Thanks for the update guys.
I've been swamped lately, but I do have plans to port my changes to the current master this week. I'll let you know how it goes.

@adamv
Owner

Asking again if this should move to the wiki/mailing list; trying to close out stale Issues but of course don't want to kill a conversation.

@mxcl
Collaborator

The problem with mailing list discussions is they aren't linkable and the wiki is not a discussion. I'd rather leave stuff open until something resolves, either we merge or we point to a separate fork.

@mikemcquaid
Owner

How is this looking? If we have a version that works on Linux now I'd be interested in looking at it again.

@abiquo-rpms

Hey Mike,

Actually I started working in forward porting changes from master to this branch a couple of days ago :). I believe I'll have something ready by the end of this week.

I believe there's some (albeit very small) user base here (including my self) so I'm pretty much interested in keeping the port alive.

@rubiojr

That was me actually, using the wrong Github account...

@mxcl
Collaborator

My play-brew2-repo has multi platform support built in.

The ever-unsolved problem is the base dependencies. I have ideas for that, but none of them are all that great.

@mikemcquaid
Owner

I think either use the base dependencies that are on a reasonably default Ubuntu/Fedora/$POPULAR_DISTRO install or bundle them somehow.

It's really tricky as if you have access to the package manager you'd probably use that for most things instead of Homebrew (due to tighter system integration). Then again some distributions won't even install ruby by default. I think relying on ruby and compilers being installed is a reasonable minimum.

@mxcl
Collaborator

It's not a working minimum for most formula that expect all the base libs.

My last thoughts on this were to make brew doctor check for all base libs etc. for Linux. But even this isn't trivial.

I guess it depends on how existing Linux Homebrew users are using it.

@mikemcquaid
Owner

Agreed that it depends. It shouldn't be too bad/hard in the longer term to add optional base dependencies for Linux-only.

@staticfloat

I have a new use case for this which might spark off some interest.

In computer lab environments (in my case, University) we are given shell accounts on linux computers that we can use, however we of course do not have root access, and as such cannot use the package managers bundled with the system. If these were macs, I would just load up brew into a writeable subdirectory and I'd be good to go, but as it stands I have no package manager that can install all the dependencies I need to compile a few select tools.

I'd love to help out testing or even some basic coding for this project, unfortunately my ruby-fu is limited to the few fomulae I've written for brew.

@ndevenish

That is indeed a similar use-case to me; remote servers that I do not and will not get root on, but still need to use tools.

With this in mind, I've been working on updating this linux port to match the current HEAD. I'm no ruby programmer, so it's pretty rough-and-ready, but it seems to work well enough - I wouldn't consider it a candidate for merging at all though, certainly some of the syntax for selection needs to be tidied (At the moment, my formulae that require it say if SystemCommand.platform == :mac along with the main code).

My fork is here: https://github.com/ndevenish/homebrew

Rather than eliminating all the Formulae and building up, I've gone for adding an extra parameter. Example:

class SomeTool < Formula
  url 'http://www.example.com/nowt-2.5.tar.gz'
  homepage 'http://www.example.com'
  platforms :mac, :linux

  def install
    ...
  end
end

Without this line Mac works exactly the same, but linux will warn you and require an extra flag --forcelinux to be passed to brew. This means that all existing formulae can be tried as-is (though might not work). With the line, the rule is absolute i.e.

platforms :mac

will prevent installations on linux (without changing the formulae) and

platforms :linux

will prevent installations on mac (without changing the formulae). I don't have any examples on this use case yet, but it's there if I wanted to give myself i.e. linux only packages.

To get this behaviour, I gave up on calling the system which and wrote a function to do it in utils.rb - for some reason the platform I was on wrote directly to the terminal for failures of which, so even piping to /dev/null didn't work.

Incidentally, for dependencies I have been doing this; it's a bit raw but works for my purposes:

depends_on 'xz' => :build unless which("xz")

which presumably also works on the mac, though it bypasses the dependency checking in that case, and would break if packages needed an explicitly newer version.

@staticfloat
@ndevenish

You certainly can open issues; that's why I turned them on (which IIRC are normally turned off on forks).

@mikemcquaid
Owner

It's worth noting that 9c16145 means that tools like brew audit now run without complaints on Linux.

@brodybits

Can anyone make a quick comparison between https://github.com/rubiojr/homebrew & https://github.com/ndevenish/homebrew and how are they working for FreeBSD?

@samueljohn

To be honest, I think we can close this pull request because the changes are way to massive if I look at the diff now. Homebrew's development pace is very high to keep up with. Further, @rubiojr has not updated his repository since a while.

Don't get me wrong: I'd love a homebrew port for linux and you put a lot of effort into this, @rubiojr!

However, it's better to do smaller pull requests if there is a certain blocker issue for a linux port. Also @ndevenish's port seems more up to date, but I have not tested it.

@ndevenish

"More up to date" meaning, "I made my port later"; You are right about Homebrew moving fast, I got it to a state where it worked for my purposes and totally neglected it since (apart from a few formula-compatibility pull requests).

@samueljohn

I didn't meant to say either port is "better" in some measure. You both did an amazing job.

My point is plainly to close this very pull request, and it's better to add smaller pull requests for specific issues that we can integrate in homebrew to support linux ports.

@ndevenish

Oh, I wasn't implying that at all - merely pointing out that mine is only "more up to date" by virtue of when I did it, rather than any actual effort to keep it up to date.

@mikemcquaid
Owner

Closing for now. I'd agree with @samueljohn that the best way to get this stuff merged is lots of small pull requests. I got brew audit and friends working a few months ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 10, 2010
  1. * Homebrew ported to Linux

    Sergio Rubio authored
  2. * include Linux install instructions

    Sergio Rubio authored
  3. * replaced hardcoded system commands

    Sergio Rubio authored
  4. do not check for macport if Linux

    Sergio Rubio authored
  5. removed untested formulas

    Sergio Rubio authored
  6. * replaced hardcoded commands with SystemCommand

    Sergio Rubio authored
  7. added expat and git formulas

    Sergio Rubio authored
  8. * ignore sync.sh

    Sergio Rubio authored
  9. code clean-up

    Sergio Rubio authored
Commits on Mar 9, 2011
  1. Patches from Kirill Maksimov <https://github.com/netoneko>

    Sergio Rubio authored
    * do not fix ruby binary path
  2. * updated jruby formula

    Sergio Rubio authored
Commits on Mar 11, 2011
  1. Patches from Kirill Maksimov <https://github.com/netoneko>

    Sergio Rubio authored
        * FreeBSD patches :)
  2. * backport commit 78f9af0

    Sergio Rubio authored
    * Added rubinius formula
  3. * Add new global constants:

    Sergio Rubio authored
    HOMEBREW_GIT_URL = 'https://github.com/rubiojr/homebrew';
    HOMEBREW_GIT_USER = 'rubiojr'
    HOMEBREW_GIT_BRANCH = 'master'
    
    This makes easier to maintain a homebrew fork
Commits on Sep 9, 2011
  1. @dannyarcher

    libtiff Formula

    dannyarcher authored
  2. @dannyarcher
  3. @dannyarcher

    added libpng

    dannyarcher authored
Commits on Sep 10, 2011
  1. @rubiojr

    Merge pull request #1 from dannyarcher/Linux

    rubiojr authored
    libtiff Formula
Commits on Sep 11, 2011
  1. @dannyarcher

    gedit as editor\

    dannyarcher authored
Commits on Sep 12, 2011
  1. @rubiojr

    Merge pull request #2 from dannyarcher/Linux

    rubiojr authored
    Thanks Danny, keep them comming!
Something went wrong with that request. Please try again.