FormulaInstaller and dependency improvements #14456

Closed
wants to merge 33 commits into
from

Projects

None yet

8 participants

@jacknagel

I'm working on some improvements to how FormulaInstaller handles dependencies, among several other things, here:

https://github.com/jacknagel/homebrew/compare/deps

There is a lot to digest, and I apologize for that, but all of these improvements touch the same areas of the code, so trying to do them separately and then rebasing them on top of each other would be a nightmare.

Major changes:

  • Dependency and requirement expansion and requirement checking are factored out into separate methods, and build-times deps and requirements are now skipped when installing bottles.

  • FormulaInstaller now takes a lock on the formula being installed and all of its dependencies. This means that two Homebrew processes cannot attempt to install the same formula concurrently. In addition to upgrade and install, uninstall, link, and unlink all attempt to take this lock as well, so if you are installing foo, you can't brew rm foo or any of foo's deps, for example.

    The locking is implemented using flock(), which is only enforced among processes that attempt to take the lock, so there is no need for manual unlocking. As soon as the process that holds the lock terminates, the lock is released.

  • Adam's work on optional/recommended dependencies. Dependencies declared with => :optional are disabled by default but generate a "--with-foo" option that can be used to enable them. Deps declared with => :recommended are enabled but generate a "--without-foo" option to disable them.

  • Formulae can now request dependencies with arbitrary build options:

    depends_on 'foo' => 'with-bar'
    depends_on 'foo' => ['with-bar', 'with-baz']
    depends_on 'foo' => %w[with-bar with-baz]
  • There is special handling for "--universal":

    If a universal build is requested, all dependencies must also be built universal, but only if they declare a universal option. This way, deps that are not compiled but merely installed (precompiled binaries, non-compiled code, other things that don't have a notion of "universal") do not need to be reinstalled. IOW, there is no need for complicated, ugly logic like

    if build.universal?
      depends_on 'foo' => :universal
    else
      depends_on 'foo'
    end

    Rather, it is implied.

@jacknagel

c.f. #3702, #8555, #13923, #14009, #14387.

(the standard "it hasn't been thoroughly tested, so things are liable to break, but I'm working on it" caveats apply.)

@adamv

There are some formulae (I think, still) that unconditionally do ENV.unviersal_binary; are these accounted for?

@jacknagel

Not yet, but eventually they will need to be converted to options.

@adamv

I can start converting them to options.

@2bits

Awesome Jack. Thanks for working on this. I wonder if an Important News feature like brew news that grabs a text file and tells the user when Homebrew is making a major change to something would be useful.

@staticfloat

This is awesome, Jack. I'm glad I saw this because I was about to start trying to put together a system have the option method auto-generate convenience methods. I've opened a pull request about my ideas, but I'll wait until this is merged before trying it so I can put it on top of these changes.

@jacknagel

The trickiest part here is getting the recursive dependency expansion right. Callers of Formula#recursive_dependencies need to be able to filter these deps, but to allow formulae to pass e.g. "with-foo" to a dep that declares depends_on 'foo' => :optional, the filter needs to inspect both the dependency and the dependent. Previously this information was lost during deps expansion.

To remedy this I have implemented da7dbad, which allows Formula#recursive_dependencies to optionally take a block and yield [dependent, dependency] pairs for each dep. The Dependency.prune method can be used to pass on a dep. For example:

f = Formula.factory("foo")
f.recursive_dependencies do |ff, dep|
  Dependency.prune if dep.recommended? and ff.build.without? dep.name
end

Combined with the new semantics of :optional and :recommended deps, this makes it simple for brew missing to inspect the install receipt and omit irrelevant deps.

@jacknagel

Would welcome suggestions on how brew deps should behave now that :optional and :recommended are implemented properly.

Ideally it would reflect this somehow, but we need to keep the standard brew deps output useable in pipelines, etc.

@2bits

Maybe the first one includes recommended, but the 2nd one is the kitchen sink.

brew deps
brew deps --all
@jacknagel

upgrade should now be working in the sense that it should be remembering and passing options to upgraded deps as well.

@mxcl
Homebrew member

Sounds great. Not looked at it for review yet though.

@MikeMcQuaid
Homebrew member

This looks amazing. Will look for review later. While we are talking bottle dependencies we could even avoid complaining/depending on Xcode for bottles too.

@jacknagel

That's implemented, the xcode dependency just needs to be internally tagged as build-time, which I keep forgetting to do.

@MikeMcQuaid
Homebrew member

Schweeeet. Make this a gentle reminder :)

This was referenced Aug 31, 2012
@MikeMcQuaid MikeMcQuaid commented on an outdated diff Aug 31, 2012
Library/Formula/mtr.rb
@@ -18,7 +14,7 @@ def install
--disable-dependency-tracking
--prefix=#{prefix}
]
- args << "--without-gtk" unless ARGV.include? "--with-gtk"
+ args << "--without-gtk" if build.without? 'gtk+'
@MikeMcQuaid
MikeMcQuaid Aug 31, 2012

Maybe just personal preference but I think unless positive is nicer than if negative (if that makes sense).

@MikeMcQuaid MikeMcQuaid and 1 other commented on an outdated diff Aug 31, 2012
Library/Homebrew/dependencies.rb
@@ -71,7 +71,9 @@ def parse_symbol_spec spec, tag
when :x11
X11Dependency.new(tag)
when :xcode
- XCodeDependency.new
+ XcodeDependency.new(tag)
@MikeMcQuaid
MikeMcQuaid Aug 31, 2012

Wants an if bottle I guess.

@jacknagel
jacknagel Aug 31, 2012

That's handled in FormulaInstaller; this is just the straight dependency processing code.

@MikeMcQuaid
MikeMcQuaid Aug 31, 2012

Aye, ok. Apply same comment there :)

@MikeMcQuaid MikeMcQuaid and 2 others commented on an outdated diff Aug 31, 2012
Library/Homebrew/dependencies.rb
@@ -71,7 +71,9 @@ def parse_symbol_spec spec, tag
when :x11
X11Dependency.new(tag)
when :xcode
- XCodeDependency.new
+ XcodeDependency.new(tag)
+ when :clt
+ CLTDependency.new(tag)
@MikeMcQuaid
MikeMcQuaid Aug 31, 2012

Does anything depend on the CLT specifically?

@mxcl
mxcl Aug 31, 2012

I came across a bunch during superenv work, but I fixed most of them (with superenv), however there are some, often because the various things are built with CLT and then CLT is removed, and this leaves them broken, like python-config.

@MikeMcQuaid MikeMcQuaid and 1 other commented on an outdated diff Aug 31, 2012
Library/Homebrew/exceptions.rb
@@ -45,6 +45,17 @@ def initialize name
end
end
+class OperationInProgressError < RuntimeError
+ def initialize name
+ message = <<-EOS.undent
+ Operation already in progress for #{name}
+ Another Homebrew process currently holds the operation lock for #{name}.
@MikeMcQuaid
MikeMcQuaid Aug 31, 2012

"operation lock" could be reworded to be a bit more user friendly. Just simply a "you can't run Homebrew in two places at once" would probably suffice.

@jacknagel
jacknagel Aug 31, 2012

Yeah it got less user friendly when I generalized it from just install to any arbitrary operation that might need a lock, been thinking about how to phrase it better.

@MikeMcQuaid MikeMcQuaid commented on the diff Aug 31, 2012
Library/Homebrew/keg.rb
@@ -50,6 +50,18 @@ def fname
parent.basename.to_s
end
+ def lock
@MikeMcQuaid
MikeMcQuaid Aug 31, 2012

What is the lock actually for? I'm sure it's needed but I'm yet to spot what exactly for.

@jacknagel
jacknagel Aug 31, 2012

So two (or more) Homebrew processes can't try to install the same formula concurrently, or uninstall deps of something that is being installed, or link/unlink at the same time, etc.

@mxcl
mxcl Aug 31, 2012

It stops you from eg. uninstalling something while you are installing it.

@MikeMcQuaid
MikeMcQuaid Aug 31, 2012

Cool. Seems reasonable. A simple "another Homebrew process is using this formula, try again later" or something might work.

@MikeMcQuaid
Homebrew member

Did a review and this looks pretty good to me. Raised a few questions but nothing worrying. Nice one!

@mxcl mxcl commented on the diff Aug 31, 2012
Library/Homebrew/cmd/uninstall.rb
@@ -7,10 +7,12 @@ def uninstall
if not ARGV.force?
ARGV.kegs.each do |keg|
- puts "Uninstalling #{keg}..."
- keg.unlink
- keg.uninstall
- rm_opt_link keg.fname
+ keg.lock do
@mxcl
mxcl Aug 31, 2012

Feels great with this syntax.

@mxcl
Homebrew member

This work deserves some git-tips.

@jacknagel

What I have left here is (a) some regression hunting and (b) clean up the interaction between deps and tabs.

When resolving dependencies, there are three things interacting: the dependency object, the dependent formula, and the dependency's tab, if it is installed already. These three things currently each have a slightly different notion of "options", which makes it difficult to follow what is happening. There's currently at least one bug in this code because of how confusing it is.

So I'm going to attempt to unify the interfaces a bit.

@jacknagel

Latest changes:

  • Since universal is now enforced across the dependency tree, formulae that previously built universal-by-default now sport universal options.

  • Reworked the locking in the installer a bit to avoid what were, in retrospect, some pretty obvious races.

@mxcl @mikemcquaid @adamv @mistydemeo @Sharpie

I've been running this branch during my day-to-day use for a couple of weeks, so I'm aiming to merge it this weekend.

@MikeMcQuaid
Homebrew member

Sounds great!

@MikeMcQuaid
Homebrew member

I'm good to merge when you are (more helpful reply).

@mxcl
Homebrew member

I think I can merge and test some installs tonight.

@jacknagel

Note, while hacking on this yesterday I broke the code that maintains the installable order of deps, so beware.

@jacknagel

..aaand fixed again.

@staticfloat

@jacknagel; How do you request the absence of an option? For example, let's say a formula allowed itself to be compiled against two different "back-end" dependencies (In my case, OpenBLAS or the Accelerate Framework for linear algebra routines). Is there a way to say depends_on foo => !'with-openblas' or something? Would I need to define without-openblas options or something along those lines?

@jacknagel

Yeah, it would need an option to explicitly disable it.

@staticfloat
@jacknagel

I think aborting is the best solution for the time being, though we can look at adding facilities for conflicting options once this lands and the kinks are ironed out.

@josegonzalez
Homebrew member

Would this allow me to specify either one of two packages to fulfill a dependency?

For example, I would like to have a php metapackage. This would be fulfilled by either the php53, php54, or upcoming php55 formulae. It would be useful for installing php binary packages.

@jacknagel

Not really. The best bet is to use a Requirement with a #satisfied? method like

def satisfied?
  %w{php53 php54}.any? do |php|
    Formula.factory(php).installed?
  end
end
@MikeMcQuaid
Homebrew member

Any ETA on this? Just excited rather than trying to put any pressure on you; goodness knows my launchd stuff is delayed :)

@jacknagel

I've been slowly merging some of the less intrusive refactoring pieces, but there are still some design issues surrounding the interaction between BuildOptions and Tab that I want to sort out before merging the new features. So... end of the month? No promises ;)

@ghost Unknown referenced this pull request Nov 6, 2012
Closed

freeDiameter 1.1.5 #15845

@ghost

:recommended does to imply that it is registering it as option, these do:

depends_on 'cmake' => :build
depends_on 'libssh2'
depends_on :mysql => :if_option #name implied (optional)
depends_on :mysql => {:if_option => 'with-mysql'}
depends_on :mysql => {:if_option => ['with-mysql', 'enable-database']}
depends_on :mysql => {:if_option => %w[with-mysql, enable-database]}
depends_on 'libidn' => :unless_option #name implied (recommended)
depends_on 'libidn' => {:unless_option => 'without-libidn'}
depends_on 'libidn' => {:unless_option => ['without-libidn', 'disable-x']}
depends_on 'libidn' => {:unless_option => %w[without-libidn, disable-x]}
@jacknagel

Some of those examples are not syntactically possible. But I prefer the scheme that is already in place anyway.

@jacknagel

Still not possible (foo => bar => baz is not legal in any context, it would have to be foo => { bar => baz }).

I can see the potential value in being able to explicitly name the associated option. But specifying dependencies is limited to a single method call, and we are already pushing the limits of what we can sanely do there. That is probably something better suited to a different (new) DSL element ("features" or "variants"), and is far beyond the scope of what is being done here.

I still prefer :optional and :recommended, especially from an aesthetics standpoint.

I also like the potential for "brew deps --optional" and "brew deps --recommended".

@ghost

This output formating would obsolete need for these switches:

brew deps x
libssh2
mysql (with --with-mysql)
libidn (unless --without-libidn or --disable-x) (installed)
@jacknagel

I'm not going to change it, sorry (note that these auto-generated options show up in brew info and brew options).

@ghost

I can also see use putting things this way:

option 'without-unixodbc' => {
  :if_depends_on => [:mysql, :postgresql, 'mariadb'],
  :unless_depends_on => 'unixodbc'
}, 'Without unix odbc'
@adamv

-1 on all the nesting and verbosity.

@ghost

It's useful for cases when many dependencies are enabled by one option. And it would discourage putting them in the if block, so they can be properly shown by brew deps.

@MikeMcQuaid
Homebrew member

Also -1 on it. We're not getting rid of :recommended or :optional, sorry.

@samueljohn

Not sure, if belongs here, @adamv:
I think the names => :optional and => :recommended could be improved. After having thought about this for a while, I would suggest => :optional and => :default because this reflects more the upcoming behavior.

Perhaps this is now the last chance to discuss this.

@MikeMcQuaid
Homebrew member

Yeh, I'd agree that :default is probably nicer but obviously we'll have to support :recommended for old stuff too. @jacknagel Let me know if and when you want any of this reviewed.

@jacknagel

Will do. Made a bunch of progress over the weekend, gradually checking things off my todo list here.

@samueljohn

Perhaps brew audit should check if the requested options are actually available at the other formula, else I fear, we might get broken builds if we rename/remove an option.

The reverse lookup would even be interesting: If I rename an option on formula x, brew audit x should tell me that formula y depends_on that option. But this code will have a bad performance, since it needs to scan through all other formulae...

@jacknagel

@samueljohn Agreed.

The reverse lookup could just be an external tool that we run occasionally to make sure options don't go out of sync.

@jacknagel

FYI the diff here is [expletive] huge because I had to update the vendored copy of multi_json to get it to respect the to_json method on Option objects. I plan to push that commit separately so it should be somewhat easier to review.

@adamv

Can the json update be pulled into core now?

Update: I fail at reading comprehension.

@MikeMcQuaid
Homebrew member

When we have CI (almost there, honest) we can run brew audit on every commit.

@adamv adamv referenced this pull request Jan 19, 2013
Closed

Wine upgraded to 1.5.22 #17185

@jacknagel

I think this is ready for some review/testing. My "after_deps" branch has the contents of this pull request plus some formula updates to utilize the new features:

https://github.com/jacknagel/homebrew/compare/after_deps

@MikeMcQuaid
Homebrew member

Looks good to me. Any particular testing requests?

@samueljohn
@jacknagel

Probably test the main features: that it passes options to deps, that it skips build-time deps when installing bottles, that it generates options for :optional and :recommended deps. There are some examples you can use for testing in the after_deps branch: https://github.com/jacknagel/homebrew/compare/after_deps

Thanks.

@ghost

+1 for :default.

@adamv

Being a spoiler: how is :default different than omitting :default? Ie, how does it communicate when a formula creator should apply it to a dep instead of not?

@ghost
depends_on 'x' => :optional
depends_on 'x' => :optional, :default => :yes # = :recommended

:optional means that option will be created, :default sets whether it will be on or off by default.

@jacknagel

Sorry, but this communicates nothing but complete confusion to users:

depends_on 'foo' => :optional, :default => :no

I don't even understand what it should mean.

I'm not going to change the flags, so let's not clutter up the thread with any more pointless discussion, thanks.

@jacknagel

I'm feeling ready to merge this, so someone yell if there are reasons I should wait.

@MikeMcQuaid
Homebrew member

Not had a chance to test but go for it.

jacknagel and others added some commits Jan 23, 2013
@jacknagel jacknagel Clean up Tab creation 76c01e9
@jacknagel jacknagel Move option comparison into BuildOptions 8d0c65e
@jacknagel jacknagel Options can be used interchangeably with Strings
We want to be able to use Option objects in place of strings and have
this be transparent. Defining to_str means that methods like
Kernel#system and Kernel#exec will be able to perform an implicit
conversion.
56f72c4
@jacknagel jacknagel Options can be dumped as JSON 7a2e4e8
@jacknagel jacknagel Refactor option handling internals
Currently we handle options in several ways, and it is hard to remember
what code needs an option string ("--foo"), what needs only the name
("foo") and what needs an Option object.

Now that Option objects can act as strings and be converted to JSON, we
can start using them instead of passing around strings between Formula
objects, Tab objects, and ARGV-style arrays.

The Options class is a special collection that can be queried for the
inclusion of options in any form: '--foo', 'foo', or Option.new("foo").
8f5ea8e
@jacknagel jacknagel BuildOptions: check has_option? for universal and 32-bit 91f15da
@jacknagel jacknagel BuildOptions: simplify setting description 7e5a869
@jacknagel jacknagel Move BuildOptions to a separate file 36939b4
@jacknagel jacknagel Tab#with? to mirror BuildOptions#with?
Eventually a common interface could be factored out into a module, but
for now this will suffice.
edffb4f
@jacknagel jacknagel Tests for BuildOptions a875e09
@jacknagel jacknagel Formula#recursive_dependencies
This behaves like recursive_deps, but the resulting list consists of
Dependency objects instead of Formula objects. The list maintains the
installable order property of recursive_deps.

While in the area, add some comments clarifying the purpose of related
methods.
b898f23
@jacknagel jacknagel Dependency#to_formula and associated helpers 778a949
@jacknagel jacknagel FormulaInstaller: factor out requirement checking 6c2b124
@jacknagel jacknagel FormulaInstaller: factor out dependency installation 0579f16
@jacknagel jacknagel FormulaInstaller: skip build-time deps for bottles fffc46c
@jacknagel jacknagel FormulaInstaller: implement installation locks
FormulaInstaller now attempts to take a lock on a "foo.brewing" file for
the formula and all of its dependencies before attempting installation.

The lock is an advisory lock implemented using flock(), and as such it
only locks out other processes that attempt to take the lock. It also
means that it is never necessary to manually remove the lock file,
because the lock is not enforced by I/O.

The uninstall, link, and unlink commands all learn to respect this lock
as well, so that the installation cannot be corrupted by a concurrent
Homebrew process, and keg operations cannot occur simultaneously.
728fcfa
@adamv adamv Formula::finalize_dsl 8f7c36a
@adamv adamv Add support for optional and recommended deps
Optional deps are not installed by default but generate a corresponding
"with-foo" option for the formula. Recommended deps _are_ installed by
default, and generate a corresponding "without-foo" option.
5e365dd
@adamv adamv Let option override with/without descriptions 48da5f1
@jacknagel jacknagel Dependency.expand_dependencies
Move Formula.expand_dependencies into the Dependency class, and extend
it to allow arbitrary filters to be applied when enumerating deps.

When supplied with a block, expand_dependencies will yield a [dependent,
dependency] pair for each dependency, allowing callers to filter out
dependencies that may not be applicable or useful in a given situation.

Deps can be skipped by simple calling Dependency.prune in the block,
e.g.:

  Dependency.expand_dependencies do |f, dep|
    Dependency.prune if dep.to_formula.installed?
  end

The return value of the method is the filtered list.

If no block is supplied, a default filter that omits optional or
recommended deps based on what the dependent formula has requested is
applied.

Formula#recursive_dependencies is now implemented on top of this,
allowing FormulaInstaller to exact detailed control over what deps are
installed. `brew missing` and `brew upgrade` can learn to use this to
apply the installed options set when expanding dependencies.

Move Formula.expand_deps and Formula#recursive_deps into compat, because
these methods do not respect the new optional and recommended tags and
thus should no longer be used.
feb6082
@jacknagel jacknagel FormulaInstaller: make tab an attr, but never a parameter d2c8424
@jacknagel jacknagel FormulaInstaller: don't install bottle if options were passed 05542a3
@jacknagel jacknagel FormulaInstaller: allow formulae to pass options to deps
Formulae can now pass build options to dependencies. The following
syntax is supported:

  depends_on 'foo' => 'with-bar'
  depends_on 'foo' => ['with-bar', 'with-baz']

If a dependency is already installed but lacks the required build
options, an exception is raised. Eventually we may be able to just stash
the existing keg and reinstall it with the combined set of used_options
and passed options, but enabling that is left for another day.
64a4c0f
@jacknagel jacknagel FormulaInstaller: construct new ARGV from an Options collection
The array of options that is passed to the spawned build process is a
combination of the current ARGV, options passed in by a dependent
formula, and an existing install receipt. The objects that are
interacting here each expect the resulting collection to have certain
properties, and the expectations are not consistent.

Clear up this confusing mess by only dealing with Options collections.
This keeps our representation of options uniform across the codebase.

We can remove BuildOptions dependency on HomebrewArgvExtension, which
allows us to pass any Array-like collection to Tab.create. The only
other site inside of FormulaInstaller that uses the array is the #exec
call, and there it is splatted and thus we can substitute our Options
collection there as well.
f6d54c0
@jacknagel jacknagel missing: ignore unused optional and recommended deps a063cad
@jacknagel jacknagel Replace usages of recursive_deps with recursive_dependencies 8488b7f
@jacknagel jacknagel upgrade: offload dependency expansion to FormulaInstaller
Now that FormulaInstaller does dependency expansion the _right_ way,
avoid duplicating the logic in upgrade. Instead, offload it to the
installer, which will generate an exception in check_install_sanity that
we can safely ignore when formulae in the outdated list are upgraded as
part of the dependency tree of another outdated formula.
40fb7df
@jacknagel jacknagel uses: utilize modern dependency API 2ae32f7
@samueljohn samueljohn Add "depends_on :clt" 845a1ba
@jacknagel jacknagel Tag Xcode and CLT requirements as build-time
This way they can be skipped when installing bottles.
59a91df
@jacknagel jacknagel Fix Dependencies -> Array conversion 81e4b46
@jacknagel jacknagel upgrade: use standard Tab accessor
Yes, the formula object does refer to a version that has not yet been
installed, but we were not looking into Formula#prefix, but #linked_keg,
which is version agnostic (since the original patch was committed, we
Tab#for_formula learned to look into #opt_prefix as well). The rest of
the logic is already embedded in the Tab accessors.
e7f3602
@jacknagel jacknagel audit: warn about nonexistent options passed to deps 5808064
@jacknagel

This is merged. I am going to bump the version to 0.9.4 but I am going to wait a day or two to allow time to address any issues.

@jacknagel jacknagel closed this Jan 26, 2013
@samueljohn
@MikeMcQuaid
Homebrew member

Amazing. Yeh, bump the version.

@staticfloat

Excellent work, Jack. You should be proud about this. :)

@dholm dholm added a commit to dholm/homebrew that referenced this pull request Jan 31, 2013
@dholm dholm Wine --devel upgraded to 1.5.22
This version includes the new experimental Mac driver which means it can be
run without X11. As the driver is still incomplete and buggy X11 is still the
default and instructions on how to switch are included in caveats.

Also, supports building a non-universal binary (i.e. only win64) support and
checking for universal dependencies thanks to #14456.
21f60f6
@adamv adamv added a commit that referenced this pull request Jan 31, 2013
@dholm dholm Wine --devel upgraded to 1.5.22
This version includes the new experimental Mac driver which means it can be
run without X11. As the driver is still incomplete and buggy X11 is still the
default and instructions on how to switch are included in caveats.

Also, supports building a non-universal binary (i.e. only win64) support and
checking for universal dependencies thanks to #14456.

Closes #17185.

Signed-off-by: Adam Vandenberg <flangy@gmail.com>
7a80519
@n1k0 n1k0 added a commit that referenced this pull request Feb 13, 2013
@dholm dholm Wine --devel upgraded to 1.5.22
This version includes the new experimental Mac driver which means it can be
run without X11. As the driver is still incomplete and buggy X11 is still the
default and instructions on how to switch are included in caveats.

Also, supports building a non-universal binary (i.e. only win64) support and
checking for universal dependencies thanks to #14456.

Closes #17185.

Signed-off-by: Adam Vandenberg <flangy@gmail.com>
4706de3
@rajeeja rajeeja pushed a commit that referenced this pull request Apr 19, 2013
@dholm dholm Wine --devel upgraded to 1.5.22
This version includes the new experimental Mac driver which means it can be
run without X11. As the driver is still incomplete and buggy X11 is still the
default and instructions on how to switch are included in caveats.

Also, supports building a non-universal binary (i.e. only win64) support and
checking for universal dependencies thanks to #14456.

Closes #17185.

Signed-off-by: Adam Vandenberg <flangy@gmail.com>
7836be7
@xu-cheng xu-cheng locked and limited conversation to collaborators Feb 16, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.