Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.
Sign upelasticsearch 2.0.0 #45644
Conversation
jasontedor
referenced this pull request
Nov 3, 2015
Closed
Homebrew-installed Elasticsearch fails on startup #14424
dakrone
referenced this pull request
Nov 3, 2015
Merged
Change tar extension back to tar.gz instead of tgz #14493
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 3, 2015
Contributor
That brew audit is failing is known to me. The issue is that Homebrew is requiring the jar files to be in libexec. However, enhancements in the Elasticsearch security manager require that the jar files be under lib; Elasticsearch will fail on startup with security exceptions if they are put in libexec. Together we'll need to find a way to work around this.
|
That |
MikeMcQuaid
reviewed
Nov 3, 2015
| @@ -128,6 +106,6 @@ def plist; <<-EOS.undent | ||
| end | ||
| test do | ||
| system "#{bin}/plugin", "--list" | ||
| system "#{bin}/plugin", "list" |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 3, 2015
Member
Can we add another test that actually starts elasticsearch and checks it's working with curl?
MikeMcQuaid
Nov 3, 2015
Member
Can we add another test that actually starts elasticsearch and checks it's working with curl?
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 3, 2015
Member
We can't install the .jar files to lib. We need to work out another location to install them to (i.e. libexec).
|
We can't install the |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
@MikeMcQuaid For Elasticsearch 2.0.0, no such change is possible. The jar files must be in lib, no exceptions; Elasticsearch 2.0.0 will intentionally fail on startup if the jar files are moved to any other location (even with an ES_CLASSPATH configuration change). This is part of the aforementioned security enhancements that have been made to lock down Elasticsearch installations.
Looking beyond Elasticsearch 2.0.0, it is extremely unlikely that a change will be made to this policy. Elasticsearch now runs under the Java Security Manager and has a very aggressive security policy to make the system harder to exploit. Part of this includes explicitly enforcing the jar files to be in lib as part of the dynamic generation of file permissions (cf. o.e.e.Environment and o.e.b.Security).
Can you help me better understand the reason that Homebrew has this requirement for jar files, especially since this deviates from the standard file system hierarchy (lib is for libraries, libexec is for helper executables)?
|
@MikeMcQuaid For Elasticsearch 2.0.0, no such change is possible. The jar files must be in Looking beyond Elasticsearch 2.0.0, it is extremely unlikely that a change will be made to this policy. Elasticsearch now runs under the Java Security Manager and has a very aggressive security policy to make the system harder to exploit. Part of this includes explicitly enforcing the jar files to be in Can you help me better understand the reason that Homebrew has this requirement for jar files, especially since this deviates from the standard file system hierarchy ( |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 4, 2015
Member
Note Debian also doesn't install these to lib: https://packages.debian.org/sid/all/libelasticsearch1.7-java/filelist
The message is fairly clear: https://github.com/Homebrew/homebrew/blob/ceab5e292c0420eaae02e3a4f9d84d4cc5d81aec/Library/Homebrew/formula_cellar_checks.rb#L41-L55
We symlink these files to lib from the Cellar. We cannot symlink all these jars because if other formula install the same files with the same names then they conflict. I don't think it's fair for Elasticsearch to expect us to change our policies just for them.
|
Note Debian also doesn't install these to The message is fairly clear: https://github.com/Homebrew/homebrew/blob/ceab5e292c0420eaae02e3a4f9d84d4cc5d81aec/Library/Homebrew/formula_cellar_checks.rb#L41-L55 We symlink these files to |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
avinassh
Nov 4, 2015
Is there any temp fix for the time being? I have installed elasticsearch using homebrew and the version is 2.0 (I don't know how, since formula still says 1.7). Anyways for the time being if you can tell me how do I get the v2 working, that would be great.
$elasticsearch
java.io.IOException: Resource not found: "org/joda/time/tz/data/ZoneInfoMap" ClassLoader: sun.misc.Launcher$AppClassLoader@43c0ae76
at org.joda.time.tz.ZoneInfoProvider.openResource(ZoneInfoProvider.java:210)
at org.joda.time.tz.ZoneInfoProvider.<init>(ZoneInfoProvider.java:127)
at org.joda.time.tz.ZoneInfoProvider.<init>(ZoneInfoProvider.java:86)
at org.joda.time.DateTimeZone.getDefaultProvider(DateTimeZone.java:514)
at org.joda.time.DateTimeZone.getProvider(DateTimeZone.java:413)
at org.joda.time.DateTimeZone.forID(DateTimeZone.java:216)
at org.joda.time.DateTimeZone.getDefault(DateTimeZone.java:151)
at org.joda.time.chrono.ISOChronology.getInstance(ISOChronology.java:79)
at org.joda.time.DateTimeUtils.getChronology(DateTimeUtils.java:266)
at org.joda.time.format.DateTimeFormatter.selectChronology(DateTimeFormatter.java:968)
at org.joda.time.format.DateTimeFormatter.printTo(DateTimeFormatter.java:672)
at org.joda.time.format.DateTimeFormatter.printTo(DateTimeFormatter.java:560)
at org.joda.time.format.DateTimeFormatter.print(DateTimeFormatter.java:644)
at org.elasticsearch.Build.<clinit>(Build.java:51)
at org.elasticsearch.node.Node.<init>(Node.java:135)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2015-11-04 13:20:26,431][INFO ][node ] [Abigail Brand] version[2.0.0], pid[3867], build[de54438/2015-10-22T08:09:48Z]
[2015-11-04 13:20:26,433][INFO ][node ] [Abigail Brand] initializing ...
Exception in thread "main" java.lang.IllegalStateException: failed to load bundle [] due to jar hell
Likely root cause: java.security.AccessControlException: access denied ("java.io.FilePermission" "/usr/local/Cellar/elasticsearch/2.0.0/libexec/antlr-runtime-3.5.jar" "read")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:372)
at java.security.AccessController.checkPermission(AccessController.java:559)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
at java.util.zip.ZipFile.<init>(ZipFile.java:206)
at java.util.zip.ZipFile.<init>(ZipFile.java:145)
at java.util.jar.JarFile.<init>(JarFile.java:154)
at java.util.jar.JarFile.<init>(JarFile.java:91)
at org.elasticsearch.bootstrap.JarHell.checkJarHell(JarHell.java:173)
at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:340)
at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:113)
at org.elasticsearch.node.Node.<init>(Node.java:144)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Refer to the log for complete error details.
avinassh
commented
Nov 4, 2015
|
Is there any temp fix for the time being? I have installed elasticsearch using homebrew and the version is 2.0 (I don't know how, since formula still says 1.7). Anyways for the time being if you can tell me how do I get the v2 working, that would be great. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
rmuir
Nov 4, 2015
We symlink these files to lib from the Cellar. We cannot symlink all these jars because if other formula install the same files with the same names then they conflict. I don't think it's fair for Elasticsearch to expect us to change our policies just for them.
We aren't changing to libexec. I don't think its fair for homebrew to expect us to change our code just for them.
rmuir
commented
Nov 4, 2015
We aren't changing to libexec. I don't think its fair for homebrew to expect us to change our code just for them. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
Note Debian also doesn't install these to lib: https://packages.debian.org/sid/all/libelasticsearch1.7-java/filelist
- That's a pre-2.0.0 version (1.7) of Elasticsearch but the Java Security Manager usage and
librequirement were not put in place until version 2.0.0 of Elasticsearch. - That's only the libelasticsearch package, which itself depends on many other lib* packages (which all get installed in
/usr/share/java). - The officially supported Debian package for Elasticsearch 2.0.0 puts all of the jars in
lib(/usr/share/elasticsearch/lib).
The message is fairly clear: https://github.com/Homebrew/homebrew/blob/ceab5e292c0420eaae02e3a4f9d84d4cc5d81aec/Library/Homebrew/formula_cellar_checks.rb#L41-L55
I already understand that.
We symlink these files to lib from the Cellar. We cannot symlink all these jars because if other formula install the same files with the same names then they conflict.
These days, almost all jars come with versions in their names so the conflicts that you speak of are effectively nonexistent. And having different versions of the same artifacts in /usr/local/lib only becomes a conflict if someone tries to add all of /usr/local/lib to their classpath or multiple versions of the same artifact, both of which would are bad practices. No installed package should do that, and no developer should be doing that. Symlinking all the jars from all the installed packages into one location is just setting up for trouble.
What are the reasons for doing this? Maybe the answer is as simple as not symlinking jars from the package-installed lib folders?
I don't think it's fair for Elasticsearch to expect us to change our policies just for them.
Elastic hasn't asked for anything. The way I see the conversation right now is merely exploring the issue and seeing what options, if any, fall out.
I already understand that.
These days, almost all jars come with versions in their names so the conflicts that you speak of are effectively nonexistent. And having different versions of the same artifacts in What are the reasons for doing this? Maybe the answer is as simple as not symlinking jars from the package-installed
Elastic hasn't asked for anything. The way I see the conversation right now is merely exploring the issue and seeing what options, if any, fall out. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
catalandres
Nov 4, 2015
Contributor
@avinassh Have you tried brew edit elasticsearch and then do the edits in the diff corresponding to the commit? I did that and ES compiled and started without complaining. It's a dirty hack and I have not verified that the software is 100% operational.
|
@avinassh Have you tried |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
avinassh
commented
Nov 4, 2015
|
@catalandres Worked! Thanks. I didn't think of it earlier. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 4, 2015
Member
We aren't changing to libexec. I don't think its fair for homebrew to expect us to change our code just for them.
You'll note I'm not submitting a PR or issue to your project asking you to change your code. I'm happy for Homebrew users to remain on Elasticsearch 1.x until this is resolved.
The officially supported Debian package for Elasticsearch 2.0.0 puts all of the jars in lib (/usr/share/elasticsearch/lib).
That's not the package Debian includes. Similarly, Elasticsearch could distribute their own Homebrew formula which does not follow our audit rules. Like the Debian package it won't be installed if you brew install elasticsearch without adding an external repository, though.
These days, almost all jars come with versions in their names so the conflicts that you speak of are effectively nonexistent. And having different versions of the same artifacts in /usr/local/lib only becomes a conflict if someone tries to add all of /usr/local/lib to their classpath or multiple versions of the same artifact, both of which would are bad practices.
That's not the case because of how Homebrew works. The contents of Cellar/elasticsearch/2.0.0/lib are always symlinked into lib. If another package also wants to symlink the same files to lib: there is a conflict. We have a conflict_with DSL we use to indicate this but it just means some software can't be installed at the same time. We resolve this by installing things to libexec, as we have done with Elasticsearch in the past.
Symlinking all the jars from all the installed packages into one location is just setting up for trouble.
That's why we don't allow jars in lib. That's exactly how Homebrew works and has always worked.
What are the reasons for doing this? Maybe the answer is as simple as not symlinking jars from the package-installed lib folders?
That's not simple. It would require fundamentally changing the way Homebrew installs software. This would potentially break other software that implicitly relies on the way Homebrew currently installs software, require us to write and test a bunch of code and discuss how best to deal with the new problems that would result.
You'll note I'm not submitting a PR or issue to your project asking you to change your code. I'm happy for Homebrew users to remain on Elasticsearch 1.x until this is resolved.
That's not the package Debian includes. Similarly, Elasticsearch could distribute their own Homebrew formula which does not follow our
That's not the case because of how Homebrew works. The contents of
That's why we don't allow jars in
That's not simple. It would require fundamentally changing the way Homebrew installs software. This would potentially break other software that implicitly relies on the way Homebrew currently installs software, require us to write and test a bunch of code and discuss how best to deal with the new problems that would result. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
You'll note I'm not submitting a PR or issue to your project asking you to change your code.
That's how open source works.
I'm happy for Homebrew users to remain on Elasticsearch 1.x until this is resolved.
I don't understand this as that doesn't seem in the best interest of Homebrew's users.
That's not the package Debian includes.
What does that matter?
Elasticsearch could distribute their own Homebrew formula which does not follow our
auditrules. Like the Debian package it won't be installed if youbrew install elasticsearchwithout adding an external repository, though.
That's the most likely outcome of this, as well as opening a pull request to remove the Elasticsearch formula from the homebrew repository since it's currently broken for latest stable and HEAD and not fixable given the conversation here.
That's why we don't allow jars in
lib. That's exactly how Homebrew works and has always worked.
You don't allow them because symlinking would be setting up for trouble. So just don't symlink them?
That's not simple. It would require fundamentally changing the way Homebrew installs software. This would potentially break other software that implicitly relies on the way Homebrew currently installs software, require us to write and test a bunch of code and discuss how best to deal with the new problems that would result.
How is it not simple to exclude jars from being symlinked if they are in lib? It breaks nothing that exists today because they aren't allowed in lib today.
Thank you for taking the time to review the pull request and your comments on adding a test. Sincerely, this has been helpful and enlightening.
That's how open source works.
I don't understand this as that doesn't seem in the best interest of Homebrew's users.
What does that matter?
That's the most likely outcome of this, as well as opening a pull request to remove the Elasticsearch formula from the homebrew repository since it's currently broken for latest stable and
You don't allow them because symlinking would be setting up for trouble. So just don't symlink them?
How is it not simple to exclude jars from being symlinked if they are in Thank you for taking the time to review the pull request and your comments on adding a test. Sincerely, this has been helpful and enlightening. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
Is there any temp fix for the time being?
@avinassh Yup, as @catalandres mentioned (thanks!), you can apply the changes in this pull request locally. That will get you up and running for now. We will very likely be establishing an official Elasticsearch tap going forward. Whatever the outcome, you can track the status of this on elastic/elasticsearch#14424.
I don't know how, since formula still says 1.7
There was a very brief period of time where the official formula was pointing at 2.0.0.
@avinassh Yup, as @catalandres mentioned (thanks!), you can apply the changes in this pull request locally. That will get you up and running for now. We will very likely be establishing an official Elasticsearch tap going forward. Whatever the outcome, you can track the status of this on elastic/elasticsearch#14424.
There was a very brief period of time where the official formula was pointing at 2.0.0. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 4, 2015
Member
You'll note I'm not submitting a PR or issue to your project asking you to change your code.
That's how open source works.
It's worth noting here that Homebrew is exclusively run by volunteers in their free time and that's not the case for Elastic.
I'm happy for Homebrew users to remain on Elasticsearch 1.x until this is resolved.
I don't understand this as that doesn't seem in the best interest of Homebrew's users.
With respect, I've been a Homebrew maintainer for just under 7 years and I think I've got a pretty good grasp of what's in our users best interests. To merge this as-is will cause additional issues. None of our audit rules are there for no reason, they have all been added to prevent previous issues from recurring.
That's not the package Debian includes.
What does that matter?
My point there was that it's unclear you've convinced other package managers to package Elasticsearch in the way you want us to (e.g. put all the jars in lib).
Elasticsearch could distribute their own Homebrew formula which does not follow our audit rules. Like the Debian package it won't be installed if you brew install elasticsearch without adding an external repository, though.
That's the most likely outcome of this, as well as opening a pull request to remove the Elasticsearch formula from the homebrew repository since it's currently broken for latest stable and HEAD and not fixable given the conversation here.
It's not currently broken, it's just not up-to-date. It is fixable but requires us to find us a middle solution together.
That's why we don't allow jars in lib. That's exactly how Homebrew works and has always worked.
You don't allow them because symlinking would be setting up for trouble. So just don't symlink them?That's not simple. It would require fundamentally changing the way Homebrew installs software. This would potentially break other software that implicitly relies on the way Homebrew currently installs software, require us to write and test a bunch of code and discuss how best to deal with the new problems that would result.
How is it not simple to exclude jars from being symlinked if they are in lib? It breaks nothing that exists today because they aren't allowed in lib today.How is it not simple to exclude jars from being symlinked if they are in lib? It breaks nothing that exists today because they aren't allowed in lib today.
Technically the code is simple to just special-case this one thing. I hope you appreciate that with >3000 formulae writing making these type changes based on one formula quickly becomes unsustainable.
I'm still not sure I completely understand what the requirement is here with lib. Would it be possible to use symlinks and/or a wrapper script and keep installing to libexec? If not, can you explain why this is not possible so we can try and work out another solution?
It's worth noting here that Homebrew is exclusively run by volunteers in their free time and that's not the case for Elastic.
With respect, I've been a Homebrew maintainer for just under 7 years and I think I've got a pretty good grasp of what's in our users best interests. To merge this as-is will cause additional issues. None of our
My point there was that it's unclear you've convinced other package managers to package Elasticsearch in the way you want us to (e.g. put all the jars in
It's not currently broken, it's just not up-to-date. It is fixable but requires us to find us a middle solution together.
Technically the code is simple to just special-case this one thing. I hope you appreciate that with >3000 formulae writing making these type changes based on one formula quickly becomes unsustainable. I'm still not sure I completely understand what the requirement is here with |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
With respect, I've been a Homebrew maintainer for just under 7 years and I think I've got a pretty good grasp of what's in our users best interests.
I don't doubt that for a second. I've been a Homebrew user for a long time, and it is the only package manager that I have installed on my systems; that should tell you the level of respect and appreciation that I have for the work of the Homebrew maintainers including yourself.
Saying that
I'm happy for Homebrew users to remain on Elasticsearch 1.x until this is resolved.
when prior to your recent comment it was appearing that we weren't going to get anywhere, you were effectively banishing Homebrew users to a rapidly-aging version of Elasticsearch that will soon be out of support.
My point there was that it's unclear you've convinced other package managers to package Elasticsearch in the way you want us to (e.g. put all the jars in
lib).
First, we have our own official packages that we support. Second, the package that you were pointing to is an old version (1.7) of Elasticsearch that didn't have the requirement that jars be in lib. If an attempt were to be made today by an outside organization (and to be clear, Elastic welcomes such community involvement), it would also have to observe the lib requirement because it will not work otherwise.
It's not currently broken, it's just not up-to-date. It is fixable but requires us to find us a middle solution together.
It is de facto broken for HEAD, and effectively broken since it can't be brought up to latest stable without breaking.
It is fixable but requires us to find us a middle solution together.
That's all I've wanted all along. Do you have any proposals?
Technically the code is simple to just special-case this one thing. I hope you appreciate that with >3000 formulae writing making these type changes based on one formula quickly becomes unsustainable.
Your concern is conflicting jars from symlinks in /usr/local/lib to package-installed jars in package lib folders. I'm of the mindset that there isn't a good reason to be doing this anyway, Making that change doesn't break anything that exists today (since it is impossible for there to be any jars there right now). I'm not proposing that Elasticsearch be special-cased, I'm proposing that jars just not be symlinked in /usr/local/lib. I think this avoids all the problems that concern you, and it's a good practice anyway.
I'm still not sure I completely understand what the requirement is here with
lib.
It's part of our integration with the Java Security Manager. We are going to great lengths to minimize Elasticsearch exploit vectors. In this particular case, with filesystem protections in place and requiring that jars be in lib, we remove some exploit vectors. At Elasticsearch startup, we create a special set of permissions for only the jars in the lib folder.
Would it be possible to use symlinks and/or a wrapper script and keep installing to
libexec?
The only place that such a symlink could come from is lib, no other symlink would work. But that still fails the brew audit, doesn't avoid the issues that concern you, and we are possibly going to remove the ability for lib to be a symlink anyway.
If not, can you explain why this is not possible so we can try and work out another solution?
I hope I clarified this, but if not, let me know? Thanks again for your time.
I don't doubt that for a second. I've been a Homebrew user for a long time, and it is the only package manager that I have installed on my systems; that should tell you the level of respect and appreciation that I have for the work of the Homebrew maintainers including yourself. Saying that
when prior to your recent comment it was appearing that we weren't going to get anywhere, you were effectively banishing Homebrew users to a rapidly-aging version of Elasticsearch that will soon be out of support.
First, we have our own official packages that we support. Second, the package that you were pointing to is an old version (1.7) of Elasticsearch that didn't have the requirement that jars be in
It is de facto broken for
That's all I've wanted all along. Do you have any proposals?
Your concern is conflicting jars from symlinks in
It's part of our integration with the Java Security Manager. We are going to great lengths to minimize Elasticsearch exploit vectors. In this particular case, with filesystem protections in place and requiring that jars be in
The only place that such a symlink could come from is
I hope I clarified this, but if not, let me know? Thanks again for your time. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 4, 2015
Member
The only place that such a symlink could come from is lib, no other symlink would work. But that still fails the brew audit, doesn't avoid the issues that concern you, and we are possibly going to remove the ability for lib to be a symlink anyway.
What would happen if we installed the libraries to libexec/lib and make libexec the home (thanks @xu-cheng)? We could then install everything in there and use wrapper scripts and/or symlinks just into bin.
What would happen if we installed the libraries to |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
DomT4
Nov 4, 2015
Member
Applying this diff to your formula works for me, locally, with the test you wrote and also starting/stopping the elasticsearch server manually.
diff --git a/Library/Formula/elasticsearch.rb b/Library/Formula/elasticsearch.rb
index d1b16a2..38583f7 100644
--- a/Library/Formula/elasticsearch.rb
+++ b/Library/Formula/elasticsearch.rb
@@ -32,12 +32,10 @@ class Elasticsearch < Formula
rm_f Dir["bin/*.exe"]
# Install everything else into package directory
- prefix.install "bin"
- prefix.install "config"
- prefix.install "lib"
+ libexec.install "bin", "config", "lib"
# Set up Elasticsearch for local development:
- inreplace "#{prefix}/config/elasticsearch.yml" do |s|
+ inreplace "#{libexec}/config/elasticsearch.yml" do |s|
# 1. Give the cluster a unique name
s.gsub!(/#\s*cluster\.name\: .*/, "cluster.name: #{cluster_name}")
@@ -46,21 +44,24 @@ class Elasticsearch < Formula
s.sub!(%r{#\s*path\.logs: /path/to.+$}, "path.logs: #{var}/log/elasticsearch/")
end
- inreplace "#{bin}/elasticsearch.in.sh" do |s|
+ inreplace "#{libexec}/bin/elasticsearch.in.sh" do |s|
# Configure ES_HOME
- s.sub!(%r{#\!/bin/sh\n}, "#!/bin/sh\n\nES_HOME=#{prefix}")
+ s.sub!(%r{#\!/bin/sh\n}, "#!/bin/sh\n\nES_HOME=#{libexec}")
end
# Move config files into etc
- (etc/"elasticsearch").install Dir[prefix/"config/*"]
- (prefix/"config").rmtree
+ (etc/"elasticsearch").install Dir[libexec/"config/*"]
+ (libexec/"config").rmtree
+
+ # Exec script executables back to top-level bin
+ bin.write_exec_script Dir[libexec/"bin/*"]
end
def post_install
# Make sure runtime directories exist
(var/"elasticsearch/#{cluster_name}").mkpath
(var/"log/elasticsearch").mkpath
- ln_s etc/"elasticsearch", prefix/"config"
+ ln_s etc/"elasticsearch", libexec/"config"
end
def caveats; <<-EOS.undent|
Applying this diff to your formula works for me, locally, with the test you wrote and also starting/stopping the diff --git a/Library/Formula/elasticsearch.rb b/Library/Formula/elasticsearch.rb
index d1b16a2..38583f7 100644
--- a/Library/Formula/elasticsearch.rb
+++ b/Library/Formula/elasticsearch.rb
@@ -32,12 +32,10 @@ class Elasticsearch < Formula
rm_f Dir["bin/*.exe"]
# Install everything else into package directory
- prefix.install "bin"
- prefix.install "config"
- prefix.install "lib"
+ libexec.install "bin", "config", "lib"
# Set up Elasticsearch for local development:
- inreplace "#{prefix}/config/elasticsearch.yml" do |s|
+ inreplace "#{libexec}/config/elasticsearch.yml" do |s|
# 1. Give the cluster a unique name
s.gsub!(/#\s*cluster\.name\: .*/, "cluster.name: #{cluster_name}")
@@ -46,21 +44,24 @@ class Elasticsearch < Formula
s.sub!(%r{#\s*path\.logs: /path/to.+$}, "path.logs: #{var}/log/elasticsearch/")
end
- inreplace "#{bin}/elasticsearch.in.sh" do |s|
+ inreplace "#{libexec}/bin/elasticsearch.in.sh" do |s|
# Configure ES_HOME
- s.sub!(%r{#\!/bin/sh\n}, "#!/bin/sh\n\nES_HOME=#{prefix}")
+ s.sub!(%r{#\!/bin/sh\n}, "#!/bin/sh\n\nES_HOME=#{libexec}")
end
# Move config files into etc
- (etc/"elasticsearch").install Dir[prefix/"config/*"]
- (prefix/"config").rmtree
+ (etc/"elasticsearch").install Dir[libexec/"config/*"]
+ (libexec/"config").rmtree
+
+ # Exec script executables back to top-level bin
+ bin.write_exec_script Dir[libexec/"bin/*"]
end
def post_install
# Make sure runtime directories exist
(var/"elasticsearch/#{cluster_name}").mkpath
(var/"log/elasticsearch").mkpath
- ln_s etc/"elasticsearch", prefix/"config"
+ ln_s etc/"elasticsearch", libexec/"config"
end
def caveats; <<-EOS.undent |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
Thanks for the suggestion @MikeMcQuaid. It's hacky, it's goes further against the standard filesystem hierarchy, but it might work. I'm still working through the implications locally (Elasticsearch can start, but plugins do not work right now).
|
Thanks for the suggestion @MikeMcQuaid. It's hacky, it's goes further against the standard filesystem hierarchy, but it might work. I'm still working through the implications locally (Elasticsearch can start, but plugins do not work right now). |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
@DomT4 Yes, I have the same working locally for Elasticsearch server, but plugins do not work at the moment. I've almost worked through the implications and a fix for that as well. Thanks for taking a look, and any additional feedback that you provide is most welcomed.
|
@DomT4 Yes, I have the same working locally for Elasticsearch server, but plugins do not work at the moment. I've almost worked through the implications and a fix for that as well. Thanks for taking a look, and any additional feedback that you provide is most welcomed. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
DomT4
Nov 4, 2015
Member
Yes, I have the same working locally for Elasticsearch server, but plugins do not work at the moment.
Can you provide an example of how we'd discover this breakage and what the breakage specifically is? Elasticsearch is not something I use incredibly regularly myself.
Curious on what that breakage is given presumably elasticsearch doesn't care what prefix you put it in, as long as the home is set correctly and all the elements remain inside the same prefix untouched? As I said I don't use it much, so feel free to ELI5.
Can you provide an example of how we'd discover this breakage and what the breakage specifically is? Elasticsearch is not something I use incredibly regularly myself. Curious on what that breakage is given presumably |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
@MikeMcQuaid @DomT4 I've pushed b3d24e5 incorporating what we've discussed. This pull request passes brew audit now.
|
@MikeMcQuaid @DomT4 I've pushed b3d24e5 incorporating what we've discussed. This pull request passes |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
@jasontedor |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 4, 2015
Member
It looks like the test is failing but should be relatively easy to fix: https://travis-ci.org/Homebrew/homebrew/jobs/89335368#L313
You can test locally with brew test --sandbox --verbose elasticsearch
|
It looks like the test is failing but should be relatively easy to fix: https://travis-ci.org/Homebrew/homebrew/jobs/89335368#L313 You can test locally with |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 4, 2015
Contributor
You can test locally with
brew test --sandbox --verbose elasticsearch�
Didn't know that, thank you. But I think I figured out one issue from the CI logs and pushed 5ee3978 to address but it looks like there will be more. I'll iterate locally. Thanks again for your time and help.
Didn't know that, thank you. But I think I figured out one issue from the CI logs and pushed 5ee3978 to address but it looks like there will be more. I'll iterate locally. Thanks again for your time and help. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
@MikeMcQuaid With 1b4c8d0 sandboxed tests are passing locally. |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 5, 2015
Contributor
@MikeMcQuaid With green on the CI now, is this ready for the rebase/squash dance into a final single commit for integration into master?
|
@MikeMcQuaid With green on the CI now, is this ready for the rebase/squash dance into a final single commit for integration into master? |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 5, 2015
Member
@jasontedor That would be fantastic, thanks. Thanks for all your work here and apologies for not recognising a better solution earlier.
|
@jasontedor That would be fantastic, thanks. Thanks for all your work here and apologies for not recognising a better solution earlier. |
jasontedor
changed the title from
Fix elasticsearch formula
to
Fix Elasticsearch formula
Nov 5, 2015
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
|
@MikeMcQuaid Done and done. Thanks again. |
jasontedor
changed the title from
Fix Elasticsearch formula
to
elasticsearch 2.0.0
Nov 5, 2015
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 5, 2015
Contributor
@MikeMcQuaid It occurred to me earlier this morning that if Homebrew is okay with installing into subdirectories and with the bin.write_exec_script trick, we can just install Elasticsearch components into home instead of libexec. I've pushed 69fa592. Let me know what you think and I'll squash one last time before this is integrated into master?
|
@MikeMcQuaid It occurred to me earlier this morning that if Homebrew is okay with installing into subdirectories and with the |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 5, 2015
Member
@MikeMcQuaid It occurred to me earlier this morning that if Homebrew is okay with installing into subdirectories and with the bin.write_exec_script trick, we can just install Elasticsearch components into home instead of libexec. I've pushed 69fa592. Let me know what you think and I'll squash one last time before this is integrated into master?
@jasontedor We'd probably prefer libexec just because it's ended up being a Homebrew convention so it's more obvious to maintainers why that's been done. If you could go back to libexec and squash I'll merge this ASAP.
@jasontedor We'd probably prefer |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
jasontedor
Nov 5, 2015
Contributor
We'd probably prefer
libexecjust because it's ended up being a Homebrew convention so it's more obvious to maintainers why that's been done.
Understood.
If you could go back to
libexecand squash I'll merge this ASAP.
Done. Thanks a lot!
Understood.
Done. Thanks a lot! |
This comment has been minimized.
Show comment
Hide comment
This comment has been minimized.
MikeMcQuaid
Nov 5, 2015
Member
Pushed! Thanks for your contribution to Homebrew, @jasontedor, and all your hard work here! Without people like you submitting PRs we couldn't run this project. You rock! Thanks also to @DomT4 for the suggested patch.
Just so you don't 
: I also added a revision here so anyone on the previous (broken) 2.0.0 version we shipped will see an upgrade to the new 2.0.0_1 version.
|
Pushed! Thanks for your contribution to Homebrew, @jasontedor, and all your hard work here! Without people like you submitting PRs we couldn't run this project. You rock! Thanks also to @DomT4 for the suggested patch. Just so you don't |
jasontedor commentedNov 3, 2015
This commit addresses several issues that exist in the Elasticsearch
formula.
Elasticsearch
HEADbuilds to require Java 8 (cf.elastic/elasticsearch@1131433)
HEADbuilds to use gradle for assembling (due toElasticsearch build system change, cf.
elastic/elasticsearch@c86100f)
Elasticsearch; cf.
elastic/elasticsearch@052cf14)
elastic/elasticsearch@09b4d0e)
Closes elastic/elasticsearch#14424