New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build ElasticSearch as Debian package #1231

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
9 participants
@nhuray
Contributor

nhuray commented Aug 11, 2011

Hi Shay,

I'd like to submit a patch to the elascticsearch project for building elasticsearch as a debian package.

The debian package is built with jdeb (https://github.com/tcurdt/jdeb) and is integrated with the gradle script (task named "deb").

The debian package installation has the following structure :

/usr/share/elasticsearch : elasticsearch home directory
/etc/elasticsearch : elasticsearch configuration directory
/etc/default/elasticsearch : default service configuration file
/etc/init.d/elasticsearch : service file (sysvinit)
/var/lib/elasticsearch : elasticsearch data directory (for indices)
/var/log/elasctisearch : elasticsearch logs directory

Regards,

Nicolas

@kimchy

This comment has been minimized.

Show comment
Hide comment
@kimchy

kimchy Aug 11, 2011

Member

Hey, looks good. Can you move this into a pkg top level directory, and move the jdeb jar file into a lib folder under pkg/lib? If we have other packaging, we can place it there.

Also, some notes:

  1. elasticsearch requires java 6, so listing java 5 locations should not be done.
  2. I think that if the sun jdk is installed, it should come before the openjdk one.
Member

kimchy commented Aug 11, 2011

Hey, looks good. Can you move this into a pkg top level directory, and move the jdeb jar file into a lib folder under pkg/lib? If we have other packaging, we can place it there.

Also, some notes:

  1. elasticsearch requires java 6, so listing java 5 locations should not be done.
  2. I think that if the sun jdk is installed, it should come before the openjdk one.
@nhuray

This comment has been minimized.

Show comment
Hide comment
@nhuray

nhuray Aug 11, 2011

Contributor

Hi Shay,

Ok no problem to move the 'debian' directory to 'pkg/debian' and the 'jdeb' jar to 'pkg/lib'.

However what do you mean about "elasticsearch requires java 6, so listing java 5 locations should not be done." ?

For package dependencies (Depends header in "control" file), the "sun-java6-jre" package is defined as an alternative to the "openjdk-6-jre-headless". If any one of the alternative packages is installed, that part of the dependency is considered to be satisfied. For more information show http://www.debian.org/doc/debian-policy/ch-relationships.html.

Regards,

Nicolas

Contributor

nhuray commented Aug 11, 2011

Hi Shay,

Ok no problem to move the 'debian' directory to 'pkg/debian' and the 'jdeb' jar to 'pkg/lib'.

However what do you mean about "elasticsearch requires java 6, so listing java 5 locations should not be done." ?

For package dependencies (Depends header in "control" file), the "sun-java6-jre" package is defined as an alternative to the "openjdk-6-jre-headless". If any one of the alternative packages is installed, that part of the dependency is considered to be satisfied. For more information show http://www.debian.org/doc/debian-policy/ch-relationships.html.

Regards,

Nicolas

@kimchy

This comment has been minimized.

Show comment
Hide comment
@kimchy

kimchy Aug 11, 2011

Member

Regarding the jdk versions, I was referring to this one: https://github.com/elasticsearch/elasticsearch/pull/1231/files#L8R49.

Member

kimchy commented Aug 11, 2011

Regarding the jdk versions, I was referring to this one: https://github.com/elasticsearch/elasticsearch/pull/1231/files#L8R49.

@kimchy

This comment has been minimized.

Show comment
Hide comment
@kimchy

kimchy Aug 18, 2011

Member

Cam you squash the commits into a single one? simpler to apply

Member

kimchy commented Aug 18, 2011

Cam you squash the commits into a single one? simpler to apply

@nhuray

This comment has been minimized.

Show comment
Hide comment
@nhuray

nhuray Aug 19, 2011

Contributor

Hi Shay

You can merge the pull request now.

Nicolas

Contributor

nhuray commented Aug 19, 2011

Hi Shay

You can merge the pull request now.

Nicolas

@dadoonet

This comment has been minimized.

Show comment
Hide comment
@dadoonet

dadoonet Aug 19, 2011

Member

Hi all,

Just a question about the stop script. Is it the best way to stop a node ? I mean that is not a curl -XPOST 'http://IP:PORT/_cluster/nodes/_local/_shutdown' command safer ?

I suppose that start-stop-daemon --stop is also safe, right ?

Thanks

Member

dadoonet commented Aug 19, 2011

Hi all,

Just a question about the stop script. Is it the best way to stop a node ? I mean that is not a curl -XPOST 'http://IP:PORT/_cluster/nodes/_local/_shutdown' command safer ?

I suppose that start-stop-daemon --stop is also safe, right ?

Thanks

@nhuray

This comment has been minimized.

Show comment
Hide comment
@nhuray

nhuray Aug 19, 2011

Contributor

Hi dadoonet

I suppose that "start-stop-daemon --stop" kill the process safely, it send a SIGTERM signal.

See this discussion : http://elasticsearch-users.115913.n3.nabble.com/how-to-shut-down-a-node-td451493.html

It's possible to send a SIGKILL signal (like CTRL+C) with the --signal option : http://man.cx/start-stop-daemon(8)

Regards,

Nicolas

Contributor

nhuray commented Aug 19, 2011

Hi dadoonet

I suppose that "start-stop-daemon --stop" kill the process safely, it send a SIGTERM signal.

See this discussion : http://elasticsearch-users.115913.n3.nabble.com/how-to-shut-down-a-node-td451493.html

It's possible to send a SIGKILL signal (like CTRL+C) with the --signal option : http://man.cx/start-stop-daemon(8)

Regards,

Nicolas

@dadoonet

This comment has been minimized.

Show comment
Hide comment
@dadoonet

dadoonet Aug 19, 2011

Member

Thanks.

Member

dadoonet commented Aug 19, 2011

Thanks.

@kimchy

This comment has been minimized.

Show comment
Hide comment
@kimchy

kimchy Aug 29, 2011

Member

Heya, sorry for coming back to this late... . One last change, at least in terms of build structure, can we fetch jdeb from maven central as a dependency and not add it as a jar file? Not sure if its possible with gradle, I think it should be...

Member

kimchy commented Aug 29, 2011

Heya, sorry for coming back to this late... . One last change, at least in terms of build structure, can we fetch jdeb from maven central as a dependency and not add it as a jar file? Not sure if its possible with gradle, I think it should be...

@nhuray

This comment has been minimized.

Show comment
Hide comment
@nhuray

nhuray Aug 29, 2011

Contributor

Hi Shay

I don't know if it's possible with Gradle to do it. Anyway I'm on holiday
for now so I'll couldn't do it for the next three weeks.

Contributor

nhuray commented Aug 29, 2011

Hi Shay

I don't know if it's possible with Gradle to do it. Anyway I'm on holiday
for now so I'll couldn't do it for the next three weeks.

@kimchy

This comment has been minimized.

Show comment
Hide comment
@kimchy

kimchy Sep 1, 2011

Member

@nhuray, I think we should be able to do it, I just don't want to have the jar file in the git repo... . When you come back from the vacation, ping me I will try and help.

Member

kimchy commented Sep 1, 2011

@nhuray, I think we should be able to do it, I just don't want to have the jar file in the git repo... . When you come back from the vacation, ping me I will try and help.

@purem

This comment has been minimized.

Show comment
Hide comment
@purem

purem Sep 9, 2011

Contributor

Why /var/lib/elasticsearch for data?

Contributor

purem commented Sep 9, 2011

Why /var/lib/elasticsearch for data?

@nhuray

This comment has been minimized.

Show comment
Hide comment
@nhuray

nhuray Sep 12, 2011

Contributor

Actually it's a linux convention to store persistent data : http://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

Contributor

nhuray commented Sep 12, 2011

Actually it's a linux convention to store persistent data : http://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard

@dstendardi

View changes

Show outdated Hide outdated build.gradle
tarfileset(dir: explodedDistDir, prefix: "/usr/share/elasticsearch", includes: "*.txt, *.textile", username: "root", group: "root")
tarfileset(dir: explodedDistBinDir, prefix: "/usr/share/elasticsearch/bin", excludes: "*.bat", filemode: "755", username: "root", group: "root")
tarfileset(dir: explodedDistLibDir, prefix: "/usr/share/elasticsearch/lib", includes: "*.jar, sigar/*", username: "root", group: "root")
tarfileset(dir: explodedDistConfigDir, prefix: "/etc/elasticsearch", username: "root", group: "root")

This comment has been minimized.

@dstendardi

dstendardi Sep 19, 2011

wrong indent here

@dstendardi

dstendardi Sep 19, 2011

wrong indent here

# Maximum Heap memory to run ElasticSearch
ES_MAX_MEM=1g

This comment has been minimized.

@dhardy92

dhardy92 Sep 19, 2011

Hi,
Maybe you should add a ulimit -n 65535 here to extand the default 1024 max open files to a higher value like 65535 or other.
As said in http://www.elasticsearch.org/guide/reference/setup/installation.html

start-stop-daemon does not use PAM so it couldn't be set in /etc/security/limits.conf

@dhardy92

dhardy92 Sep 19, 2011

Hi,
Maybe you should add a ulimit -n 65535 here to extand the default 1024 max open files to a higher value like 65535 or other.
As said in http://www.elasticsearch.org/guide/reference/setup/installation.html

start-stop-daemon does not use PAM so it couldn't be set in /etc/security/limits.conf

This comment has been minimized.

@nhuray

nhuray Sep 26, 2011

Contributor

Hi Shay,

Do you think we should add this ulimit in the /etc/init.d/elasticsearch script ?

@nhuray

nhuray Sep 26, 2011

Contributor

Hi Shay,

Do you think we should add this ulimit in the /etc/init.d/elasticsearch script ?

This comment has been minimized.

@kimchy

kimchy Sep 26, 2011

Member

Yes, I think it makes sense.

@kimchy

kimchy Sep 26, 2011

Member

Yes, I think it makes sense.

This comment has been minimized.

@nhuray

nhuray Sep 27, 2011

Contributor

Hi Shay,

I added the ulimit and merged your last commits. You can merged the pull request when you want.

@nhuray

nhuray Sep 27, 2011

Contributor

Hi Shay,

I added the ulimit and merged your last commits. You can merged the pull request when you want.

@kimchy

This comment has been minimized.

Show comment
Hide comment
@kimchy

kimchy Sep 27, 2011

Member

Gerat, thanks!. Pushed the changes. Hoping for more eyeballs on this and tests, as I am not an expert on formal deb packaging...

Member

kimchy commented Sep 27, 2011

Gerat, thanks!. Pushed the changes. Hoping for more eyeballs on this and tests, as I am not an expert on formal deb packaging...

@kimchy kimchy closed this Sep 27, 2011

@medcl

This comment has been minimized.

Show comment
Hide comment
@medcl

medcl Sep 28, 2011

Member

cool stuff~

Member

medcl commented Sep 28, 2011

cool stuff~

@nickhoffman

This comment has been minimized.

Show comment
Hide comment
@nickhoffman

nickhoffman Oct 7, 2011

Hey guys. How does one use what's in this awesome commit to build an ES Debian package?

nickhoffman commented Oct 7, 2011

Hey guys. How does one use what's in this awesome commit to build an ES Debian package?

@nhuray

This comment has been minimized.

Show comment
Hide comment
@nhuray

nhuray Oct 7, 2011

Contributor

Hi Nick

You can build the project with the gradle wrapper ./gradlew or with gradle if it was installed on your system with gradle -q deb.

The package will be created under build/distributions.

Contributor

nhuray commented Oct 7, 2011

Hi Nick

You can build the project with the gradle wrapper ./gradlew or with gradle if it was installed on your system with gradle -q deb.

The package will be created under build/distributions.

@nickhoffman

This comment has been minimized.

Show comment
Hide comment
@nickhoffman

nickhoffman Oct 7, 2011

Thanks, @nhuray. Much appreciated!

nickhoffman commented Oct 7, 2011

Thanks, @nhuray. Much appreciated!

@nickhoffman

This comment has been minimized.

Show comment
Hide comment
@nickhoffman

nickhoffman Oct 7, 2011

BTW, I just noticed that this commit isn't in the 0.17 branch. Will it be included in the next major or minor release or tagged version, or will we have to use HEAD?

nickhoffman commented Oct 7, 2011

BTW, I just noticed that this commit isn't in the 0.17 branch. Will it be included in the next major or minor release or tagged version, or will we have to use HEAD?

@kimchy

This comment has been minimized.

Show comment
Hide comment
@kimchy

kimchy Oct 8, 2011

Member

@nickhoffman: its there now in the repo, when there is a release, its tagged, so you can always switch to the tagged source and build it. I am still not sure if it will be provided as a "download" in the download page, or people will simply close the repo, and use the tag to build it. I guess time will tell on how best to provide it.

Member

kimchy commented Oct 8, 2011

@nickhoffman: its there now in the repo, when there is a release, its tagged, so you can always switch to the tagged source and build it. I am still not sure if it will be provided as a "download" in the download page, or people will simply close the repo, and use the tag to build it. I guess time will tell on how best to provide it.

@nickhoffman

This comment has been minimized.

Show comment
Hide comment
@nickhoffman

nickhoffman Oct 9, 2011

@kimchy Am I correct in assuming that this commit will be in 0.17.9 (or 0.18.0 if that's the next tagged release)?

It can't really hurt to provide an i386 build of the Debian package on elasticsearch.org . If people want it built for a different platform, they can always clone the repo and build it themselves, as you said. I would imagine that most people run ES on i386 due to the extra memory usage on x64.

nickhoffman commented Oct 9, 2011

@kimchy Am I correct in assuming that this commit will be in 0.17.9 (or 0.18.0 if that's the next tagged release)?

It can't really hurt to provide an i386 build of the Debian package on elasticsearch.org . If people want it built for a different platform, they can always clone the repo and build it themselves, as you said. I would imagine that most people run ES on i386 due to the extra memory usage on x64.

@kimchy

This comment has been minimized.

Show comment
Hide comment
@kimchy

kimchy Oct 13, 2011

Member

@nickhoffman: its in master now, which means when 0.18 will be tagged (and any future versions). I think we should let it bake a bit before providing it as a full download, and have clear instructions on how to build it for now.

Member

kimchy commented Oct 13, 2011

@nickhoffman: its in master now, which means when 0.18 will be tagged (and any future versions). I think we should let it bake a bit before providing it as a full download, and have clear instructions on how to build it for now.

@nickhoffman

This comment has been minimized.

Show comment
Hide comment
@nickhoffman

nickhoffman commented Oct 13, 2011

@kimchy Sounds good!

@gabbyz

This comment has been minimized.

Show comment
Hide comment
@gabbyz

gabbyz Aug 7, 2012

i can't connect the elasticsearch in mysql database using cakephp

do i missing something?..

gabbyz commented Aug 7, 2012

i can't connect the elasticsearch in mysql database using cakephp

do i missing something?..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment