From f88acb695663c0a3f06bbb5afc082133b4b249ce Mon Sep 17 00:00:00 2001 From: Aled Sage Date: Wed, 3 Aug 2016 22:57:28 +0100 Subject: [PATCH] Move entropy/ulimit docs to troubleshooting They were previously in website/documentation/increase-entropy.md and in the faq. Now they are top-level pages in troubleshooting. --- guide/ops/locations/_GCE.md | 2 +- guide/ops/locations/_ssh-keys.md | 2 +- guide/ops/production-installation.md | 5 ++-- guide/ops/requirements.md | 7 ++++- .../ops/troubleshooting}/increase-entropy.md | 17 +++++++++-- .../increase-system-resource-limits.md | 29 +++++++++++++++++++ guide/ops/troubleshooting/index.md | 2 ++ .../ops/troubleshooting/slow-unresponsive.md | 5 +++- website/documentation/faq.md | 28 ------------------ 9 files changed, 60 insertions(+), 37 deletions(-) rename {website/documentation => guide/ops/troubleshooting}/increase-entropy.md (72%) create mode 100644 guide/ops/troubleshooting/increase-system-resource-limits.md diff --git a/guide/ops/locations/_GCE.md b/guide/ops/locations/_GCE.md index e4d25ea1..8539bbf3 100644 --- a/guide/ops/locations/_GCE.md +++ b/guide/ops/locations/_GCE.md @@ -91,4 +91,4 @@ To configure the location to use this, you can include a location configuration ### Entropy GCE images often have little entropy. One option to work around this is to use `installDevUrandom: true`. -For more information, see [Increase Entropy]({{ site.path.website }}/documentation/increase-entropy.html). +For more information, see [Linux kernel entropy]({{ site.path.guide }}/ops/troubleshooting/increase-entropy.html). diff --git a/guide/ops/locations/_ssh-keys.md b/guide/ops/locations/_ssh-keys.md index a929acfa..acf30d71 100644 --- a/guide/ops/locations/_ssh-keys.md +++ b/guide/ops/locations/_ssh-keys.md @@ -85,4 +85,4 @@ If this isn't the case, see below. if command-line `ssh` and `scp` work, but Brooklyn/java does not, check the versions enabled in Java and on both servers. * Missing entropy: creating and using ssh keys requires randomness available on the servers, - usually in `/dev/random`; see [here]({{ site.path.website }}/documentation/increase-entropy.html) for more information + usually in `/dev/random`; see [here]({{ site.path.guide }}/ops/troubleshooting/increase-entropy.html) for more information diff --git a/guide/ops/production-installation.md b/guide/ops/production-installation.md index 37487129..d04517f3 100644 --- a/guide/ops/production-installation.md +++ b/guide/ops/production-installation.md @@ -27,10 +27,11 @@ Then configure the server as follows: * install Java JRE or JDK (version 7 or later) * install an [SSH key]({{ site.path.guide }}/ops/locations/index.html#ssh-keys), if not available -* enable [passwordless ssh login]({{ site.path.guide }}/ops/locations/index.html#ssh-keys) +* if the "localhost" location will be used, enable [passwordless ssh login]({{ site.path.guide }}/ops/locations/index.html#ssh-keys) * create a `~/.brooklyn` directory on the host with `$ mkdir ~/.brooklyn` * check your `iptables` or other firewall service, making sure that incoming connections on port 8443 is not blocked -* check that the [linux kernel entropy]({{ site.path.website }}/documentation/increase-entropy.html) is sufficient +* check that the [linux kernel entropy]({{ site.path.guide }}/ops/troubleshooting/increase-entropy.html) is sufficient +* check that the [ulimit values]({{ site.path.guide }}/ops/troubleshooting/increase-system-resource-limits.html) are sufficiently high * ensure external libraries are up-to-date, including `nss` for SSL. diff --git a/guide/ops/requirements.md b/guide/ops/requirements.md index 86a186a0..a194fc7e 100644 --- a/guide/ops/requirements.md +++ b/guide/ops/requirements.md @@ -110,4 +110,9 @@ It is normally recommended that Brooklyn run as a non-root user with keys instal ### Linux Kernel Entropy -Check that the [linux kernel entropy]({{ site.path.website }}/documentation/increase-entropy.html) is sufficient. +Check that the [linux kernel entropy]({{ site.path.guide }}/ops/troubleshooting/increase-entropy.html) is sufficient. + + +### System Resource Limits + +Check that the [ulimit values]({{ site.path.guide }}/ops/troubleshooting/increase-system-resource-limits.html) are sufficiently high. diff --git a/website/documentation/increase-entropy.md b/guide/ops/troubleshooting/increase-entropy.md similarity index 72% rename from website/documentation/increase-entropy.md rename to guide/ops/troubleshooting/increase-entropy.md index c269f329..a6b58729 100644 --- a/website/documentation/increase-entropy.md +++ b/guide/ops/troubleshooting/increase-entropy.md @@ -1,11 +1,18 @@ --- -title: Increase Entropy layout: website-normal +title: Increase Entropy +toc: /guide/toc.json --- -If you are installing Apache Brooklyn on a virtual machine, you may find it useful to increase the Linux kernel entropy to speed up the ssh connections to the managed entities. You can install and configure `rng-tools` or just use /dev/urandom`. + +If you are installing Apache Brooklyn on a virtual machine, you may find it useful to increase the +Linux kernel entropy to speed up the ssh connections to the managed entities. You can install and +configure `rng-tools` or just use /dev/urandom`. + ### Installing rng-tool + if you are using a RHEL-based OS: + {% highlight bash %} yum -y -q install rng-tools echo "EXTRAOPTIONS=\"-r /dev/urandom\"" | cat >> /etc/sysconfig/rngd @@ -13,15 +20,19 @@ echo "EXTRAOPTIONS=\"-r /dev/urandom\"" | cat >> /etc/sysconfig/rngd {% endhighlight %} if you are using a Debian-based OS: + {% highlight bash %} apt-get -y install rng-tools echo "HRNGDEVICE=/dev/urandom" | cat >> /etc/default/rng-tools /etc/init.d/rng-tools start {% endhighlight %} -The following links contain further [information for RHEL or CentOS](http://my.itwnik.com/how-to-increase-linux-kernel-entropy/), and [Ubuntu](http://www.howtoforge.com/helping-the-random-number-generator-to-gain-enough-entropy-with-rng-tools-debian-lenny). +The following links contain further [information for RHEL or CentOS](http://my.itwnik.com/how-to-increase-linux-kernel-entropy/), +and [Ubuntu](http://www.howtoforge.com/helping-the-random-number-generator-to-gain-enough-entropy-with-rng-tools-debian-lenny). + ### Using /dev/urandom + You can also just mv /dev/random then create it again linked to /dev/urandom, by issuing the following commands: {% highlight bash %} diff --git a/guide/ops/troubleshooting/increase-system-resource-limits.md b/guide/ops/troubleshooting/increase-system-resource-limits.md new file mode 100644 index 00000000..666bd4e3 --- /dev/null +++ b/guide/ops/troubleshooting/increase-system-resource-limits.md @@ -0,0 +1,29 @@ +--- +layout: website-normal +title: Increase System Resource Limits +toc: /guide/toc.json +--- + +If you encounter the error below, e.g. when running with many entities, please consider **increasing the ulimit**: + + java.lang.OutOfMemoryError: unable to create new native thread + +On the VM running Apache Brooklyn, it is recommended that nproc and nofile are reasonably high +(e.g. 16384 or higher; a value of 1024 is often the default). + +If you want to check the current limits run `ulimit -a`. Alternatively, if Brooklyn is run as a +different user (e.g. with user name "adalovelace"), then instead run `ulimit -a -u adalovelace`. + +For RHEL (and CentOS) distributions, you can increase the limits by running +`sudo vi /etc/security/limits.conf` and adding (if it is "adalovelace" user running Apache Brooklyn): + + adalovelace soft nproc 16384 + adalovelace hard nproc 16384 + adalovelace soft nofile 16384 + adalovelace hard nofile 16384 + +Generally you do not have to reboot to apply ulimit values. They are set per session. +So after you have the correct values, quit the ssh session and log back in. + +For more details, see one of the many posts such as +[http://tuxgen.blogspot.co.uk/2014/01/centosrhel-ulimit-and-maximum-number-of.html](http://tuxgen.blogspot.co.uk/2014/01/centosrhel-ulimit-and-maximum-number-of.html). diff --git a/guide/ops/troubleshooting/index.md b/guide/ops/troubleshooting/index.md index bb3715c1..436cd846 100644 --- a/guide/ops/troubleshooting/index.md +++ b/guide/ops/troubleshooting/index.md @@ -6,6 +6,8 @@ children: - { path: deployment.md, title: Deployment } - { path: connectivity.md, title: Server Connectivity } - { path: slow-unresponsive.md, title: Brooklyn Slow or Unresponsive } +- { path: increase-entropy.md, title: Increase Entropy } +- { path: increase-system-resource-limits.md, title: Increase System Resource Limits } - { path: detailed-support-report.md, title: Detailed Support Report } - { path: softwareprocess.md, title: SoftwareProcess Entities } - { path: going-deep-in-java-and-logs.md, title: Going Deep in Java and Logs } diff --git a/guide/ops/troubleshooting/slow-unresponsive.md b/guide/ops/troubleshooting/slow-unresponsive.md index 6a8e6f9c..2b0716d6 100644 --- a/guide/ops/troubleshooting/slow-unresponsive.md +++ b/guide/ops/troubleshooting/slow-unresponsive.md @@ -58,6 +58,9 @@ ulimit -a -u adalovelace Of particular interest is the limit for "open files". +See [Increase System Resource Limits]({{ site.path.guide }}/ops/troubleshooting/increase-system-resource-limits.html) +for more information. + #### Disk Space @@ -102,7 +105,7 @@ netstat -an | grep ESTABLISHED | wc -l A lack of entropy can cause random number generation to be extremely slow. This can cause tasks like ssh to also be extremely slow. See -[linux kernel entropy]({{ site.path.website }}/documentation/increase-entropy.html) +[linux kernel entropy]({{ site.path.guide }}/ops/troubleshooting/increase-entropy.html) for details of how to work around this. diff --git a/website/documentation/faq.md b/website/documentation/faq.md index 483d6869..25d38649 100644 --- a/website/documentation/faq.md +++ b/website/documentation/faq.md @@ -20,31 +20,3 @@ Supplying the answers are a TODO. ## How do I supply answers? Click the "Edit this Page" link in the bottom right. - - -# Common Problems: - -## java.lang.OutOfMemoryError: unable to create new native thread - -You could encounter this error when running with many entities. - -Please **increase the ulimit** if you see such error: - -On the VM running Apache Brooklyn, we recommend ensuring nproc and nofile are reasonably high (e.g. higher than 1024, which is often the default). -We recommend setting it limits to a value of 16384 or higher. - -If you want to check the current limits run `ulimit -a`. - -Here are instructions for how to increase the limits for RHEL like distributions. -Run `sudo vi /etc/security/limits.conf` and add (if it is "brooklyn" user running Apache Brooklyn): - - brooklyn soft nproc 16384 - brooklyn hard nproc 16384 - brooklyn soft nofile 16384 - brooklyn hard nofile 16384 - - -Generally you do not have to reboot to apply ulimit values. They are set per session. -So after you have the correct values, quit the ssh session and log back in. - -For more details, see one of the many posts such as http://tuxgen.blogspot.co.uk/2014/01/centosrhel-ulimit-and-maximum-number-of.html