Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

12311 auto memory docs #819

Merged
merged 19 commits into from Jul 10, 2014
Merged

12311 auto memory docs #819

Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
42202a1
Move jvm settings to adv. configuration (See #12311)
joshmoore Jun 10, 2014
c3a06ea
Formatting fixes & grammar improvements
joshmoore Jun 11, 2014
ecfb53e
Merge 'origin/dev_5_0' into 12311-auto-memory
joshmoore Jun 23, 2014
c8a0a2c
Fix troubleshooting
joshmoore Jun 23, 2014
f4a1f0f
Clarify 'may want to tune' section
joshmoore Jun 23, 2014
da6d7e1
First draft of updated docs
joshmoore Jun 23, 2014
2dddb16
Add :file: markup for indexer, etc
joshmoore Jun 26, 2014
e574072
Fix ending periods
joshmoore Jun 26, 2014
8ac1cb1
Improve psutil/jmx paragraph
joshmoore Jun 26, 2014
62d0b3c
Remove use of 'percentage'
joshmoore Jun 26, 2014
901d1fc
Point server-upgrade at jvm_memory_settings
joshmoore Jul 8, 2014
be3b345
Implement feedback from PR
joshmoore Jul 8, 2014
b781fe8
Remove adaptive strategy
joshmoore Jul 8, 2014
10b8b59
Merge 'origin/dev_5_0' into 12311-auto-memory
joshmoore Jul 8, 2014
8fbc548
Clarify 'append'
joshmoore Jul 9, 2014
3433c47
Other wording suggestions
joshmoore Jul 9, 2014
ffc0dc1
Remove :file: for server names
joshmoore Jul 9, 2014
0b76e4c
Unify unix/windows mem blurbs
joshmoore Jul 9, 2014
c57d915
Fix 'use use' typo
joshmoore Jul 9, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
132 changes: 130 additions & 2 deletions omero/sysadmins/server-advanced-configuration.txt
View file
Open in desktop
Expand Up @@ -56,6 +56,134 @@ Your database installation will need to be configured to accept *at
least* as many, preferably more, connections as the value of
"omero.db.poolsize"

.. _jvm_memory_settings:

omero.mem.*
^^^^^^^^^^^

A number of configuration properties starting with ``omero.mem``
control the calculation of how much memory to allocate to
various OMERO services on startup, most importantly:

* blitz
* indexer
* pixeldata

Configuration properties
""""""""""""""""""""""""

Configuration properties can either be applied to all three service
types at the same time by omitting the service type (e.g.
:term:`omero.mem.strategy`) or to each individually by including it
(e.g. `omero.mem.blitz.strategy`).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No idea how messy this would look but Sphinx allows to define multiple terms in glossary e.g.

.. glossary::

    omero.mem.strategy
    omero.mem.blitz.strategy
    omero.mem.indexer.strategy
    omero.mem.pixeldata.strategy
        Selects which memory strategy to use.


.. glossary::

omero.mem.strategy
Selects which memory strategy to use.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link to the section below?


omero.mem.heap_size
Sets the -Xmx value for the JVM.

omero.mem.heap_dump
Toggles on or off heap dumps on OOMs.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not clear what the expected value is True/False, 1/0 ?


omero.mem.perm_gen
Sets the MaxPermSize value for the JVM.

omero.mem.append
Is parsed breaking on non-escaped whitespace
and appended to the rest of the parameters.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description of omero.mem.append doesn't make too much sense to me...

omero.mem.max_total
Maximum amount of memory that OMERO will
see. By default, 16 GB.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I managed to confuse myself over max_total, min_total and use_total. Could you change max_total to something along the lines of Maximum amount of memory that OMERO will use, subject to use_total and omero.mem.strategy?


omero.mem.min_total
Minimum amount of memory that OMERO will
see. By default, 3.4 GB.

omero.mem.use_total
Specify how much memory OMERO *thinks* your system has.
If unset, an attempt will be made to detect the actual
amount: first by using the Python library `psutil` and
if that is not installed, by running a Java process. If
neither works, 4.0 GB is assumed.

For example, the default strategy set is equivalent to making the call:

::

$ bin/omero config set omero.mem.strategy adaptive
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not any more....


This could be changed to the ManualStrategy for a single server via:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why manual strategy is capitalised and concatenated? Maybe to avoid confusion, make it a link to the description of the strategy?


::

$ bin/omero config set omero.mem.indexer.strategy manual


Strategies
""""""""""

A number of strategies are available for calculating the effective
JVM settings from the provided configuration properties.

.. glossary::

AdaptiveStrategy
Default strategy which uses the system resources to scale
how much memory will be used. AdaptiveStrategy is a specific
type of :term:`PercentStrategy`.

PercentStrategy
Reads the `percent` configuration property which can also be
set globally or on a service-type basis. This percent (0-100)
of the system memory is used for the process.
:term:`omero.mem.use_total`, :term:`omero.mem.min_total`, and
:term:`omero.mem.max_total` are all used for defining the
system memory seen. The default percents are: blitz and
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

percents -> percentages ?

pixeldata 15%, indexer 10%.


ManualStrategy
Simply provides the values given as the JVM settings. If no
value is set for a particular configuration property, then
the default is used: heap_size=512m and perm_gen=128m
These values are equivalent to the defaults in OMERO 5.0.2
and earlier.

::

$ bin/omero config set omero.mem.blitz.percent 50
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth adding an omero.mem.percent to the glossary above?


would raise the blitz percent to 50.

::

$ bin/omero config set omero.mem.max_total 24000

would allow an OMERO installation to make use of 24GB
of system memory.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps add: For example, using the default PercentStrategy would assign 15% of this (3600MB) to Blitz.


Tips
""""

After modifying any memory settings, be sure to restart your server.

::

$ bin/omero admin restart

.. seealso::

http://www.openmicroscopy.org/community/viewtopic.php?f=4&t=7400
Forum thread on PixelData JVM memory settings

:ref:`gridconfiguration`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having a highlighted box with See also referring to the section immediately below looks a bit strange. Not a blocker for merging.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's see if that goes away in the re-working.

Section of the advanced server configuration documentation describing
:file:`etc/grid/templates.xml`.

.. _gridconfiguration:

Grid configuration
Expand All @@ -71,8 +199,8 @@ Windows systems and both make use of
:source:`templates.xml <etc/grid/templates.xml>`.

The default OMERO application descriptor deploys multiple server instances
(Blitz-0, FileServer, Indexer-0, PixelData-0…) on a single node. Each server
instance is defined by a ``server-template`` element in
(Blitz-0, FileServer, :file:`Indexer-0`, :file:`PixelData-0.log`, ...) on
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PixelData-0.log -> PixelData-0 and maybe remove or add fully the :file: markup?

a single node. Each server instance is defined by a ``server-template`` element in
:source:`templates.xml <etc/grid/templates.xml>` with is own parameters.

Modifying the application descriptors
Expand Down
35 changes: 5 additions & 30 deletions omero/sysadmins/troubleshooting.txt
View file
Open in desktop
Expand Up @@ -163,37 +163,12 @@ OutOfMemoryError / PermGen space errors in OMERO.server logs
Out of memory or permanent generation (PermGen) errors can be caused by
many things. You may be asking too much of the server. Or you may
require an increase in the maximum Java heap or the permanent generation
space. This can be done by modifying the IceGrid configuration for your
OMERO.server as follows:
space. This can be done by modifying the configuration for your
OMERO.server. See :ref:`jvm_memory_settings`.

- In ``etc/grid/templates.xml``:

::

…

<server-template id="BlitzTemplate">
<parameter name="index"/>
<parameter name="config" default="default"/>
<server id="Blitz-${index}" exe="java" activation="always" pwd="${OMERO_HOME}">
<!--
Debugging options:
<option>-Xdebug</option>
<option>-Xrunjdwp:server=y,transport=dt_socket,address=8787,suspend=n</option>
-->
<option>-Xmx512M</option>
<option>-Djava.awt.headless=true</option>
<option>-Dlog4j.configuration=${OMERO_ETC}log4j.xml</option>
…

Replace ``-Xmx512M`` initially with ``-Xmx1024M`` or greater as required,
(use ``-Xmx1024M -XX:MaxPermSize=128m`` as an alternative).

Alternatively, if you are finding out of memory errors in one of the other
process logs (e.g. Indexer-0.log or PixelData-0.log), look for the entry in
``etc/grid/templates.xml`` corresponding to that process and change the ``-Xmx``
option there (e.g. under ``<server-template id="IndexerTemplate">`` or
``<server-template id="PixelDataTemplate">``, respectively).
Similarly, if you are finding out of memory errors in one of the other
process logs (e.g. :file:`Indexer-0.log` or :file:`PixelData-0.log`),
you might try optimizing the JVM memory settings.

Furthermore, under certain conditions access of images greater than 4GB
can be problematic on 32-bit platforms due to certain bugs within the
Expand Down
35 changes: 6 additions & 29 deletions omero/sysadmins/unix/server-installation.txt
View file
Open in desktop
Expand Up @@ -698,35 +698,12 @@ Configuration
JVM memory settings
-------------------

To make your server production-ready, you will likely need to increase the JVM
memory settings of your application descriptors. This is not done by default
since it would prevent starting the server on some sites' test instance but
for production use a setting higher than 512MB is **highly** recommended.

To increase the JVM memory settings, you need to edit the
:file:`etc/grid/templates.xml` file under the OMERO.server directory.

It is highly recommended to increase the ``BlitzTemplate`` default JVM memory
settings from ``-Xmx512M`` to ``-Xmx2048M`` and from ``XX:MaxPermSize=128m``
to ``XX:MaxPermSize=256M`` in production.

If working with large images generates ``OutOfMemory`` exceptions in the
:file:`var/log/PixelData-0.log` file, you may consider increasing the
``PixelDataTemplate`` default JVM memory settings to ``-Xmx1024M``.

Once you have modified the :file:`templates.xml` file, restart the server
using::

$ bin/omero admin restart

.. seealso::

http://www.openmicroscopy.org/community/viewtopic.php?f=4&t=7400
Forum thread on PixelData JVM memory settings

:ref:`gridconfiguration`
Section of the advanced server configuration documentation describing
:file:`etc/grid/templates.xml`.
The OMERO server starts a number of Java services. Memory settings
for these are calculated on a system-by-system basis. An attempt
has been made to have usable settings out of the box, but if you
can afford to provide OMERO with more memory, it will certainly
improve your overall performance. See :ref:`jvm_memory_settings`
on how to tune the JVM.

OMERO.web and administration
----------------------------
Expand Down
32 changes: 4 additions & 28 deletions omero/sysadmins/windows/server-installation.txt
View file
Open in desktop
Expand Up @@ -675,35 +675,11 @@ Installation
JVM memory settings
-------------------

To make your server production-ready, you will likely need to increase the JVM
memory settings of your application descriptors. This is not done by default
since it would prevent starting the server on some sites' test instance but
for production use a setting higher than 512MB is **highly** recommended.
The OMERO server starts a number of Java services. Memory settings
for these are calculated on a system-by-system basis.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whitespace between sentences can be removed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I follow.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a new line between the end of the first sentence (... basis.) and the start of the next sentence (To increase...). The sentences are thematically similar, so it would look better, if the second would follow the first without extra new lines.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah! I was looking in the wrong server-installation.txt. Sorry about that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pushed.

To increase the JVM memory settings, you need to edit the
:file:`etc\\grid\\templates.xml` file under the OMERO.server directory.

It is highly recommended to increase the ``BlitzTemplate`` default JVM memory
settings from ``-Xmx512M`` to ``-Xmx2048M`` and from ``XX:MaxPermSize=128m``
to ``XX:MaxPermSize=256M`` in production.

If working with large images generates ``OutOfMemory`` exceptions in the
:file:`var\log\PixelData-0.log` file, you may consider increasing the
``PixelDataTemplate`` default JVM memory settings to ``-Xmx1024M``.

Once you have modified the :file:`templates.xml` file, restart the server
using::

C:\OMERO.server> bin\omero admin restart

.. seealso::

http://www.openmicroscopy.org/community/viewtopic.php?f=4&t=7400
Forum thread on PixelData JVM memory settings

:ref:`gridconfiguration`
Section of the advanced server configuration documentation describing
:file:`etc\\grid\\templates.xml`.
To increase the performance of your system, you may want to tune
the JVM. See :ref:`jvm_memory_settings`.

OMERO.web and administration
----------------------------
Expand Down