/
ScalingOmero.txt
69 lines (50 loc) · 2.05 KB
/
ScalingOmero.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
Scaling Omero
=============
There are several ways that OMERO, or any server system, can scale.
Optimizing your system for more than one of these factors is
non-trivial, but we try to lay out some guidelines below for what has
worked, what almost certainly will not work, and what -- under the right
circumstances -- might be optimal.
Concurrent invocations
----------------------
The bottlenecks for concurrent invocations are:
- database connections
- server threads
- the router
Database connections
~~~~~~~~~~~~~~~~~~~~
Database servers, in general, have a maximum number of allowed
connections. In postgres, the default ``max_connections`` is 100, though
in many cases this will be significantly lower due to the available
shared memory (SHMMAX). If OMERO were to use direct connections to the
database, after ``max_connections`` invocations, all further attempts to
connect to the server would fail with "too many connection" exceptions.
Instead, OMERO uses a **connection pool** in front of Postgres, which
manages many more simultaneous attempts to connect to the database.
With the default ``max_connection`` set to 64,
it is possible to execute 500 queries simultaneously without database
exceptions. Instead, one receives server exceptions.
Server threads
~~~~~~~~~~~~~~
In :doc:`/developers/server-blitz`, too many (500+ on the default
configuration) simultaneous invocations will result in
``ConnectionLost`` exceptions. We are currently working on ways to
extend the number of single invocations on one server, but a simpler
solution is to start another :doc:`/developers/server-blitz` server.
Total throughput
----------------
The bottlenecks for throughput are:
- maximum message size
- server memory
- IO
- network
--------------
.. seealso::
:doc:`/sysadmins/unix/server-postgresql`
Instructions about OMERO.server and PostgreSQL under UNIX & UNIX-like
platforms.
:doc:`/sysadmins/windows/server-postgresql`
Instructions about OMERO.server and PostgreSQL under Windows
platforms.
|OmeroGrid|
:ticket:`906`