Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
244 lines (231 sloc) 7.57 KB
---
# Copyright 2017 Yahoo Holdings. Licensed under the terms of the Apache 2.0 license. See LICENSE in the project root.
title: "Files, Processes and Ports"
---
<p>
This is a reference of directories used in a Vespa installation,
processes that run on the Vespa nodes and ports used.
It is not necessary to know the information
in this file to operate Vespa in production as the system
automatically manages its files and processes.
See also <a href="logs.html">log files</a>.
</p>
<h2 id="directories">Directories</h2>
<table class="table">
<thead><tr><th>Directory</th><th>Description</th></tr></thead><tbody>
<tr>
<td>$VESPA_HOME/bin/</td>
<td>Command line utilities and scripts</td>
</tr>
<tr>
<td>$VESPA_HOME/sbin/</td>
<td>Server programs, daemons, etc</td>
</tr>
<tr>
<td>$VESPA_HOME/lib64/</td>
<td>Dynamically linked libraries, typically third-party
libraries</td>
</tr>
<tr>
<td>$VESPA_HOME/lib/jars/</td>
<td>Java archives</td>
</tr>
<tr>
<td>$VESPA_HOME/logs/vespa/</td>
<td>Log files</td>
</tr>
<tr>
<td>$VESPA_HOME/var/db/vespa/config_server/serverdb/</td>
<td>Config server database and user applications</td>
</tr>
<tr>
<td>$VESPA_HOME/share/vespa/</td>
<td>A directory with config definitions and XML schemas for
application package validation</td>
</tr>
<tr>
<td>$VESPA_HOME/conf/vespa</td>
<td>Various config files used by Vespa or libraries Vespa depend on</td>
</tr>
</tbody>
</table>
<h2 id="processes-and-ports">Processes and ports</h2>
<p>
The following is an overview of which ports and port ranges
are used by the different services in a Vespa system.
Note that for services capable of running multiple instances on the same node,
all instances will run within the listed port range.
</p><p>
Many services are allocated ports dynamically.
So even though the allocation is deterministic,
i.e. the same system will get the same ports on subsequent startups,
a particular service instance may get different ports
when the overall system setup is changed through <a href="services.html">services.xml</a>.
Use <a href="vespa-cmdline-tools.html#vespa-model-inspect">vespa-model-inspect</a>
to see port allocations.
</p>
<ul>
<li>
The number of ports used in a range depends on number of instances that are running
</li><li>
Not all ports within a range are used,
but they are assigned each service to support future extensions
</li><li>
The range from 19100 is used for internal communication ports,
i.e. ports that are not necessary to use from an external API
</li><li>
See <a href="../jdisc/http-server-and-filters.html">
Configuring Http Servers and Filters</a> for how to configure Container ports and
<a href="services.html">services.xml</a> for how to configure other ports
</li>
</ul>
<table class="table">
<thead><tr>
<th>Process</th><th>Host</th><th>Port/range</th><th>ps</th><th>Function</th></tr></thead><tbody>
<tr>
<td><a href="../cloudconfig/configuration-server.html">Config server</a></td>
<td>Config server nodes</td>
<td>19070 - 19071</td>
<td>java (...) -jar $VESPA_HOME/lib/jars/standalone-container-jar-with-dependencies.jar</td>
<td>Vespa Configuration server</td>
</tr>
<tr>
<td><a href="../config-sentinel.html">Config sentinel</a></td>
<td>All nodes</td>
<td>19098</td>
<td>$VESPA_HOME/sbin/vespa-config-sentinel</td>
<td>Sentinel that starts and stops vespa services and makes sure they
are running unless they are manually stopped</td>
</tr>
<tr>
<td><a href="config-proxy.html">Config proxy</a></td>
<td>All nodes</td>
<td>19090</td>
<td>java (&hellip;) com.yahoo.vespa.config.proxy.ProxyServer</td>
<td>Communication liaison between Vespa processes and config
server. Caches config in memory</td>
</tr>
<tr>
<td><a href="../slobrok.html">Slobrok</a></td>
<td>Admin nodes</td>
<td>19099 for RPC port, HTTP port dynamically allocated in 19100 - 19899 range</td>
<td>$VESPA_HOME/sbin/vespa-slobrok</td>
<td>Service location object broker</td>
</tr>
<tr>
<td><a href="logs.html#logd">logd</a></td>
<td>All nodes</td>
<td></td>
<td>$VESPA_HOME/sbin/vespa-logd</td>
<td>Reads local log files and sends them to log server</td>
</tr>
<tr>
<td><a href="logs.html#log-server">Log server</a></td>
<td>Log server node</td>
<td>19081 and 19083</td>
<td>java (...) -jar lib/jars/logserver-jar-with-dependencies.jar</td>
<td>Vespa Log server</td>
</tr>
<tr>
<td><a href="../distributor.html">Distributor</a></td>
<td>Content cluster</td>
<td>dynamically allocated in 19100 - 19899 range</td>
<td>$VESPA_HOME/sbin/vespa-distributord-bin</td>
<td>Content layer distributor processes</td>
</tr>
<tr>
<td><a href="../clustercontroller.html">Cluster controller</a></td>
<td>Content cluster</td>
<td>dynamically allocated in 19100 - 19899 range</td>
<td>java (...) -jar
$VESPA_HOME/lib/jars/container-disc-jar-with-dependencies.jar</td>
<td>Cluster controller processes, manages state for content nodes</td>
</tr>
<tr>
<td><a href="../querying-vespa.html#dispatch">Dispatch</a></td>
<td>Content cluster</td>
<td>dynamically allocated in 19100 - 19899 range</td>
<td>$VESPA_HOME/sbin/vespa-dispatch-bin</td>
<td>Dispatcher, communicates between container and content nodes.
Can be multi-level in a hierarchy</td>
</tr>
<tr>
<td><a href="../proton.html">proton</a></td>
<td>Content cluster</td>
<td>dynamically allocated in 19100 - 19899 range</td>
<td>$VESPA_HOME/sbin/vespa-proton-bin</td>
<td>Searchnode process, receives queries from the dispatcher and
returns results from the indexes. Also receives feed and indexes
documents</td>
</tr>
<tr>
<td><a href="../jdisc/index.html">container</a></td>
<td>Container cluster</td>
<td>8080</td>
<td>java (...) -jar $VESPA_HOME/lib/jars/container-disc-with-dependencies.jar</td>
<td>Container running servers, handlers and processing components</td>
</tr>
</tbody>
</table>
<h2 id="vespa-system-limits">System limits</h2>
<p>
The startup scripts in the next section will ensure that some of the system limits are
set to a value greater than or equal to the table below.
</p>
<table class="table">
<thead><tr>
<th>Name</th><th>Limit</th><th>Command executed</th></thead><tbody>
<tr>
<td>Max user processes</td>
<td>409600</td>
<td>ulimit -u 409600</td>
</tr>
<tr>
<td>Open files</td>
<td>262144</td>
<td>ulimit -n 262144</td>
</tr>
<tr>
<td>Core file size</td>
<td>unlimited</td>
<td>ulimit -c unlimited</td>
</tr>
</tbody>
</table>
<p>
If the startup scripts are not able to set the maximum number of open files Vespa will not start.
</p>
<h2 id="vespa-start-stop">Vespa start / stop</h2>
<p>
Start and stop <em>all</em> services on a node:
<pre>
$ $VESPA_HOME/bin/vespa-start-services
$ $VESPA_HOME/bin/vespa-stop-services
</pre>
Likewise, for the config server:
<pre>
$ $VESPA_HOME/bin/vespa-start-configserver
$ $VESPA_HOME/bin/vespa-stop-configserver
</pre>
Learn more about which processes are run at
<a href="../config-sentinel.html">Vespa startup</a>.
</p>
<h3 id="pre-start-check">Pre-start check</h3>
<p>
To run an automated node check before Vespa is started, provide a check script,
returning 0 if <em>service vespa-services start</em> is allowed to run.
This is useful when autostarting:
<pre>
$ export VESPA_VALIDATIONSCRIPT=/tmp/foo.bar.sh
$ cat /tmp/foo.bar.sh
#!/bin/sh
echo detect some problem
exit 1
$ $VESPA_HOME/bin/vespa-start-services
... starting ...
detect some problem
validation script '/tmp/foo.bar.sh' failed
$VESPA_HOME/bin/vespa-start-services failed: exit code 1 from command:
$VESPA_HOME/libexec/vespa/start-vespa-base.sh
</pre>
</p>