This config resides in /etc/lithos/sandboxes/NAME.yaml
(by default). Where NAME
is the name of a sandbox.
The configuration file contains security and resource limits for the container. Including:
- A directory where image resides
- Set of directories that are mounted inside the container (i.e. all writable directories for the container, the
/tmp
...) - User and group limits
See overview <sandbox-overview>
for guidelines.
config-file
The path for the processes config<process_config>
. In most cases should be left unset. Default is null
which is results into /etc/lithos/processes/NAME.yaml
with all other settings being defaults.
image-dir
Directory where application images are. Every subdir of the image-dir
may be mounted as a root file system in the container. Required.
image-dir-levels
(default 1
) A number of directory components required for image name in image-dir
log-file
The file name where to put supervisor log of the container. Default is /var/log/lithos/SANDBOX_NAME.yaml
.
log-level
(default warn
). The logging level of the supervisor.
readonly-paths
The mapping of virtual_directory: host_system_directory
of folders which are visible for the container in read-only mode. (Note currently if you have submounts in the source directory, thay may be available as writeable). See Volumes
for more details.
writable-paths
The mapping of virtual_directory: host_system_directory
of folders which are visible for the container in writable mode. See Volumes
for more details.
allow-users
List of ranges of user ids which can be used by container. For containers without user namespaces, it's just a limit of the user-id
setting.
Example:
allow-users: [1, 99, 1000-2000]
For containers which have uid maps enabled in sandbox this is a list of users available after uid mapping applied. For example, the following maps uid 100000 as root in namespace (e.g. for file permissions), but doesn't allow to start process as root (even if it's 100000 ouside):
uid-map: [{outside: 100000, inside: 0, count: 65536}]
allow-users: [1-65535]
For containers which do have uid maps enabled in container config, it limits all the user ids available to the namespace (i.e. for the outside setting of the uid map).
default-user
(no default) A user id used in the container if no user-id
is specified in container config. By default user-id
is required.
Note: default-user
value must be contained in the allow-users
range
allow-groups
List of ranges of group ids for the container. Works similarly to allow-users
.
default-group
(default 0
) A group id used in the container if no group-id
is specified in container config.
Note: default-group
value must be contained in the allow-users
range
allow-tcp-ports
List of ranges of allowed TCP ports for container. This is currently not enforced in any way except:
- Ports < 1024 are restricted by OS for non-root (but may be allowed here)
- It restricts
bind-port
setting in container config
Note
if you have overlapping TCP port for different sandboxes, only single file descriptor will be used for each port. The config for opening port will be used arbitrary from single config amonst all users, which have obvious security implications.
Warning
tcp-ports
bind at port in host namespace, i.e. it effectively discards bridged-network
for that port this is both the feature and might be a pitfall. So most of the time you should avoid non-empty allow-tcp-ports
if using bridged-network.
additional-hosts
Mapping of hostname: ip
for names that will be added to /etc/hosts
file. This is occasinally used for cheap but static service discovery.
uid-map, gid-map
The list of mapping for uids(gids) in the user namespace of the container. If they are not specified the user namespace is not used. This setting allows to run processes with uid
zero without the risk of being the root
on host system.
Here is a example of maps:
uid-map:
- {inside: 0, outside: 1000, count: 1}
- {inside: 1, outside: 1, count: 1}
gid-map:
- {inside: 0, outside: 100, count: 1}
Note
Currently you may have uid-map either in a sandbox or in a container config, not both.
used-images-list
(optional) A text file that is used by lithos_clean
to keep images alive. It's not used by any other means except lithos_clean
utility.
Each line of the file should contain image name relative to the image_dir
.
It's expected that the list is kept up by some orchestration system or by deployment scripts or by any other tool meaningful for ops team.
This setting is only useful if auto-clean
is true
(default)
auto-clean
(default true
) Clean images of this sandbox when running lithos_clean
. This is a subject of the following caveats:
- Lithos clean is not run by lithos automatically, you ought to run it using cron tab
- If same
image-dir
is used for multiple sandboxes it will be cleaned if at least one of them has non-falsyauto-clean
.
resolv-conf
(default /etc/resolv.conf
) default place to copy resolv.conf
from for containers.
Note: Container itself can override it's own resolv.conf file, but can't read original /etc/resolv.conf
if this setting is changed.
hosts-file
(default /etc/hosts
) default place to copy hosts
from for containers.
Note: Container itself can override it's own hosts
file, but can't read original /etc/hosts
if this setting is changed.
bridged-network
(default is absent) a network bridge configuration for all the cotainers in the bridge
Example:
bridged-network:
bridge: br0
network: 10.0.0.0/24
default_gateway: 10.0.0.1
after-setup-command: [/usr/bin/arping, -U, -c1, '@{container_ip}']
Note
when bridged network is active your process_config
should contain a list of ip addresses one for each container.
Note
this setting does not affect tcp-ports
. So usually you should keep allow-tcp-ports
setting empty when using bridged network.
Options:
after-setup-command
Command to run after setting up container namespace but before running actual container. The example shown above sends unsolicited arp packet to notify router and other machines on the network that MAC address corresponding to container's IP is changed.
Command must have absolute path, and has almost empty environment, so don't assume PATH
is there if you're writing a script. Command runs in container's network namespace but with all other namespaces in host system (in particular in host filesystem and with permissions of root in host system)
Replacement variables that work in command-line:
@{container_ip}
-- replaced with IP address of a container being set up
Few examples:
[/usr/bin/arping, -U, -c1, '@{container_ip}']
-- default in v0.17.x. This notifies other peers that MAC address for this IP changed.[/usr/bin/arping, -c1, '10.0.0.1']
-- other way to do that, that often does the same as in (1) a side-effect (where 10.0.0.1 is a default gateway)[/usr/bin/ping, -c1, '10.0.0.1']
-- doing same as (2) but using ICMP instead of ARP directly
Most of the time containers should work with empty after-setup-command
, but because container gets new MAC address each time it starts, there might be a small delay (~ 5 sec) after container's start where packets going to that IP are lost (so it appears that host is unavailable).
secrets-private-key
(default is absent) Use the specified private key(s) to decode secrets in container's secret-environ
setting.
The key in this file is openssh-compatible ed25519 private key (RSA keys are not supported). File can contain multiple keys (concatenated), if secret matches any of them it will be decoded.
To create a key use normal ssh-keygen
and leave the password empty (password-protected keys aren't supported):
ssh-keygen -t ed25519 -t /etc/lithos/keys/secret.key
Note: the key must be owned by root with permissions of 0600 (default for ssh-keygen).
secrets-namespaces
(default is [""]) allow only secrets with listed namespaces. Useful only if secrets-private-key
is set.
For example:
secrets-namespaces:
- project1.web
- project1.celery
The idea is you might want to use single secret private key for a whole cluster. But diferent services having different "namespaces". This means you can use single public key for encyption and specify different namespace for each service. With this setup user can't just copy a key from one service to another if that another service isn't authorized to read the namespace using secrets-namespaces
.
To encrypt secret for a specific namespace use:
lithos_crypt encrypt -k key.pub -d "secret" -n "project1.web"
By default both lithos_crypt
and secrets-namespaces
specify empty string as a namespace. This is good enough if you don't have multiple teams sharing the same cluster.
Currently namespaces are limited to a regexp ^[a-zA-Z0-9_.-]*$
See encrypted-vars
for more info.