forked from knowrob/docker
-
Notifications
You must be signed in to change notification settings - Fork 0
/
openease_storage_concept.txt
78 lines (69 loc) · 3.68 KB
/
openease_storage_concept.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
############################################
# Docker data storage concept for openEASE #
############################################
0. About this document
This document describes the data storage locations for all components of openEASE. It is the reference for all questions regarding the creation/usage/deletion of new docker data containers, as well as backup and restore strategies.
ANY ADDITIONS or CHANGES to the docker data container structure of openEASE or the usage of external data sources MUST be documented in this document first!
1. Data from mounted host directories
The openEASE platform uses the following host directories and files:
- /var/run/docker.sock:
The unix socket file for the docker API. Access to this file basically is equivalent to having ROOT access to the HOST, so be careful! This file is needed to dynamically control easeapp and user containers from the dockerbridge. Also the docker-gen container has access to monitor container spawns/shutdowns for adjusting the nginx reverse proxy on the fly.
Used by: dockerbridge, docker-gen
- $DOCKER_REPO/nginx-proxy:
Contains the nginx config template file for docker-gen.
Used by: docker-gen
- $DOCKER_REPO/nginx-proxy/certs:
Contains the certificates for TLS used by nginx. Keep confidential!
Used by: nginx
2. Data containers and their volumes
*** ATTENTION: Changes made to the structure of data containers that need backup requires the adjustment of any backup and restore scripts! See 4. and 5. chapter of this document! ***
user_data
- /home/ros/user_data
Used for common data files only needed by the easeapp containers, like the prolog query history files for each user.
Used by: All easeapp containers.
mesh_data
- /home/ros/mesh_data
Used for storing mesh_data that is automatically downloaded by the easeapp/knowrob container
Used by: easeapp/knowrob container.
episode_data
- /episodes
Used for common data files needed in every user container and the easeapp containers, like common experiment data.
Used by: easeapp/knowrob container and (read-only) all easeapp/knowrob user containers.
ease_secret
- /etc/ease_secret
Used for storing the SECRET_KEY for the flask session.
Used by: All easeapp containers (read only).
lft_data
- /tmp/openEASE/dockerbridge
A folder, where easeapp containers temporarily stores large files and folders. Transfers of large files from the user data containers to the easeapp containers and back will are stored here.
Used by: All easeapp containers, intermediate containers created on the fly for every large file copy action
data_* (user data containers, * is the users user name)
- /home/ros/user_data
Used for data required by the user container, like custom prolog modules or generated files.
Used by: The corresponding user container, intermediate containers created on the fly for every large file copy action for the corresponding user
- /etc/rosauth
Contains rosauth websocket authentication secret individual for each user.
mongo_data
- /data/db
Contains the mongo dbs for shared experiment data
Used by: mongo_db
postgres_data
- /var/lib/postgresql/data
Contains the postgres databases for user account data and user roles
Used by: postgres_db
3. Data that needs to be backed up
On the host:
$DOCKER_REPO/nginx-proxy/certs (Do this manually and keep the backup at a safe place, the certificates must not be made public!)
Data containers:
user_data
- /home/ros/user_data
data_* (all user data containers using the knowrob/user_data base image)
- /home/ros/user_data
mongo_data
- /data/db
postgres_data
- /var/lib/postgresql/data
4. Backup
TBD
5. Restore
TBD