Skip to content
Backend.AI Manager and API Gateway Daemon
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
alembic Execute query with bindparam only when updates exist Jul 14, 2019
config Add "run-as-user" configuration Jul 3, 2019
docs refs #5: Implemented a Redis based instance/kernel mgmt. Jul 30, 2015
sample-configs feature/#149 Two types of vfolder: User and Group (#152) Jun 18, 2019
scripts Fix propagation of sslcert validation check option to manager.cli.etcd Feb 14, 2019
src/ai/backend Register public interfaces only when app exists (#8) Jul 15, 2019
tests tests: Update broken cases Jul 11, 2019
.dockerignore refs lablup/ Add Dockerfile and make it dockerizable. Apr 29, 2017
.editorconfig Add some sample config files Jul 1, 2019
.gitignore Update README Jun 25, 2019
.travis.yml feature/config-revamp (#155) Jun 25, 2019
CHANGELOG.rst Update changelog and bump version to 19.06.0b2 Jul 14, 2019
Dockerfile refs lablup/ Update Dockerfile for better build caching Apr 29, 2017
LICENSE refs lablup/ Add license and classifier info. Oct 6, 2016 README: Add recommended system/kernel parameters Jul 9, 2019 Update Jan 13, 2018
alembic.ini.sample Update alembic.ini.sample Aug 15, 2018
requirements-ci.txt feature/config-revamp (#155) Jun 25, 2019
requirements-dev.txt setup: Remove editable installation of common Sep 12, 2018
requirements.txt setup: Remove dependency on custom aiohttp fork Jun 6, 2018
setup.cfg setup: Update the minimum required version of Jul 14, 2019 repo: Restructure to use src directory and static setup.cfg metadata Aug 22, 2018

Backend.AI Manager with API Gateway

Package Structure

  • ai.backend
    • manager: Abstraction of agents and computation kernels
    • gateway: User and Admin API (REST/GraphQL) gateway based on aiohttp


Please visit the installation guides.

Kernel/system configuration

Recommended resource limits:


root hard nofile 512000
root soft nofile 512000
root hard nproc 65536
root soft nproc 65536
user hard nofile 512000
user soft nofile 512000
user hard nproc 65536
user soft nproc 65536


net.ipv4.ip_local_port_range="10000 65000"
net.ipv4.tcp_rmem=4096 12582912 16777216
net.ipv4.tcp_wmem=4096 12582912 16777216

For development


  • libnsappy-dev or snappy-devel system package depending on your distro
  • Python 3.6 or higher with pyenv and pyenv-virtualenv (optional but recommneded)
  • Docker 18.03 or later with docker-compose (18.09 or later is recommended)

Common steps

Clone the meta repository and install a "halfstack" configuration. The halfstack configuration installs and runs several dependency daemons such as etcd in the background.

$ git clone halfstack
$ cd halfstack
$ docker-compose -f docker-compose.halfstack.yml up -d

Then prepare the source clone of the agent as follows. First install the current working copy.

$ git clone manager
$ cd manager
$ pyenv virtualenv venv-manager
$ pyenv local venv-manager
$ pip install -U pip setuptools
$ pip install -U -r requirements-dev.txt

From now on, let's assume all shell commands are executed inside the virtualenv.

Halfstack (single-node development & testing)

Recommended directory structure

Install as an editable package in the manager (and the agent) virtualenvs to keep the codebase up-to-date.

$ cd manager
$ pip install -U -e ../common


Copy (or symlink) the halfstack configs:

$ cp config/halfstack.toml ./manager.toml
$ cp config/halfstack.alembic.ini ./alembic.ini

Set up Redis:

$ python -m ai.backend.manager.cli etcd put config/redis/addr

Set up the public Docker registry:

$ python -m ai.backend.manager.cli etcd put config/docker/registry/ ""
$ python -m ai.backend.manager.cli etcd put config/docker/registry/ "lablup"
$ python -m ai.backend.manager.cli etcd rescan-images

Set up the vfolder paths:

$ mkdir -p "$HOME/vfroot/local"
$ python -m ai.backend.manager.cli etcd put volumes/_mount "$HOME/vfroot"
$ python -m ai.backend.manager.cli etcd put volumes/_default_host local

Set up the database:

$ python -m ai.backend.manager.cli schema oneshot
$ python -m ai.backend.manager.cli fixture populate sample-configs/example-keypairs.json
$ python -m ai.backend.manager.cli fixture populate sample-configs/example-resource-presets.json

Then, run it (for debugging, append a --debug flag):

$ python -m ai.backend.gateway.server

To run tests:

$ python -m flake8 src tests
$ python -m pytest -m 'not integration' tests

Now you are ready to install the agent. Head to the README of Backend.AI Agent.

NOTE: To run tests including integration tests, you first need to install and run the agent on the same host.



Put a TOML-formatted manager configuration (see the sample in config/sample.toml) in one of the following locations:

  • manager.toml (current working directory)
  • ~/.config/ (user-config directory)
  • /etc/ (system-config directory)

Only the first found one is used by the daemon.

Also many configurations shared by both manager and agent are stored in etcd. As you might have noticed above, the manager provides a CLI interface to access and manipulate the etcd data. Check out the help page of our etcd command set:

$ python -m ai.backend.manager.cli etcd --help

If you run etcd as a Docker container (e.g., via halfstack), you may use the native client as well. In this case, PLEASE BE WARNED that you must prefix the keys with "/sorna/{namespace}" manaully:

$ docker exec -it ${ETCD_CONTAINER_ID} /bin/ash -c 'ETCDCTL_API=3 etcdctl ...'

Running from a command line

The minimal command to execute:

python -m ai.backend.gateway.server

For more arguments and options, run the command with --help option.

Writing a wrapper script

To use with systemd, crontab, and other system-level daemons, you may need to write a shell script that executes specific CLI commands provided by Backend.AI modules.

The following example shows how to set up pyenv and virtualenv for the script-local environment. It runs the gateway server if no arguments are given, and execute the given arguments as a shell command if any. For instance, you may get/set configurations like: python -m ai.backend.manager.etcd ... where the name of scripts is

#! /bin/bash
if [ -z "$HOME" ]; then
  export HOME="/home/devops"
if [ -z "$PYENV_ROOT" ]; then
  export PYENV_ROOT="$HOME/.pyenv"
  export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
pyenv activate venv-bai-manager

if [ "$#" -eq 0 ]; then
  exec python -m ai.backend.gateway.server
  exec "$@"


The manager and agent should run in the same local network or different networks reachable via VPNs, whereas the manager's API service must be exposed to the public network or another private network that users have access to.

The manager requires access to the etcd, the PostgreSQL database, and the Redis server.

User-to-Manager TCP Ports Usage
manager:{80,443} Backend.AI API access
Manager-to-X TCP Ports Usage
etcd:2379 etcd API access
postgres:5432 Database access
redis:6379 Redis API access

The manager must also be able to access TCP ports 6001, 6009, and 30000 to 31000 of the agents in default configurations. You can of course change those port numbers and ranges in the configuration.

Manager-to-Agent TCP Ports Usage
6001 ZeroMQ-based RPC calls from managers to agents
6009 HTTP watcher API
30000-31000 Port pool for in-container services
You can’t perform that action at this time.