The next-generation Unbit hosting platform
Python Shell Perl C
Switch branches/tags
Nothing to show
Clone or download
Latest commit f852a97 Jun 17, 2018
Permalink
Failed to load latest commit information.
distros added nginx to artful Jun 17, 2018
services added support for ec2 ubuntu images and openstack clouds Nov 17, 2016
snippets Create nginx.md Jun 17, 2018
uwsgi_it_api Update README Jul 28, 2017
.gitignore api to retrive domains in a specific container and May 5, 2017
API.md Update API.md Jan 9, 2014
Alarms.md grammar fixes in Alarms documentation Nov 3, 2014
Changelog.md Update Changelog.md Feb 4, 2015
CustomDistros.md Update CustomDistros.md Jun 30, 2015
CustomerQuickstart.md Update CustomerQuickstart.md Aug 4, 2015
FreeBSD.md Create FreeBSD.md Jan 20, 2014
FullServerCustomerQuickstart.md Update FullServerCustomerQuickstart.md Sep 13, 2016
INSTALL.md Update INSTALL.md Jan 19, 2017
LICENSE Initial commit Dec 19, 2013
PORTS.md Update PORTS.md Feb 7, 2014
README.md Update README.md Nov 7, 2016
build_uwsgi.sh added slack plugin Jul 19, 2015
calc_ip.c automatically assign ip to the tuntap peers Dec 24, 2013
collector.pl increased timeout in configurators Jun 3, 2018
configurator.pl increased timeout in configurators Jun 3, 2018
dgram_router.c fixed dgram router Jan 12, 2014
dominator.pl increased timeout in configurators Jun 3, 2018
emperor.conf better emperor.conf Nov 19, 2014
emperor.ini more gentle quotaon in emperor Mar 30, 2017
emperor.service added systemd support Sep 23, 2016
firewall.sh improved firewall and added subscription for uid 33 Jan 12, 2014
generate_30000.py added domain collector Apr 25, 2014
local.ini added env Dec 24, 2013
loopbox.c fixed include for xenial Sep 23, 2016
loopboxer.pl fixed loopboxer Jun 3, 2018
nsswitch.conf added utopic Oct 25, 2014
openat_alarm.c implemented openat alarm Oct 17, 2014
rand_pid.c added rand_pid plugin Jan 13, 2014
shortcuts.ini added certificate authentication shortcut Nov 16, 2014

README.md

uwsgi.it

The latest Unbit hosting platform, sponsored by:

unbit.com

20Tab S.r.l.

Holey S.r.l.

Consulthink S.p.a.

Intro

contrary to the old unbit.it hosting platform, the new one is:

  • fully open source (oldest platforms was more-or-less proprietary software)
  • can be installed on vanilla kernels
  • everyone can build it on his/her systems (and eventually buy commercial support from unbit.com ;)
  • will not rely on apache (so .htaccess will not be supported, unless you install apache in your container and proxy it via uWSGI routing)

Features and Goals

  • each customer has a pool of containers
  • each container has an associated disk quota, a cpu share and a fixed amount of memory
  • each container has an associated Emperor
  • best possible isolation between containers
  • each container can be mapped to a different distribution (both 32 and 64 bit)
  • each container has its dedicated firewall based on the tuntap router plugin
  • ssh access is governed by the container emperor using the pam-unbit project
  • uid/gid mapping is managed using nss-unbit project
  • each container runs with its own uid/gid
  • each container has its own /etc/hosts, /etc/hostname and /etc/resolv.conf
  • each vassal in the container subscribes to a central http router with a specific key (domain)
  • containers' Emperor by default configure alarms for: disk quota, oom, memory thresholds, restarts
  • gather metrics and generate graphs
  • SNI is the only https/spdy supported approach
  • cron and external processes (like dbs) are managed as vassals
  • native (as uWSGI plugins) supported languages are Perl, CPython, PyPy, Ruby, Lua, Java and php (yes php apps works even without .htaccess...)
  • you can easily proxy (in offloaded mode too) to external services, like nodejs, meteor, zope, elasticsearch ...
  • Websockets support (in the routers/proxy) is enabled by default
  • Simple clustering and load-balancing
  • Sending emails is not part of the infrastructure (read:no SMTP services), but each container has transparent support for the nullmailer spool service (so you can use it to asynchronously send mails to external smtp services like mandrill and sendgrid)
  • /usr/local must be user-writable to allows custom installation/compilation (is bind-mounted to the container's home)
  • customers can buy a whole server, and create containers without supplier intervention
  • the unbit nss module exposes a name resolution facility to map container.local to the relevant ip
  • /run/shm (/dev/shm) is automatically mapped to the whole container memory
  • /var/run/utmp only exports sessions running in a container
  • xattrs and acls
  • allows mounting and managing loop block device via api
  • The HTTP router can forward RTSP and chunked input requests
  • support for old CGI's (via uWSGI cgi plugin)
  • SFTP virtual chroot/jail is managed via the pysftpserver project
  • support in home etc/rc.local for distros without a uWSGI installation
  • support for custom distributions/images (https://github.com/unbit/uwsgi.it/blob/master/CustomDistros.md)

Status

Currently the platform is in-production for unbit.it services, and working on hetzner and ovh hardware.

You still need a bit of work to install on your systems. Contact info@unbit.it for more infos.

How it works

On server startup the emperor.ini is run. This Emperor starts 4 services:

an http/https router

a fastrouter

a legion manager

a containers manager

The 4th service manage vassals in /etc/uwsgi/vassals

Each vassal is spawned in a new Linux namespace and cgroup (all is native, no lxc is involved)

An external app (well it could be hosted on the same infrastructure too) serves the api (a django app)

All the customers vassals are created by the api.

Each vassal spawns a sub-Emperor with uid and gid > 30000, the user (the owner of the container) can now use this sub-Emperor to spawn its vassals.

The user can enter the container via ssh (a pam module calls setns() to attach to the running container)

The user can only view (and access) processes generated by the sub-Emperor (even the sub-Emperor is hidden)

Domains to containers mapping is done via the uWSGI secured subscription system. An RSA key pair is generated by the control webapp (a Django app) for each registered customer. This key is used by the user to subscribe to the http/https router

Subscriptions can pass SSL certificates to the router that reconfigure it to map them to domains (via SNI)

Four perl processes manage the infrastruture configuration:

  • configurator.pl -> manage containers vassal files
  • dominator.pl -> manage domains to rsa key mappings
  • collector.pl -> gather statistics from the various exposed metrics
  • loopboxer.pl -> manage loop block devices (loopbox)

TODO

  • Map public ipv4/ipv6 addresses directly to a container (via NAT)
  • Implement sub-users concept for the api (maybe as subuser@user)
  • Add support for https://github.com/unbit/spockfs mounts (maybe in the same way as loopboxes ?)
  • fastrouter-only implementation for nginx integration
  • is SPDY/HTTP2 support whorty ?

LONG TERM GOALS

  • FreeBSD jail support in addition to Linux namespaces ?