Merge pull request #705 from minrk/actual-services

WIP: implement services API

docs/rest-api.yml

@@ -312,6 +312,30 @@ paths:
       responses:
         '200':
           description: The users have been removed from the group
+  /services:
+    get:
+      summary: List services
+      responses:
+        '200':
+          description: The service list
+          schema:
+            type: array
+            items:
+              $ref: '#/definitions/Service'
+  /services/{name}:
+    get:
+      summary: Get a service by name
+      parameters:
+        - name: name
+          description: service name
+          in: path
+          required: true
+          type: string
+      responses:
+        '200':
+          description: The Service model
+          schema:
+            $ref: '#/definitions/Service'
   /proxy:
     get:
       summary: Get the proxy's routing table
@@ -436,3 +460,26 @@ definitions:
         description: The names of users who are members of this group
         items:
           type: string
+  Service:
+    type: object
+    properties:
+      name:
+        type: string
+        description: The service's name
+      admin:
+        type: boolean
+        description: Whether the service is an admin
+      url:
+        type: string
+        description: The internal url where the service is running
+      prefix:
+        type: string
+        description: The proxied URL prefix to the service's url
+      pid:
+        type: number
+        description: The PID of the service process (if managed)
+      command:
+        type: array
+        description: The command used to start the service (if managed)
+        items:
+          type: string

docs/source/services.md

@@ -0,0 +1,88 @@
+# JupyterHub services
+
+JupyterHub 0.7 adds the notion of Services.
+A Service is a process that interacts with the Hub REST API.
+Services may perform actions such as shutting down user servers that have been idle for some time,
+or registering additional web servers that should also use the Hub's authentication
+and be served behind the Hub's proxy.
+
+There are two main characteristics of services:
+
+1. Is it **managed** by JupyterHub?
+2. Does it have a web server that should be added to the proxy?
+
+If a `command` is specified for launching the service, it will be started and managed by the Hub.
+If a `url` is specified for where the service runs its own webserver,
+it will be added to the Hub's proxy at `/service/:service-name`.
+
+## Managed services
+
+**Managed** services are services that the Hub starts and is responsible for.
+These can only be local subprocesses of the Hub,
+and the Hub will take care of starting these processes and restarting them if they stop.
+
+While there are similarities with notebook Spawners,
+there are no plans to support the same spawning abstractions as notebook.
+If you want to run these services in docker or other environments,
+you can register it as an external service below.
+
+A managed service is characterized by the `command` specified for launching the service.
+
+
+```python
+c.JupyterHub.services = [
+    {
+        'name': 'cull-idle',
+        'admin': True,
+        'command': ['python', '/path/to/cull-idle.py', '--interval']
+    }
+]
+```
+
+In addition to `command`, managed services can take additional optional parameters,
+to describe the environment in which to start the process:
+
+- `env: dict` additional environment variables for the service.
+- `user: str` name of the user to run the server as if different from the Hub.
+    Requires Hub to be root.
+- `cwd: path` directory in which to run the service, if different from the Hub directory.
+
+When the service starts, the Hub will pass the following environment variables:
+
+```
+JUPYTERHUB_SERVICE_NAME: the name of the service ('cull-idle' above)
+JUPYTERHUB_API_TOKEN: API token assigned to the service
+JUPYTERHUB_API_URL: URL for the JupyterHub API (http://127.0.0.1:8080/hub/api)
+JUPYTERHUB_BASE_URL: Base URL of the Hub (https://mydomain[:port]/)
+JUPYTERHUB_SERVICE_PREFIX: URL path prefix of this service (/services/cull-idle/)
+```
+
+## External services
+
+You can use your own service management tools, such as docker or systemd, to manage JupyterHub services.
+These are not subprocesses of the Hub, and you must tell JupyterHub what API token the service is using to perform its API requests.
+Each service will need a unique API token because the Hub authenticates each API request,
+identifying the originating service or user.
+
+An example of an externally managed service with admin access and running its own web server:
+
+```python
+c.JupyterHub.services = [
+    {
+        'name': 'my-web-service',
+        'url': 'https://10.0.1.1:1984',
+        'api_token': 'super-secret',
+    }
+]
+```
+
+
+## Writing your own services
+
+TODO
+
+### Authenticating with the Hub
+
+TODO
+
+JupyterHub 0.7 introduces some utiltiies for you to use that allow you to use the Hub's authentication mechanism.

examples/cull-idle/cull_idle_servers.py

@@ -9,10 +9,21 @@
 - single-user websocket ping interval (default: 30s)
 - JupyterHub.last_activity_interval (default: 5 minutes)
 
-Generate an API token and store it in `JPY_API_TOKEN`:
+You can run this as a service managed by JupyterHub with this in your config::
 
-    export JPY_API_TOKEN=`jupyterhub token`
-    python cull_idle_servers.py [--timeout=900] [--url=http://127.0.0.1:8081/hub]
+
+    c.JupyterHub.services = [
+        {
+            'name': 'cull-idle',
+            'admin': True,
+            'command': 'python cull_idle_servers.py --timeout=3600'.split(),
+        }
+    ]
+
+Or run it manually by generating an API token and storing it in `JUPYTERHUB_API_TOKEN`:
+
+    export JUPYTERHUB_API_TOKEN=`jupyterhub token`
+    python cull_idle_servers.py [--timeout=900] [--url=http://127.0.0.1:8081/hub/api]
 """
 
 import datetime
@@ -34,7 +45,7 @@ def cull_idle(url, api_token, timeout):
     auth_header = {
             'Authorization': 'token %s' % api_token
         }
-    req = HTTPRequest(url=url + '/api/users',
+    req = HTTPRequest(url=url + '/users',
         headers=auth_header,
     )
     now = datetime.datetime.utcnow()
@@ -47,7 +58,7 @@ def cull_idle(url, api_token, timeout):
         last_activity = parse_date(user['last_activity'])
         if user['server'] and last_activity < cull_limit:
             app_log.info("Culling %s (inactive since %s)", user['name'], last_activity)
-            req = HTTPRequest(url=url + '/api/users/%s/server' % user['name'],
+            req = HTTPRequest(url=url + '/users/%s/server' % user['name'],
                 method='DELETE',
                 headers=auth_header,
             )
@@ -60,15 +71,15 @@ def cull_idle(url, api_token, timeout):
         app_log.debug("Finished culling %s", name)
 
 if __name__ == '__main__':
-    define('url', default='http://127.0.0.1:8081/hub', help="The JupyterHub API URL")
+    define('url', default=os.environ.get('JUPYTERHUB_API_URL'), help="The JupyterHub API URL")
     define('timeout', default=600, help="The idle timeout (in seconds)")
     define('cull_every', default=0, help="The interval (in seconds) for checking for idle servers to cull")
 
     parse_command_line()
     if not options.cull_every:
         options.cull_every = options.timeout // 2
 
-    api_token = os.environ['JPY_API_TOKEN']
+    api_token = os.environ['JUPYTERHUB_API_TOKEN']
 
     loop = IOLoop.current()
     cull = lambda : cull_idle(options.url, api_token, options.timeout)

examples/cull-idle/jupyterhub_config.py

@@ -0,0 +1,8 @@
+# run cull-idle as a service
+c.JupyterHub.services = [
+    {
+        'name': 'cull-idle',
+        'admin': True,
+        'command': 'python cull_idle_servers.py --timeout=3600'.split(),
+    }
+]

jupyterhub/apihandlers/__init__.py

@@ -1,11 +1,6 @@
 from .base import *
-from .auth import *
-from .hub import *
-from .proxy import *
-from .users import *
-from .groups import *
-from . import auth, hub, proxy, users
+from . import auth, hub, proxy, users, groups, services
 
 default_handlers = []
-for mod in (auth, hub, proxy, users, groups):
+for mod in (auth, hub, proxy, users, groups, services):
     default_handlers.extend(mod.default_handlers)

jupyterhub/apihandlers/proxy.py

@@ -28,7 +28,7 @@ def get(self):
     @gen.coroutine
     def post(self):
         """POST checks the proxy to ensure"""
-        yield self.proxy.check_routes(self.users)
+        yield self.proxy.check_routes(self.users, self.services)
 
 
     @admin_only
@@ -59,7 +59,7 @@ def patch(self):
             self.proxy.auth_token = model['auth_token']
         self.db.commit()
         self.log.info("Updated proxy at %s", server.bind_url)
-        yield self.proxy.check_routes(self.users)
+        yield self.proxy.check_routes(self.users, self.services)
 
 
 

jupyterhub/apihandlers/services.py

@@ -0,0 +1,64 @@
+"""Service handlers
+
+Currently GET-only, no actions can be taken to modify services.
+"""
+
+# Copyright (c) Jupyter Development Team.
+# Distributed under the terms of the Modified BSD License.
+
+import json
+
+from tornado import web
+
+from .. import orm
+from ..utils import admin_only
+from .base import APIHandler
+
+def service_model(service):
+    """Produce the model for a service"""
+    return {
+        'name': service.name,
+        'admin': service.admin,
+        'url': service.url,
+        'prefix': service.server.base_url if service.server else '',
+        'command': service.command,
+        'pid': service.proc.pid if service.proc else 0,
+    }
+
+class ServiceListAPIHandler(APIHandler):
+    @admin_only
+    def get(self):
+        data = {name: service_model(service) for name, service in self.services.items()}
+        self.write(json.dumps(data))
+
+
+def admin_or_self(method):
+    """Decorator for restricting access to either the target service or admin"""
+    def decorated_method(self, name):
+        current = self.get_current_user()
+        if current is None:
+            raise web.HTTPError(403)
+        if not current.admin:
+            # not admin, maybe self
+            if not isinstance(current, orm.Service):
+                raise web.HTTPError(403)
+            if current.name != name:
+                raise web.HTTPError(403)
+        # raise 404 if not found
+        if name not in self.services:
+            raise web.HTTPError(404)
+        return method(self, name)
+    return decorated_method
+
+class ServiceAPIHandler(APIHandler):
+
+    @admin_or_self
+    def get(self, name):
+        service = self.services[name]
+        self.write(json.dumps(service_model(service)))
+
+
+default_handlers = [
+    (r"/api/services", ServiceListAPIHandler),
+    (r"/api/services/([^/]+)", ServiceAPIHandler),
+]