83 docs/pages/administration/database/Oracle-Integration.md
View file
@@ -0,0 +1,83 @@
[Documentation](.) > [Administration](Administration) > [Database Integration](Database Integration) > Oracle Integration

# Oracle Integration

## Quick Setup

From a terminal, run the following to setup the Crossbar.io Oracle database integration:

```
cd database
sqlplus sys/oracle@localhost:1521/orcl as sysdba @create_users.sql
sqlplus cbdb/crossbar@localhost:1521/orcl @install.sql
```

> Note: Above assumes Oracle database is listening on `localhost` port `1521` using the service name `ORCL`, and that `SYS` still has the default password `oracle`.
Then start Crossbar.io

```
crossbar start
```

## Installation

By default, the installation scripts generated will create two database users:

* `CBADAPTER`
* `CBDB`

The `CBADAPTER` user is used by Crossbar.io to connect to Oracle. Other than `CREATE SESSION`, this user only has `EXECUTE` permission on the public API (the stored procedures) exposed by the Crossbar.io database integration.

The `CBDB` user is the owning schema of the database objects making up the Crossbar.io database integration.

To create those user, you must be connected as `SYS` and run the following script (from within the `database` directory generated):

```console
$ sqlplus sys/oracle@localhost:1521/orcl as sysdba @create_users.sql

SQL*Plus: Release 12.1.0.2.0 Production on Mon Oct 27 12:49:40 2014

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

...

No errors.
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
```

The next step creates all the database objects that together make up the Crossbar.io database integration.

To create those objects, you must be connected as `CBDB` and run the following script (from within the `database` directory generated):

```console
$ sqlplus cbdb/crossbar@localhost:1521/orcl @install.sql

SQL*Plus: Release 12.1.0.2.0 Production on Mon Oct 27 12:50:22 2014

Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

****** Installing Crossbar.io Oracle Integration *******

...

****** Installation complete *******
No errors.
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
```




6 docs/pages/administration/database/PostgreSQL-Integration-Callee.md
View file
@@ -0,0 +1,6 @@
[Documentation](.) > [Administration](Administration) > [Database Integration](Database Integration) > PostgreSQL Integration Callee

# PostgreSQL Integration - Callee

-- write me --

6 docs/pages/administration/database/PostgreSQL-Integration-Caller.md
View file
@@ -0,0 +1,6 @@
[Documentation](.) > [Administration](Administration) > [Database Integration](Database Integration) > PostgreSQL Integration Caller

# PostgreSQL Integration - Caller

-- write me --

212 docs/pages/administration/database/PostgreSQL-Integration-Publisher.md
View file
@@ -0,0 +1,212 @@
[Documentation](.) > [Administration](Administration) > [Database Integration](Database Integration) > [PostgreSQL Integration](PostgreSQL Integration) > PostgreSQL Publisher

# PostgreSQL Publisher

### Node Configuration

#### Running inside a Container Worker

For running the **Publisher** integration, here is how you would configure your node (only relevant parts are shown - [here](https://github.com/crossbario/crossbarexamples/blob/master/database/postgresql/publisher/.crossbar/config_using_container.json) is complete config) to have the adapter run in a separate worker process:

```json
{
"workers": [
{
"type": "router",
...
"transports": [
{
"type": "web",
"endpoint": {
"type": "tcp",
"port": 8080
},
"paths": {
...
"ws": {
"type": "websocket"
}
}
}
]
},
{
"type": "container",
"options": {
"reactor": {
"win32": "select"
}
},
"components": [
{
"type": "class",
"classname": "crossbar.adapter.postgres.PostgreSQLPublisher",
"realm": "realm1",
"transport": {
"type": "websocket",
"endpoint": {
"type": "tcp",
"host": "127.0.0.1",
"port": 8080
},
"url": "ws://127.0.0.1:8080/ws"
},
"extra": {
"database": {
"host": "127.0.0.1",
"port": 5432,
"database": "cdc",
"user": "crossbar",
"password": "crossbar"
}
}
}
]
}
]
}
```

Essentially, the PostgreSQL integration for Publisher is provided as a WAMP component `crossbar.adapter.postgres.PostgreSQLPublisher` which can be run and connected to a router worker as any other WAMP component.

The integration component however requires an `extra` configuration with the following attributes:

option | description
---|---
**`database`** | A dictionary with database connection configuration (*required*)
**`database.host`** | The hostname or IP address of the target PostgreSQL database cluster to connect to (*required*).
**`database.port`** | The TCP listening port on the PostgreSQL database cluster to connect to (*required*).
**`database.database`** | The database on the PostgreSQL cluster to connect to (*required*).
**`database.user`** | The name of the user to connect to the PostgreSQL cluster (*required*).
**`database.password`** | The password of the user to connect to the PostgreSQL cluster (*required*).


#### Running inside a Router Worker

Since the PostgreSQL integration is provided as a regular WAMP component (`crossbar.adapter.postgres.PostgreSQLPublisher`), you can also run the integration within a *router worker* (side-by-side). Here are relevant parts of a config (complete config [here](https://github.com/crossbario/crossbarexamples/blob/master/database/postgresql/publisher/.crossbar/config.json)):

```json
{
"workers": [
{
"type": "router",
...
"components": [
{
"type": "class",
"classname": "crossbar.adapter.postgres.PostgreSQLPublisher",
"realm": "realm1",
"role": "anonymous",
"extra": {
"database": {
"host": "127.0.0.1",
"port": 5432,
"database": "cdc",
"user": "crossbar",
"password": "crossbar"
}
}
}
]
}
]
}
```

#### Database Connection Parameters from Environment Variables

Instead of providing database connection configuration using literal values in the node configuration, you can also use **environment variables**:

```json
"extra": {
"database": {
"host": "127.0.0.1",
"port": 5432,
"database": "cdc",
"user": "crossbar",
"password": "$PGPASSWORD"
}
}
```

Crossbar.io will now try to read the `extra.database.password` attribute from the environment variable `PGPASSWORD`:

```console
export PGPASSWORD="crossbar"
```

You can use any names for your environment variables, as long as the name match the regular expression `^\$([A-Z0-9_]+)$`.

However, PostgreSQL has a number of [standard environment variables](http://www.postgresql.org/docs/current/static/libpq-envars.html), and it might be wise to reuse those, as these are usually used by other tools as well (such as `psql`):

* `PGHOST`
* `PGPORT`
* `PGDATABASE`
* `PGUSER`
* `PGPASSWORD`


### Using the PostgreSQL Integration

Publishing events from within PostgreSQL is done by calling the stored procedure `crossbar.publish()` provided by the integration:

```sql
FUNCTION crossbar.publish (
topic TEXT,
args JSONB DEFAULT NULL,
kwargs JSONB DEFAULT NULL,
options JSONB DEFAULT NULL,
autonomous BOOLEAN DEFAULT FALSE
) RETURNS BIGINT
```

with these parameters

* `topic` The WAMP URI the event should be published to (*required*)
* `args` Any positional payload for the event.
* `kwargs` Any keyword-based payload for the event.
* `options` WAMP publish options (see below).
* `autonomous`: If `TRUE`, do the publish in an autonomous transaction, otherwise do the publish within the current transaction context (*[NOT YET IMPLEMENTED](https://github.com/crossbario/crossbar/issues/223)*)

Here are a couple of examples:

```sql
SELECT crossbar.publish('com.example.topic1');
```

This will publish an event to topic `com.example.topic1` with no payload.

Of course you can supply payload as well:

```sql
SELECT crossbar.publish('com.example.topic1', json_build_array('hello world!', 23)::jsonb);
```

Here, the event is published with (positional) payload consisting of two arguments `'hello world!'` and `23`.

You can supply both positional and keyword-based payload for events

```sql
SELECT crossbar.publish('com.example.topic1',
json_build_array(23, 7, 'hello world!')::jsonb,
json_build_object('foo', 'bar', 'baz', 42)::jsonb
);
```

#### Publish Options

The PostgreSQL Publisher integration supports WAMP options for publishing events:

```sql
SELECT crossbar.publish('com.example.topic1',
options := json_build_object(
'exclude', json_build_array(1968608799)::jsonb
)::jsonb
);
```

The following options are supported:

* `exclude` - a list of WAMP session IDs (integers)
* `eligible` - a list of WAMP session IDs (integers)
* `acknowledge` - *[NOT YET IMPLEMENTED](https://github.com/crossbario/crossbar/issues/357)*
5 docs/pages/administration/database/PostgreSQL-Integration-Subscriber.md
View file
@@ -0,0 +1,5 @@
[Documentation](.) > [Administration](Administration) > [Database Integration](Database Integration) > PostgreSQL Integration Subscriber

# PostgreSQL Integration - Subscriber

-- write me --
160 docs/pages/administration/database/PostgreSQL-Integration.md
View file
@@ -0,0 +1,160 @@
[Documentation](.) > [Administration](Administration) > [Database Integration](Database Integration) > PostgreSQL Integration

# PostgreSQL Integration

## Introduction

The PostgreSQL integration provided by Crossbar.io extends WAMP directly into the database. Using the integration services, you can

* **[publish](PostgreSQL-Integration-Publisher)** WAMP events directly from within the database (e.g. a database trigger or stored procedure)
* **[subcribe](PostgreSQL-Integration-Subscriber)** database stored procedures to receive WAMP events on a topic
* **[register](PostgreSQL-Integration-Callee)** database stored procedures to be called transparently like any other WAMP procedure
* **[call](PostgreSQL-Integration-Caller)** any WAMP procedure from within the database (e.g. a database trigger or stored procedure)

You can find complete, working examples in the [Crossbar.io examples repository](https://github.com/crossbario/crossbarexamples/):

* [PostgreSQL Publisher Example](https://github.com/crossbario/crossbarexamples/tree/master/database/postgresql/publisher)
* [PostgreSQL Subscriber Example](https://github.com/crossbario/crossbarexamples/tree/master/database/postgresql/subscriber)
* [PostgreSQL Caller Example](https://github.com/crossbario/crossbarexamples/tree/master/database/postgresql/caller)
* [PostgreSQL Callee Example](https://github.com/crossbario/crossbarexamples/tree/master/database/postgresql/callee)

**Why?**

This is an incredibly powerful feature, as it allows you to (selectively) move business logic into the database, close to your data, greatly reducing latencies, round-trips and increasing security, as you have full and application-independent control and a clean API to your database, that encaspulates and abstracts away SQL, shielding you from schema changes.

And the best: doing so is fully transparent to all consumers. A WAMP caller that calls into the database is completely *unaware* that the callee actually is a stored procedure running inside a database.


## Installation

### Prerequisites

You will need to have Crossbar.io installed with PostgreSQL support:

```console
pip install crossbar[postgres]
```

or

```console
pip install crossbar[all]
```

> Note: On Debian/Ubuntu, you might need the PostgreSQL client library with development headers: `sudo apt-get install -y libpq-dev`

### Database Setup

To use the PostgreSQL integration services of Crossbar.io, the following needs to be run **once** as a database superuser (e.g. `postgres`) connected to the database where you want to have the intgration:

```sql
CREATE ROLE crossbar LOGIN INHERIT;
ALTER ROLE crossbar ENCRYPTED PASSWORD 'crossbar';
CREATE SCHEMA crossbar AUTHORIZATION crossbar;
GRANT USAGE ON SCHEMA crossbar TO public;
```

**For production, you will want to change the password 'crossbar' above to something safe!** The examples assume the password 'crossbar' though.

> Above commands will setup a database schema `crossbar` where Crossbar.io will create database objects (tables, stored procedures, ..) needed for the integration, and it will create a database user `crossbar` under which Crossbar.io will connect to your PostgreSQL database.
To remove everything again:

```sql
DROP OWNED BY crossbar CASCADE;
DROP SCHEMA IF EXISTS crossbar CASCADE;
DROP ROLE IF EXISTS crossbar;
```

## Notes on PostgreSQL

For convenience, here are some notes regarding PostgreSQL on Debian/Ubuntu.

Further information can be found in the [Ubuntu/PostgreSQL documentation](https://help.ubuntu.com/community/PostgreSQL) and the [Linux/PostgreSQL documentation](http://www.postgresql.org/download/linux/ubuntu/).

### Installation

To install PostgreSQL on Debian/Ubuntu

```console
sudo apt-get install -y postgresql-9.4
```

If your Debian/Ubuntu is too old and lacks PostgreSQL 9.4, here is how you can add the PostgreSQL project's binary package repository which always contains the latest version.

Add the PostgreSQL package repository

```console
sudo vi /etc/apt/sources.list.d/pgdg.list
```

and add the following line in this file

```
deb http://apt.postgresql.org/pub/repos/apt/ trusty-pgdg main
```

Then, add the PostgreSQL package repository key

```console
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
```

update the package list

```console
sudo apt-get update
```

and finally install PostgreSQL

```console
sudo apt-get install -y postgresql-9.4
```

This will create a new PostgreSQL database cluster with these directories

* `/etc/postgresql/9.4/main` for configuration
* `/var/lib/postgresql/9.4/main` for data

For example, the configuration file for authorization is here `/etc/postgresql/9.4/main/pg_hba.conf`.

### Administration

**Connect**

This will connect as PostgreSQL superuser (`postgres` by default) to the service database `postgres`

```console
sudo -u postgres psql postgres
```

**Control**

Control the PostgreSQL server by doing

```console
sudo -u postgres pg_ctlcluster 9.4 main <action>
```

where `<action>` is one of `start`, `stop`, `restart`, `reload`, `status` or `promote`.

**Change Password**

To change (or set) the password of the PostgreSQL default superuser (`postgres`)

```console
sudo -u postgres psql postgres
```

and run

```sql
ALTER ROLE postgres ENCRYPTED PASSWORD '123456';
```
132 docs/pages/administration/http-bridge/HTTP-Bridge-Callee.md
View file
@@ -0,0 +1,132 @@
[Documentation](.) > [Administration](Administration) > [HTTP Bridge](HTTP Bridge) > HTTP Bridge Callee

# HTTP Bridge Callee

> The *HTTP Callee* feature is available starting with Crossbar **0.10.3**.
* The *HTTP Callee* is a service that translates WAMP procedures to HTTP requests.


## Try it

Clone the [Crossbar.io examples repository](https://github.com/crossbario/crossbarexamples), and go to the `rest/callee` subdirectory.

Now start Crossbar:

```console
crossbar start
```

This example is configured to register a WAMP procedure named `com.myap.rest`, which sends requests to `httpbin.org`.
The procedure's complete keyword arguments are detailed further down, but if we use a kwargs of `{"url": "get", "method": "GET"}`, Crossbar will send a HTTP GET request to `httpbin.org/get` and respond with the result.
You can test this using the [HTTP Caller](HTTP Bridge Caller) configured in the example:

```shell
curl -H "Content-Type: application/json" \
-d '{"procedure": "com.myapp.rest", "kwargs": {"url": "get", "method": "GET"}}' \
http://127.0.0.1:8080/call
```

This will call the procedure and print the web response to the terminal.


## Configuration

The *HTTP Callee* is configured as a WAMP component.
Here it is as part of a Crossbar configuration:

```javascript
{
"type": "container",
"options": {
"pythonpath": [".."]
},
"components": [
{
"type": "class",
"classname": "crossbar.adapter.rest.RESTCallee",
"realm": "realm1",
"extra": {
"procedure": "com.myapp.rest",
"baseurl": "https://httpbin.org/"
},
"transport": {
"type": "websocket",
"endpoint": {
"type": "tcp",
"host": "127.0.0.1",
"port": 8080
},
"url": "ws://127.0.0.1:8080/ws"
}
}
]
}
```

The callee is configured through the `extra` dictionary:

option | description
---|---
**`procedure`** | The WAMP procedure name to register the callee as. (*required*)
**`baseurl`** | The base URL that the callee will use. All calls will work downward from this URL. If you wish to call any URL, set it as an empty string `""`. This URL must contain the protocol (e.g. `"https://"`) (*required*)

When making calls to the registered WAMP procedure, you can use the following keyword arguments:

argument | description
---|---
**`method`** | The HTTP method. (*required*)
**`url`** | The url which will be appended to the configurd base URL. For example, if the base URL was `"http://example.com"`, providing `"test"` as this argument would send the request to `http://example.com/test`. (optional, uses the configured base URL if not provided)
**`body`** | The body of the request as a string. (optional, empty if not provided)
**`headers`** | A dictionary, containing the header names as the key, and a *list* of header values as the value. For example, to send a `Content-Type` of `application/json`, you would use `{"Content-Type": ["application/json"]}` as the argument. (optional)
**`params`** | Request parameters to send, as a dictionary. (optional)


## Examples

### Wikipedia

Wikipedia has a web API that we can use for this demonstration.

Configure the `RESTCallee` WAMP component:

```javascript
"extra": {
"procedure": "org.wikipedia.en.api",
"baseurl": "http://en.wikipedia.org/w/api.php"
}
````

This code snippet calls the procedure with the parameters to look up the current revision of the Twisted Wikipedia page, reads the web response as JSON, and then pretty prints the response to the terminal.

```python
import json
from twisted.internet import reactor
from twisted.internet.defer import inlineCallbacks
from autobahn.twisted.wamp import ApplicationSession, ApplicationRunner
class AppSession(ApplicationSession):
@inlineCallbacks
def onJoin(self, details):
res = yield self.call("org.wikipedia.en.api",
method="GET",
url="",
params={
"format": "json",
"action": "query",
"titles": "Twisted (software)",
"prop": "revisions",
"rvprop": "content"
})
pageContent = json.loads(res["content"])
print(json.dumps(pageContent, sort_keys=True,
indent=4, separators=(',', ': ')))
reactor.stop()
if __name__ == '__main__':
from autobahn.twisted.wamp import ApplicationRunner
runner = ApplicationRunner("ws://127.0.0.1:8080/ws", "realm1")
runner.run(AppSession)
```
123 docs/pages/administration/http-bridge/HTTP-Bridge-Caller.md
View file
@@ -0,0 +1,123 @@
[Documentation](.) > [Administration](Administration) > [HTTP Bridge](HTTP Bridge) > HTTP Bridge Caller

# HTTP Bridge Caller

## Introduction

> The *HTTP Caller* feature is available starting with Crossbar **0.10.3**.
The *HTTP Caller* is a service that allows clients to perform WAMP calls via HTTP/POST requests.
Crossbar will forward the call to the performing server and return the result.

## Try it

Clone the [Crossbar.io examples repository](https://github.com/crossbario/crossbarexamples), and go to the `rest/caller` subdirectory.

Now start Crossbar:

```console
crossbar start
```

This will register a simple procedure that takes two integers, adds them together, and returns the result.

To test this out, you can use [curl](http://curl.haxx.se/):

```console
curl -H "Content-Type: application/json" \
-d '{"procedure": "com.example.add2", "args": [1, 2]}' \
http://127.0.0.1:8080/call
```

...or any other HTTP/POST capable tool or library.


## Configuration

The *HTTP Caller* is configured on a path of a Web transport - here is part of a Crossbar configuration:

```javascript
{
"workers": [
{
"type": "router",
...
"transports": [
{
"type": "web",
...
"paths": {
...
"call": {
"type": "caller",
"realm": "realm1",
"role": "anonymous"
}
}
}
]
}
]
}
```

The service dictionary has the following parameters:

option | description
---|---
**`type`** | MUST be `"caller"` (*required*)
**`realm`** | The realm to which the forwarding session is attached that will inject the submitted events, e.g. `"realm1"` (*required*)
**`role`** | The fixed (authentication) role the forwarding session is authenticated as when attaching to the router-realm, e.g. `"role1"` (*required*)
**`options`** | A dictionary of options (optional, see below).

The `options` dictionary has the following configuration parameters:

option | description
---|---
**`key`** | A string that when present provides the *key* from which request signatures are computed. If present, the `secret` must also be provided. E.g. `"myapp1"`.
**`secret`** | A string with the *secret* from which request signatures are computed. If present, the `key` must also be provided. E.g. `"kkjH68GiuUZ"`).
**`post_body_limit`** | An integer when present limits the length (in bytes) of a HTTP/POST body that will be accepted. If the request body exceed this limit, the request is rejected. If 0, accept unlimited length. (default: **0**)
**`timestamp_delta_limit`** | An integer when present limits the difference (in seconds) between a signature's timestamp and current time. If 0, allow any divergence. (default: **0**).
**`require_ip`** | A list of strings with single IP addresses or IP networks. When given, only clients with an IP from the designated list are accepted. Otherwise a request is denied. E.g. `["192.168.1.1/255.255.255.0", "127.0.0.1"]` (default: **-**).
**`require_tls`** | A flag that indicates if only requests running over TLS are accepted. (default: **false**).
**`debug`** | A boolean that activates debug output for this service. (default: **false**).


## Making Requests

To call WAMP procedures through Crossbar, issue a HTTP/POST request to the URL of the Crossbar HTTP Caller service with:

1. Content type `application/json`
2. Body containing a JSON object
3. Two query parameters: `timestamp` and `seq`

For a call to a HTTP Caller service, the body MUST be a JSON object with the following attributes:

* `procedure`: A string with the URI of the procedure to call.
* `args`: An (optional) list of positional event payload arguments.
* `kwargs`: An (optional) dictionary of keyword event payload arguments.


### Signed Requests

Signed requests work like unsigned requests, but have the following additional query parameters. All query parameters (below and above) are mandatory for signed requests.

* `key`: The key to be used for computing the signature.
* `nonce`: A random integer from [0, 2^53]
* `signature`: See below.

The signature computed as the Base64 encoding of the following value:

```
HMAC[SHA256]_{secret} (key | timestamp | seq | nonce | body)
```

Here, `secret` is the secret shared between the publishing application and Crossbar. This value will never travel over the wire.

The **HMAC[SHA256]** is computed w.r.t. the `secret`, and over the concatenation

```
key | timestamp | seq | nonce | body
```

The `body` is the JSON serialized event. You can look at working code [here](https://github.com/crossbario/crossbarconnect/blob/master/python/lib/crossbarconnect/client.py#L197).
262 docs/pages/administration/http-bridge/HTTP-Bridge-Publisher.md
View file
@@ -0,0 +1,262 @@
[Documentation](.) > [Administration](Administration) > [HTTP Bridge](HTTP Bridge) > HTTP Bridge Publisher

# HTTP Bridge Publisher

## Introduction

> The *HTTP Publisher* (formerly *HTTP Pusher*) feature is available starting with Crossbar **0.9.5**.
The *HTTP Publisher* is a service that allows clients to submit PubSub events via HTTP/POST requests.
Crossbar will receive the event data via the request and forward the event via standard WAMP to any connected subscribers in real-time.

![WAMP Publish via REST bridge](/static/img/docs/crossbar_http_publish.png)
## Try it

Clone the [Crossbar.io examples repository](https://github.com/crossbario/crossbarexamples), and go to the `rest/publisher` subdirectory.

Now start Crossbar:

```console
crossbar start
```

and open [http://localhost:8080](http://localhost:8080) in your browser. Open the JavaScript console to see events received.

To submit events via HTTP/POST, you can use [curl](http://curl.haxx.se/):

```console
curl -H "Content-Type: application/json" \
-d '{"topic": "com.myapp.topic1", "args": ["Hello, world"]}' \
http://127.0.0.1:8080/publish
```

...or any other HTTP/POST capable tool or library.

## Using Python

To make using the *HTTP Publisher* service even easier, we've created a (trivial) library which you can install by doing:

```console
pip install crossbarconnect
```

> `crossbarconnect` does *not* depend on `crossbar`, `autobahn`, `twisted` or `asyncio`. It only uses the Python standard library. It only does HTTP/POST requests.
You can publish events from Python like this:

```python
import crossbarconnect
client = crossbarconnect.Client("http://127.0.0.1:8080/publish")
client.publish("com.myapp.topic1", "Hello, world!", 23)
```

The example also contains two Python scripts for testing unsigned requests:

```console
python publish.py
```

and signed requests:

```console
python publish_signed.py
```

## Configuration

The *HTTP Publisher* is configured on a path of a Web transport - here is part of a Crossbar configuration:

```javascript
{
"workers": [
{
"type": "router",
...
"transports": [
{
"type": "web",
...
"paths": {
...
"publish": {
"type": "publisher",
"realm": "realm1",
"role": "anonymous"
}
}
}
]
}
]
}
```

The service dictionary has the following parameters:

option | description
---|---
**`type`** | MUST be `"publisher"` (*required*)
**`realm`** | The realm to which the forwarding session is attached that will inject the submitted events, e.g. `"realm1"` (*required*)
**`role`** | The fixed (authentication) role the forwarding session is authenticated as when attaching to the router-realm, e.g. `"role1"` (*required*)
**`options`** | A dictionary of options (optional, see below).

The `options` dictionary has the following configuration parameters:

option | description
---|---
**`key`** | A string that when present provides the *key* from which request signatures are computed. If present, the `secret` must also be provided. E.g. `"myapp1"`.
**`secret`** | A string with the *secret* from which request signatures are computed. If present, the `key` must also be provided. E.g. `"kkjH68GiuUZ"`).
**`post_body_limit`** | An integer when present limits the length (in bytes) of a HTTP/POST body that will be accepted. If the request body exceed this limit, the request is rejected. If 0, accept unlimited length. (default: **0**)
**`timestamp_delta_limit`** | An integer when present limits the difference (in seconds) between a signature's timestamp and current time. If 0, allow any divergence. (default: **0**).
**`require_ip`** | A list of strings with single IP addresses or IP networks. When given, only clients with an IP from the designated list are accepted. Otherwise a request is denied. E.g. `["192.168.1.1/255.255.255.0", "127.0.0.1"]` (default: **-**).
**`require_tls`** | A flag that indicates if only requests running over TLS are accepted. (default: **false**).
**`debug`** | A boolean that activates debug output for this service. (default: **false**).

## Running Standalone

If you only want to run WebSocket and the HTTP Publisher Service (and no other Web path services), here is an example configuration:

```javascript
{
"workers": [
{
"type": "router",
"realms": [
{
"name": "realm1",
"roles": [
{
"name": "anonymous",
"permissions": [
{
"uri": "*",
"allow": {
"call": true,
"register": true,
"publish": true,
"subscribe": true
},
}
]
}
]
}
],
"transports": [
{
"type": "websocket",
"endpoint": {
"type": "tcp",
"port": 9000
}
},
{
"type": "web",
"endpoint": {
"type": "tcp",
"port": 8080
},
"paths": {
"/": {
"type": "publisher",
"realm": "realm1",
"role": "anonymous"
}
}
}
]
}
]
}
```

This will run:

1. a WAMP-over-WebSocket endpoint on `ws://localhost:9000`
2. a HTTP Push Bridge endpoint on `http://localhost:8080`

You can test this using

```html
<!DOCTYPE html>
<html>
<body>
<script src="autobahn.min.js"></script>
<script>
var connection = new autobahn.Connection({
url: "ws://127.0.0.1:9000",
realm: "realm1"
});
connection.onopen = function (session) {
console.log("Connected");
function onevent (args, kwargs) {
console.log("Got event:", args, kwargs);
}
session.subscribe('com.myapp.topic1', onevent);
};
connection.onclose = function () {
console.log("Connection lost", arguments);
}
connection.open();
</script>
</body>
</html>
```

and publishing from curl:

```console
curl -H "Content-Type: application/json" \
-d '{"topic": "com.myapp.topic1", "args": ["Hello, world"]}' \
http://127.0.0.1:8080/
```

## Making Requests

To submit events through Crossbar, issue a HTTP/POST request to the URL of the Crossbar HTTP Publisher service with:

1. Content type `application/json`
2. Body containing a JSON object
3. Two query parameters: `timestamp` and `seq`

For a call to a HTTP Publisher service, the body MUST be a JSON object with the following attributes:

* `topic`: A string with the URI of the topic to publish to.
* `args`: An (optional) list of positional event payload arguments.
* `kwargs`: An (optional) dictionary of keyword event payload arguments.
* `options`: An (optional) dictionary of WAMP publication options (see below).

### Signed Requests

Signed requests work like unsigned requests, but have the following additional query parameters. All query parameters (below and above) are mandatory for signed requests.

* `key`: The key to be used for computing the signature.
* `nonce`: A random integer from [0, 2^53]
* `signature`: See below.

The signature computed as the Base64 encoding of the following value:

```
HMAC[SHA256]_{secret} (key | timestamp | seq | nonce | body)
```

Here, `secret` is the secret shared between the publishing application and Crossbar. This value will never travel over the wire.

The **HMAC[SHA256]** is computed w.r.t. the `secret`, and over the concatenation

```
key | timestamp | seq | nonce | body
```

The `body` is the JSON serialized event. You can look at working code [here](https://github.com/crossbario/crossbarconnect/blob/master/python/lib/crossbarconnect/client.py#L197).

## PHP - Symfony Publisher Bundle

For PHP/Symfony users, there is a bundle which makes publishing via HTTP comfortable - [Crossbar HTTP Publisher Bundle](https://github.com/facile-it/crossbar-http-publisher-bundle) (thanks to [peelandsee](https://github.com/peelandsee) for providing this).
113 docs/pages/administration/http-bridge/HTTP-Bridge-Subscriber.md
View file
@@ -0,0 +1,113 @@
[Documentation](.) > [Administration](Administration) > [HTTP Bridge](HTTP Bridge) > HTTP Bridge Subscriber

# HTTP Bridge Subscriber

## Introduction

> The *HTTP Subscriber* feature is available starting with Crossbar **0.10.3**.
The *HTTP Subscriber* is a service that forwards PubSub events to HTTP endpoints.


## Try it

Clone the [Crossbar.io examples repository](https://github.com/crossbario/crossbarexamples), and go to the `rest/subscriber` subdirectory.

Now start Crossbar:

```console
crossbar start
```

This example is configured to subscribe all events sent to the `com.myapp.topic1` topic to `httpbin.org/post`.
If you publish a message using the [HTTP Publisher](HTTP Bridge Publisher) configured in the example, it will forward the message and print the response of the message in Crossbar's debug log:

```shell
curl -H "Content-Type: application/json" \
-d '{"topic": "com.myapp.topic1", "args": ["Hello, world"]}' \
http://127.0.0.1:8080/publish
```


## Configuration

The *HTTP Subscriber* is configured as a WAMP component.
Here it is as part of a Crossbar configuration:

```javascript
{
"workers": [
{
"type": "container",
"options": {
"pythonpath": [".."]
},
"components": [
{
"type": "class",
"classname": "crossbar.adapter.rest.MessageForwarder",
"realm": "realm1",
"extra": {
"subscriptions": [
{"url": "https://httpbin.org/post",
"topic": "com.myapp.topic1"}
],
"method": "POST",
"expectedcode": 200,
"debug": true
},
"transport": {
"type": "websocket",
"endpoint": {
"type": "tcp",
"host": "127.0.0.1",
"port": 8080
},
"url": "ws://127.0.0.1:8080/ws"
}
}
]
}
]
}
```

The subscriber is configured through the `extra` dictionary:

option | description
---|---
**`subscriptions`** | A list of dictionaries which each MUST contain `"url"` and `"topic"` keys. The `"url"` key is a full URL with `http` or `https` (for example, `"https://example.org/endpoint"`), and the topic is the exact topic which events will be forwarded from. (*required*)
**`method`** | The HTTP method which the forwarding requests will be made with. (optional, `"POST"` by default)
**`expectedcode`** | The HTTP status code which is expected from the requests. If none is given, the status code is not checked. (optional)
**`debug`** | If `true`, then the response body will be printed to Crossbar's debug log. (optional, `false` by default)


## Handling Forwarded Events

The Subscriber, upon recieving a PubSub event that it has been configured to subscribe to, will send a request to the URL associated with the topic.
The body will be a JSON encoded dictionary and contain two keys, `"args"` and `"kwargs"` from the PubSub event.
Here is an example Flask application that prints the pubsub event to the terminal:

```python
import json
from flask import Flask, request
app = Flask(__name__)
@app.route("/", methods=["POST"])
def message():
body = json.loads(request.get_data())
print("args:", body["args"], "kwargs:", body["kwargs"])
return b"OK"
if __name__ == "__main__":
app.run()
```

When this server is started, Crossbar is configured to forward the event to it, and the example event at the top of the page is published, you should see:

```console
$ python ~/example.py
* Running on http://127.0.0.1:5000/
('args:', [u'Hello, world'], 'kwargs:', {})
127.0.0.1 - - [21/Apr/2015 21:01:05] "POST / HTTP/1.1" 200 -
```
132 docs/pages/administration/http-bridge/HTTP-Bridge-Webhook.md
View file
@@ -0,0 +1,132 @@
[Documentation](.) > [Administration](Administration) > [HTTP Bridge](HTTP Bridge) > HTTP Bridge Webhook

# HTTP Bridge Webhook

## Introduction

> The *HTTP Webhook* feature is available starting with Crossbar **0.11.0**.
The *HTTP Webhook Service* broadcasts incoming HTTP/POST requests on a fixed WAMP topic.

Webhooks are a method of "push notification" used by services such as GitHub and BitBucket to notify other services when events have happened through a simple HTTP POST.
The HTTP Webhook Service allows you to consume these events (providing it is accessible by the external service) through a WAMP PubSub channel (allowing potentially many things to occur from one webhook notification).


## Try it

Clone the [Crossbar.io examples repository](https://github.com/crossbario/crossbarexamples), and go to the `rest/webhooks` subdirectory.

Now start Crossbar:

```console
crossbar start
```

and open [http://localhost:8080](http://localhost:8080) in your browser.
Open the JavaScript console to see events received.

To submit an example webhook via HTTP/POST, you can use [curl](http://curl.haxx.se/):

```console
curl -H "Content-Type: text/plain" \
-d 'fresh webhooks!' \
http://127.0.0.1:8080/webhook
```

## Configuration

The *HTTP Webhook Service* is configured on a path of a Web transport - here is part of a Crossbar configuration:

```javascript
{
"workers": [
{
"type": "router",
...
"transports": [
{
"type": "web",
...
"paths": {
...
"webhook": {
"type": "webhook",
"realm": "realm1",
"role": "anonymous",
"options": {
"topic": "com.myapp.topic1"
}
}
}
}
]
}
]
}
```

The service dictionary has the following parameters:

option | description
---|---
**`type`** | MUST be `"webhook"` (*required*)
**`realm`** | The realm to which the forwarding session is attached that will inject the submitted events, e.g. `"realm1"` (*required*)
**`role`** | The fixed (authentication) role the forwarding session is authenticated as when attaching to the router-realm, e.g. `"role1"` (*required*)
**`options`** | A dictionary of options (required, see below).

The `options` dictionary has the following configuration parameters:

option | description
---|---
**`topic`** | The topic to which the forwarded events will be sent.
**`post_body_limit`** | An integer when present limits the length (in bytes) of a HTTP/POST body that will be accepted. If the request body exceed this limit, the request is rejected. If 0, accept unlimited length. (default: **0**)


## With GitHub

If you set up Crossbar to have a Webhook service, and make it externally available, you can configure GitHub to send events to it.
Underneath Settings and "Services & Webhooks", you can add a new webhook, which just requires the URL of the externally-accessible Webhook service.
You can configure GitHub to send certain events, or all events.

When you have configured it, it will send a 'ping' for you to verify it.
As you have configured the Webhook service, you will recieve a message similar to this (most of the body cut out for brevity) on the WAMP topic it was configured with.


```json
{
"body": "{\"zen\":\"Design for failure.\",[...more json...]}",
"headers": {
"Content-Length": [
"6188"
],
"X-Github-Event": [
"ping"
],
"X-Github-Delivery": [
"7e87c300-462c-11e5-8008-e7623fda32a6"
],
"Accept": [
"*/*"
],
"User-Agent": [
"GitHub-Hookshot/4963429"
],
"Host": [
"atleastfornow.net:8080"
],
"Content-Type": [
"application/json"
]
}
}
```

The message on the WAMP topic will be a dict containing the body as a string, and the headers as a dictionary of lists.

You will also see the following in the logs:

```
2015-08-19T04:44:43+0000 [Router 490] Successfully sent webhook from 192.30.252.34 to com.myapp.topic1
```

For more information on Webhooks, please see GitHub's [Webhooks Guide](https://developer.github.com/webhooks/).
28 docs/pages/administration/http-bridge/HTTP-Bridge.md
View file
@@ -0,0 +1,28 @@
[Documentation](.) > [Administration](Administration) > HTTP Bridge

# HTTP Bridge

## Quick Links

* [HTTP Bridge Publisher](HTTP Bridge Publisher)
* [HTTP Bridge Subscriber](HTTP Bridge Subscriber)
* [HTTP Bridge Caller](HTTP Bridge Caller)
* [HTTP Bridge Callee](HTTP Bridge Callee)
* [HTTP Bridge Webhook](HTTP Bridge Webhook)

## Background

Imagine you have an existing application which isn't based on WAMP components -- say, a REST or classical Web application using HTTP.

Now what if you just want to *add* some real-time features *without* changing your existing app to use WAMP or migrate from synchronous, blocking to asynchronous, non-blocking code?
Or if you want to access your existing HTTP services using WAMP?

This is where the *HTTP bridge services* of Crossbar can help.
They provide WAMP components which provide interoperability with existing code by using HTTP.

* The [HTTP Publisher](HTTP Bridge Publisher) is a service that allows clients to submit PubSub events via HTTP/POST requests.
Crossbar will receive the event data via the request and forward the event via standard WAMP to any connected subscribers in real-time.
* The [HTTP Caller](HTTP Bridge Caller) is a service that allows clients to perform WAMP calls via HTTP/POST requests.
Crossbar will forward the call to the performing server and return the result.
* The [HTTP Subscriber](HTTP Bridge Subscriber) is a service that forwards WAMP PubSub events to HTTP endpoints.
* The [HTTP Callee](HTTP Bridge Callee) is a service that translates WAMP procedure calls to HTTP requests.
245 docs/pages/administration/production/Automatic-startup-and-restart.md
View file
@@ -0,0 +1,245 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > Automatic Startup and Restart

# Automatic Startup and Restart

For running Crossbar.io in production, you might want to:

* automatically start Crossbar.io at system boot as a daemon (background service)
* automatically restart Crossbar.io if it exits (either deliberately, or by accident)

There are different approaches and tools to accomplish above under Unix-like operating systems:

* [systemd](#systemd): recommended for **Ubuntu >= 15.04**
* [upstart](#upstart): recommended for **Ubuntu 14.04 LTS**
* [daemontools](#daemontools): recommended for **FreeBSD**


## systemd

Create a systemd service file `/etc/systemd/system/crossbar.service`

```
[Unit]
Description=Crossbar.io
After=network.target
[Service]
Type=simple
User=ubuntu
Group=ubuntu
StandardInput=null
StandardOutput=journal
StandardError=journal
Environment="MYVAR1=foobar"
ExecStart=/opt/crossbar/bin/crossbar start --cbdir=/home/ubuntu/mynode1/.crossbar
Restart=on-abort
[Install]
WantedBy=multi-user.target
```

> Adjust the path to the Crossbar.io executable `/opt/crossbar/bin/crossbar` and your Crossbar.io node directory `/home/ubuntu/mynode1/.crossbar` in above.
Then do:

```console
sudo systemctl daemon-reload
```

To make Crossbar.io start automatically at boot time:

```console
sudo systemctl enable crossbar.service
```

To start, stop, restart and get status for Crossbar.io:

```console
sudo systemctl start crossbar
sudo systemctl stop crossbar
sudo systemctl restart crossbar
sudo systemctl status crossbar
```

To get log output:

```console
journalctl -f -u crossbar
```

---


## upstart

Create an upstart job file `/etc/init/crossbar.conf`

```
description "Crossbar.io"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
respawn limit 20 5
setuid ubuntu
setgid ubuntu
env MYVAR1=foobar
exec /opt/crossbar/bin/crossbar start --cbdir=/home/ubuntu/mynode1/.crossbar
```

> Adjust the path to the Crossbar.io executable `/opt/crossbar/bin/crossbar` and your Crossbar.io node directory `/home/ubuntu/mynode1/.crossbar` in above.
Then do

```console
sudo initctl reload-configuration
```

To start, stop, restart and get status for Crossbar.io:

```console
sudo start crossbar
sudo stop crossbar
sudo restart crossbar
sudo status crossbar
```

To get log output:

```console
sudo tail -f /var/log/upstart/crossbar.log
```

---


## daemontools

The following describes how to monitor and restart Crossbar.io automatically using [Daemontools](http://cr.yp.to/daemontools.html). **Daemontools** is a simple, effective, highly secure tool create by [Dan Bernstein](http://en.wikipedia.org/wiki/Daniel_J._Bernstein) (aka "djb").

> Note: There is also [runit](http://smarden.org/runit/), which is a Daemontools clone that some people [prefer](http://www.sanityinc.com/articles/init-scripts-considered-harmful/).
### Installation

To install Daemontools on Debian based systems (Ubuntu et al):

```
sudo apt-get install csh daemontools daemontools-run
```

This will install a couple of tools including

```
/usr/bin/svc
/usr/bin/svstat
/usr/bin/svscanboot
/usr/bin/setuidgid
```

### Configuration

Create a Daemontools service directory for Crossbar.io:

```
sudo mkdir /etc/service/crossbar
```

Create a service run script

```
sudo vi /etc/service/crossbar/run
```

with the following content:

```
#!/bin/sh
exec /usr/bin/setuidgid ubuntu \
/home/ubuntu/pypy-2.2.1-linux64/bin/crossbar start \
--cbdir /home/ubuntu/cbdemo/.crossbar \
--logdir /home/ubuntu/cbdemo/.crossbar/log
```

Above assumes:

* you are using PyPy under the specified path
* you want to run Crossbar.io under the dedicated Unix user `ubuntu` (which fits for a Amazon EC2 Ubuntu Server AMI)
* you have a Crossbar.io node created in the specified node directory
* you want Crossbar.io log to the specified subdirectory within the node directory

Make the run script executable:

```
sudo chmod +x /etc/service/crossbar/run
```

To make Daemontools start automatically at system boot:

```
sudo vi /etc/rc.local
```

and add the following to the end of that file:

```
/bin/csh -cf '/usr/bin/svscanboot &'
exit 0
```

Reboot your system and check the Crossbar.io has been started:

```
ubuntu@ip-10-229-126-122:~$ sudo svstat /etc/service/crossbar
/etc/service/crossbar: up (pid 1006) 91391 seconds
```

### Administration

To stop Crossbar.io:

```
sudo svc -d /etc/service/crossbar
```

To (manually) start again:

```
sudo svc -u /etc/service/crossbar
```

To restart:

```
sudo svc -t /etc/service/crossbar
```

To check status:

```
sudo svstat /etc/service/crossbar
```

By default - if given `--logdir` option - Crossbar.io will create daily rotated log files in the directory specified:

```
ubuntu@ip-10-229-126-122:~$ ls -la /home/ubuntu/cbdemo/.crossbar/log
total 28
drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 18 04:15 .
drwxrwxr-x 3 ubuntu ubuntu 4096 Mar 17 16:14 ..
-rw-r--r-- 1 ubuntu ubuntu 2737 Mar 18 08:13 node.log
-rw-r--r-- 1 ubuntu ubuntu 13915 Mar 17 16:14 node.log.2014_3_17
```

To watch the log file:

```
tail -f /home/ubuntu/cbdemo/.crossbar/log/node.log
```

---
74 docs/pages/administration/production/Browser-Support.md
View file
@@ -0,0 +1,74 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > Browser Support

# Browser Support

**The final RFC spec**

WebSocket, as an [IETF standard](http://tools.ietf.org/html/rfc6455), and with a W3C browser API, is fully supported by all modern browsers:

* Chrome 16 + (incl. Chrome for Android)
* Firefox 11 + (incl. Firefox for Android)
* Internet Explorer 10+ (incl. Internet Explorer Mobile on Windows Phone 8)
* Safari 6 +
* Opera 12.1 + (incl. Opera Mobile)
* iOS 6.0 +
* Blackberry 7 +

Generally speaking, on the desktop outside of crazy legacy systems you should be OK. On mobile, older Android versions are the largest (but rapidly shrinking) gap.


**Adding WebSocket support**

***Internet Explorer***

For IE < 10, you can use Adobe Flash Bridge to bring WebSocket Supporting.

IE10+ has native WebSocket support. It is the default browser on Windows 8, and has been rolled out as an update for Windows 7. It is also the browser for Windows Phone 8.


***Android***

We recommend:

* Chrome for Android on Android ICS+.
* Firefox Mobile or Opera Mobile on sub-ICS devices,
* Opera Mobile on ARMv6 devices

The above browsers all offer native WebSocket support. When none of the above is an option, Flash Bridge may be.

Flash is increasing rare on Android devices. It is not supported anymore from 4.1 (Jelly Bean) onward. Other devices, e.g. ARMv6 devices (like the Samsung Galaxy ACE) mostly never had Flash support.

If Flash fallback is needed, obviously, the Flash Plugin must be installed and active.
The setting for Plugins on the Android built-in browser can be changed under "Options->Settings->Active Plugins" and should be left at the default "Always On".

Firefox Mobile runs on ARMv7+ devices, with only experimental support for ARMv6.

Opera Mobile works, and is currently the only option on devices that do not have Flash (ARMv6 e.g. Samsung Galaxy ACE) or that do not have the option to run Chrome Mobile or Firefox.

Opera Mini is NOT supported, since it has only very limited support for JavaScript.

The Android WebView supports WebSocket only starting at Android 4.4, since at this point it was switched to the Chromium engine from the Android browser.


***Opera***

Opera 12.1+ (both desktop and mobile) has full WebSocket support, and of course the newer Operas, which are based on Google's Blink engine, share its WebSocket support.

Versions 11.0 - 12.0 have support for an older version (Hixie76), and this
is deactivated by default.

To activate, [follow this link](opera:config#Enable%20WebSocket) (or enter "opera:config" into the browser address bar and search for "websocket"), check the enable flag and restart the browser.

***Safari***

* Safari 6.0+ has full WebSocket support.
* Safari 5.0.1+ (desktop/mobile) has native support for an older version (Hixie76).


***Supporting Flash bridge***

When using browsers without any native WebSocket support, such support can be added via a flash shim such as [web-socket-js](https://github.com/gimite/web-socket-js/).

In order for this to work, 'Flash Policy Server' in Crossbar.io needs to be active. This requires a restart to take effect.

Additionally, the 'Flash Policy Port' should be set correctly to '843'.
41 docs/pages/administration/production/Cryptobox-Payload-Encryption.md
View file
@@ -0,0 +1,41 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > Cryptobox Payload Encryption

# Cryptobox Payload Encryption

> Note: This feature is experimental at present. Feedback is highly welcome!
> Note: This feature presently only works with Autobahn|Python as a client library.
Crossbar.io is normally seen as trusted, and transport encryption provides security for data in transit.

In some situations, you may want to reduce the access the router has to the information users transmit, e.g. when you are using a hosted Crossbar.io instance and want to minimize information leakage to the hosting provider.

Cryptobox payload encryption allows this.

## Basics

Cryptobox uses curve25519-based asymetric key encryption.

Both the sender of a message and its recipient(s) have a key pair, which is set per-URI. The sender uses the recipient's public key to encrypt the message payload, and its own private key to sign it. This allows to separate the possibility to e.g. publish to a topic from being able to unencrpyt publishes to the topic.

> Note: Key distribution is presently outside of the scope of Cryptobox.
Crpytobox relies on **payload transparency**: The entire encrypted payload is transmitted as a new, additional argument in the WAMP message.

## Disadvantages

This does not render client traffic fully opaque to the router. By necessity, the router needs to have routing information (registrations, subscriptions, URIs in the messages). Depending on the application, this may still represent an inacceptable amount of information leakage when using a third-party controlled Crossbar.io instance.

> Note: We are thinking of implementing URI scrambling, which would further avoid the information leakage from nice, human readable and systematically assigned URIs.
Since the payload is encrypted, the router can no longer access the content.

This presently prevents the use of different serializations between the sender of a message and Crossbar.io and Crossbar.io and the recipient(s).

It will further prevent payload validation (which is on the roadmap).

As is always the case, security and convenience are a trade off.

## Examples

There is an example in the [Crossbarexamples](https://github.com/crossbario/crossbarexamples/tree/master/encryption/cryptobox) which will track the progress on Cryptobox. Until we have a final version, this example will provide more up-to-date information than this documentation page.
18 docs/pages/administration/production/Going-to-Production.md
View file
@@ -0,0 +1,18 @@
[Documentation](.) > [Administration](Administration) > Going to Production

# Going to Production

This section covers security issues and fine-tuning of advanced options. Tips for a performant and secure production configuration.

* [Running on privileged ports](Running on privileged ports)
* [Secure WebSocket and HTTPS](Secure WebSocket and HTTPS)
* [Automatic startup and restart](Automatic startup and restart)
* [Network Tuning](Network Tuning)
* [Reverse Proxies](Reverse Proxies)
* [SSL/TLS Interception Proxies](SSL-TLS-Interception-Proxies)

Testing your instance & browser support.

* [Browser Support](Browser Support)
* [WebSocket Compliance Testing](WebSocket Compliance Testing)
* [Stream Testee](Stream Testee)
117 docs/pages/administration/production/Network-Tuning.md
View file
@@ -0,0 +1,117 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > Network Tuning

# Network Tuning

## Linux

Linux TCP networking is tuned as in the following. This (or similar) is *required*, since we are really pushing the system.

Add the following to the end of `/etc/sysctl.conf` and do `sysctl -p`:

net.core.somaxconn = 8192
net.ipv4.tcp_max_orphans = 8192
net.ipv4.tcp_max_syn_backlog = 8192
net.core.netdev_max_backlog = 262144

net.ipv4.ip_local_port_range = 1024 65535

#net.ipv4.tcp_low_latency = 1
#net.ipv4.tcp_window_scaling = 0
#net.ipv4.tcp_syncookies = 0

fs.file-max = 16777216
fs.pipe-max-size = 134217728

Further system level tuning:

Modify `/etc/security/limits.conf` for the following

# wildcard does not work for root, but for all other users
* soft nofile 1048576
* hard nofile 1048576
# settings should also apply to root
root soft nofile 1048576
root hard nofile 1048576

and add the following line

session required pam_limits.so

to both of these files at the end:

/etc/pam.d/common-session
/etc/pam.d/common-session-noninteractive

Reboot (or at least I don't know how to make it immediate without reboot).

Check that you actually got large (`1048576`) FD limit:

ulimit -n

Probably also check that above `sysctl` settings actually are in place (`sysctl -a | grep ..` or such). I am paranoid.


## FreeBSD

Here are a couple of background articles:

* https://pleiades.ucsc.edu/hyades/FreeBSD_Network_Tuning
* https://blog.whatsapp.com/196/1-million-is-so-2011?
* https://wiki.freebsd.org/NetworkPerformanceTuning
* https://calomel.org/freebsd_network_tuning.html
* https://www.freebsd.org/doc/handbook/configtuning-kernel-limits.html


Add the following to `/boot/loader.conf`:

```
boot_verbose="YES"
# increase max. open sockets / files
kern.ipc.maxsockets=2400000
kern.maxfiles=3000000
kern.maxfilesperproc=2700000
kern.maxproc=16384
# tune up for high connection numbers
net.inet.tcp.tcbhashsize=524288
net.inet.tcp.hostcache.hashsize=4096
net.inet.tcp.hostcache.cachelimit=131072
net.inet.tcp.hostcache.bucketlimit=120
# misc
kern.hwpmc.nbuffers=32
kern.hwpmc.nsamples=64
kern.timecounter.smp_tsc=1
kern.random.sys.harvest.ethernet=0
```

Add the following to `/etc/sysctl.conf`

```
# increase range of ephemeral ports
net.inet.ip.portrange.first=1024
net.inet.ip.portrange.last=65535
net.inet.ip.portrange.randomized=0
# allow binding of ports <1024 by non-root processes
net.inet.ip.portrange.reservedhigh=0
# increase backlog
kern.ipc.somaxconn=32768
# set to 128MB
kern.ipc.maxsockbuf=134217728
# set autotuning maximum to 128MB too
net.inet.tcp.sendbuf_max=134217728
net.inet.tcp.recvbuf_max=134217728
# enable send/recv autotuning
net.inet.tcp.sendbuf_auto=1
net.inet.tcp.recvbuf_auto=1
# increase autotuning step size
net.inet.tcp.sendbuf_inc=16384
net.inet.tcp.recvbuf_inc=16384
```
18 docs/pages/administration/production/Reverse-Proxies.md
View file
@@ -0,0 +1,18 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > Reverse Proxies

# Reverse Proxies

Running Crossbar.io behind [reverse proxies](http://en.wikipedia.org/wiki/Reverse_proxy).

## HAProxy

Write me.

## Nginx

Write me. You can find some Nginx examples and hints [here](https://github.com/nicokaiser/nginx-websocket-proxy).

## Apache

Apache supports proxying WebSocket connections using [mod_proxy_wstunnel](http://httpd.apache.org/docs/2.4/mod/mod_proxy_wstunnel.html).

48 docs/pages/administration/production/Running-on-privileged-ports.md
View file
@@ -0,0 +1,48 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > Running on Privileged Ports

# Running on Privileged Ports

For production, you might want to:

* make Crossbar.io listen on ports 80/443, which are the standard ports for both HTTP(S) and (secure) WebSocket
* run Crossbar.io under a dedicated non-root service user

However, Unix-like operating system by default do not allow programs that run non-root to listen on TCP/IP ports <1024.

There are different ways of achieving above, and those ways depend on the OS flavor you use (Linux, FreeBSD, etc).

## Linux

Here we describe one way that works using [Linux Capabilities](http://linux.die.net/man/7/capabilities) on kernels >= 2.6.24.

Install `libcap2`:

```
sudo apt-get install libcap2-bin
```

Now allow the Crossbar.io and PyPy executables to bind privileged ports:

```
sudo setcap cap_net_bind_service=+ep `which crossbar`
sudo setcap cap_net_bind_service=+ep `which pypy`
```

> Note that with above, any user on the host that is able to execute PyPy (or Crossbar) will be able to bind privileged ports *with any Python script*. If the host is used by others as well, you might want to restrict *execution permissions* on the binaries again.
> Also note that using capabilities will disable searching directories for shared libraries from `LD_LIBRARY_PATH`. See [here](http://stackoverflow.com/questions/9843178/linux-capabilities-setcap-seems-to-disable-ld-library-path)
>
## FreeBSD

On FreeBSD, the range of privileged ports which only may be opened by root-owned processes may be modified by the `net.inet.ip.portrange.reservedlow` and `net.inet.ip.portrange.reservedhigh` **sysctl** settings.

The values default to the traditional range, `0` through `IPPORT_RESERVED - 1` (`0` through `1023`), respectively.

To temporarily allow non-root process to bind ports <1024:

sysctl net.inet.ip.portrange.reservedhigh=0

To make that setting persist reboots:

echo "net.inet.ip.portrange.reservedhigh=0" >> /etc/sysctl.conf
17 docs/pages/administration/production/SSL-TLS-Interception-Proxies.md
View file
@@ -0,0 +1,17 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > SSL-TLS Interception Proxies

# SSL-TLS Interception Proxies

SSL-TLS Interception Proxies sit between browsers and Web/WebSocket servers, unwrapping a secure TLS connection to look inside the payload of the traffic. Here is an [introduction](http://www.secureworks.com/cyber-threat-intelligence/threats/transitive-trust/).

Interception proxies are usually operating transparently to the user, impersonating target sites and servers to do their content inspection thing. They are one class of devices that in the context of WebSocket are called *intermedaries*.

## Products

### Barracuda Web Filter

According to a Barracuda sales engineer (2014/07/08), the [Barracuda Web Filter](https://www.barracuda.com/products/webfilter) does **NOT** support WebSocket. Hence, WAMP-over-WebSocket will not work.

It is unclear whether the appliance allows to exclude (via configuration) certain target URLs from interception, and if doing so would make WebSocket work on such configured target addresses.

It is unclear whether WAMP-over-HTTP/longpoll would work for that appliance (precisely, whether the appliance would allow long standing HTTP/POST requests to persist and immediately forward data sent downstream).
37 docs/pages/administration/production/Scaling-Crossbar.io.md
View file
@@ -0,0 +1,37 @@
# Scaling Crossbar.io

The following discusses Crossbar.io scalability in terms of

* scale-up: utilizing faster and more cores on a single machine
* scale-out: utilizing multiple machines

and with regard to

* scaling WAMP application components
* scaling WAMP routing

## Scaling Application Components

Crossbar.io can host WAMP application components (which connect to WAMP routers) and supports scale-up and scale-out.

Application components are run in worker processes, and hence multiple cores on a single machine can be utilized by starting multiple, functionally different components or multiple instances of the same component.

Application components can also be spread across multiple machines, all connecting to the same router. Doing so allows you to scale-out and utilize the resources of multiple machines.

Above features already work today with the current Crossbar.io release.

## Scaling Routers

A Crossbar.io router worker process can manage multiple, independent realms and a single Crossbar.io node can run multiple router worker processes managing independent realms.

A *single* Crossbar.io router worker process already scales to 100-200k concurrently active connections.

You can utilize multiple cores on one machine for routing by starting *multiple* router worker processes, each managing *independent* realms.

The same works for scale-out, by running router workers on different machines, all managing different, independent realms.

Above features already work today with the current Crossbar.io release.

However, if you need to scale beyond 100-200k concurrently active connection on a *single realm*, this is not yet possible today.

For this, router workers will need to work together as a single logical router, managing the same realm. This feature is under development and currently scheduled for Q1/2016.
174 docs/pages/administration/production/Secure-WebSocket-and-HTTPS.md
View file
@@ -0,0 +1,174 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > Secure WebSocket and HTTPS

# Secure WebSocket and HTTPS

For production use, it is **strongly recommended** to always run WebSocket over [TLS](https://en.wikipedia.org/wiki/Transport_Layer_Security) ("secure WebSocket"). This is mainly for two reasons:

* keeping your and your user's data confidential and untampered
* avoiding issues with WebSocket on networks that employ so-called intermediaries (proxies, caches, firewalls)

> The latter is especially important in locked down enterprise environments and on mobile operator networks. By using secure WebSocket ("wss"), WebSocket will work in almost all circumstances (exceptions potentially being TLS interception / MITM proxies).
Crossbar.io has full support for running secure WebSocket and HTTPS. We discuss configuration:

* [WebSocket Transport Configuration](#websocket-transport-configuration)
* [Endpoint TLS Configuration](#endpoint-tls-configuration)

To actually use TLS, you will need a **certificate** for your server. This guide describes the three main options:

1. [Using self-signed certificates](#using-self-signed-certificates)
2. [Using certificates from commercial CAs](#using-commercial-certificates)
3. [Creating and using your own CA](#creating-your-own-certificate-authority)

We also **strongly recommend** to test your server using the [SSL Server Test](https://www.ssllabs.com/ssltest/) provided by Qualys SSL Labs. This will point out weaknesses in your configuration or issues with your certificate.

## WebSocket Transport Configuration

To configure a WebSocket transport for TLS, include a `tls` dictionary with (mandatory) attributes `key` and `certificate` in your transport configuration. Here is an example:

```javascript
{
"type": "websocket",
"endpoint": {
"type": "tcp",
"port": 443,
"tls": {
"key": "server_key.pem",
"certificate": "server_cert.pem"
}
},
"url": "wss://example.com"
}
```

The `key` must point to the server's private key file (PEM format, no passphrase), and the `certificate` must point to the server's certificate file (PEM format). The paths can be relative to the node directory, or absolute.

To configure a Web transport for TLS, here is an example:

```javascript
{
"type": "web",
"endpoint": {
"type": "tcp",
"port": 443,
"tls": {
"key": "server_key.pem",
"certificate": "server_cert.pem"
}
},
"paths": {
"/": {
"type": "static",
"directory": ".."
},
"ws": {
"type": "websocket",
"url": "wss://example.com/ws"
}
}
}
```

---

## Endpoint TLS Configuration

The TLS configuration has a couple of options:

```javascript
{
"type": "websocket",
"endpoint": {
"type": "tcp",
"port": 443,
"tls": {
"key": "server_key.pem",
"certificate": "server_cert.pem",
"ca_certificates": [
"ca.cert.pem",
"intermediate.cert.pem"
],
"dhparam": "dhparam.pem",
"ciphers": "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AES:RSA+3DES:!ADH:!AECDH:!MD5:!DSS"
}
},
"url": "wss://example.com"
}
```
where
* `key` is the filesystem path to the server private key file (PEM format, no passphrase) (**mandatory**)
* `certificate` is the filesystem path to the server certificate file (PEM format) (**mandatory**)
* `ca_certificates` when set requires that a connecting client's certificate be issued by one of the listed CAs, otherwise the connection establishment will be denied (**optional**)
* `dhparam` is the filesystem path to a Diffie-Hellman parameter file - see explanation below (**optional**)
* `ciphers` is a list of ciphers the server is willing to use with a client - see explanation below (**optional**)

### Diffie-Hellman

To use [Diffie-Hellman](http://en.wikipedia.org/wiki/Diffie%E2%80%93Hellman_key_exchange) based key exchange, you need to generate a parameter file:

openssl dhparam -2 4096 -out .crossbar/dhparam.pem

The use of Diffie-Hellman key exchange is desirable, since this provides [Perfect Forward Secrecy (PFS)](http://en.wikipedia.org/wiki/Forward_secrecy). Without a DH parameter file, no Diffie-Hellman based ciphers will be used, even if configured to do so.

### Elliptic Curve Ciphers

Using elliptic curve based ciphers ("ECDH/ECDHE") is generally considered desirable, since shorter keys than RSA support strong encryption already consuming less CPU cycles.

#### Prerequisites for EC Support

EC crypto is fully supported by Crossbar.io, if the underlying OpenSSL library supports EC **and** you have pyOpenSSL >= 0.15 running.

You can check like this:

openssl ecparam -list_curves

To install pyOpenSSL trunk (since 0.15 isn't released yet - 2014/07/09):

cd ~
git clone git@github.com:pyca/pyopenssl.git
cd pyopenssl
python setup.py install

Crossbar.io uses the `prime256v1` curve by default.

`prime256v1`(X9.62/SECG) is an elliptic curve over a 256 bit prime field. This is elliptic curve "NIST P-256" from [here](http://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.186-4.pdf).

This seems to be the most [widely used curve](http://crypto.stackexchange.com/questions/11310/with-openssl-and-ecdhe-how-to-show-the-actual-curve-being-used) and researchers [think](https://twitter.com/hyperelliptic/status/394258454342148096) it is "ok" (other than wrt timing attacks etc that might lurk inside OpenSSL itself).

### Ciphers

Crossbar.io will by default run a very strong and conservative set of ciphers:

```text
ECDHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA
```

Above configuration activates exactly 6 ciphers to be used, all of which provide **Forward Secrecy**.

**Note that the default configuration does not support Windows XP!**. If you must support XP, you will need to modify the ciphers configuration.

In general, you should only change the `ciphers` if you know what you are doing.

The `ciphers` parameter must be in the format as used by OpenSSL, and the OpenSSL library version installed on the system must support the ciphers configured to make same actually available. If your OpenSSL version installed does not support a configured cipher (e.g. ECDH elliptic curve based), the ciphers not available will simply be skipped.


## TLS Certificates

We provide help for creation and handling of TLS certificates on the [TLS Certificates page](TLS Certificates).


## Examples


* [Sample configuration](https://github.com/crossbario/crossbarexamples/tree/master/encryption/tls)
* [Python example for using TLS with Crossbar.io](https://github.com/crossbario/crossbarexamples/tree/master/wss/python)
* [TLS Client Cert Authentication examples](https://github.com/crossbario/crossbarexamples/tree/master/authentication/tls)
* [Crossbar.io demo instance production configuration](https://github.com/crossbario/crossbarexamples/blob/master/demos/_demo_launcher/.crossbar/config_prod_cdc.json) - an example of recommended strongly secure settings



## Resources

* [OpenSSL man page](http://linux.die.net/man/1/dhparam)
* [OpenSSL API documentation](http://linux.die.net/man/3/ssl_ctx_set_tmp_dh)
* [The Most Common OpenSSL Commands](https://www.sslshopper.com/article-most-common-openssl-commands.html)
7 docs/pages/administration/production/Security.md
View file
@@ -0,0 +1,7 @@
Write me.

## Library

* [A New Approach to IoT Security](http://www.pubnub.com/static/papers/IoT_Security_Whitepaper_Final.pdf)
* [Inside-Out Security for the IoT](https://www.altera.com/solutions/technology/system-design/articles/_2014/security-iot.smartphone.highResolutionDisplay.html)
* [TUV SUD IoT Honeynet Project (in German)](http://www.tuev-sued.de/tuev-sued-konzern/presse/pressemeldungen/potenzielle-angreifer-sind-ueberall)
284 docs/pages/administration/production/Stream-Testee.md
View file
@@ -0,0 +1,284 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > Stream Testee

# Stream Testee

Crossbar.io includes a *testee* (pseudo) transport, which just echo's back anything at a stream level (TCP or Unix domain socket).

**This feature is for debugging, development and benchmarking purposes.**

Here is an example configuration:

```json
{
"workers": [
{
"type": "router",
"transports": [
{
"type": "stream.testee",
"endpoint": {
"type": "tcp",
"port": 9000,
"backlog": 1024
}
}
]
}
]
}
```

Now you can connect e.g. via telnet to the host on port 9000.

## Configuration

option | description
---|---
**`id`** | ID of the transport within the running node (default: **`transport<N>`** where `N` is numbered automatically starting from `1`)
**`type`** | Type of transport - must be `"stream.testee"`.
**`endpoint`** | Listening endpoint for transport. See [Transport Endpoints](Transport Endpoints) for configuration
**`debug`** | Turn on debug logging for this transport instance (default: **`false`**).

---

## Results

These tests were performed on 2 boxes running FreeBSD 10.1 / x86-64:

```console
[oberstet@brummer1 ~]$ sysctl -a | egrep -i 'hw.model|hw.ncpu|hw.physmem'
hw.model: Intel(R) Xeon(R) CPU E3-1240 v3 @ 3.40GHz
hw.ncpu: 8
hw.physmem: 34276188160
```

The boxes are networked via 10GbE.

## Netperf

The following is a dump of performance results using [netperf](http://linux.die.net/man/1/netperf). [Here](https://gist.github.com/cgbystrom/985475) are some numbers to compare the results with. [Link](/static/img/docs/netperf_benchmark_results.png) to full picture.

![Crossbar.io raw TCP performance](/static/img/docs/netperf_benchmark_results.png)

### TCP_RR

Over **10GbE** to netserver:

```console
[oberstet@brummer2 ~]$ netperf -H 10.0.0.10 -t TCP_RR -l 60
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.0.10 () port 0 AF_INET : histogram : interval : dirty data : demo : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec

32768 65536 1 1 60.01 26851.95
32768 65536
```

Over **10GbE** to Crossbar.io / PyPy 2.4:

```console
[oberstet@brummer2 ~]$ netperf -N -H 10.0.0.10 -t TCP_RR -l 60 -- -P 9000
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 9000 AF_INET to 10.0.0.10 () port 9000 AF_INET : no control : histogram : interval : dirty data : demo : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec

32768 65536 1 1 60.03 20524.63
0 0
```

Over **10GbE** to Crossbar.io / CPython 2.7.9:

```console
[oberstet@brummer2 ~]$ netperf -N -H 10.0.0.10 -t TCP_RR -l 60 -- -P 9000
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 9000 AF_INET to 10.0.0.10 () port 9000 AF_INET : no control : histogram : interval : dirty data : demo : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec

32768 65536 1 1 60.01 13342.51
0 0
```

Over **loopback** to netserver:

```console
[oberstet@brummer1 ~]$ netperf -H 127.0.0.1 -t TCP_RR -l 60
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : histogram : interval : dirty data : demo : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec

32768 65536 1 1 60.02 114822.69
32768 65536
```

Over **loopback** to Crossbar.io / PyPy 2.4:

```console
[oberstet@brummer1 ~]$ netperf -N -H 127.0.0.1 -t TCP_RR -l 60 -- -P 9000
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 9000 AF_INET to 127.0.0.1 () port 9000 AF_IN ET : no control : histogram : interval : dirty data : demo : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec

32768 65536 1 1 60.02 63572.31
0 0
```

Over **loopback** to Crossbar.io / CPython 2.7.9:

```console
[oberstet@brummer1 ~]$ netperf -N -H 127.0.0.1 -t TCP_RR -l 60 -- -P 9000
TCP REQUEST/RESPONSE TEST from 0.0.0.0 (0.0.0.0) port 9000 AF_INET to 127.0.0.1 () port 9000 AF_INET : no control : histogram : interval : dirty data : demo : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec

32768 65536 1 1 60.00 25266.67
0 0
```

### TCP_STREAM

Over **10GbE** to netserver:

```console
[oberstet@brummer2 ~]$ netperf -H 10.0.0.10 -t TCP_STREAM -l 60
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.0.10 () port 0 AF_INET : histogram : interval : dirty data : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

65536 32768 32768 60.00 9918.16
```

Over **10GbE** to Crossbar.io / PyPy 2.4:

```console
[oberstet@brummer2 ~]$ netperf -N -H 10.0.0.10 -t TCP_STREAM -l 60 -- -P 9000
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 9000 AF_INET to 10.0.0.10 () port 9000 AF_INET : no control : histogram : interval : dirty data : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

0 32768 32768 60.00 1202.82
```

Over **10GbE** to Crossbar.io / CPython 2.7.9:

```console
[oberstet@brummer2 ~]$ netperf -N -H 10.0.0.10 -t TCP_STREAM -l 60 -- -P 9000
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 9000 AF_INET to 10.0.0.10 () port 9000 AF_INET : no control : histogram : interval : dirty data : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

0 32768 32768 60.02 1368.46
```

Over **loopback** to netserver:

```console
[oberstet@brummer1 ~]$ netperf -H 127.0.0.1 -t TCP_STREAM -l 60
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : histogram : interval : dirty data : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

65536 32768 32768 60.01 46675.42
```

Over **loopback** to Crossbar.io / PyPy 2.4:

```console
[oberstet@brummer1 ~]$ netperf -N -H 127.0.0.1 -t TCP_STREAM -l 10 -- -P 9000
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 9000 AF_INET to 127.0.0.1 () port 9000 AF_INET : no control : histogram : interval : dirty data : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

0 32768 32768 10.04 11420.84
```

Over **loopback** to Crossbar.io / CPython 2.7.9:

```console
[oberstet@brummer1 ~]$ netperf -N -H 127.0.0.1 -t TCP_STREAM -l 10 -- -P 9000
TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 9000 AF_INET to 127.0.0.1 () port 9000 AF_INET : no control : histogram : interval : dirty data : demo
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec

0 32768 32768 10.02 10335.18
```

### TCP_CRR

Over **10GbE** to netserver:

```console
[oberstet@brummer2 ~]$ netperf -H 10.0.0.10 -t TCP_CRR -l 60
TCP Connect/Request/Response TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 10.0.0.10 () port 0 AF_INET : histogram : interval : dirty data : demo
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec

32768 65536 1 1 60.02 13206.10
32768 65536
```

Over **loopback** to netserver:

```console
[oberstet@brummer1 ~]$ netperf -H 127.0.0.1 -t TCP_CRR -l 60
TCP Connect/Request/Response TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to 127.0.0.1 () port 0 AF_INET : histogram : interval : dirty data : demo
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec

32768 65536 1 1 60.04 45586.91
32768 65536
```

## Accept Rate

These tests were performed on a notebook running Windows 7.

* http://stackoverflow.com/a/1824817/884770
* http://www.lenholgate.com/blog/2005/11/windows-tcpip-server-performance.html

```
oberstet@THINKPAD-T410S /c/Temp
$ ./EchoServerTest.exe -server 127.0.0.1 -port 9000 -connections 60000 -connectionBatchSize 1000 -connectionBatchDelay 600 -hold -pause
Creating 60000 connections
1000 connections created
2000 connections created
...
59000 connections created
60000 connections created
All connections in progress
All connections complete in 47545ms
60000 established. 0 failed.
Press return to close connections
```

## Resources

* [Why is TCP accept performance so bad under Xen?](http://serverfault.com/questions/272483/why-is-tcp-accept-performance-so-bad-under-xen)
* [Why virtualization reduces network performance](https://news.ycombinator.com/item?id=2574702)
417 docs/pages/administration/production/TLS-Certificates.md
View file

Large diffs are not rendered by default.

94 docs/pages/administration/production/WebSocket-Compliance-Testing.md
View file
@@ -0,0 +1,94 @@
[Documentation](.) > [Administration](Administration) > [Going to Production](Going to Production) > WebSocket Compliance Testing

# WebSocket Compliance Testing

Crossbar.io has best-in-class compliance to the WebSocket protocol (RFC6455).

> Compliance is testified via the [**Autobahn**Testsuite](http://autobahn.ws/testsuite/), the [industry standard](http://autobahn.ws/testsuite/#users) WebSocket compliance testsuite which includes more than 500 automated test cases. Crossbar.io passed *all* tests - 100% strict. No matter what WebSocket server you use, we encourage you to run the testsuite against it and compare.
Protocol compliance is very important for two reasons:
* interoperability
* security

You don't want an evil client disturb or break your servers, or fail to serve clients because of interoperability issues.

**Check out the compliance reports for Crossbar.io [here](http://crossbar.io/download/reports/20140319/index.html).**

## Testing yourself

Install the testsuite:

```
pip install -U autobahntestsuite
```

Create a Crossbar.io node with a node configuration starting a WebSocket testee transport:

```json
{
"workers": [
{
"type": "router",
"transports": [
{
"type": "websocket.testee",
"endpoint": {
"type": "tcp",
"port": 9001,
"backlog": 1024
},
"options": {
"compression": {
"deflate": {
}
}
}
}
]
}
]
}
```

Now create a file `fuzzingclient.json`:

```json
{
"servers": [
{
"agent": "Crossbar.io",
"url": "ws://127.0.0.1:9001"
}
],
"cases": ["*"],
"exclude-cases": [],
"exclude-agent-cases": {}
}
```

This test specification defines which test cases to run against what servers.

Then, start Crossbar.io in a first terminal

```
crossbar start
```

and start the testsuite in a second terminal

```
wstest -m fuzzingclient -s fuzzingclient.json
```

Testing will take some time. It runs over 500 test cases. In the end, it'll generate HTML report files. Open the `reports/servers/index.html` overview page in your browser - click on the green "Pass" links to view the case detail reports.

## Configuration

option | description
---|---
**`id`** | ID of the transport within the running node (default: **`transport<N>`** where `N` is numbered automatically starting from `1`)
**`type`** | Type of transport - must be `"websocket.testee"`.
**`endpoint`** | Listening endpoint for transport. See [Transport Endpoints](Transport Endpoints) for configuration
**`debug`** | Turn on debug logging for this transport instance (default: **`false`**).
**`url`** | The WebSocket server URL to use (default: `null`)
**`options`** | See [WebSocket Options](WebSocket-Options)
54 docs/pages/administration/router/Router-Components.md
View file
@@ -0,0 +1,54 @@
[Documentation](.) > [Administration](Administration) > [Processes](Processes) >[Router Configuration](Router Configuration) > Router Components
# Router Components

*Routers* can run WAMP application components written in Python *side-by-side*, i.e. within the same system process.

Here's an example configuration:

```javascript
{
"controller": {
},
"workers": [
{
"type": "router",
"options": {
"pythonpath": [".."]
},
"components": [
{
"type": "class",
"realm": "realm1",
"classname": "hello.MySession"
}
],
// ... rest of router configuration ...
}
]
}
```

The common parameters for components are:

parameter | description
---|---
**`id`** | The (optional) component ID - this must be unique within the router this components runs in (default: **"componentN"** where N is numbered starting with 1)
**`type`** | The type of component, must be one of `"class"` or `"wamplet"` (**required**)
**`realm`** | The realm on the router to attach this component to, e.g. "realm1" (**required**)
**`role`** | The (optional) role for which the component is authenticated, e.g. "role1", if none give authentication is as "anonymous"
**`extra`** | Optional data provided to the class when instantiating.

For components of `type == "class"`, the following parameters must be provided:

parameter | description
---|---
**`classname`** | The (fully qualified) class name of a class that derives from `ApplicationSession` (**required**)


For components of `type == "wamplet"`, the following parameters must be provided:

parameter | description
---|---
**`package`** | The name of the Python package ("distribution") containing the WAMPlet (**required**)
**`entrypoint`** | The factory entry point within the package (**required**)

83 docs/pages/administration/router/Router-Realms.md
View file
@@ -0,0 +1,83 @@
[Documentation](.) > [Administration](Administration) > [Processes](Processes) >[Router Configuration](Router Configuration) > Router Realms

# Router Realms

Crossbar.io uses *Realms* as domains for separation of routing and administration.

Every WAMP session between Crossbar.io and a *Client* is always attached to a specific *Realm*. Since the routing of calls and events is separate for each realm, sessions attached to different realms won't "see" each other.

For example, consider

* Client 1 attached to `realm1`
* Client 2 attached to `realm1`
* Client 3 attached to `realm2`

on the *same* Crossbar.io router, and both Client 2 and Client 3 subscribed to topic `com.example.mytopic`, when Client 1 publishes an event to `com.example.mytopic`, only Client 2, which is attached to the *same realm* as Client 1, will receive this event.

The realm for the session is selected as part of session establishment.

For example, when creating a new connection to a WAMP router using [Autobahn|JS](http://autobahn.ws/js), the realm the session (running over the connection) should attach to is specified like this:


```javascript
var connection = new autobahn.Connection({url: 'ws://127.0.0.1:9000/', realm: 'realm1'});
```

## Configuring Realms on a Router

Realms are created on a *Router* as part of the *Router* configuration`:
```javascript
{
"version": 2,
"controller": {
},
"workers": [
{
"type": "router",
"options": {
// any router options
},
"realms": [
{
"name": "realm1",
"roles": [
{
"name": "anonymous",
"permissions": [
{
"uri": "*",
"allow": {
"call": true,
"register": true,
"publish": true,
"subscribe": true
}
}
]
}
]
}
],
"transports": [
// transports run by this router
],
"components": [
// app components running side-by-side with this router
],
"manhole": {
// log live into running router via SSH (for debugging)
}
}
]
}
```

In the above example configuration, the *Router* starts up with a single *Realm* (`realm1`).

Authorization is configured on a per-realm basis. Authorization is role-based. Above, clients connecting as anonymous have full permissions for all URIs. (This makes sense for starting out development of a WAMP application.)

*Read more:*

* [[Router Configuration]]
* [[Authorization]]
48 docs/pages/administration/router/Router-Transports.md
View file
@@ -0,0 +1,48 @@
[Documentation](.) > [Administration](Administration) > Router Transports

# Router Transports

Transports are necessary for allowing incoming connections to *Routers*. This applies to WAMP connections as well as for other services that *Routers* provide, such as [Web Services](Web Services).

Crossbar.io provides the following transports for WAMP

* [WebSocket Transport](WebSocket Transport)
* [RawSocket Transport](RawSocket Transport)

as well as

* [Web Transport and Services](Web Transport and Services)

which include WebSocket as one suboption.

All of above is running over [Transport Endpoints](Transport Endpoints), so you need that as well to get a fully working transport.

> For completeness, there is also the [Flash Policy auxiliary transport](Flash Policy Transport), and special transports for [TCP Benchmarks](Stream Testee) and [WebSocket Testing](WebSocket Compliance Testing).

## Background

[WAMP](http://wamp.ws/) runs over any transport with the following characteristics (see the [spec](https://github.com/tavendo/WAMP/blob/master/spec/basic.md#transports)):

1. message-based
2. reliable
3. ordered
4. bidirectional (full-duplex)

Over which WAMP transport an application component is connected to a router does not matter. It's completely transparent from the application component point of view.

The [WAMP spec](https://github.com/wamp-proto/wamp-proto/blob/master/rfc/draft-oberstet-hybi-tavendo-wamp.html) currently defines these transports:

* [WAMP-over-WebSocket Transport]
* [WAMP-over-RawSocket Transport]
* [WAMP-over-Longpoll Transport]

Crossbar.io currently supports **18** WAMP transports in total:

![Crossbar.io: supported WAMP Transports](/static/img/docs/gen/crossbar_transports_1.png)

The most common transports are the following:

![Common WAMP Transports](/static/img/docs/gen/crossbar_transports_2.png)

---
148 docs/pages/administration/router/transport/Cookie-Tracking.md
View file
@@ -0,0 +1,148 @@
[Documentation](.) > [Administration](Administration) > [Router Transports](Router Transports) > [WebSocket Transport](WebSocket Transport) > Cookie Tracking

# Cookie Tracking

Cookie tracking identifies and tracks WAMP-over-WebSocket client connections using HTTP cookies.

Cookie tracking can be enabled on [WebSocket-](WebSocket-Transport) and [Web-Transport](Web-Transport-and-Services). It is not available on other transport types such as [RawSocket](RawSocket-Transport).

> While enabling cookie tracking is a prerequisite for cookie-based WAMP authentication, it can be used without authentication.

## How it works

Cookie tracking is backed by a configurable cookie store. Currently we have two types of store:

* memory-backed
* file-backed

> In the future, we'll have an LMDB backed cookie store as well.
The stored information for a cookie includes the cookie ID as well as authentication information (see [Cookie Authentication](Cookie-Authentication)).

With a memory-backed cookie store, cookies are stored in in-memory objects, and, obviously, those cookies will be gone after stopping Crossbar.io

With a file-backed cookie store, cookies are stored in an append-only, on-disk file.


## Cookie Tracking without Authentication

Cookie tracking can be enabled without using cookie-based authentication as well.

This is the case when

1. no authentication is configured at all
2. only anonymous authentication is configured
3. only non-cookie based authentication is configured

With 1) and 2) and cookie tracking enabled, Crossbar.io will automatically use the cookie ID as the authentication ID (`authid`) for the client.

This way, you still can **identify** clients across reconnects using WAMP `authid`. Without cookies, in case of 1) and 2), a WAMP client will get a random `authid` **each time** it connects.

On the other hand, with 3), the authentication ID (`authid`) still comes from the respective authentication method used.


## Cookie Tracking with Authentication

Please see [Cookie Authentication](Cookie-Authentication).


## Configuration
The following parameters are all optional and shared between different backing stores:

option | description
---|---
**`name`** | The field name where Crossbar.io will store its (random) tracking ID within the Cookie set. The default is `"cbtid"`. Must match the regular expression `^[a-z][a-z0-9_]+$`.
**`length`** | The length of the value for the tracking ID. The default is 24 (which amounts to 144 bits of randomness). The default should be large enough to reduce the collision probability to essentially zero. Must be between 6 and 64.
**`max_age`**| The maximum Cookie lifetime in seconds. The default is 1 day. Must be between 1 second and 10 years.
**`store`** | A dictionary with cookie store configuration (see below).

The `store` is a dictionary with the following attributes for a **memory-backed** cookie store:

attribute | description
---|---
**`type`** | Must be `"memory"`.

and for a **file-backed** cookie store:

attribute | description
---|---
**`type`** | Must be `"file"`.
**`filename`** | Either an absolute path or a relative path (relative to the node directory)

---

## Examples

To configure a memory-backed cookie store:

```json
{
"transports": [
{
"type": "web",
"endpoint": {
"type": "tcp",
"port": 8080
},
"paths": {
"/": {
"type": "static",
"directory": "../web"
},
"ws": {
"type": "websocket",
"cookie": {
"name": "cbtid",
"length": 24,
"max_age": 864000,
"store": {
"type": "memory"
}
}
}
}
}
]
}
```

To configure a file-backed cookie store:

```json
{
"transports": [
{
"type": "web",
"endpoint": {
"type": "tcp",
"port": 8080
},
"paths": {
"/": {
"type": "static",
"directory": "../web"
},
"ws": {
"type": "websocket",
"cookie": {
"name": "cbtid",
"length": 24,
"max_age": 864000,
"store": {
"type": "file",
"filename": "cookies.dat"
}
}
}
}
}
]
}
```

In above example, the cookie store would reside in `.crossbar/cookies.dat` for a default node directory.

> Note that the cookie file is "growing forever". There is no purging whatsoever, as the file is written append-only. The LMDB cookie store will provide a more advanced store.
---