Skip to content

Commit

Permalink
Merge pull request #33 from tgpfeiffer/review-tutorial
Browse files Browse the repository at this point in the history
review tutorial and fix smaller formatting issues
  • Loading branch information
tgpfeiffer committed Mar 15, 2016
2 parents ca28d16 + 02840e2 commit 6e1fbd0
Show file tree
Hide file tree
Showing 6 changed files with 301 additions and 255 deletions.
14 changes: 7 additions & 7 deletions source/bql/operators.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Operators

This chapter introduces operators used in BQL.

Arithmetic operators
Arithmetic Operators
====================

BQL provides following arithmetic operators:
Expand All @@ -33,7 +33,7 @@ example, ``3 + 5 * 2.5`` is valid.
because ``--`` and succeeding characters are parsed as a comment. For
example, ``---3`` is parsed as ``--`` and a comment body ``-3``.

String operators
String Operators
================

BQL provides following string operators:
Expand All @@ -48,7 +48,7 @@ an error. When one operand is ``NULL``, the result is also ``NULL``.
For instance, ``NULL || "str"``, ``"str" || NULL``, and ``NULL || NULL`` result
in ``NULL``.

Comparison operators
Comparison Operators
====================

BQL provides following comparison operators:
Expand Down Expand Up @@ -102,7 +102,7 @@ There's one exception. When one operand of comparison operators except

.. _bql_operators_null_comparison:

``NULL`` comparison
``NULL`` Comparison
-------------------

In a three-valued logic, comparing any value with ``NULL`` results in ``NULL``.
Expand Down Expand Up @@ -131,7 +131,7 @@ expression is ``NULL``.
value. Unlike ``NULL``, comparing infinite floating point values
always result in ``false``.

Logical operators
Logical Operators
=================

BQL provides following logical operators:
Expand All @@ -146,8 +146,8 @@ BQL provides following logical operators:
Logical operators also follow the three-valued logic. For example,
``true AND NULL`` and ``NULL OR false`` result in ``NULL``.

JSON operators
JSON Operators
==============

Operator precedence
Operator Precedence
===================
6 changes: 3 additions & 3 deletions source/bql/queries.rst
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ In terms of BQL syntax, the emit operator keyword is given after the ``SELECT``
The following subsections describe how each operator works.
To illustrate the effects of each operator, a visual example is provided afterwards.

``RSTREAM`` operator
``RSTREAM`` Operator
^^^^^^^^^^^^^^^^^^^^

When ``RSTREAM`` is specified, all tuples in the relation are emitted.
Expand All @@ -104,7 +104,7 @@ In contrast,
emits (at most) 100 tuples for every tuple in ``src``.


``ISTREAM`` operator
``ISTREAM`` Operator
^^^^^^^^^^^^^^^^^^^^

When ``ISTREAM`` is specified, all tuples in the relation *that have not been in the previous relation* are emitted.
Expand Down Expand Up @@ -133,7 +133,7 @@ To illustrate the multiplicity counting,
will emit three times :math:`1` and then nothing (because after the first three tuples processed, both the previous and the current relation always look like :math:`\{1, 1, 1\}`.)


``DSTREAM`` operator
``DSTREAM`` Operator
^^^^^^^^^^^^^^^^^^^^

The ``DSTREAM`` operator is very similar to ``ISTREAM``, except that it emits all tuples in the *previous* relation that are not also contained in the current relation.
Expand Down
4 changes: 2 additions & 2 deletions source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to SensorBee's documentation!
Welcome to SensorBee's Documentation!
=====================================

Contents:
Expand All @@ -29,7 +29,7 @@ Contents:
ref.rst


Indices and tables
Indices and Tables
==================

* :ref:`genindex`
Expand Down
55 changes: 30 additions & 25 deletions source/preface.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,19 @@ This document is structured as follows:
What is SensorBee?
******************

SensorBee is an open source lightweight stateful streaming data processing
engine for IoT. SensorBee is designed to be used for streaming ETL
SensorBee is an open source, lightweight, stateful streaming data processing
engine for the Internet of Things (IoT). SensorBee is designed to be used for streaming ETL
(Extract/Transform/Load) at the edge of the network including
`Fog <http://www.cisco.com/c/dam/en_us/solutions/trends/iot/docs/computing-overview.pdf>`_.
`Fog Computing <http://www.cisco.com/c/dam/en_us/solutions/trends/iot/docs/computing-overview.pdf>`_.
In ETL operations, SensorBee mainly focuses on data transformation and data
enrichment, especially using machine learning. SensorBee is very small (< 30MB)
enrichment, especially using machine learning. SensorBee is very small (stand-alone executable file size < 30MB)
and runs on small computers such as Raspberry Pi.

The processing flow in SensorBee is written in BQL, a dialect of CQL
(Continuous Query Language), which is similar to SQL but extended for streaming
data processing. Its internal data structure (tuple) is compatible to JSON
data processing. Its internal data structure (tuple) is compatible to JSON documents
rather than rows in RDBMSs. Therefore, in addition to regular SQL expressions,
BQL implements JSON notation and type conversions that works well with JSON.
BQL implements JSON notation and type conversions that work well with JSON.
BQL is also schemaless at the moment to support rapid prototyping and
integration.

Expand All @@ -45,31 +45,31 @@ integration.

SensorBee manages user-defined states (UDSs) and BQL utilizes those states to
perform stateful processing on streaming data. An example of stateful processing
is machine learning. With the Python extension, SensorBee supports deep learning
using `Chainer <http://chainer.org/>`_, which is a flexible deep learning
framework also developed by Preferred Networks, Inc. and Preferred
Infrastructure, Inc.. The combination of SensorBee and Chainer enables users to
is machine learning. Via a Python extension, SensorBee supports deep learning
using `Chainer <http://chainer.org/>`_, a flexible deep learning
framework developed by `Preferred Networks, Inc. <https://www.preferred-networks.jp/>`_ and
`Preferred Infrastructure, Inc. <https://preferred.jp/>`_ The combination of SensorBee and Chainer enables users to
support not only online analysis but also online training of deep learning
models at the edge of the network with a help of GPUs. Preprocessing of data
models at the edge of the network with the help of GPUs. Preprocessing of data
and feature extraction from preprocessed results can be written in BQL. The
results can be computed in an online manner and directly connected to deep
learning models implemented with Chainer.

By combining JSON-like data structure of BQL and machine learning, SensorBee
becomes good at handling unstructured data such as text written in natural
languages and even video streams, which are not well supported by majority of
data processing engines. Therefore, SensorBee can be located, for example,
languages and even video streams, which are not well supported by most
data processing engines. Therefore, SensorBee can operate, for example,
between a video camera and Cloud-based (semi-structured) data analytics
services so that those services don't have to analyze raw video images and
can only utilize the information extracted from them by SensorBee.

SensorBee can be extended to work with existing databases or data processing
solutions by developing data source or sink plugins. For example, it officially
provides plugins for `fluentd <http://www.fluentd.org/>`_, which is an open
source data collector and has various input and output plugins for major
provides plugins for `fluentd <http://www.fluentd.org/>`_, an open
source data collector, and has various input and output plugins for major
databases and Cloud services.

SensorBee is **not** designed for:
SensorBee has **not** been designed for:

* very large scale data processing
* massively parallel streaming data processing
Expand All @@ -93,6 +93,9 @@ The following conventions are used in the synopsis of a command:

* Dots (``...``) mean that the preceding element can be repeated.

* Commands that are to be run in a normal system shell are prefixed with a
dollar sign (``$``).

Types and keywords in BQL are written with ``fixed-size fonts``.

*******************
Expand All @@ -103,18 +106,20 @@ Besides this documentation, there're other resources about SensorBee:

Website

`<http://sensorbee.io/>`_ has general information of SensorBee.
`<http://sensorbee.io/>`_ has general information about SensorBee.

github
Github

The `sensorbee <https://github.com/sensorbee>`_ organization has the core
source code of SensorBee and its official plugins.
The `sensorbee <https://github.com/sensorbee>`_ organization contains SensorBee's core
source code repository and its official plugins.

Godoc
.. Godoc
SensorBee is written in Go and the document of its source code can be found
at (TODO: godoc)
SensorBee is written in Go and the document of its source code can be found
at (TODO: godoc link)
Mailing lists
Mailing Lists

TODO
There are two Google Groups for discussion and questions about SensorBee:
https://groups.google.com/forum/#!forum/sensorbee (English) and
https://groups.google.com/forum/#!forum/sensorbee-ja (Japanese).

0 comments on commit 6e1fbd0

Please sign in to comment.