Add RESTful web server and webhook parser

docs/configuration-reference.md

@@ -105,6 +105,18 @@ skills:
 
 See [module options](#module-options) for installing custom skills.
 
+### `web`
+
+Configure the REST API in opsdroid.
+
+By default opsdroid will start a web server on port `8080` accessible only to localhost. For more information see the [REST API docs](rest-api).
+
+```yaml
+web:
+  host: '127.0.0.1'  # set to '0.0.0.0' to allow all traffic
+  port: 8080
+```
+
 ## Module options
 
 All modules are installed from git repositories. By default if no additional options are specified opsdroid will look for the repository at `https://github.com/opsdroid/<moduletype>-<modulename>.git`.

docs/parsers/crontab.md

@@ -6,6 +6,7 @@ The crontab parser is a bit different to other parsers. This parser doesn't take
 
 ```python
 from opsdroid.matchers import match_crontab
+from opsdroid.message import Message
 
 @match_crontab('* * * * *')
 async def mycrontabskill(opsdroid, config, message):

docs/parsers/overview.md

@@ -3,9 +3,11 @@
 When writing skills for opsdroid there are multiple parsers you can use for matching messages to your functions.
 
  * [Regular Expression](parsers/regex)
-  * `opsdroid.skills.match_regex`
+    * `opsdroid.skills.match_regex`
  * [API.AI](parsers/api.ai)
-  * `opsdroid.skills.match_apiai_action`
-  * `opsdroid.skills.match_apiai_intent`
+    * `opsdroid.skills.match_apiai_action`
+    * `opsdroid.skills.match_apiai_intent`
  * [Crontab](parsers/crontab)
-  * `opsdroid.skills.match_crontab`
+    * `opsdroid.skills.match_crontab`
+ * [Webhook](parsers/webhook)
+    * `opsdroid.skills.match_webhook`

docs/parsers/webhook.md

@@ -0,0 +1,33 @@
+# Webhook Parser
+
+Similar to the crontab parser this parser doesn't take a message as an input, it takes a webhook instead. It allows you to trigger the skill by calling a specific URL endpoint.
+
+## Example
+
+```python
+from aiohttp.web import Request
+
+from opsdroid.matchers import match_webhook
+from opsdroid.message import Message
+
+@match_webhook('examplewebhook')
+async def mycrontabskill(opsdroid, config, message):
+
+    if type(message) is not Message and type(message) is Request:
+      # Capture the request POST data and set message to a default message
+      request = await message.post()
+      message = Message("", None, connector.default_room,
+                        opsdroid.default_connector)
+
+    # Respond
+    await message.respond('Hey')
+```
+
+**Config**
+
+```yaml
+skills:
+  - name: "exampleskill"
+```
+
+The above skill would be called if you send a POST to `http://localhost:8080/skill/exampleskill/examplewebhook`. As the skill is being triggered by a webhook the `message` argument being passed in will be set to the [aiohttp Request](http://aiohttp.readthedocs.io/en/stable/web_reference.html#aiohttp.web.BaseRequest), this means you need to create an empty message to respond to. You will also need to know which connector, and possibly which room, to send the message back to. For this you can use the `opsdroid.default_connector` and `opsdroid.default_connector.default_room` properties to get some sensible defaults.

docs/rest-api.md

@@ -0,0 +1,64 @@
+# REST API
+
+There is a RESTful API for opsdroid which by default is only accessible to `localhost` on port `8080`. See the [configuration reference](configuration-reference#web) for config options.
+
+## Methods
+
+### `/` _[GET]_
+
+A test url you can use to check whether the API is running.
+
+**Example response**
+
+```json
+{
+  "timestamp": "2017-02-05T10:12:51.622981",
+  "status": 200,
+  "result": {
+    "message": "Welcome to the opsdroid API"
+  }
+}
+```
+
+### `/stats/` _[GET]_
+
+This method returns runtime statistics which could be useful in monitoring.
+
+**Example response**
+
+```json
+{
+  "timestamp": "2017-02-05T10:14:37.494541",
+  "status": 200,
+  "result": {
+    "version": "0.6.0",
+    "messages": {
+      "total_parsed": 164,
+      "webhooks_called": 28
+    },
+    "modules": {
+      "skills": 13,
+      "connectors": 1,
+      "databases": 0
+    }
+  }
+}
+```
+
+### `/skill/{skillname}/{webhookname}` _[POST]_
+
+This method family will call skills which have been decorated with the [webhook matcher](parsers/webhook). The URI format includes the name of the skill from the `configuration.yaml` and the name of the webhook set in the decorator.
+
+The response includes information on whether a skill was successfully triggered or not.
+
+**Example response**
+
+```json
+{
+  "timestamp": "2017-02-04T16:25:01.956323",
+  "status": 200,
+  "result": {
+    "called_skill": "examplewebhookskill"
+  }
+}
+```

mkdocs.yml

@@ -6,6 +6,7 @@ pages:
   - Home: 'index.md'
   - Getting Started: 'getting-started.md'
   - Configuration Reference: 'configuration-reference.md'
+  - REST API: 'rest-api.md'
   - Extending opsdroid:
     - Adding skills: 'extending/skills.md'
     - Adding connectors: 'extending/connectors.md'
@@ -14,4 +15,6 @@ pages:
     - Overview: 'parsers/overview.md'
     - Regular Expressions: 'parsers/regex.md'
     - API.AI: 'parsers/api.ai.md'
+    - Crontab: 'parsers/crontab.md'
+    - Webhook: 'parsers/webhook.md'
   - Contributing: 'contributing.md'

opsdroid/__main__.py

@@ -7,6 +7,7 @@
 
 from opsdroid.core import OpsDroid
 from opsdroid.const import LOG_FILENAME
+from opsdroid.web import Web
 
 
 _LOGGER = logging.getLogger("opsdroid")
@@ -99,6 +100,7 @@ def main():
     with OpsDroid() as opsdroid:
         opsdroid.load()
         configure_logging(opsdroid.config)
+        opsdroid.web_server = Web(opsdroid)
         opsdroid.start_loop()
         opsdroid.exit()
 

opsdroid/core.py

@@ -37,6 +37,11 @@ def __init__(self):
         self.memory = Memory()
         self.loader = Loader(self)
         self.config = {}
+        self.stats = {
+            "messages_parsed": 0,
+            "webhooks_called": 0
+        }
+        self.web_server = None
         _LOGGER.info("Created main opsdroid object")
 
     def __enter__(self):
@@ -95,6 +100,7 @@ def start_loop(self):
         self.setup_skills(skills)
         self.start_connector_tasks(connectors)
         self.eventloop.create_task(parse_crontab(self))
+        self.web_server.start()
         try:
             self.eventloop.run_forever()
         except (KeyboardInterrupt, EOFError):
@@ -147,6 +153,7 @@ def start_databases(self, databases):
 
     async def parse(self, message):
         """Parse a string against all skills."""
+        self.stats["messages_parsed"] = self.stats["messages_parsed"] + 1
         tasks = []
         if message.text.strip() != "":
             _LOGGER.debug("Parsing input: " + message.text)

opsdroid/matchers.py

@@ -1,6 +1,12 @@
 """Decorator functions to use when creating skill modules."""
 
+import logging
+
 from opsdroid.core import OpsDroid
+from opsdroid.web import Web
+
+
+_LOGGER = logging.getLogger(__name__)
 
 
 def match_regex(regex):
@@ -49,3 +55,29 @@ def matcher(func):
                                     opsdroid.loader.current_import_config})
         return func
     return matcher
+
+
+def match_webhook(webhook):
+    """Return webhook match decorator."""
+    def matcher(func):
+        """Add decorated function to skills list for webhook matching."""
+        for opsdroid in OpsDroid.instances:
+            config = opsdroid.loader.current_import_config
+            opsdroid.skills.append({"webhook": webhook, "skill": func,
+                                    "config": config})
+
+            async def wrapper(req, opsdroid=opsdroid, config=config):
+                """Wrap up the aiohttp handler."""
+                _LOGGER.info("Running skill %s via webhook", webhook)
+                opsdroid.stats["webhooks_called"] = \
+                    opsdroid.stats["webhooks_called"] + 1
+                await func(opsdroid, config, req)
+                return Web.build_response(200, {"called_skill": webhook})
+
+            opsdroid.web_server.web_app.router.add_post(
+                "/skill/{}/{}".format(config["name"], webhook), wrapper)
+            opsdroid.web_server.web_app.router.add_post(
+                "/skill/{}/{}/".format(config["name"], webhook), wrapper)
+
+        return func
+    return matcher

opsdroid/web.py

@@ -0,0 +1,84 @@
+"""Submodule to handle web requests in opsdroid."""
+
+import json
+import logging
+from datetime import datetime
+
+from aiohttp import web
+
+from opsdroid.const import __version__
+
+
+_LOGGER = logging.getLogger(__name__)
+
+
+class Web:
+    """Web server for opsdroid."""
+
+    def __init__(self, opsdroid):
+        """Setup web object."""
+        self.opsdroid = opsdroid
+        try:
+            self.config = self.opsdroid.config["web"]
+        except KeyError:
+            self.config = {}
+        self.web_app = web.Application(loop=self.opsdroid.eventloop)
+        self.web_app.router.add_get('/', self.web_index_handler)
+        self.web_app.router.add_get('', self.web_index_handler)
+        self.web_app.router.add_get('/stats', self.web_stats_handler)
+        self.web_app.router.add_get('/stats/', self.web_stats_handler)
+
+    @property
+    def get_port(self):
+        """Return port from config or the default."""
+        try:
+            port = self.config["port"]
+        except KeyError:
+            port = 8080
+        return port
+
+    @property
+    def get_host(self):
+        """Return host from config or the default."""
+        try:
+            host = self.config["host"]
+        except KeyError:
+            host = '127.0.0.1'
+        return host
+
+    def start(self):
+        """Start web servers."""
+        _LOGGER.debug(
+            "Starting web server with host %s and port %s",
+            self.get_host, self.get_port)
+        web.run_app(self.web_app, host=self.get_host,
+                    port=self.get_port, print=_LOGGER.info)
+
+    @staticmethod
+    def build_response(status, result):
+        """Build a json response object."""
+        return web.Response(text=json.dumps({
+            "timestamp": datetime.now().isoformat(),
+            "status": status,
+            "result": result
+        }))
+
+    def web_index_handler(self, request):
+        """Handle root web request."""
+        return self.build_response(200, {
+            "message": "Welcome to the opsdroid API"})
+
+    def web_stats_handler(self, request):
+        """Handle stats request."""
+        return self.build_response(200, {
+            "version": __version__,
+            "messages": {
+                "total_parsed": self.opsdroid.stats["messages_parsed"],
+                "webhooks_called": self.opsdroid.stats["webhooks_called"]
+            },
+            "modules": {
+                "skills": len(self.opsdroid.skills),
+                "connectors": len(self.opsdroid.connectors),
+                "databases": len(self.opsdroid.memory.databases)
+            }
+        })

tests/test_core.py

@@ -38,6 +38,7 @@ def test_load_config(self):
     def test_start_loop(self):
         with OpsDroid() as opsdroid:
             mockconfig = {}, {}, {}
+            opsdroid.web_server = mock.Mock()
             opsdroid.loader = mock.Mock()
             opsdroid.loader.load_config = mock.Mock(return_value=mockconfig)
             opsdroid.start_databases = mock.Mock()