Bot platform adapter for Botkit

[twardnw]

It seems that Hubot is practically abandoned by Github at this point, there is a small community effort to try and keep it alive, but that is not progressing very well. botkit has a very substantial following and very active development. It would be quite nice if RocketChat could build a connector for botkit ( per howdyai/botkit#1126 they request that the platform wanting the connector start the effort ). Unfortunately, this is well beyond my JS skills.

[JSzaszvari]

@rocket-cat label add enhancement

[timkinnane]

The recent release of our Node.js SDK with a platform agnostic driver for building bots on Rocket.Chat should unlock potential for this issue. We're hoping to make progress from upcoming Google Summer of Code projects but also encourage any interested community members to check out the SDK and start building for their preferred platform.

Get the SDK

See examples implementation in Hubot adapter

[bizzbyster]

In addition to the changes in Rocket.chat to support Botkit, we'll need to write botkit middleware that formats messages into a structure that Rocket.chat understands. See https://github.com/howdyai/botkit/blob/master/docs/readme-pipeline.md#format. So just pointing out that this issue involves both Rocket.chat changes and a piece of middleware that we'd ideally publish to the botkit repo.

Just to be clear, the nice thing about Botkit is, for instance, you can write a single "rich message" that includes buttons and postbacks and this will work on multiple chat platforms due to the middleware layer doing the re-formatting.

[bizzbyster]

@timkinnane Is the node.js bot SDK intended to run in its own process similar to the hubot architecture? Or, potentially, could this be implemented as a Rocket.chat.app and run within the Rocket.chat server? Forgive me if I'm way off base architecturally with this question.

[timkinnane]

I think you're close @bizzbyster but it's a bit easier than that.
You can take the hubot adapter (develop branch) as an example implementation of the SDK.
Hubot also requires messages be in a different format, but that's handled by using the message params that come from Rocket.Chat in the constructor for Hubot's own message class (see new TextMessage). So a similar approach would work for Botkit without any changes required to Rocket.Chat.

The adapter / connector model does exactly that kind of translation. Nearly all message platforms and bot frameworks follow that convention, of having a layer between them passing things back and forth in the required formats so neither has to be fitted to each other in core.

"Rich messages" won't be supported in Rocket.Chat yet, but you're aware of the work on that front.

The SDK is not intended to run as a bot, but it does provide most of the features that bots and their adapters need to work with Rocket.Chat. It is intended to be re-used as a dependency, to keep things DRY. If there's any particular features needed for Botkit that weren't in there, we can improve the SDK so those features are available to other platforms too.

[bizzbyster]

"The adapter / connector model does exactly that kind of translation. Nearly all message platforms and bot frameworks follow that convention, of having a layer between them passing things back and forth in the required formats so neither has to be fitted to each other in core."

Ok this makes sense. Currently we are using hubot to integrate RC with our Dialogflow bot using https://www.npmjs.com/package/hubot-apiai with modifications to the coffee file. I suppose in this case then our Hubot is architecturally the same as a Botkit "middleware" for RC. In your opinion, is there any reason we don't just implement our botkit middleware using Hubot? It might be doubt be easier since Hubot is already interfacing with Rocket.chat. But that said it'd probably add a layer of cruft that would not exist if we just implemented Botkit middleware using the SDK. Perhaps a better way forward would be to fork and modify https://github.com/watson-developer-cloud/botkit-middleware and just add RC support using bot SDK. Thoughts?

One thing that concerns me is the scalability of this "middleware" model -- we'd want to make sure that the middleware service itself has all the goodness of redundancy for scale and high availability and automatic monitoring, etc. But perhaps that should be seen as a secondary issue at this point.

[timkinnane]

@bizzbyster - I think the middleware is something different in this context. E.g. Watson is an NLP provider that can be used to add attributes to messages being processed by Botkit.

In Botkit terminology, there would be a Rocket.Chat "connector" that negotiates the connection and message formatting, between a Rocket.Chat message stream and the bot's internal logic.

So, taking the example of a Botkit bot running on Rocket.Chat with Watson middleware, it looks like this...

1. Botkit bot connects to Rocket.Chat via connector (aka adapter).
2. Rocket.Chat user sends message to bot user.
3. Rocket.Chat emits message in streamer.
4. Botkit bot receives message, parsed via connector into the Botkit standard format.
5. Botkit tries to process message intent via Watson middleware.
6. Botkit forms response based on internal scripting and gathered context from middleware.
7. Botkit sends response to Rocket.Chat room using the connector.

But there's no requirement for Hubot. Hubot and Botkit perform essentially the same role in the stack, so Hubot passing messages to Botkit is essentially having one bot operate another.

In your scenario, you could replace Watson in the example above with API.ai, but that's only if you're using API.ai primarily for it's NLP. If you're using it to compose whole conversation flows, then it is essentially your bot, so using Botkit as well is another example of redundant bots driving bots. What you need in that case is a Rocket.Chat to Dialogflow adapter. Which is also possible, but we're focusing on open source adapters first.

[bizzbyster]

Thanks Tim. I need to digest this. But I get what you mean now. Middleware is definitely the NLP (Watson, Wit.ai, api.ai, etc.) okay got it.

Your step (1) above (botkit bot connects to Rocket.Chat via connector/adapter) is essentially this step: https://github.com/watson-developer-cloud/botkit-middleware/blob/master/examples/simple-bot/simple-bot-slack.js#L30 but for RC instead of Slack, is that correct?

[timkinnane]

Yeah. So Botkit has a built-in connector for Slack and a few other platforms, but we can build our own for Rocket.Chat as an addon to allow Botkit conversational methods to interface through Rocket.Chat. I'm not too sure of the particulars, just need to start hacking and see what makes sense.

Ben from Botkit pointed us at this guide: https://botkit.ai/docs/howto/build_connector.html
So that's a definite step 1.

Step 2 would be a starter package, similar to our hubot-rocketchat-boilerplate, providing the end to end scaffold for a basic bot using Botkit and Rocket.Chat. For that we have this example to go by: https://github.com/howdyai/botkit-starter-slack

[ear-dev]

@timkinnane So if our use case were to talk to/with pre-configured botkit bots, hosted in the cloud, then Rocket.Chat just needs to implement an adapter, and not an actual botkit bot. As I understand it, that would be different than the current implementation with hubot, where you have implemented both a connector and a hubot bot itself. Am I understanding that correctly?

[timkinnane]

@ear-dev Not exactly. I don't think Rocket.Chat needs to implement anything, it's all done on the Botkit side. The pre-configured Botkit bots would need to be updated to require and use the Rocket.Chat connector module. I am going to set up the connector project repo early next week to kick things off. I won't be able to complete the development work alone, but hopefully it will make more sense when there's something tangible to look at.

[timkinnane]

I've started two repos for the required components, with some declaration in the readme about how they should fit together. I haven't made any working examples, just putting the resources up so the two communities have something to put shared focus on:

• botkit-rocketchat-connector
• botkit-starter-rocketchat

Anyone who is able to contribute, please make PRs to those repos. If you're contributing often I can give direct write access. Thanks!

[timkinnane]

@arthurTemporim - I'd like to assign you this issue as part of your GSOC project but I think you need to comment or subscribe or something first because I can't select you from the list.