[Gitter] Handle bad tokens more gracefully, so that all gateways don't go down

[patcon]

Is your feature request related to a problem? Please describe.
Gitter token went stale and brought down the whole matterbridge app.

Describe the solution you'd like
Would be great if handled this gracefully and just logged an error

Describe alternatives you've considered
None.

Additional context
This is what happened recently with Gitter. But tokens can go stale for many reasons (a coworker resetting them, etc.) and would be nice if this didn't break the other gateways :)
http://blog.gitter.im/2018/06/11/gitter-token-leak-security-issue-notification/

Thanks for everything!

[42wim]

It crashed and paniced ?
Do you maybe have a debug or log?

[patcon]

Ah shoot, don't have the logs that far back anymore, so will have to make time to repro. thanks! (I didn't notice "panick" in the output, but also noticed in another log file that it's easy to miss)

[akhmerov]

We had a similar problem. I believe this is due to how matterbridge handles not being able to setup a relay (it refuses to continue).

I believe setting up a relay to a private mattermost channel and booting the bot out of it would have the same effect.

As a possible fix I suggest to introduce a possibly configurable way for matterbridge to handle failures to set up a gateway. This is especially important because it is not visible to the users whether the bridge is running or not.

As a sane default I'd say matterbridge could ignore all failures. Alternatively it could report changes in its status to a separate channel (relay initiated/relay failed/shutting down).

[42wim]

Matterbridge refuses to start-up if a bridge is down. This seems like a sane default?

When all bridges are ok, matterbridge doesn't do anymore checks, it's the different libraries that handle the reconnects.

[akhmerov]

Indeed, I was assuming a certain usage pattern, which isn't enough to reason what could be the best default, my apologies.

Yet, for some patterns of usage I believe a different treatment of failure would be more suitable. Let me describe our situation and explain why it is a relevant feature.

• We use a single matterbridge instance to handle multiple gateways (about 5 between mattermost and gitter, and about 20 between different teams within mattermost).
• The matterbridge is dockerised and configured via ansible
• Whenever a new relay is added, the matterbridge container is restarted (or deleted and started anew).

If one gateway cannot be set up (can happen because gitter invalidates a token or because the bot user is booted from a private channel), all gateways silently go down (the only way for the users to observe that something doesn't work is to see that the messages stop being relayed).

This inability to see the failure, combined with all channels failing can be rather disruptive, and therefore I believe for our use case matterbridge continuing to work after failing to setup one gateway would be better.

[42wim]

Ok, I'll make an option to continue even if a bridge has errors

[akhmerov]

Thanks!

[Arinerron]

@42wim What was the option (and how do I use it)? I don't know go.

[Arinerron]

Figured it out. Add IgnoreFailureOnStart="true" right after [general] in the toml conf file