Automating Huginn?

[elsurudo]

Huginn is great for automation, which begs the question: how can I automate Huginn?

I know there is the CommandAgent to enable/disable Agents and update configuration, but what about creating agents and updating configuration from an outside system?

For example, I have a few workflows where several agents feed into an email digest, and each of these agents is similar, but with slightly different config (an example of one workflow: each watches for changes from a different URL). So every time I want to watch a different URL, for example, I need to clone an agent, change some config, and hook it up to the main workflow. This is manual and error-prone work that I'd like to avoid, if possible.

It would be cool if I could do this via API, or even a bookmarklet.

Here, I see some logic in a GET-only API like Pinboard has, which makes it easy to interact with via URLs and bookmarklets.

---

Basically I'm more looking to spark a discussion here – has there ever been any discussion around this I may have missed, and how do people work around such issues currently?

One thought that I just had: what about extending CommandAgent to create new agents with a template for config, or being able to clone agents – then I could maybe do what I want by using the Webhook agent -> Command agent.

[dsander]

Hi!

I think almost all (if not all) HTTP methods that are exposed and usable by the user also respond to JSON requests, so in theory Huginn has an API for everything. The problem is that we don't have documentation for any of the methods. You would need to "reverse engineer" the parameters used by the HTML "fromtend" and convert them into JSON requests.

Logging into Huginn automatically is a bit tedious, since we don't have API Keys you need to create a session and send the cookie to authenticate your requests. Here is an example of Huginn logging into itself using a few Agents: https://huginnio.herokuapp.com/scenarios/19

One thought that I just had: what about extending CommandAgent to create new agents with a template for config, or being able to clone agents – then I could maybe do what I want by using the Webhook agent -> Command agent.

That should be doable, the only problem I see with that is that after the Agent is cloned it can not be controlled by the commander because it is not connected to it anymore. You should be able to extend the scenario I liked to make a call to the create Agent endpoint.

[elsurudo]

Logging into Huginn automatically is a bit tedious, since we don't have API Keys you need to create a session and send the cookie to authenticate your requests. Here is an example of Huginn logging into itself using a few Agents: https://huginnio.herokuapp.com/scenarios/19

Thanks for the tip, but on the other hand this seems like a needlessly-complex workaround.

Would you guys be opposed to a PR that added API key auth (with a simple interface to manage API keys) at some point?

[elsurudo]

Interestingly, I just noticed that this fork has the start of a true API: https://github.com/automaticmode/active_workflow#rest-api

With some discussion of porting that part upstream here: automaticmode/active_workflow#2 (comment)

[dsander]

Would you guys be opposed to a PR that added API key auth (with a simple interface to manage API keys) at some point?

That sounds like a good idea to me. Do you think a user needs to have more than one API key? My main concern with token authentication is timing attacks. We had some discussion about it here #628. Since almost all application that I use and issue API keys don't use an email or login in combination with the token I am not sure how real timing attacks really are.

Interestingly, I just noticed that this fork has the start of a true API: https://github.com/automaticmode/active_workflow#rest-api

I don't feel like maintaining two APIs for Huginn, if we start to document the existing API we would be sort of bound to our "no breaking changes" promise. That could get annoying, but if we get to a point which would create a breaking change we could start actually versioning the API then.

[elsurudo]

Do you think a user needs to have more than one API key?

For my own personal use, I would be satisfied with just one (for now). But in case I ever had a use-case to hand over an API key to a service I couldn't fully trust, it would make sense to have a scheme where I could generate & revoke tokens at will (one per service, ideally).

So I think a good start could be to have just one by default and display that, but at least keep it in a one-to-many relationship in the DB, so the functionality could be extended later.

My main concern with token authentication is timing attacks. We had some discussion about it here #628. Since almost all application that I use and issue API keys don't use an email or login in combination with the token I am not sure how real timing attacks really are. We had some discussion about it here #628. Since almost all application that I use and issue API keys don't use an email or login in combination with the token I am not sure how real timing attacks really are.

Regarding this, I agree with @cantino here:

[...] If someone has hacked your Huginn and stolen your API keys, hasn't all of your data been compromised anyway? Given that, I'm not sure there's any reason to store only the hash instead of the full key, since it's not like you'll be reusing it on other sites, like you might be with a password.

It should be fine to simply store the API token in the DB as-is and not hash it, in our case (thus a timing attack isn't an issue). Hashing and salting are only really needed to protect passwords from dictionary attacks in case of leakage. In Huginn, if you DB was leaked, you'd simply reset all API tokens, and the old ones would be useless and meaningless to an attacker.

I don't feel like maintaining two APIs for Huginn, if we start to document the existing API we would be sort of bound to our "no breaking changes" promise. That could get annoying, but if we get to a point which would create a breaking change we could start actually versioning the API then.

Fair enough, it does entail some overhead. Just using the standard Rails controllers' JSON responses could be fine for a start. Until someone complains, and in that case, they could do the work of creating a separate API :P

[dsander]

So I think a good start could be to have just one by default and display that, but at least keep it in a one-to-many relationship in the DB, so the functionality could be extended later.

Sounds good to me, I initially thought we could use a devise feature when we did not put the keys in a separate table, but they removed that some time ago (more on it later).

It should be fine to simply store the API token in the DB as-is and not hash it, in our case (thus a timing attack isn't an issue). Hashing and salting are only really needed to protect passwords from dictionary attacks in case of leakage. In Huginn, if you DB was leaked, you'd simply reset all API tokens, and the old ones would be useless and meaningless to an attacker.

Right, that is correct. My concern isn't about hashing but the timing attack a SELECT * from api_keys WHERE key == 'somekey' (in theory) enables. As far as I know there aren't any secure_compare alternatives in SQL.

Devise removed their Token Authenticatable module acknowledging that the previous solution was vulnerable to timing attacks and they list alternatives to the removed feature, check the second example which should be safe.

But since the post is super old, we talked about the possible issues almost 5 years ago and major cloud providers still use the "classic" Bearer auth method it's probably safe enough for Huginn as well.