Collection triggers

[ganigeorgiev]

This was previously discussed in #34, #245 and several other places.
Another user (emilydev) also mailed me with very helpful and detailed examples how other platforms tackle this.

Although I still think that for more advanced scenarios the available PocketBase event hooks should be preferred, not everyone is familiar with Go and this ended up being a blocker for a lot of users (the documentation currently is also bit lacking on "How to use as framework" but that will be improved in the near future).

Based on the user feedback, I think we can try to simplify things a little and introduce "Collection Triggers". This triggers will be fired after a successfull create/update action and initially we will support only 2 types of triggers:

• Webhook - trigger that will send a single http request to another url. The record data will be available as placeholders (eg. {field1}, {field2.subfield}, etc.) so that users will be able to use them to construct a request body. This trigger could be also used to maintain another collection (there will be an option to authorize as admin when configuring the http request).

• Mail - (inspired by this Firebase extension) trigger that will send a single email based on the created/updated record data. This trigger will have 3 main configurable fields - to, subject and body. The record data will be available as placeholders (eg. {field1}, {field2.subfield}, etc.), so that users will be able to use them as values in the mail fields.

If you have other ideas or suggestions, please share your feedback below. I plan to implement this feature for v0.7.0 or v0.8.0 release.

[Baoqi]

I'd like to suggest an integration with https://github.com/openfaas/of-watchdog/ project, which is a sub component for OpenFaaS. which is a simple program written in Go, but by supporting it, we can integrate a lot of different programming languages as "functions".

Seems supabase and nhost also support some kinds of "functions"

[ganigeorgiev]

@Baoqi Thanks for the suggestion.

I'm familiar with openfaas, but for now I'm not planning to add serverless function support to PocketBase because it will be hard to implement while still keeping everything simple and within a single executable.

PocketBase in its core is designed as a Go framework/toolkit and that is one of the main difference (and one of the biggest benefit in my opinion) compared to other similar solutions because this way you can execute any custom code without the constraints that usually comes with cloud functions (access to globals, what packages are you allowed to import, limited control over the request lifecycle, etc.) and still have a single binary at the end by just running go build.

While the goal of this feature request is to help developers not familiar with Go, it is not intended to be a full replacement for the cases when you really need to write custom business logic.

Eventual support in the future for a "Workflow builder" with various plugins/blocks, one of which could be a scripting interpreter like goja, may be added but for now this is out of the scope for v1.0.0.

[Sagor64]

Nice

[pchasco]

From my point of view, webhooks may be the best fit for PocketBase rather than embedding scripting or a workflow engine. If an application needs custom business logic on the server side, I think they would do better to simply host a separate service that presents a REST endpoint, and/or responds to PocketBase webhook events. One enhancement may be to support API key authentication for server-to-server connections to PocketBase to make this integration easier.

From my own experience maintaining an existing custom workflow engine: I would avoid if possible. Non-developers do not understand them, and developers prefer to write code rather than work within the constraints of the engine.

[pchasco]

Another idea I had: A response from a web hook could return a JSON structure that may contain instructions to update or modify the record, or possibly even a record on another collection. Care would need to be taken to prevent runaway event storms and/or cycles. A feature like this may cover many use cases when a web hook wants to interact with a third-party service and store some new data in PocketBase.

[osseonews]

Do you have any timetable for when the Collection triggers might be available? Thanks.

[ganigeorgiev]

@osseonews No, sorry. As mentioned in #123 (comment), I don't want to limit myself with hard deadlines or ETAs. Currently, the users management refactoring from #376 is with highest priority and I plan to make a v0.8 pre-release in the next 2-3 weeks, but once again - no hard deadlines.

If you haven't already, you could check PocketBase v1.0.0 roadmap and the current features priority here - https://github.com/orgs/pocketbase/projects/2.

[jkdoshi]

For anyone interested in this issue, you might want to checkout this project of mine - https://github.com/spinspire/pocketbase-sveltekit-starter/. It is a general purpose starter-kit with pocketbase backend and a static SveltKit frontend.

But the main thing that is relevant to the audience here is the support for "config-driven hooks" that support both command execution and webhook posts tied to specific events (insert/update/delete) on specific tables. Implementation details here

Discussion here and documenation here

[ktown007]

@ganigeorgiev, In addition to webhooks a run command would also be a good idea. It is very powerful that hooks can be compiled in for performance. But it makes me nervous that any time an external hook changes you could have the re-compile and deploy the server. Real time Webhooks and Command are much higher priority for integration use cases. In my view emails are lower in priority since they can always be polled via rest api.

It would be helpful for the hooks to have filter/rules to only fire when needed based on the values of the record.

[jkdoshi]

@ganigeorgiev some suggested features:

In addition to webhooks and mail ...

• Now that we have a JavaScript VM embedded into PocketBase, may be allow writing triggers in JS? In general, I think JS should be used to do most of the extending of PocketBase. This might cover most use-cases where we are extending via Golang and using PocketBase as a framework.
• Possibly allow execution of arbitrary programs (such as shell scripts) as triggers? This is probably same as the "run command" idea above by @ktown007. I know this can get out of hand. But if we run these things in a goroutine then it shouldn't block anything else.

Also, regarding webhook: Most webhooks will require some kind of auth. It would be interesting to see how you provide credentials to the code that posts to webhooks. Also, most of the times you might have to transform the JSON payload before POST'ing it to a webhook. This can be done with a jq-like expression language. But even here JavaScript might come handy.

[ktown007]

@jkdoshi It has been my experience that "server side logic" is often more than a transformation or template. I tired this is the past with XSLT. The webhook or command or socket adds the flexibility to build this on any server(less) or any tool/script language and be decoupled from db server. This type of trigger/hook before creation/update or after after creation/update is a powerful way to add secure server logic without having to place a full server between the client and the database. I agree the webhook will need configurable http headers in addition to json body, some outer joins in the body would reduce the need for hook to make additional db lookups.

[hungcrush]

@ganigeorgiev What do you think if we add Mail Template for each collection ? Do you have any Idea similar it? 😬

[ganigeorgiev]

@hungcrush You can check the discussion in #1547 (comment).

[ganigeorgiev]

Old issues cleanup: I'm closing the issue as there no longer plans to work on it since the JS pb_hooks cover this and other more complex use cases.

[c-nv-s]

please can an example be added to the docs to demonstrate a safe way to send a custom webhook which includes a private API key in the header e.g. Authorization: Bearer xxxx if this is covered by the existing capabilities.

maybe even include an example of a custom email if it is also possible.
it would be greatly appreciated.

[pchasco]

What do you mean by “safe?” If you use HTTPS the headers are encrypted.

…

On Tue, Mar 26, 2024 at 9:03 AM c-nv-s ***@***.***> wrote: please can an example be added to the docs to demonstrate a safe way to send a custom webhook which includes a private API key in the header e.g. Authorization: Bearer xxxx if this is covered by the existing capabilities. maybe even include an example of a custom email if it is also possible. it would be greatly appreciated. — Reply to this email directly, view it on GitHub <#317 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAUFIAGCKTPDJDAGGJ7GQJTY2FWZRAVCNFSM56RY3GE2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBSGA2TCNBWGQYA> . You are receiving this because you commented.Message ID: ***@***.***>

[ktown007]

I think he means a '.env' or secrets so the token is not hardcoded in js code.