Reduce payload formatter script size

[johanstokking]

Summary

Replaces https://github.com/TheThingsIndustries/lorawan-stack/issues/2675

Reduce the maximum script size for payload formats to 4 KB.

Why do we need this?

Because it takes too much memory otherwise.

We will allow larger scripts via the Device Repository. We have optimizations for that planned, see https://github.com/TheThingsIndustries/lorawan-stack/issues/2264.

What is already there? What do you see now?

No limits. We already see scripts larger than 200 KB.

What is missing? What do you want to see?

Limits

How do you propose to implement this?

This applies to user defined scripts on the application and device level. This does not affect the message processor parameter field (for now), as we still allow larger values from device repository.

Setting this to a new minor as this is a somewhat breaking behavioral API change for users. Some scripts are for example already stored, and they cannot be updated or changed, and new devices and applications cannot be assigned a script that existing devices or applications are using.

How do you propose to test this?

Local testing

Can you do this yourself and submit a Pull Request?

Can review

[descartes]

On v2 when the servers were under extreme pressure the JS decoder would time out so the HTTP Integration, MQTT & Data Storage only had the raw payload - didn't bother me, I use the JS decoder for debugging only - but it did tend to cause a stir for the TTN users.

It would be foolish to say that this could never happen on v3, and 4KB is more than plenty of code which will keep processing times down, but is it worth having a status field so that if decoding does time out or otherwise fail, that it's clear in the relayed JSON that decoded_payload is going to be empty?

[johanstokking]

@descartes good suggestion to include this somehow in the payload, potentially with the already existing errors field of decoded_payload.

We actually see V3 also already failing on large scripts. We measure the 100 ms execution timeout to include the evaluation time of the script, so with large scripts and CPU pressure, that fails just like in V2. Although we use a different JavaScript engine, this is a very similar design in V3.