[example tests] generate "set -e" to exit scripts upon failure

[savil]

Summary

RFC: we should have failures in the devbox-scripts reflect in a non-zero exit status
from devbox run <script>.

There is some risk that customers whose code was previously running will now fail. However, their scripts were indeed failing and its just that this failure will now be exposed.

Update: Based on John's feedback in a comment below I am adding a featuregate that is activated in examples tests. We'll make a plan for rolling this out to users, in a way that is not a big surprise for them.

TODO:

• fix examples which have failures that were previously masked. Done in previous PRs in the stack.

How was it tested?

Removed curl from lepp-stack packages, and ran:

DEVBOX_EXAMPLE_TESTS=1 DEVBOX_DEBUG=0 go test -count 1 -v -run TestExamples/stacks_lepp ./testscripts/...

This failed, demonstrating that the set -e was working as intended.

I then added the curl back and the above command passed.

[savil]

Current dependencies on/for this PR:

• main
  • PR set HOME for testscripts #875
    • PR [devbox] Make stack-examples using Postgres actually work #874
      • PR [examples] enable stacks-django and rust examples to pass #880
        • PR [example tests] generate "set -e" to exit scripts upon failure #869 👈

This comment was auto-generated by Graphite.

[Lagoja]

Is this done only for testscripts? or for all devbox scripts?

[savil]

@Lagoja this is for all users. Thoughts?

[Lagoja]

Was thinking about this a bit -- since -e makes a script exit immediately upon an error, I worry that this could cause a big wave of unexpected errors if it's automatically pushed to users.

I think we'd need to either make this something configurable, or do it in a roll out where it's explicitly described and developers have a chance to adapt.

I do think forcing it on our examples and testscripts is a good idea -- we should hold those to a really high standard

[savil]

Sounds good. Let me hold off on this then.

[savil]

I'll revive this and gate it for example tests