Agent: Bundle WASM artifacts

[philipp-spiess]

Closes #3302

This PR ensures that cody agent always has access to the WASM artifacts. While building this, we found out that it is necessary to bundle the wasm inside the binary:

----stderr----
Aborted(Error: File '/**/dist/tree-sitter.wasm' was not included into executable at compilation stage. Please recompile adding it as asset or script.)
--------------
----stderr----
failed to asynchronously prepare wasm: RuntimeError: Aborted(Error: File '/**/dist/tree-sitter.wasm' was not included into executable at compilation stage. Please recompile adding it as asset or script.). Build with -sASSERTIONS for more info.
Aborted(RuntimeError: Aborted(Error: File '/**/dist/tree-sitter.wasm' was not included into executable at compilation stage. Please recompile adding it as asset or script.). Build with -sASSERTIONS for more info.)
--------------
----stderr----
Uncaught exception: RuntimeError: Aborted(RuntimeError: Aborted(Error: File '/**/dist/tree-sitter.wasm' was not included into executable at compilation stage. Please recompile adding it as asset or script.). Build with -sASSERTIONS for more info.). Build with -sASSERTIONS for more info.
    at abort (/snapshot/dist/agent.js)
    at /snapshot/dist/agent.js
--------------

Since we have more than one agent build, this would result in a huge increase in bundle size. We offset that by adding gzip compression. I tried brotli but due to the nature of not having centralized builds yet, it's not worth the additional build time.

Here are the final bundle size changes (for the aarch build):

• Currently the agent binaries are ~82mb
• Without compression they would grow to almost ~146mb
• With compression (gzip) ~64mb

Test plan

I added a new script that we can use to play around with the binary under agent/scripts/test-agent-binary.ts. You can use it to see that WASM works:

• cd agent/
• pnpm run build-agent-binaries
• pnpm run test-agent-binary

This will output something like this. Notice the addition of the intent which requires wasm to load 😎

[philipp-spiess]

This rename is necessary so the build artifacts are still called agent- and not cody-agent-. We don't use that name anywhere though so it should be fine.

[philipp-spiess]

We have to use to this pkg syntax and define the wasm artifact here so it'll be inlined properly.

[pkukielka]

So that test will only work on windows? Can we pickup proper binary depending on the platform?

[philipp-spiess]

Good point, I'll copy paste the right version selector here if we ever need to debug on win envs.

[pkukielka]

LGTM