feat: produce agentic chat bundle from repo

[awschristou]

This change allows us to produce agentic chat artifacts directly from the repo. A new GitHub Action has been created that will build these artifacts for every commit made to the main and feature/* branches. The artifacts will be available (for a limited time) from the GitHub commit check.

These changes were developed on an Ubuntu WSL image. There are bash scripts involved, which will not run on Windows. The intention is that these are run from a linux based CI compute.

To support this change:

• a new npm script command ci:generate:agent-standalone has been created in the workspace app/aws-lsp-codewhisperer-runtimes. This is the command that CI compute will run to produce the artifacts.
• native binary dependencies (ripgrep, workspace indexer) have been directly added to the repo using git LFS. While not ideal, this gives us a faster path to quicker build iteration. In the future we can relocate these away from the repo.
• most of the scripting has been pulled from the internal repo DEXPRuntimeServerBuildConfiguration, and is mostly unchanged. This way our resulting artifacts are identical to what is currently produced internally. It also makes it easier to review these scripts, comparing them to the internal versions. Optimizations can be made to the scripts over time if desired. Here is where each script came from:
  • download-node.sh: update-node-assets.sh
  • package.sh: package.sh
  • copy-node-assets.ts: copy-node-assets.ts
  • webpack.config.prod.js: webpack.config.js

Upcoming changes will make use of this one in order to make agentic chat builds easier to obtain.

License

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[leigaol]

We do need Windows arm64. See https://aws-toolkit-language-servers.amazonaws.com/qAgenticChatServer/0/manifest.json

[awschristou]

We're currently copying the win-x86 builds to win-arm late in the distribution pipeline. We should introduce support for this, but that should be a separate task on the backlog. My current goal is to port the existing behaviors into the repo, which can then be iterated on.

[leigaol]

What is the size limit of Github LFS? Can it suppport 10GB?

At development time, do we need to pull these?

[awschristou]

The largest file is 2gb, but I don't know the cumulative space we're allowed to take up. Ideally we would try to reduce our reliance on LFS over time, but for now the goal is the ability to quickly produce builds.

We do not need these at development time, the intent is that the changes I'm adding are only used by CI jobs.

[leigaol]

since we are using git lfs anyway, maybe we can also check in node binary?

[awschristou]

I wanted to avoid putting too much data in LFS, but then ended up adding the qserver files (500 mb). Unless there is a major concern, I'd prefer to download these for now, since our ideal state is to avoid using LFS.

[leigaol]

may need another script to automatically populate https://aws-language-servers.us-east-1.amazonaws.com/e37e261e-d046-460a-81ab-619b3c60f94a/THIRD_PARTY_LICENSES in https://aws-toolkit-language-servers.amazonaws.com/qAgenticChatServer/0/manifest.json

[awschristou]

Yes - I have a separate task to track that

[leigaol]

Maybe merge on Monday just in case this PR failed to build inside the internal pipeline