Problems with packaging and distribution of devcontainer as standalone executable

[alexanderilyin]

I'm trying to build and distribute devcontainer as standalone executable. Currently I'm on:

$ git log -1
commit 413c5f790f3a4972bf6f05b71d71ac946c87bc7c (HEAD -> main, origin/main, origin/HEAD)
Merge: ac349ef 4cb96b7
Author: Samruddhi Khandale <skhandale@microsoft.com>
Date:   Mon Jul 31 08:42:00 2023 -0700

    Merge pull request #570 from yokonao/add-buildx-cache-to
    
    Add `--cache-to` option to `devcontainer build` command

I'm packaging it with:

pkg devcontainer.js

Which generates couple warnings:

> pkg@5.8.1
> Targets not specified. Assuming:
  node18-linux-x64, node18-macos-x64, node18-win-x64
> Warning Cannot resolve 'config.extends'
  $PWD/cli/dist/spec-node/devContainersSpecCLI.js
  Dynamic require may fail at run time, because the requested file
  is unknown at compilation time and not included into executable.
  Use a string literal as an argument for 'require', or leave it
  as is and specify the resolved file name in 'scripts' option.
> Warning Cannot resolve 'path32'
  $PWD/cli/dist/spec-node/devContainersSpecCLI.js
  Dynamic require may fail at run time, because the requested file
  is unknown at compilation time and not included into executable.
  Use a string literal as an argument for 'require', or leave it
  as is and specify the resolved file name in 'scripts' option.

But it does produce executable:

$ file devcontainer-linux
devcontainer-linux: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=e8e9fc449943f935d469333b1101f224a4c0164f, for GNU/Linux 2.6.32, stripped

It seems to work locally:

$ ./devcontainer-linux --version
0.50.2

But once I deploy it to server where I want to actually use it:

+ /opt/devcontainer-linux --workspace-folder . build
+ jq .
23:51:08  @devcontainers/cli 0.50.2. Node.js v18.5.0. linux 5.15.0-71-generic x64.

It fails:

Error: File or directory '/**/cli/dist/node_modules/vscode-dev-containers/container-features' was not included into executable at compilation stage. Please recompile adding it as asset or script.

I tried the following patch, but it did not help:

diff --git a/package.json b/package.json
index 1275861..e317b78 100644
--- a/package.json
+++ b/package.json
@@ -109,5 +109,13 @@
                "vscode-dev-containers": "https://github.com/microsoft/vscode-dev-containers/releases/download/v0.245.2/vscode-dev-containers-0.245.2.tgz",
                "vscode-uri": "^3.0.7",
                "yargs": "~17.7.1"
+       },
+       "pkg": {
+               "scripts": [
+                       "dist/**/*.js"
+               ],
+               "assets": [
+                       "dist/node_modules/vscode-dev-containers/container-features/**"
+               ]
        }
 }

Any suggestions are welcome.

[alexanderilyin]

For more context here is how I try to run it:

• Add documentation on container/image clean-up for self-hosted runners ci#207 (comment)

[alexanderilyin]

Well, not sure what of those two was but now I works.

diff --git a/package.json b/package.json
index 1275861..d01b036 100644
--- a/package.json
+++ b/package.json
@@ -109,5 +109,11 @@
                "vscode-dev-containers": "https://github.com/microsoft/vscode-dev-containers/releases/download/v0.245.2/vscode-dev-containers-0.245.2.tgz",
                "vscode-uri": "^3.0.7",
                "yargs": "~17.7.1"
+       },
+       "pkg": {
+               "assets": [
+                       "node_modules/vscode-dev-containers/package.json",
+                       "dist/node_modules/vscode-dev-containers/container-features"
+               ]
        }
 }

-pkg devcontainer.js
+pkg .

[alexanderilyin]

Well… not really:

Error: File '/**/cli/scripts/updateUID.Dockerfile' was not included into executable at compilation stage. Please recompile adding it as asset or script.

[alexanderilyin]

Okay 2nd one was easy:

01:03:37  Container started
+ /opt/devcontainer exec --workspace-folder . poetry --version
Poetry (version 1.5.1)

[yuhuyoyo]

i'm trying the same thing, but seeing slightly different errors.

i got these warning when i run pkg .

> pkg@5.8.1
> Warning Cannot resolve 'config.extends'
  /$PWD/tmp/cli/dist/spec-node/devContainersSpecCLI.js
  Dynamic require may fail at run time, because the requested file
  is unknown at compilation time and not included into executable.
  Use a string literal as an argument for 'require', or leave it
  as is and specify the resolved file name in 'scripts' option.
> Warning Cannot resolve 'path32'
  /$PWD/tmp/cli/dist/spec-node/devContainersSpecCLI.js
  Dynamic require may fail at run time, because the requested file
  is unknown at compilation time and not included into executable.
  Use a string literal as an argument for 'require', or leave it
  as is and specify the resolved file name in 'scripts' option.
> Warning Failed to make bytecode node16-x64 for file /snapshot/cli/node_modules/chalk/source/index.js
> Warning Failed to make bytecode node16-x64 for file /snapshot/cli/node_modules/chalk/source/utilities.js
prebuild-install warn install No prebuilt binaries found (target=v16.16.0 runtime=node arch=x64 libc= platform=linux)

the container did start successfully but I'm getting inner error

Container started
innerError Error: Cannot find module '../build/Debug/pty.node'
Require stack:
- /snapshot/cli/node_modules/node-pty/lib/unixTerminal.js
- /snapshot/cli/node_modules/node-pty/lib/index.js
- /snapshot/cli/dist/spec-node/devContainersSpecCLI.js
- /snapshot/cli/devcontainer.js
1) If you want to compile the package/file into executable, please pay attention to compilation warnings and specify a literal in 'require' call. 2) If you don't want to compile the package/file into executable and want to 'require' it from filesystem (likely plugin), specify an absolute path in 'require' call using process.cwd() or process.execPath.
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function._resolveFilename (pkg/prelude/bootstrap.js:1951:46)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at Module.require (pkg/prelude/bootstrap.js:1851:31)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/snapshot/cli/node_modules/node-pty/lib/unixTerminal.js:30:15)
    at Module._compile (pkg/prelude/bootstrap.js:1926:22)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/snapshot/cli/node_modules/node-pty/lib/unixTerminal.js',
    '/snapshot/cli/node_modules/node-pty/lib/index.js',
    '/snapshot/cli/dist/spec-node/devContainersSpecCLI.js',
    '/snapshot/cli/devcontainer.js'
  ],
  pkg: true
}

[chrmarti]

Have you tried creating a separate package.json that depends on the CLI package and packages that? That might simplify things as you don't need to worry about includes/excludes and can just include everything from the CLI package. That will also avoid the node-pty dependency.

[timnolte]

It sure would be nice if there was a standalone devcontainer CLI officially available to install. I would much rather not require Devs that want to contribute to my projects to not have to install NodeJS in order to development and testing locally. My goal it to provide a consistent development environment for all while also including the ability to use things like GitHub Codespaces. I don't want to require Devs to use any particular IDE. Personally I use Neovim on Linux(Chromebook) and so while I wan't to support the use of VSCode I don't want it to be a requirement.

[chrmarti]

The Dev Container CLI does not depend on VS Code and can be run standalone. Since we removed the dependency on node-pty it is now much easier to install the package, see e.g. microsoft/vscode-remote-release#8698 (comment) for an idea for a script for doing that. There is also a PR in progress for bundling the CLI as a single executable: #343.