Using "features" with Bring Your Own Image

[metaskills]

Can "features" (https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/container-features/src/features.json) be used with bringing your own image? We have found that our custom CicleCI images make great development images. They are base on Ubuntu and the scripting seems to be compatible with VS Code's dev container features. For example, we wanted to use the "common" feature to make our CircleCI images feel more Codespaces'y. I ideally we could have done something like this.

"features": {
  "common": "latest"
}

But from the creation log, I saw a message that they would not be applied to the image. So we have had to resort to doing something like this in our Dockerfile.

# Install Codespaces "common" features.
RUN curl -s https://raw.githubusercontent.com/microsoft/vscode-dev-containers/main/script-library/common-debian.sh > /tmp/common-debian.sh \
    && bash /tmp/common-debian.sh true automatic automatic automatic false

Perhaps this is an opportunity for 3rd-party features? Tho I do think if the OS is compatible, many folks would love to use the "common" feature via JSON too. Maybe y'all publish it as a 3rd-party one too?

"features": {
  "@codespaces/common": "latest"
}

[Chuxel]

For example, we wanted to use the "common" feature to make our CircleCI images feel more Codespaces'y. I ideally we could have done something like this.

Some good news here - we did add common as a feature and should ship soon. You'll be able to do something like the following:

    "common": "latest"

Or for options:

    "common": {
        "username": "automatic",
        "uid": "automatic",
        "gid": "automatic",
        "installZsh": true,
        "installOhMyZsh": true,
        "upgradePackages": true,
        "nonFreePackages": false
    }

Like other features, it focuses on Debian/Ubuntu based images.

Perhaps this is an opportunity for 3rd-party features?

Yeah we are currently looking into this one as well. The goal is you'll be able to reference <org>/<repo>/<feature>@<version> in a similar way you can with GitHub Actions today. Versions would tie to GitHub releases with this syntax. We'll have a way you can reference a .tgz as well for other cases as well.

To give an example, the common script would technically be microsoft/vscode-dev-containers/common@0.206.0 current state (or @latest), and the same options would be available.

[metaskills]

we did add common as a feature and should ship soon

Awesome! Is the idea that when it "ships" that it would run on our Ubuntu images which were failing before? Just want to make sure that BYOI can use microsoft/vscode-dev-containers features?

We'll have a way you can reference a .tgz as well for other cases as well.... To give an example, the common script would technically be microsoft/vscode-dev-containers/common@0.206.0 current state (or @latest), and the same options would be available.

Awesome!!! Would the design of the system also allow for private 3rd-party features?

[Chuxel]

Awesome! Is the idea that when it "ships" that it would run on our Ubuntu images which were failing before? Just want to make sure that BYOI can use microsoft/vscode-dev-containers features?

Yeah, the idea is you'd be able to do this in devcontainer.json:

{
   "image": "your.registry.com/repository/image:tag",
   "features": {
       "common": true,
       // other features here
   }
}

You can do this now with docker-in-docker, etc and it works with a custom Dockerfile or even a Docker Compose file (it just applies the image updates to the referenced service in devcontainer.json). That said, images can have a lot of things in them, so the base features we provide may not always work in certain circumstances (at least without tweaking options), so the getting started UI drives you through adding features for definitions we've tested rather than anything. But, you can manually add it in json form to anything.

The dev container CLI then allows you to pre-build an image in CI to lock in versions, speed things up, and avoid issues due to something downstream being down (e.g. package repository hiccups outside of GitHub and Microsoft's control). You then reference pre-built image but leave the features in devcontainer.json to pick up extensions, etc. It will identify if the build has already been run and skip it.

Awesome!!! Would the design of the system also allow for private 3rd-party features?

Yeah, that is the goal, though not immediately - need to sort out the automated auth story here in a safe way, but the core mechanics are there since the release could be in a private GitHub repo.

[simkimsia]

I’m still a bit lost esp the docker in docker. Any tutorials?

I’m a GitHub team subscriber and looking at migrating my docker heavy setup for django local host env complete with Postgres to use codespaces

[Chuxel]

This should do the trick for a Debian or Ubuntu based image:

{
   "image": "your.registry.com/repository/image:tag",
   "features": {
       "common": "latest",
       "docker-in-docker": "latest"
   }
}

Or you can omit common if you don't want the stuff it adds.

More info on the docker-in-docker feature's options here: https://github.com/microsoft/vscode-dev-containers/blob/main/script-library/docs/docker-in-docker.md#feature-use

Info on referencing a Dockerfile or Docker Compose file instead of an image can be found at https://aka.ms/devcontainer.json

[shellerci]

Yeah, that is the goal, though not immediately - need to sort out the automated auth story here in a safe way, but the core mechanics are there since the release could be in a private GitHub repo.

@Chuxel , is there any word on when we might be able to start taking advantage of those mechanics? We're currently handling a bunch of our shared image configuration tasks by pulling in a private repo with a similar strategy to the one @metaskills referenced for pulling in the common-debian features in a postCreate hook :

git clone "https://${GITHUB_TOKEN}@github.com/orgname/org-setup.git" /workspaces/.org-setup
cd /workspaces/.org-setup
./install-org-common.sh

I was wondering if we could shim in our own feature from that repo by having the install-org-common.sh add an env variable along the lines of :

_VSC_INSTALL_OUR_FEATURE="/workspaces/.org-setup/features/our-feature.sh ${_BUILD_ARG_OUR_FEATURE_ARG:default}"

and then just dragging it in as desired in the devcontainer.json with :

features : {
  "our_feature": "default"
}

Ideally I'd like to implement a shim that's close enough to what the eventual 3rd party feature interface would be. That way we can switch over to the official implementation with a minimal amount of modification to our devcontainer.json files when the time comes.

[metaskills]

@Chuxel We have been following the work done here around the file naming updates:

• change features.json -> devcontainer-features.json microsoft/dev-container-features-template#1
• Add JSON schema for work-in-progress features.json microsoft/vscode-remote-release#5980

We hit one issue while testing moving some of our postCreate lifecycle scripts over to features. Namely, there is no access to either organizational or user secrets. Can you share how this is or will be handled? Our install.sh script does source this file but it appears to be empty.

set -a
. ./devcontainer-features.env
set +a

[Chuxel]

Ah interesting. Yeah there is a second docker build step that probably is not having those pulled into it. Thoughts @joshspicer @jkeech @2percentsilk ?

[joshspicer]

Great question/ask. We should have secrets available as environment variables at build-time - I think this might just require some extra plumbing in our tooling to get those to show up properly.

[joshspicer]

Looking into it!

[metaskills]

I think this might just require some extra plumbing in our tooling to get those to show up properly.

AMAZING! This would be the last blocker we have on migrating fully to features vs postCreate hooks.