Testing Codespace definition in devcontainer.json

[sobolevn]

Hi!

I am wondering if it is possible to test the codespaces definition.
Currently we are trying every new change manually, which is not perfect.

Desired workflow:

• We define base image, postCreateCommand, other settings like PATH, etc in devcontainer.json
• We push the definition to GitHub
• Some specific GitHub Action stars to run our tests inside the Codespace

Is it supported / planned? Thanks!

Related PR: HypothesisWorks/hypothesis#2880

[martinwoodward]

Feels like what you would like is an action that builds your container using your devcontainer.json exactly how it would if running a codespace and then use a docker action to run whatever you want to run inside that container and report the results - in your case report on the status of a unit test suite run?

What would actually be cool is to generate your exact codespace container image as part of your builds and then for your CI builds (not PR builds) actually store this in GitHub Container Registry so that when you come to do a 'Run in Codespace' it can just pull a pre-built container rather than having to do any post-build steps. As you want to generate that container anyway to run your tests in it, pretty efficient.

At the moment you could get part of the way there by building the container inside your CD action and dropping it into GHCR which then allows you to get rid of the postcreate steps in your devcontainer.json. But I can def see how an action that you can point at an existing devcontainer and get a docker container out would be even better.

[sobolevn]

Thanks! I have never looked at this problem from this perspective 👍
Indeed working with containers feels way easier than trying to test the Codespace service.

Should this be added to the docs? 🤔

[martinwoodward]

if you come up with a pattern that works then definitely. Think we need to test with a couple of real life examples first to make sure it's the best practise approach and work out any gotcha's before we recommend too broadly.

[cdijkstra]

Are there any updates on how to test a devcontainer.json. I would like to verify that the extensions are working, the default shell setting has been processed et cetera. I'm reading on ways to build the docker file in a pipeline and running tests there, but all custom settings from the devcontainer.json are not taken into account in that case...

[2percentsilk]

@cdijkstra if you are iterating on your devcontainer.json within a codespace, you can always use the Rebuild command to see how your changes are applied and manually test the effects of the devcontainer.json/Dockerfiles. You can read more here: https://docs.github.com/en/codespaces/customizing-your-codespace/configuring-codespaces-for-your-project#applying-changes-to-your-configuration

Does that help or are you looking for a way to automate the testing of the configuration?