Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracking] devcontainer features test improvements #139

Open
9 of 17 tasks
joshspicer opened this issue Aug 24, 2022 · 5 comments
Open
9 of 17 tasks

[Tracking] devcontainer features test improvements #139

joshspicer opened this issue Aug 24, 2022 · 5 comments
Labels
bug Something isn't working feature-request New feature or request

Comments

@joshspicer
Copy link
Member

joshspicer commented Aug 24, 2022

An issue collecting various improvements that can be made for the features test... command. The command is still experimental and under active development.

@eitsupi
Copy link

eitsupi commented Sep 24, 2022

Hello.
I developed a feature using the feature template and noticed that the test for option selection was not covered.

Is it possible to run multiple installations by referencing a pre-created options matrix (json file) when running an installation as a test?
Or the user may want to give options as CLI arguments to parallelize using the matrix on GitHub Actions.

Similarly, it would be useful to be able to specify the platform (CPU architecture).

edit: I understand that we can add scenarios.json to the folder for each feature, but it seems useless because when I set up the GitHub matrix, the tests in scenarios.json are run under all matrix conditions --skip-scenarios seems work fine.

@Clockwork-Muse
Copy link

Can we add another task? Currently, there isn't a way to supply --docker-path/--docker-compose-path to any of the features commands, which causes issues if you're using podman in ci scenarios.

@joshspicer
Copy link
Member Author

@Clockwork-Muse Do you mean a way to specify the path to the docker/docker-compose CLIs? Yes, I'll convert your request into an issue 👍 (If you're interested, we'd also accept a PR on this)

@Clockwork-Muse
Copy link

Yes, that's what I meant.

@jcbhmr
Copy link

jcbhmr commented Aug 17, 2023

If there is a scenario.json file and no test.sh, the test will not run unless --skip-autogenerated is set. We should be smarter about this

👍 id very much like to see this so that i can do this in something like gh actions:

devcontainer features test -f ${{ matrix.id }}

and NOT fail like this:
image

as a workaround I'm currently doing this:

if [[ -e test/${{ matrix.id }}/test.sh ]]; then
  devcontainer features test -f ${{ matrix.id }}
else
  devcontainer features test -f ${{ matrix.id }} --skip-autogenerated
fi

accidentally made devcontainers/spec#294 🙄

@joshspicer joshspicer removed their assignment Jul 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working feature-request New feature or request
Projects
None yet
Development

No branches or pull requests

5 participants