Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support user interaction in ci_testing #13512

Open
BD103 opened this issue May 26, 2024 · 1 comment
Open

Support user interaction in ci_testing #13512

BD103 opened this issue May 26, 2024 · 1 comment
Labels
A-Build-System Related to build systems or continuous integration A-Input Player input via keyboard, mouse, gamepad, and more C-Enhancement A new feature C-Testing A change that impacts how we test Bevy or how users test their apps D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes X-Uncontroversial This work is generally agreed upon

Comments

@BD103
Copy link
Member

BD103 commented May 26, 2024

What problem does this solve or what need does it fill?

We test examples using Github Actions, but interacting with these examples while they are running is hard. Github Action runners don't have a keyboard and mouse, nor an actual monitor. We want to test the majority of examples, but are limited to examples that do not require any sort of user interaction.

What solution would you like?

We use bevy_dev_tools's ci_testing module to specify automated actions that should be run, such as to take a screenshot at a specific frame. These actions are specified by the CiTestingEvent enum:

#[derive(Deserialize, Debug)]
enum CiTestingEvent {
Screenshot,
AppExit,
Custom(String),
}

As brought up by @mockersf in a recent presentation, this enum is lacking additional features that would enable greater interaction. Some ideas for additional features, sorted by usefulness, are:

  1. KeyboardInput
  2. MouseButtonInput
  3. GamepadEvent
  4. TouchInput

I propose KeyboardInput is added to CiTestingEvent so that we can test keyboard interactions in CI. If that is successful, we can re-evaluate whether the other 3 are useful and worth implementing.

What alternative(s) have you considered?

I think that specifying actions using ci_testing is the right direction, thought the list of supported actions is currently very weak. By not implementing this feature, we leave the testing framework underpowered. Testing is one of the best ways to ensure quality, which Rust and Bevy by extension have a reputation for. This is why I believe not implementing user inputs is not an option, though the method used to achieve may vary.

The reason I propose to only support keyboards in the very beginning is because they are very easy to implement, and likely the most useful. The mouse is difficult without moving it, which works on offsets. The gamepad and touch are more obscure are difficult to verify locally without the proper hardware.

@BD103 BD103 added C-Enhancement A new feature S-Needs-Triage This issue needs to be labelled A-Build-System Related to build systems or continuous integration A-Input Player input via keyboard, mouse, gamepad, and more X-Uncontroversial This work is generally agreed upon D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes and removed S-Needs-Triage This issue needs to be labelled labels May 26, 2024
@BD103 BD103 added the C-Testing A change that impacts how we test Bevy or how users test their apps label May 26, 2024
@BD103
Copy link
Member Author

BD103 commented May 26, 2024

A good example of where this could be used is by pressing D in #13507 to automatically add 100 samples. The example is random, but from a deterministic seed.

github-merge-queue bot pushed a commit that referenced this issue May 26, 2024
# Objective

- We use
[`ci_testing`](https://dev-docs.bevyengine.org/bevy/dev_tools/ci_testing/index.html)
to specify per-example configuration on when to take a screenshot, when
to exit, etc.
- In the future more features may be added, such as #13512. To support
this growth, `ci_testing` should be easier to read and maintain.

## Solution

- Convert `ci_testing.rs` into the folder `ci_testing`, splitting the
configuration and systems into `ci_testing/config.rs` and
`ci_testing/systems.rs`.
- Convert `setup_app` into the plugin `CiTestingPlugin`. This new plugin
is added to both `DefaultPlugins` and `MinimalPlugins`.
- Remove `DevToolsPlugin` from `MinimalPlugins`, since it was only used
for CI testing.
- Clean up some code, add many comments, and add a few unit tests.

## Testing

The most important part is that this still passes all of the CI
validation checks (merge queue), since that is when it will be used the
most. I don't think I changed any behavior, so it should operate the
same.

You can also test it locally using:

```shell
# Run the breakout example, enabling `bevy_ci_testing` and loading the configuration used in CI.
CI_TESTING_CONFIG=".github/example-run/breakout.ron" cargo r --example breakout -F bevy_ci_testing
```

---

## Changelog

- Added `CiTestingPlugin`, which is split off from `DevToolsPlugin`.
- Removed `DevToolsPlugin` from `MinimalPlugins`.

## Migration Guide

Hi maintainers! I believe `DevToolsPlugin` was added within the same
release as this PR, so I don't think a migration guide is needed.

`DevToolsPlugin` is no longer included in `MinimalPlugins`, so you will
need to remove it manually.

```rust
// Before
App::new()
    .add_plugins(MinimalPlugins)
    .run();

// After
App::new()
    .add_plugins(MinimalPlugins)
    .add_plugins(DevToolsPlugin)
    .run();
```

---------

Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
Co-authored-by: François Mockers <francois.mockers@vleue.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Build-System Related to build systems or continuous integration A-Input Player input via keyboard, mouse, gamepad, and more C-Enhancement A new feature C-Testing A change that impacts how we test Bevy or how users test their apps D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes X-Uncontroversial This work is generally agreed upon
Projects
None yet
Development

No branches or pull requests

1 participant