Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for "filtering" unit tests using a string #3268

Closed
mitchmindtree opened this issue Nov 3, 2022 · 0 comments · Fixed by #4566
Closed

Add support for "filtering" unit tests using a string #3268

mitchmindtree opened this issue Nov 3, 2022 · 0 comments · Fixed by #4566
Assignees
@kayagokalp
Labels
forc forc-test Everything related to the `forc-test` lib and `forc test` command. good first issue Good for newcomers

Comments

@mitchmindtree
Copy link
Contributor

I.e. only run tests whose functions contain the given string (same as cargo).
@mitchmindtree mitchmindtree added forc forc-test Everything related to the `forc-test` lib and `forc test` command. labels Nov 3, 2022
@mitchmindtree mitchmindtree mentioned this issue Nov 3, 2022
Merged
8 tasks
mitchmindtree added a commit that referenced this issue Nov 4, 2022
@mitchmindtree
Sway unit test (i.e. #[test] fn) compilation (#2985)
5f29c66 
OK, this should be good to go!

## Highlights

- Adds support for multiple entry points to IR. Adds a tests for
serializing entry functions to/from IR.
- Enables ASM generation for libraries (to support test function entry
points).
- Track entry points through ASM generation so we can return entry point
metadata as a part of the compilation result. This doesn't affect ASM
generation, but allows tools using `sway-core` as a library to work with
different entry points.
- Updated E2E test harness with a new test category "UnitTestsPass".
- Added E2E tests with multiple unit tests for each type of Sway program
(library, script, predicate, contract).
- Gets `forc test` working with pretty output!

Here's the output from the new `lib_multi_test`:

![Screenshot from 2022-11-03
19-00-57](https://user-images.githubusercontent.com/4587373/199700362-ba32f90d-1f0f-4f76-bf89-b191de9589af.png)


## Test running implementation

Currently, it seems like there's no publicly accessible approach to
execute a script directly from a custom entry point. As a result, for
each test, we patch the bytecode with a `JI` instruction that jumps from
after the data section setup to the test's entry point. This is a bit
hairy, but works for now!

As a follow-up, we may want to consider adding support upstream in
`fuel-vm` for executing scripts directly from a given entry point. Even
if this was only exposed from the interpreter API, this would make the
`forc-test` implementation quite a bit cleaner and avoid the need to
patch the bytecode.

Alternatively, when building projects we could return more metadata
along with the compiled output (whether in memory or bytecode header) to
indicate how to work with different entry points in a more reliable
manner (rather than the magic const offset currently used in
`forc-test`).

# TODO

- [x] Add `include_tests` flag to `BuildConfig`, allowing `forc` to
trigger compilation of test functions.
- [x] Include `#[test]` fns as entry points within dead code analysis.
- [x] IR and ASM generation for test entry points.
- [x] Add `forc build --tests`.
- [x] Add `forc test`.
- [x] Always include test fns in `TyProgram` (for DCA), but omit from IR
if not building tests.
- [x] Work out how to iterate over different entry points during `forc
test`.
- [x] Only generate tests for top-level "members" (not all
dependencies).

### Follow-up:

- #3260
- #3261
- #3262
- #3263
- #3264
- #3265
- #3266
- #3267
- #3268

Closes #1832.
@mitchmindtree mitchmindtree added the good first issue Good for newcomers label Nov 4, 2022
@sdankel sdankel mentioned this issue Jan 21, 2023
Merged
@kayagokalp kayagokalp self-assigned this May 15, 2023
@kayagokalp kayagokalp mentioned this issue May 16, 2023
Merged
7 tasks
kayagokalp added a commit that referenced this issue May 24, 2023
@kayagokalp
feat: implement forc-test filter (#4566)
2f0ee52 
## Description
closes #3268.

This PR adds filtering capability to `forc-test` based on test names. Tests are executed if their name contains the given filter phrase.
IGI-111 pushed a commit that referenced this issue May 25, 2023
@kayagokalp @IGI-111
feat: implement forc-test filter (#4566)
3c46943 
## Description
closes #3268.

This PR adds filtering capability to `forc-test` based on test names. Tests are executed if their name contains the given filter phrase.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kayagokalp kayagokalp

Labels
forc forc-test Everything related to the `forc-test` lib and `forc test` command. good first issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: implement forc-test filter FuelLabs/sway
2 participants
@mitchmindtree @kayagokalp