Skip to content

twister help/document revisit #38140

Unanswered
hakehuang asked this question in Ideas
twister help/document revisit #38140
Aug 30, 2021 · 2 answers · 9 replies

@hakehuang
hakehuang Aug 30, 2021
Collaborator

twister is getting larger and larger, can we start to do tidy the twister document and help command line:

  1. all the supported features shall be listed in help
  2. all paramenter shall have an example
  3. all parameter combination and dependency shall be well documented
  4. for given command lint all parameters shall be print out and easy to be searched(grep)
  5. need find a way that parameter is organized in one place. e.g the way with zephyr\boards\arc\nsim\support

Replies

2 suggested answers
·
9 replies

It's also good to have a simple way to:

  1. Print default state of all options
  2. Dump entire set of options with its values being effectively used including those overridden from default with user's settings.

As a good example might be GCC printing its built-in defines taking into account user-set options:

64-bit system

$ gcc -dM -E - < /dev/null | grep LP64
#define __LP64__ 1
#define _LP64 1

32-bit system (note -m32 added)

$ gcc -dM -E -m32 - < /dev/null | grep LP64

3 replies
@hakehuang

hakehuang Sep 6, 2021
Collaborator Author

it is to provide a clear list on all what are set for give commands.
take zephyr\boards\arc\nsim\support as examples

@wangnuannuan

https://github.com/zephyrproject-rtos/zephyr/blob/main/scripts/pylib/twister/twisterlib.py#L2145 here the twister reads the CMakeCache.txt, we can extract variables from it

@hakehuang

hakehuang Sep 20, 2021
Collaborator Author

and at the end of parse_arguments, call print(vars(parser)), then we can get all option settings for twister

this is too meta, if you find issues with documentation please file a specific bug or submit a PR to fix it, if you have new ideas for how things can be done better, same thing. The tool itself documents all available options with --help, there is also lots of docs in doc/guides/test/twister.rst.

6 replies
@hakehuang

hakehuang Sep 17, 2021
Collaborator Author

"-B", "--subset",
help="Only run a subset of the tests, 1/4 for running the first 25%%, "
"3/5 means run the 3rd fifth of the total. "
"This option is useful when running a large number of tests on "
"different hosts to speed up execution time.")

this option can only allow first 1/4 case to run, then how to run the rest 3/4? this option is usful when we have multiply boards to run.

Then we can use -B 1/4 ; -B 2/4; -B 3/4; -B 4/4 run them on different boards or buildkite agents task

@hakehuang

hakehuang Sep 17, 2021
Collaborator Author

--unrecognized-section-test

what is this options for? detect unrecognized binary sections?

@hakehuang

hakehuang Sep 17, 2021
Collaborator Author

parser.add_argument(
"--filter", choices=['buildable', 'runnable'],
default='buildable',
help="""Filter tests to be built and executed. By default everything is
built and if a test is runnable (emulation or a connected device), it
is run. This option allows for example to only build tests that can
actually be run. Runnable is a subset of buildable.""")

is this option in use? if set to runnable, seems all cases will run

@PerMac

Just to clarify "--subset": It is heavily used in CIs, where execution can be parallelized between multiple independent agents. The idea is to divide a list of tests that will be executed into X parts and each of them is executed by calling twister --subset i/X X times, for i in range(X). If you first run twister --subset 1/4 to get the rest it would require calling the remaining 3 parts with 3 commands. I don't see a point to script an option for splitting the execution into uneven parts (like 25% first, and 75% later). I think this option doesn't help with "usful when we have multiply boards to run."

@PerMac

Runnable will filter out all tests which cannot be executed, e.g build-only tests. The other one (buildable) looks like being useless as it equals to not using any filter, all tests are buildable. It is just a false dichotomy, also in the code. In fact, runnable is a subset of buildable. These are not disjunctive groups. I believe the "buildable" can be removed from the code with very little effort.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
💡
Ideas
Labels
None yet
5 participants