[doc] Add a sw/device/tests markdown #800
Conversation
|
adding some description to the software tests per @martin-lueker request. I'm re-using @sriyerg testplanner stuff, since it seemed useful. Let me know if it's not quite the right place to use and should be taken out. Also, it seems a little weird to me the testplan stuff is hardcoded into |
| @@ -0,0 +1,32 @@ | |||
| { | |||
There was a problem hiding this comment.
Ok to keep this here for now, however I am intending for this to be moved to hw/top_earlgrey/data area.
There was a problem hiding this comment.
o i can move it there. Not a problem. Need a better name though... if you're okay with the conetent, how about we call these standalone_sw tests?
util/build_docs.py
Outdated
| @@ -72,6 +72,7 @@ | |||
| "hw/ip/i2c/data/i2c_testplan.hjson", | |||
| "hw/ip/rv_timer/data/rv_timer_testplan.hjson", | |||
| "hw/ip/uart/data/uart_testplan.hjson", | |||
| "sw/device/tests/self_contained_testplan.hjson", | |||
There was a problem hiding this comment.
I didn't realize these need to be manually added to this script. Is this going to be fixed in future?
@gkelly
There was a problem hiding this comment.
would it help to auto-gather these files if we long term renamed them to hjson.testplan?
sw/device/tests/index.md
Outdated
| These platforms include - UVM DV, Verilator DV, FPGA, and eventually silicon. | ||
|
|
||
| To enable this cross-platform testing, each test is self-contained and do not require additional host or backend resources to verify correct behavior. | ||
| This suggests that any testing that would involve emulation of host capability, such as an external spi host or an external host to encrypt/decrypt data, is not appropriate for these types of self-contained tests. |
sw/device/tests/index.md
Outdated
| ## Self Checking Mechanism | ||
|
|
||
| Currently, the self-checking mechanism is accomplished through a console message. | ||
| When the tests pass, it will output 'Pass!'. |
There was a problem hiding this comment.
Actually, 'PASS!\r\n' is the magic string. You'll also want to wrap it in backticks so it renders as code.
|
Thanks for putting this together @tjaychen! One question: Does the described behavior accurately reflect the current test behavior, or is this planned behavior? When I loaded these tests into SPI the other day, I did not see a "PASS!" message on the UART. (Though, I certainly wouldn't rule out PEBKAC, in this case yet.) This is is pretty good confirmation of what I should be expecting in these tests. Also, should there a convention for broadcasting a message even if the test does not pass? Maybe an announcement naming the test. |
There was a problem hiding this comment.
Thanks for putting this together @tjaychen! One question: Does the described behavior accurately reflect the current test behavior, or is this planned behavior? When I loaded these tests into SPI the other day, I did not see a "PASS!" message on the UART. (Though, I certainly wouldn't rule out PEBKAC, in this case yet.) This is is pretty good confirmation of what I should be expecting in these tests. Also, should there a convention for broadcasting a message even if the test does not pass? Maybe an announcement naming the test.
(I know this is a documentation PR, so maybe this should be raised as a broader issue elsewhere, though regardless I think this test-launch-announcement could be useful.)
yes you should see PASS! on all 3 of these tests right now. IF you don't, that either means the binary didn't run..or something else.
Can you actually file an issue with what you are seeing? Then we can take it from there.
|
o sorry i forgot to answer your other question. But this only works if you invoke the tests with the |
|
@gkelly |
sw/device/tests/index.md
Outdated
| The capturing and generation of this message will differ per testing target. | ||
| On FPGA / silicon, they will feed directly to a host machine through a physical UART conection, where the host can decipher the message correctness. | ||
| On Verilator, they will feed to a virtual UART terminal where the host can do the same. | ||
| On DV, they will feed to a fixed memory location where the DV backend can efficiently pickup the message without significant parsing overhead. |
No description provided.