test: fix apitest, add new unit tests and benchmarks, normalize output#1467
test: fix apitest, add new unit tests and benchmarks, normalize output#1467KernelDeimos merged 1 commit intoHeyPuter:mainfrom
Conversation
XiaochenCui
force-pushed
the
test/apitest
branch
from
39b1e93 to
be71aed
Compare
KernelDeimos
left a comment
KernelDeimos
left a comment
There was a problem hiding this comment.
Looks good, only a couple documentation-related comments - will merge once those are addressed.
| ' --config=<path> (required) Path to configuration file\n' + | ||
| ' --report=<path> (optional) Output file for full test results\n' + | ||
| ' --suite=<name> (optional) Run only tests with matching suite name\n' + | ||
| ' --stop-on-failure (optional) Stop execution on first test failure\n' + |
There was a problem hiding this comment.
missing --onlycase here - what does it do?
There was a problem hiding this comment.
--onlycase doesn't work, I propose to remove this arg.
A bigger problem is that the current implementation doesn't support run a specific case - since case can be nested and we don't know a case's children before running it. e.g:
// we cannot skip 'test_a' when filter=test_b, since test_b might be a child of 'test_a'
await t.case('test_a', async () => {
await t.case('test_b', async () => {
...
});a good news is --suite now supported:
-
Filter tests by suite name:
node ./tools/api-tester/apitest.js --config=./tools/api-tester/config.yml --unit --suite=mkdir
-
Filter benchmarks by name:
node ./tools/api-tester/apitest.js --config=./tools/api-tester/config.yml --bench --suite=stat_intensive_1
| // TODO (xiaochen): `t.mkdir('a/b/c')` throws 409/422, unify the | ||
| // behavior of these two cases. | ||
| expect(e.response.status).equal(422); | ||
| expect(e.response.status).oneOf([409, 422]); |
There was a problem hiding this comment.
Is this error code dependent on the filesystem implementation? (whether it's memoryfs or puterfs)
There was a problem hiding this comment.
The error code depends on the state left by previous tests. I left a TODO here to (plan to) make it more deterministic and meaningful.
2 participants
this PR updates the apitest: