Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refine config and docs for sytest #714

Merged
merged 11 commits into from Jun 21, 2019

Conversation

@Cnly
Copy link
Collaborator

commented May 30, 2019

This PR adds the "passed but expected fail" tests to testfile, as well as instructions on finding which tests to add for new PRs.

Currently, there are 4 tests still marked with # TODO passed but expected fail even if they've been added to testfile:

ok 15 (expected fail) POST /register rejects registration of usernames with '£' # TODO passed but expected fail
ok 16 (expected fail) POST /register rejects registration of usernames with 'é' # TODO passed but expected fail
ok 427 (expected fail) Should reject keys claiming to belong to a different user # TODO passed but expected fail
ok 698 (expected fail) Enabling an unknown default rule fails with 404 # TODO passed but expected fail

Pull Request Checklist

  • I have added any new tests that need to pass to testfile as specified in docs/sytest.md
  • Pull request includes a sign off
@anoadragon453

This comment has been minimized.

Copy link
Member

commented May 30, 2019

Thanks for doing this! Odd to hear those tests still being marked as expected fail. I'll look into this.

Add passed but expected fail tests to testfile
Signed-off-by: Alex Chen <minecnly@gmail.com>

@Cnly Cnly force-pushed the Cnly:refine-sytest-config-docs branch from 2e87e87 to 6ce865c May 30, 2019

Cnly added 3 commits Jun 3, 2019
Show tests to add to testfile in CircleCI
Signed-off-by: Alex Chen <minecnly@gmail.com>
Remove not ok test from testfile
Signed-off-by: Alex Chen <minecnly@gmail.com>
Fix path for show-expected-fail-tests.sh
Signed-off-by: Alex Chen <minecnly@gmail.com>
@Cnly

This comment has been minimized.

Copy link
Collaborator Author

commented Jun 4, 2019

Now this PR should cover #715.

@anoadragon453
Copy link
Member

left a comment

Looking great! Sorry have asked for a bit of a rewrite, but hopefully shouldn't be too bad.

docs/sytest.md Show resolved Hide resolved
show-expected-fail-tests.sh Show resolved Hide resolved
Better output format for show-expected-fail-tests.sh
Signed-off-by: Alex Chen <minecnly@gmail.com>
Cnly added 3 commits Jun 19, 2019
Refine test guides in docs/sytest.md
Signed-off-by: Alex Chen <minecnly@gmail.com>
Revert changes for .circleci/config.yml - included in sytest docker i…
…mage

Signed-off-by: Alex Chen <minecnly@gmail.com>
Update testfile
Signed-off-by: Alex Chen <minecnly@gmail.com>
anoadragon453 added a commit to matrix-org/sytest that referenced this pull request Jun 19, 2019
Show new tests to add to testfile for dendrite (#626)
This is considered part of matrix-org/dendrite#714. We automatically run it in the container so it'll be easier for people to run tests themselves.
@anoadragon453
Copy link
Member

left a comment

Nearly there. Just some small changes.

docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
@anoadragon453
Copy link
Member

left a comment

lgtm otherwise!

docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
docs/sytest.md Outdated Show resolved Hide resolved
Apply suggestions to sytest.md from code review
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

@anoadragon453 anoadragon453 merged commit adaec7c into matrix-org:master Jun 21, 2019

1 check passed

ci/circleci: dendrite Your tests passed on CircleCI!
Details

@Cnly Cnly deleted the Cnly:refine-sytest-config-docs branch Jun 22, 2019

anoadragon453 added a commit to serra-allgood/dendrite that referenced this pull request Jun 25, 2019
Merge branch 'master' into sca/631-alias-lookup
* master:
  Fix testfile path hardcoded in show-expected-fail-tests.sh (matrix-org#719)
  Make federation state request 404 when event not in the room - fixes matrix-org#625 (matrix-org#716)
  Not all systems have bash (matrix-org#720)
  Update dependency gomatrixserverlib to 178ed5e (matrix-org#695)
  Refine config and docs for sytest (matrix-org#714)
  Fix pipeline, emoji and syntax (matrix-org#713)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.