Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

autobahn test failures #147

Closed
belm0 opened this issue Nov 14, 2020 · 4 comments
Closed

autobahn test failures #147

belm0 opened this issue Nov 14, 2020 · 4 comments

Comments

@belm0
Copy link
Member

belm0 commented Nov 14, 2020

Dozens of test cases are failing in the client suite.

All the cases I've checked are due to "The connection was failed by the wrong endpoint (FAILED)".

I haven't been able to run the server suite.

@belm0
Copy link
Member Author

belm0 commented Nov 15, 2020

Regarding running the server suite, the instructions are outdated, and don't take platform differences into consideration.

  • in the docker invocation, I needed to open the port to the server running on the host with -p 9000:9000
  • on OS X or Windows, the host in fuzzingclient.json must be changed from 172.17.0.1 to host.docker.internal

The server suite also has dozens of failures with "The connection was failed by the wrong endpoint (FAILED)".

belm0 added a commit that referenced this issue Nov 15, 2020
@belm0
Copy link
Member Author

belm0 commented Nov 15, 2020

I went back to trio-websocket 0.6.0 release, and client test suite still fails on a number of tests with "The connection was failed by the wrong endpoint (FAILED)":

  • 9.1.{4,5,6}
  • 9.2.{4,5,6}
  • 9.{3,4}

--> these were only due to insufficient limit on message length

So for the other failures seen in head, it seems there is a regression since 0.6.0.

@belm0
Copy link
Member Author

belm0 commented Nov 15, 2020

All I know so far is that there's something nondeterministic about the client shutdown on error. Sometimes it matches the test case expectation on the server end, sometimes not.

belm0 added a commit that referenced this issue Nov 15, 2020
@belm0
Copy link
Member Author

belm0 commented Nov 15, 2020

The regression seems to be from trio-websocket 0.7.0-- specially PR #106, which was a significant refactoring for upgrade to wsproto 0.14 API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant