Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: run (some) servers on "any port" #5247

Merged
merged 9 commits into from Apr 19, 2020
Merged

tests: run (some) servers on "any port" #5247

merged 9 commits into from Apr 19, 2020

Conversation

@bagder
Copy link
Member

@bagder bagder commented Apr 16, 2020

Makes the test servers (sws and ftpserver) for HTTP, Gopher, FTP, IMAP, POP3, SMTP listen on a currently unused port and runtests adapts to that! Removed the need and use of a fixed port number for these tests.

Test case preprocessing

In order to work correctly with flexible port number for all test cases and all protocols, runtests now preprocesses every test case before it is started and the updated test case file is stored as log/test$num. Each protocol test server should now look for the preprocessed version first, then fall back to the original test file only if not found. This allows proper variable substitution in the entire test case and also makes it work for all protocols and test servers at once.

@bagder bagder added the tests label Apr 16, 2020
@bagder bagder force-pushed the bagder/tests-on-dynport branch from fda3f03 to ce5ccbc Apr 17, 2020
@bagder bagder marked this pull request as ready for review Apr 17, 2020
@xquery
Copy link
Member

@xquery xquery commented Apr 18, 2020

what is the interaction with absolute test numbers eg. take for example test2054 ... do we want to randomise that port ?

Loading

@bagder
Copy link
Member Author

@bagder bagder commented Apr 18, 2020

I don't think so. That test verifies that the logic works with host + (fixed) port number. I don't think that's a problem. It won't connect to those ports anyway. The port number it actually connects to will be "random" though...

Loading

@xquery
Copy link
Member

@xquery xquery commented Apr 18, 2020

apologies for being slow on the uptake - do these changes mean there is now no way to absolutely specify a port ?

Loading

@bagder
Copy link
Member Author

@bagder bagder commented Apr 18, 2020

Sure, just specify a number and that number will be used. But the port numbers for test servers should never be specified like that, not now and not before either. Up until now however, such mistakes were easily let through since it works for most test runs, while this PR will make them never work (well, except for one in many thousands).

Loading

bagder added 5 commits Apr 18, 2020
Makes the test servers for HTTP and Gopher pop up on a currently unused
port and runtests adapts to that!

Closes #5247
The runtests script now always performs variable replacement on the
entire test source file before the test gets executed, and saves the
updated version in a temporary file (log/test[num]) so that all test
case readers/servers can use that version (if present) and thus enjoy
the powers of test case variable substitution.

This is necessary to allow complete port number freedom.

Test 309 is updated to work with a non-fixed port number thanks to this.
@bagder bagder force-pushed the bagder/tests-on-dynport branch from a05fa6c to f99b007 Apr 18, 2020
bagder added 4 commits Apr 18, 2020
FTP, IMAP, POP3, SMTP and their IPv6 versions are now all on dynamic
ports

Test 842-845 are unfortunately a bit hard to move over to this concept
right now and require "default port" still...
@bagder bagder changed the title tests: run the sws server on "any port" tests: run (some) servers on "any port" Apr 19, 2020
@bagder bagder closed this in 80d6515 Apr 19, 2020
@bagder bagder merged commit 003a7b5 into master Apr 19, 2020
42 of 48 checks passed
Loading
@bagder bagder deleted the bagder/tests-on-dynport branch Apr 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants