Add ChannelLiveServerTestCase #497
Conversation
|
1: I don't think there's a need for multiple aliases for this one - in fact, raise an error if someone tries to set it. 2: I don't know what it's for either. 3: I believe Twisted will traceback during listen if you try to listen on a port that's already bound. Best to verify this by trying to make it listen on something deliberately already used and checking what happens. 4: If we can make sure Daphne lives in a separate thread but the worker lives in the test thread somehow (though I'm not sure that's possible), the overrides would work. Otherwise, yes, we will need to at least restart the worker for each test. I also think the test case class should be in a new module, not |
|
I can't comment on the implementation details related to Channels, because I haven't used it yet. Some comments on the file organization:
|
|
@bittner I absolutely agree with you. Its better to keep channel.test as primary package intended for users. We should provide @andrewgodwin This is my thoughts so far:
Thank for feed back guys! |
websocket.receive should point to some consumer not to the null_handler.
We need to apply routes before live server threads start so its channel layers will contain right routing table.
|
@andrewgodwin Do you agree with @proofit404's thoughts on the Is there anything I can do to help this PR move forward? |
|
@bittner sorry for the huge delay. I can continue working on this PR when I comeback from my business trip on 13th of February. |
|
I do think separating out tests and test infrastructure would be a good idea, yes. |
* master: (23 commits) fix tox (#516) Update Binding to support models with UUIDField as primary key (#528) Typo in Example using class-based consumer (#526) Fix two typos (#521) Docs: Python 3.3 is not supported any more (#519) Typo in changelog Releasing 1.0.3 Simplify testing infrastructure (#515) Fixed #512: Give rundelay a configurable sleep interval Add code indent Handle slight ordering not being set Remove slight ordering from generics docs Fixed #509: Docs for enforce_ordering now mirror post-1.0 Sort imports correctly. Remove optional multiplexer arg in generics docs Fixed #462: Don't actually close DB connections during tests Fixed #505: Add classifiers to setup.py Fixed #477: Only re-save session if it's not empty Installable benchmark package. (#501) Fix typo (#500) ...
|
Hello guys, I stuck. Channels test cases use in memory layer implementation. Twisted reactor can be started and stopped only once during whole process live circle. If we use threads for live server, we can't write more than one test case at all. If we use multiprocessing for live server, we can't use in memory layer. Any suggestions? |
|
Maybe running behave against real channels installation on developers machine is the best option here. |
|
Yes, I think given that limitation it might have to use a cross-process channel backend so we can run the server properly? |
|
Yes, for live server test case we can substitute project settings with |
|
I don't want to encourage use of |
|
It is possible to ignore channels test mixin. But that means we can't give certain guaranties about our environment. For example RabbitMQ layer doesn't provide flush extension. In this situation one test can influence into another through messages in the layer. I can provide test mixin for RabbitMQ which must be used manually in each test case. But this wont be a general solution. |
|
Yeah, the flush is kind of important for tests. Maybe the test case could look for the flush extension and then error if it's not present? Both IPC and Redis provide it, so at least there's options. |
|
Current RabbitmqChannelLayer create new virtual host for each test. This make guaranties that we do not overlap test data even during parallel execution. I can move this in to test helper and make it part of asgi_rabbitmq package. So if user doesn't inherit from this mixin in test, test will fail due to the lack of flush extension. If user inherit from this mixin, test will be executed in the sandbox environment. Does this sound like a proper solution? |
* master: Separate tests into own directory. (#531)
|
I haven't had a chance to look at this yet, it's pretty big and the last couple of weeks have been very busy. I'll try to get to it soon. |
channels/test/liveserver.py
Outdated
| # new connection will be established. | ||
|
|
||
|
|
||
| class ProcessSetupMixin(object): |
There was a problem hiding this comment.
This doesn't need to be a mixin - both places it is used also inherit from multiprocessing.Process, so you can just make this inherit from that and have single inheritance.
docs/releases/1.1.0.rst
Outdated
| @@ -15,6 +15,9 @@ Major Changes | |||
| work but will trigger a deprecation warning, and ``channels.tests`` will be | |||
| removed completely in version 1.3. | |||
|
|
|||
| * Add ``ChannelLiveServerTestCase`` for integration testing. It is | |||
| possible to run Selenium with JavaScript which operates WebSockets. | |||
There was a problem hiding this comment.
This now needs removing from these release notes as we're further along; don't worry about adding release notes, I'll do that when I make the release!
| infrastructure is ready default web browser will be also started. You | ||
| can open your website in the real browser which can execute JavaScript | ||
| and operate on WebSockets. ``live_server_ws_url`` property is also | ||
| provided if you decide to run messaging directly from Python. |
There was a problem hiding this comment.
There's a few grammar and presentation fixes here but I'll fix those post-merge.
testproject/chtest/consumers.py
Outdated
| @@ -2,5 +2,6 @@ | |||
|
|
|||
|
|
|||
| def ws_message(message): | |||
| "Echoes messages back to the client" | |||
| """Echoes messages back to the client.""" | |||
There was a problem hiding this comment.
Why was this changed? There's no need for """ unless you're doing multiple lines.
|
And we done. Any thoughts on testing extension or asgi_redis test mixin? |
|
What do you mean by those? (also, it looks like master is still failing to test, so this probably will too - I'll fix that and then you can rebase) |
|
Oh, you mean the idea of having a separate test channel layer to run tests on. I'd rather not make this an extension but have people configure a channel layer specifically for tests and pass its alias as an argument to the test class, I think. Trying to make it automatic is probably not possible on all backends, and if you want this level of advanced testing a bit more setup is fine. |
* master: Remove unused import Add explicit checks for asgi library versions
|
Hooray test passes. |
|
Hi @proofit404! It works very well with the last asgi_rabbitmq, but it shows error in teardown process. psycopg2.OperationalError: SSL error: sslv3 alert bad record mac====================================================================== ERROR: test_multichat_functionality (functional_tests.test_channels_page.MultichatFunctionalTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/balnir/.virtualenvs/multichat/lib/python3.5/site-packages/django/db/backends/utils.py", line 62, in execute return self.cursor.execute(sql) psycopg2.OperationalError: SSL error: sslv3 alert bad record macThe above exception was the direct cause of the following exception: Traceback (most recent call last): The actual sql variable is "DROP DATABASE 'test_multichat'" command. That database is being dropped at the end of the test suite, but if I duplicate tests the other won't pass: detailstest_multichat_functionality (functional_tests.test_channels_page.MultichatFunctionalTest) ... ok ERROR test_multichat_functionality2 (functional_tests.test_channels_page.MultichatFunctionalTest) ... ERROR ERRORRegards! |
|
I just tried this against a 1.11 installation and it looks like I remember we discussed running integration tests in Travis a while back - did I say no, or was there a technical reason we rejected it? |
|
Turns out Regarding integration tests we decided to exclude them into channel layers. Running this tests requires real layer implementation. Because of that it go against channel policy not use any layer package as a dependency for channels. I use this live server test cases for integration testing in RabbitMQ layer. Also there is an open PR in Redis layer. But you can't merge it yet because of this lines before next channels release. Ideally we need to run layers tests against channels master after each push to it. Probably we can do it with TravisCI API. Running special script in the |
|
We can definitely get Travis running the layer tests as well after every commit, using the Redis library. I'm going to merge this, and then the things left I'd like one of us to look at are:
|
|
Wahoo! 😲 This is merged, great! 🎉 Thanks @proofit404, thanks thanks thanks! (Now let's make sure this stay stable and usable for everyone.) |
|
I'm glad we merge it! Regarding further work on this test case:
First two items of this list already clean enough. I'll implement them. |
|
Ah, yes, I missed the static stuff. We should have a way to turn it on/off easily for sure as people might not want static files on. |
|
So far we have
|
Adds a new test case base class that fires up a live Daphne server and workers for the test case to use, like Django's LiveServerTestCase.
|
Hi @proofit404, I am currently using ChannelsLiveServerTestCase to spin up a runserver for behave, I am using StaticLiveServerTestCase at first but have to change to ChannelsLiveServerTestCase for the websocket support but for some reason now my test for my upload-feature is failing. Does ChannelsLiveServerTestCase support uploading file like StaticLiveServerTestCase? |
|
I left channels project a long time ago. I can't even recall what all those things have to mean. Sorry 😞 Regards, Artem. |
As we discussed earlier in #494 we need live server test case for example to run selenium against it. This is work in progress PR, but I want some feedback already.
Few questions we need to answer before merging it.
Channels documentation mention test_channel_aliases as ability to test complex layers setup. I don't know what to do in this case for live server test. It is possible to spawn multiple live server instances. But its not clear to me what user API we should provide.
Regular live server thread contains connection override logic. I don't understand right now what problem does it solve. It will be awesome if any one can describe the reason it present there or provide test case for it.
How we can validate that twisted bind port successfully? For now I select first port from the possible range without any validation.
apply_routes setup test instance with overriding setUp and tearDown methods. Live server spawns its threads in the test class setup (as original one do). So apply routes hook works after we start live server threads and overrides not available in those threads. For now I change apply_routes decorator to work on class basis. But we also can consider running live server for each individual test method however this will slow things down and inconsistent with django behavior.
We need to write docs.
cc @bittner