Asyncio RawSocket #679

Merged
merged 17 commits into from May 31, 2016

Conversation

Projects
None yet
4 participants
@izderadicka
Contributor

izderadicka commented May 23, 2016

Hi,

this is my first attempt for rawsocket transport for aio.

Few comments for current code:

  • It's python 3 only now - it's bit intentional - since trollius is officially deprecated it does not make much sense for me to support it here - anybody willing to work with asyncio will use python3 anyhow
  • I've tried to make it basically independent on remaining code, self-contained - so basically no changes to other parts of code, that can break something else.
  • This lead to some duplication - mainly in utils module ( which probably should be shared with remaining codebase) and WampRawSocketMixinGeneral in rawsocket module, which contains exactly same code as rawsocket transport for twisted.
  • There is also new runner class ApplicationRunnerRawSocket - again made not to infer with existing code. Potentially it can be easily integrated into current runner - just use additional URI schemes tcp:// and unix:
  • no ssl support yet - I do not need it - main motivation was to has backend asyncio component connected on same computer - so unix socket is what I need.
  • testing - I've written unittests for this addition. Also tested in python 3 against crossbar - example added. Not sure if you have some more advanced tests - quickly looked to testsuite but it looks like tests are mostly focused on websocket.

Let me know your thoughts - thanks
Ivan

autobahn/asyncio/rawsocket.py
+ peer = transport.get_extra_info('peername')
+ self.peer = peer2str(peer)
+ self.log.debug('RawSocker Asyncio: Connection made with peer {peer}'.format(peer=self.peer))
+ self._buffer = b''

This comment has been minimized.

@oberstet

oberstet May 23, 2016

Member

We use structured logging, not string interpolation

@oberstet

oberstet May 23, 2016

Member

We use structured logging, not string interpolation

This comment has been minimized.

@izderadicka

izderadicka May 23, 2016

Contributor

You mean like this?

self.log.debug('RawSocker Asyncio: Connection made with peer {peer}', peer=self.peer)
@izderadicka

izderadicka May 23, 2016

Contributor

You mean like this?

self.log.debug('RawSocker Asyncio: Connection made with peer {peer}', peer=self.peer)

This comment has been minimized.

@meejah

meejah May 23, 2016

Member

yes, exactly

@meejah

meejah May 23, 2016

Member

yes, exactly

autobahn/asyncio/wamp.py
+
+# TODO - unify with previous class
+class ApplicationRunnerRawSocket(object):
+ """

This comment has been minimized.

@oberstet

oberstet May 23, 2016

Member

No, we don't want to introduce a class for this

@oberstet

oberstet May 23, 2016

Member

No, we don't want to introduce a class for this

This comment has been minimized.

@izderadicka

izderadicka May 23, 2016

Contributor

So what would be right solution - extend runner class for broader URI support - tcp:// ...
Or keep it our completely just stick with WS runner?

@izderadicka

izderadicka May 23, 2016

Contributor

So what would be right solution - extend runner class for broader URI support - tcp:// ...
Or keep it our completely just stick with WS runner?

This comment has been minimized.

@oberstet

oberstet May 23, 2016

Member

A better approach would be to teach ApplicationRunner how to establish different transports ("better" as in until we get anywhere with the "new API" discussion;)

@oberstet

oberstet May 23, 2016

Member

A better approach would be to teach ApplicationRunner how to establish different transports ("better" as in until we get anywhere with the "new API" discussion;)

This comment has been minimized.

@meejah

meejah May 23, 2016

Member

On the one hand, ApplicationRunner is a convenience API and needs improvement for more use-cases, but there should be just one way to run any WAMP connection; e.g. see #674 (and "LTS-API" tagged bugs).

On the other hand of course, there's not currently a way to ask ApplicationRunner to instantiate anything except WebSocket :/

The "real" solution is to make sure Component understands raw vs web socket. However, there is only a Component in the twisted/ packages right now as that API isn't finalized (but, it currently does properly distinguish ws/tcp transports). I don't think this PR should be stalled/blocked on that though ... maybe we should just not change ApplicationRunner etc at all in this PR and do it separately?

For testing purposes if you have crossbar start your session, it can be configured with a raw-socket transport (with type: "rawsocket" vs type: "websocket").

@meejah

meejah May 23, 2016

Member

On the one hand, ApplicationRunner is a convenience API and needs improvement for more use-cases, but there should be just one way to run any WAMP connection; e.g. see #674 (and "LTS-API" tagged bugs).

On the other hand of course, there's not currently a way to ask ApplicationRunner to instantiate anything except WebSocket :/

The "real" solution is to make sure Component understands raw vs web socket. However, there is only a Component in the twisted/ packages right now as that API isn't finalized (but, it currently does properly distinguish ws/tcp transports). I don't think this PR should be stalled/blocked on that though ... maybe we should just not change ApplicationRunner etc at all in this PR and do it separately?

For testing purposes if you have crossbar start your session, it can be configured with a raw-socket transport (with type: "rawsocket" vs type: "websocket").

This comment has been minimized.

@izderadicka

izderadicka May 24, 2016

Contributor

Re: testing with crossbar - it's already done in example - backend and frontend over raw socket - backend using unix socket, frontend tcp socket

@izderadicka

izderadicka May 24, 2016

Contributor

Re: testing with crossbar - it's already done in example - backend and frontend over raw socket - backend using unix socket, frontend tcp socket

@@ -0,0 +1,3 @@
+/key.priv
+/key.pub
+/socket1

This comment has been minimized.

@oberstet

oberstet May 23, 2016

Member

These 3 should be excluded

@oberstet

oberstet May 23, 2016

Member

These 3 should be excluded

This comment has been minimized.

@izderadicka

izderadicka May 23, 2016

Contributor

??? they are excluded - this is .gitignore

@izderadicka

izderadicka May 23, 2016

Contributor

??? they are excluded - this is .gitignore

This comment has been minimized.

@oberstet

oberstet May 23, 2016

Member

Ah, right. Sorry;)

@oberstet

oberstet May 23, 2016

Member

Ah, right. Sorry;)

@oberstet

This comment has been minimized.

Show comment
Hide comment
@oberstet

oberstet May 23, 2016

Member

Thanks for working on this! As requirement would be Python 3 + 2 support plus all tests green. As you mentioned, you see some potential for avoiding code duplication .. this is highly welcome! Maybe you could break this change up in multiple smaller PRs? Easier/quicker to review ..

Member

oberstet commented May 23, 2016

Thanks for working on this! As requirement would be Python 3 + 2 support plus all tests green. As you mentioned, you see some potential for avoiding code duplication .. this is highly welcome! Maybe you could break this change up in multiple smaller PRs? Easier/quicker to review ..

@izderadicka

This comment has been minimized.

Show comment
Hide comment
@izderadicka

izderadicka May 23, 2016

Contributor

OK - I'll look into clean CI tests firsts - fixed pep8 already will look into python2

Contributor

izderadicka commented May 23, 2016

OK - I'll look into clean CI tests firsts - fixed pep8 already will look into python2

@izderadicka

This comment has been minimized.

Show comment
Hide comment
@izderadicka

izderadicka May 23, 2016

Contributor

I have interesting problem with unittests - in twisted tests I got this error:

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/twisted/trial/runner.py", line 511, in loadPackage
    module = modinfo.load()
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/twisted/python/modules.py", line 389, in load
    return self.pathEntry.pythonPath.moduleLoader(self.name)
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/twisted/python/reflect.py", line 300, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/twisted/python/reflect.py", line 239, in _importAndCheckStack
    return __import__(importName)
  File "/home/ivan/workspace/autobahn-python/autobahn/asyncio/__init__.py", line 36, in <module>
    from autobahn.asyncio.websocket import \
  File "/home/ivan/workspace/autobahn-python/autobahn/asyncio/websocket.py", line 32, in <module>
    txaio.use_asyncio()
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/txaio/__init__.py", line 117, in use_asyncio
    raise RuntimeError("Explicitly using '{}' already".format(_explicit_framework))
builtins.RuntimeError: Explicitly using 'twisted' already

autobahn.asyncio.test.test_rawsocket
-------------------------------------------------------------------------------

It basically happens even before my test module is imported - parent modules autobahn.asyncio is imported first.

Would removing of /autobahn/asyncio/websocket.py", line 32, in
txaio.use_asyncio()

be safe? Because it seems as cause of problem. Or how to exclude searching for tests in autobahn.asycio in this case - I use env.var in test, but this happens before test module is loaded!

Contributor

izderadicka commented May 23, 2016

I have interesting problem with unittests - in twisted tests I got this error:

===============================================================================
[ERROR]
Traceback (most recent call last):
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/twisted/trial/runner.py", line 511, in loadPackage
    module = modinfo.load()
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/twisted/python/modules.py", line 389, in load
    return self.pathEntry.pythonPath.moduleLoader(self.name)
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/twisted/python/reflect.py", line 300, in namedAny
    topLevelPackage = _importAndCheckStack(trialname)
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/twisted/python/reflect.py", line 239, in _importAndCheckStack
    return __import__(importName)
  File "/home/ivan/workspace/autobahn-python/autobahn/asyncio/__init__.py", line 36, in <module>
    from autobahn.asyncio.websocket import \
  File "/home/ivan/workspace/autobahn-python/autobahn/asyncio/websocket.py", line 32, in <module>
    txaio.use_asyncio()
  File "/home/ivan/workspace/autobahn-python/.tox/py34-twtrunk/lib/python3.4/site-packages/txaio/__init__.py", line 117, in use_asyncio
    raise RuntimeError("Explicitly using '{}' already".format(_explicit_framework))
builtins.RuntimeError: Explicitly using 'twisted' already

autobahn.asyncio.test.test_rawsocket
-------------------------------------------------------------------------------

It basically happens even before my test module is imported - parent modules autobahn.asyncio is imported first.

Would removing of /autobahn/asyncio/websocket.py", line 32, in
txaio.use_asyncio()

be safe? Because it seems as cause of problem. Or how to exclude searching for tests in autobahn.asycio in this case - I use env.var in test, but this happens before test module is loaded!

@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 23, 2016

Member

@izderadicka regarding the testing, there's a USE_TWISTED=1 environment variable that should control whether the twisted tests are run or not...what might be happening here is that it's effectively trying to run both Twisted and asyncio tests in one process, which won't work.

Removing the .use_asyncio() also won't really work -- that's there so that you're forced to choose twisted vs. asyncio based on imports (e.g. autobahn.twisted.* vs. autobahn.asyncio.*), and it's impossible to do both asyncio and twisted things in the same python process. I will look at the unit-tests in this branch and suggest something useful in the next few hours...

Member

meejah commented May 23, 2016

@izderadicka regarding the testing, there's a USE_TWISTED=1 environment variable that should control whether the twisted tests are run or not...what might be happening here is that it's effectively trying to run both Twisted and asyncio tests in one process, which won't work.

Removing the .use_asyncio() also won't really work -- that's there so that you're forced to choose twisted vs. asyncio based on imports (e.g. autobahn.twisted.* vs. autobahn.asyncio.*), and it's impossible to do both asyncio and twisted things in the same python process. I will look at the unit-tests in this branch and suggest something useful in the next few hours...

@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 23, 2016

Member

Also, thanks for the PR! This is definitely a nice missing feature to have :)

Member

meejah commented May 23, 2016

Also, thanks for the PR! This is definitely a nice missing feature to have :)

@izderadicka

This comment has been minimized.

Show comment
Hide comment
@izderadicka

izderadicka May 24, 2016

Contributor

@meejah - I cannot solve problem with unittest just by using USE_TWISTED or USE_ASYNCIO - problem is that when running twisted tests ( for instance tox -c tox.ini -e py34-twtrunk) the test runner still tries to import everything that start with test_ . Which means - if I put my test into autobahn.asyncio.test, where it should logically belong ( at least based on other tests placement in code)' - test runner will try to import it first - which means it'll first import autobahn.asyncio and module's __init__ imports both websocket and wamp submodules, each runs txaio.use_asyncion during the import. So form my perspective I cannot stop this unless I place test in different directory - would this be acceptable - put it for instance in autobahn.test?

Contributor

izderadicka commented May 24, 2016

@meejah - I cannot solve problem with unittest just by using USE_TWISTED or USE_ASYNCIO - problem is that when running twisted tests ( for instance tox -c tox.ini -e py34-twtrunk) the test runner still tries to import everything that start with test_ . Which means - if I put my test into autobahn.asyncio.test, where it should logically belong ( at least based on other tests placement in code)' - test runner will try to import it first - which means it'll first import autobahn.asyncio and module's __init__ imports both websocket and wamp submodules, each runs txaio.use_asyncion during the import. So form my perspective I cannot stop this unless I place test in different directory - would this be acceptable - put it for instance in autobahn.test?

Fixed remaining pep8
Python2 compatibility
@codecov-io

This comment has been minimized.

Show comment
Hide comment
@codecov-io

codecov-io May 24, 2016

Current coverage is 59.29%

Merging #679 into master will increase coverage by 1.94%

  1. 7 files (not in diff) in autobahn/wamp were modified. more
    • Misses -16
    • Partials -44
    • Hits +60
  2. 2 files (not in diff) in autobahn/twisted were modified. more
  3. 2 files (not in diff) in autobahn were modified. more
    • Partials -22
    • Hits +22
@@             master       #679   diff @@
==========================================
  Files            63         66     +3   
  Lines         10278      10739   +461   
  Methods           0          0          
  Messages          0          0          
  Branches       1667       1643    -24   
==========================================
+ Hits           5894       6367   +473   
- Misses         3824       3860    +36   
+ Partials        560        512    -48   

Powered by Codecov. Last updated by 1c86fef...f66411e

codecov-io commented May 24, 2016

Current coverage is 59.29%

Merging #679 into master will increase coverage by 1.94%

  1. 7 files (not in diff) in autobahn/wamp were modified. more
    • Misses -16
    • Partials -44
    • Hits +60
  2. 2 files (not in diff) in autobahn/twisted were modified. more
  3. 2 files (not in diff) in autobahn were modified. more
    • Partials -22
    • Hits +22
@@             master       #679   diff @@
==========================================
  Files            63         66     +3   
  Lines         10278      10739   +461   
  Methods           0          0          
  Messages          0          0          
  Branches       1667       1643    -24   
==========================================
+ Hits           5894       6367   +473   
- Misses         3824       3860    +36   
+ Partials        560        512    -48   

Powered by Codecov. Last updated by 1c86fef...f66411e

@izderadicka

This comment has been minimized.

Show comment
Hide comment
@izderadicka

izderadicka May 24, 2016

Contributor

Fixed all issues with clean build. Theoretically should be compatible with python2, but example is for python3 (yield from).
Had to move unit test module - due to issue explained above.
Now decide What to do with Application Runner - I propose to merge with my solution - so it'll be possible to use URLs:

  • ws://host:port/path - websocket
  • wss://host:port/path - secure websocket
  • tcp://host:port - raw socket over TCP
  • unix:/path - raw socket over unix domain socket
Contributor

izderadicka commented May 24, 2016

Fixed all issues with clean build. Theoretically should be compatible with python2, but example is for python3 (yield from).
Had to move unit test module - due to issue explained above.
Now decide What to do with Application Runner - I propose to merge with my solution - so it'll be possible to use URLs:

  • ws://host:port/path - websocket
  • wss://host:port/path - secure websocket
  • tcp://host:port - raw socket over TCP
  • unix:/path - raw socket over unix domain socket
@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 24, 2016

Member

@izderadicka using the "whole file" os.environ guard like you have already, I can move the file back to autobahn/asyncio/test.

Also, what about something like this: meejah@356bcec

(Seems to work for me, but I'm not entirely sure why the pytest import doesn't fail in e.g. the py27 envs ...). The idea being: bail out early and avoid a "whole file" indent-level...

Member

meejah commented May 24, 2016

@izderadicka using the "whole file" os.environ guard like you have already, I can move the file back to autobahn/asyncio/test.

Also, what about something like this: meejah@356bcec

(Seems to work for me, but I'm not entirely sure why the pytest import doesn't fail in e.g. the py27 envs ...). The idea being: bail out early and avoid a "whole file" indent-level...

@izderadicka

This comment has been minimized.

Show comment
Hide comment
@izderadicka

izderadicka May 24, 2016

Contributor

@meejah And it works in py34-twtrunk or in other twisted test environments? It did not for me. The other tip looks fine - nice way how to avoid ident

Contributor

izderadicka commented May 24, 2016

@meejah And it works in py34-twtrunk or in other twisted test environments? It did not for me. The other tip looks fine - nice way how to avoid ident

@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 24, 2016

Member

With changes as on the branch from #679 (comment) all the environments pass for me .. not entirely sure why? Hmm, maybe the "pytest-twisted" req for the Twisted environments ends up installing pytest in them, too...

In any case, I think your first instinct to put these tests in autobahn/asyncio/test/* is the better place -- if the above code works, feel free to add it to this PR (better to do it yourself so it doesn't look like I wrote that whole file ;) by deleting whitespace)...

IMO, the ApplicationRunner stuff is best dealt with in a follow-up PR; then at least we can get this code in and worry about the best syntax for ApplicationRunner separately. FWIW, your proposed URL syntax looks fine -- but there might be implications I'm not thinking of...

Member

meejah commented May 24, 2016

With changes as on the branch from #679 (comment) all the environments pass for me .. not entirely sure why? Hmm, maybe the "pytest-twisted" req for the Twisted environments ends up installing pytest in them, too...

In any case, I think your first instinct to put these tests in autobahn/asyncio/test/* is the better place -- if the above code works, feel free to add it to this PR (better to do it yourself so it doesn't look like I wrote that whole file ;) by deleting whitespace)...

IMO, the ApplicationRunner stuff is best dealt with in a follow-up PR; then at least we can get this code in and worry about the best syntax for ApplicationRunner separately. FWIW, your proposed URL syntax looks fine -- but there might be implications I'm not thinking of...

@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 24, 2016

Member

p.s. just to clarify explicitly: IMO it's totally fine to have the asyncio example only work with python3 (i.e. "yield from" is fine for the examples).

Member

meejah commented May 24, 2016

p.s. just to clarify explicitly: IMO it's totally fine to have the asyncio example only work with python3 (i.e. "yield from" is fine for the examples).

izderadicka added some commits May 26, 2016

Removed ApplicationRunnerRawSocket (actually moved to example)
Returned test back to autobahn.asyncio.test (see what'll do)
@izderadicka

This comment has been minimized.

Show comment
Hide comment
@izderadicka

izderadicka May 26, 2016

Contributor

I removed ApplicationRunnerRawSocket.
I really cannot make all tests run if unittest in in autobahn.asyncio.test. Problem is in already in collection phase - because autobahn.asyncio imports from autobahn.asyncio.websocket and autobahn.asyncio.wamp - both packages run txaio.use_asyncio() on top level. So it's run during collection - it does not matter that all test is skipped.
Try this (trial is used to run tests for twisted):

USE_TWISTED=1 trial  autobahn

or this

USE_TWISTED=1 py.test --twisted -k 'not asyncio/test' autobahn

Both fail for me because of imports in collection phase. Before it was not problem because there were no tests in autobahn.asyncio.

It about organization of imports - question: are imports in autobahn.asyncio package needed? Why for instance import there form websocket, if I'll use only rawsocket. But that's for you to decide. From my perspective I've tried implementation that will not disrupt existing code.

@meejah - if you really can make test working in autobahn.asyncio.test I'll need more detailed guidance - I've tried both proposed ways, with no luck. But as I explained I think problem is elsewere.

Contributor

izderadicka commented May 26, 2016

I removed ApplicationRunnerRawSocket.
I really cannot make all tests run if unittest in in autobahn.asyncio.test. Problem is in already in collection phase - because autobahn.asyncio imports from autobahn.asyncio.websocket and autobahn.asyncio.wamp - both packages run txaio.use_asyncio() on top level. So it's run during collection - it does not matter that all test is skipped.
Try this (trial is used to run tests for twisted):

USE_TWISTED=1 trial  autobahn

or this

USE_TWISTED=1 py.test --twisted -k 'not asyncio/test' autobahn

Both fail for me because of imports in collection phase. Before it was not problem because there were no tests in autobahn.asyncio.

It about organization of imports - question: are imports in autobahn.asyncio package needed? Why for instance import there form websocket, if I'll use only rawsocket. But that's for you to decide. From my perspective I've tried implementation that will not disrupt existing code.

@meejah - if you really can make test working in autobahn.asyncio.test I'll need more detailed guidance - I've tried both proposed ways, with no luck. But as I explained I think problem is elsewere.

@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 26, 2016

Member

@izderadicka Ah, okay, I was missing a __init__ in asyncio/test which is why these appeared to be working for me. With the __init__.py then I can repeat your failures as per the above comment.

I'll see if I can make it work, but for now let's stick with the one which does work -- i.e. putting the tests in the "non ideal" place; we can always fix it later if there's a nicer way. Alternatively, if you'd prefer to keep them in the "ideal" place, you can keep it under the if os.environ(..) instead.

Member

meejah commented May 26, 2016

@izderadicka Ah, okay, I was missing a __init__ in asyncio/test which is why these appeared to be working for me. With the __init__.py then I can repeat your failures as per the above comment.

I'll see if I can make it work, but for now let's stick with the one which does work -- i.e. putting the tests in the "non ideal" place; we can always fix it later if there's a nicer way. Alternatively, if you'd prefer to keep them in the "ideal" place, you can keep it under the if os.environ(..) instead.

@izderadicka

This comment has been minimized.

Show comment
Hide comment
@izderadicka

izderadicka May 26, 2016

Contributor

@meejah I see - that's cool trick - if test is not package, then it'll not import parent packages. I leave decision to you as maintainers - what is smaller evil - test in non-ideal place or in right place, but not as package ( all other test directories are packages - e.g. have __init__.py)

Contributor

izderadicka commented May 26, 2016

@meejah I see - that's cool trick - if test is not package, then it'll not import parent packages. I leave decision to you as maintainers - what is smaller evil - test in non-ideal place or in right place, but not as package ( all other test directories are packages - e.g. have __init__.py)

@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 26, 2016

Member

I'm still not really sure what's going on, specifically why the Twisted tests work fine with a similar setup (including __init__.py). Paging @hawkowl as IIRC she did "something" to make this work nicely ...

Member

meejah commented May 26, 2016

I'm still not really sure what's going on, specifically why the Twisted tests work fine with a similar setup (including __init__.py). Paging @hawkowl as IIRC she did "something" to make this work nicely ...

@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 26, 2016

Member

Ah, I think I know what's happening: the mere presence of a test_*.py file anywhere below twisted.asyncio.* causes trial to import twisted.asyncio which itself causes txaio.use_asyncio() to be called (after we already selected twisted). But, I think I have a solution that allows asycio tests in "the right spot".

Member

meejah commented May 26, 2016

Ah, I think I know what's happening: the mere presence of a test_*.py file anywhere below twisted.asyncio.* causes trial to import twisted.asyncio which itself causes txaio.use_asyncio() to be called (after we already selected twisted). But, I think I have a solution that allows asycio tests in "the right spot".

@meejah

This comment has been minimized.

Show comment
Hide comment
Member

meejah commented May 26, 2016

@izderadicka

This comment has been minimized.

Show comment
Hide comment
@izderadicka

izderadicka May 27, 2016

Contributor

Merged @meejah solution for unittest. Two more things were needed - adding autobahn.asyncio.test as package_data to setup.py and also modifying coverage command in tox.ini - because otherwise coverage data are not collected.

Frankly spoken after playing a while with tests, maybe the simplest solution was not to try to put this test into autobahn.asyncio. Longer term solution could be to have all tests separate from package as proposed here http://pytest.org/latest/goodpractices.html#goodpractices
(now we combine inline tests with and without init file, which causes some issues).

Contributor

izderadicka commented May 27, 2016

Merged @meejah solution for unittest. Two more things were needed - adding autobahn.asyncio.test as package_data to setup.py and also modifying coverage command in tox.ini - because otherwise coverage data are not collected.

Frankly spoken after playing a while with tests, maybe the simplest solution was not to try to put this test into autobahn.asyncio. Longer term solution could be to have all tests separate from package as proposed here http://pytest.org/latest/goodpractices.html#goodpractices
(now we combine inline tests with and without init file, which causes some issues).

@meejah

This comment has been minimized.

Show comment
Hide comment
@meejah

meejah May 27, 2016

Member

Personally, I like separating tests from "real code". Some people prefer the opposite.

But it might make the whole asyncio vs twisted vs generic test-organization a lot nicer to do something like:

  • autobahn/* (existing code, as now)
    • twisted/*.py
    • asyncio/*.py
  • tests/* (move test-code to here)
    • /generic/test_*.py
    • /asyncio/test_*.py
    • /twisted/test_*.py

I'm not sure if this would remove all the hack-ish things to get around test-collection etc. That is, trial would still have to be "tricked" into not importing tests/asyncio/* unless we make tests/* "not a module" (i.e. no __init__). Then we could be explicit when running things and eliminate the environment-variables (i.e. run like trial tests/generic tests/twisted or py.test tests/generic/ tests/asyncio).

There's also the "distribute the test-code with your module or not" question (e.g. that's one of the reasons people prefer tests "in" with the code, so they always come along). Since Autobahn has "small" systems as a target audience this might be a good reason to not include that code at all (i.e. you only get the tests/* code if you do a Git checkout; a pip install wouldn't include them).

Maybe for this PR @izderadicka's original solution is really the best? :/

...and we could consider a wholesale move of the tests to their own tree as a separate ticket?

Member

meejah commented May 27, 2016

Personally, I like separating tests from "real code". Some people prefer the opposite.

But it might make the whole asyncio vs twisted vs generic test-organization a lot nicer to do something like:

  • autobahn/* (existing code, as now)
    • twisted/*.py
    • asyncio/*.py
  • tests/* (move test-code to here)
    • /generic/test_*.py
    • /asyncio/test_*.py
    • /twisted/test_*.py

I'm not sure if this would remove all the hack-ish things to get around test-collection etc. That is, trial would still have to be "tricked" into not importing tests/asyncio/* unless we make tests/* "not a module" (i.e. no __init__). Then we could be explicit when running things and eliminate the environment-variables (i.e. run like trial tests/generic tests/twisted or py.test tests/generic/ tests/asyncio).

There's also the "distribute the test-code with your module or not" question (e.g. that's one of the reasons people prefer tests "in" with the code, so they always come along). Since Autobahn has "small" systems as a target audience this might be a good reason to not include that code at all (i.e. you only get the tests/* code if you do a Git checkout; a pip install wouldn't include them).

Maybe for this PR @izderadicka's original solution is really the best? :/

...and we could consider a wholesale move of the tests to their own tree as a separate ticket?

@oberstet oberstet merged commit c54bc9e into crossbario:master May 31, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@oberstet

This comment has been minimized.

Show comment
Hide comment
@oberstet

oberstet May 31, 2016

Member

@izderadicka cool! thanks a lot!

Member

oberstet commented May 31, 2016

@izderadicka cool! thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment