Run by paremeters #1

cleberzavadniak · 2016-06-28T13:21:17Z

Got rid of prettyconf dependencies;
Tests were thoroughly revised.

rhenter · 2016-06-28T15:58:19Z

loafer/manager.py

-from .route import Route
-from .utils import import_callable
+from .aws.consumer import Consumer as AWSConsumer
+


💄 Muitas linhas dos imports

rhenter · 2016-06-28T16:17:13Z

👍

allisson · 2016-06-28T19:40:08Z

Alguns testes estão quebrando:

(loafer) allisson@allisson-olist:~/Projects/loafer$ py.test tests --cov loafer
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/allisson/Projects/loafer, inifile: 
plugins: asyncio-0.4.1, env-0.6.0, cov-2.2.1
collected 78 items 

tests/test_dispatcher.py FFFFFFFFFFFFF
tests/test_manager.py ....F.
tests/test_message_translator.py .
tests/test_route.py .......
tests/test_utils.py ........
tests/aws/test_consumer.py ......
tests/aws/test_message_translator.py ........................
tests/aws/test_publisher.py ....
tests/example/test_consumer.py ..
tests/example/test_jobs.py .....
tests/example/test_message_translator.py ..
------------------------------------------------------------------------------ coverage: platform linux, python 3.5.1-final-0 ------------------------------------------------------------------------------
Name                                   Stmts   Miss  Cover
----------------------------------------------------------
loafer/__init__.py                         1      0   100%
loafer/aws/__init__.py                     0      0   100%
loafer/aws/consumer.py                    42      0   100%
loafer/aws/message_translator.py          28      0   100%
loafer/aws/publisher.py                   20      0   100%
loafer/dispatcher.py                      65     50    23%
loafer/example/__init__.py                 0      0   100%
loafer/example/consumer.py                13      0   100%
loafer/example/jobs.py                    17      0   100%
loafer/example/message_translator.py      10      0   100%
loafer/exceptions.py                       8      0   100%
loafer/manager.py                         47     17    64%
loafer/message_translator.py               6      0   100%
loafer/route.py                           23      4    83%
loafer/utils.py                           28      0   100%
----------------------------------------------------------
TOTAL                                    308     71    77%

================================================================================================= FAILURES =================================================================================================
__________________________________________________________________________________________ test_without_consumers __________________________________________________________________________________________

route = <Mock spec='Route' id='140013064921552'>

    def test_without_consumers(route):
>       dispatcher = LoaferDispatcher(routes=[route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:15: TypeError
___________________________________________________________________________________________ test_with_consumers ____________________________________________________________________________________________

route = <Mock spec='Route' id='140013065234584'>

    def test_with_consumers(route):
        consumer = Mock()
>       dispatcher = LoaferDispatcher(routes=[route], consumers=[consumer])

tests/test_dispatcher.py:22: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <loafer.dispatcher.LoaferDispatcher object at 0x7f5754ff0748>, routes = [<Mock spec='Route' id='140013065234584'>], consumers = [<Mock id='140013064882048'>], max_jobs = 10

    def __init__(self, routes, consumers, max_jobs=10):
        self.routes = routes
        self.consumers = consumers or []
>       self._semaphore = asyncio.Semaphore(self.max_jobs)
E       AttributeError: 'LoaferDispatcher' object has no attribute 'max_jobs'

loafer/dispatcher.py:16: AttributeError
________________________________________________________________________________________ test_get_consumer_default _________________________________________________________________________________________

route = <Mock spec='Route' id='140013064924240'>

    def test_get_consumer_default(route):
>       dispatcher = LoaferDispatcher(routes=[route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:28: TypeError
_________________________________________________________________________________________ test_get_consumer_custom _________________________________________________________________________________________

route = <Mock spec='Route' id='140013064602736'>

    def test_get_consumer_custom(route):
        consumer = Mock(source=route.source)
>       dispatcher = LoaferDispatcher(routes=[route], consumers=[consumer])

tests/test_dispatcher.py:35: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <loafer.dispatcher.LoaferDispatcher object at 0x7f5754fac6a0>, routes = [<Mock spec='Route' id='140013064602736'>], consumers = [<Mock id='140013064603184'>], max_jobs = 10

    def __init__(self, routes, consumers, max_jobs=10):
        self.routes = routes
        self.consumers = consumers or []
>       self._semaphore = asyncio.Semaphore(self.max_jobs)
E       AttributeError: 'LoaferDispatcher' object has no attribute 'max_jobs'

loafer/dispatcher.py:16: AttributeError
__________________________________________________________________________________ test_get_consumer_default_with_custom ___________________________________________________________________________________

route = <Mock spec='Route' id='140013064656544'>

    def test_get_consumer_default_with_custom(route):
        consumer = Mock(source='other-source')
>       dispatcher = LoaferDispatcher(routes=[route], consumers=[consumer])

tests/test_dispatcher.py:44: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <loafer.dispatcher.LoaferDispatcher object at 0x7f5754fb9828>, routes = [<Mock spec='Route' id='140013064656544'>], consumers = [<Mock id='140013064656880'>], max_jobs = 10

    def __init__(self, routes, consumers, max_jobs=10):
        self.routes = routes
        self.consumers = consumers or []
>       self._semaphore = asyncio.Semaphore(self.max_jobs)
E       AttributeError: 'LoaferDispatcher' object has no attribute 'max_jobs'

loafer/dispatcher.py:16: AttributeError
__________________________________________________________________________________________ test_dispatch_message ___________________________________________________________________________________________

route = <Mock spec='Route' id='140013064697056'>

    @pytest.mark.asyncio
    async def test_dispatch_message(route):
        route.deliver = CoroutineMock(return_value='receipt')
>       dispatcher = LoaferDispatcher([route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:53: TypeError
________________________________________________________________________________ test_dispatch_message_without_translation _________________________________________________________________________________

route = <Mock spec='Route' id='140013064635616'>

    @pytest.mark.asyncio
    async def test_dispatch_message_without_translation(route):
        route.deliver = CoroutineMock(return_value=None)
>       dispatcher = LoaferDispatcher([route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:67: TypeError
________________________________________________________________________________ test_dispatch_message_error_on_translation ________________________________________________________________________________

route = <Mock spec='Route' id='140013064852424'>

    @pytest.mark.asyncio
    async def test_dispatch_message_error_on_translation(route):
        route.deliver = CoroutineMock(return_value=None)
>       dispatcher = LoaferDispatcher([route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:82: TypeError
________________________________________________________________________________ test_dispatch_message_task_reject_message _________________________________________________________________________________

route = <Mock spec='Route' id='140013064882440'>

    @pytest.mark.asyncio
    async def test_dispatch_message_task_reject_message(route):
        route.deliver = CoroutineMock(side_effect=RejectMessage)
>       dispatcher = LoaferDispatcher([route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:97: TypeError
________________________________________________________________________________ test_dispatch_message_task_ignore_message _________________________________________________________________________________

route = <Mock spec='Route' id='140013064643024'>

    @pytest.mark.asyncio
    async def test_dispatch_message_task_ignore_message(route):
        route.deliver = CoroutineMock(side_effect=IgnoreMessage)
>       dispatcher = LoaferDispatcher([route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:111: TypeError
_____________________________________________________________________________________ test_dispatch_message_task_error _____________________________________________________________________________________

route = <Mock spec='Route' id='140013064654072'>

    @pytest.mark.asyncio
    async def test_dispatch_message_task_error(route):
        route.deliver = CoroutineMock(side_effect=Exception)
>       dispatcher = LoaferDispatcher([route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:125: TypeError
____________________________________________________________________________________ test_dispatch_message_task_cancel _____________________________________________________________________________________

route = <Mock spec='Route' id='140013064712936'>

    @pytest.mark.asyncio
    async def test_dispatch_message_task_cancel(route):
        route.deliver = CoroutineMock(side_effect=asyncio.CancelledError)
>       dispatcher = LoaferDispatcher([route])
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:139: TypeError
_________________________________________________________________________________________ test_dispatch_consumers __________________________________________________________________________________________

route = <Mock spec='Route' id='140013064593760'>, consumer = <CoroutineMock id='140013064597120'>

    @pytest.mark.asyncio
    async def test_dispatch_consumers(route, consumer):
        routes = [route]
>       dispatcher = LoaferDispatcher(routes)
E       TypeError: __init__() missing 1 required positional argument: 'consumers'

tests/test_dispatcher.py:152: TypeError
_____________________________________________________________________________________________ test_dispatcher ______________________________________________________________________________________________

event_loop = <_UnixSelectorEventLoop running=False closed=False debug=False>

    def test_dispatcher(event_loop):
        manager = LoaferManager('test-queue', event_loop=event_loop)
>       assert isinstance(manager.get_dispatcher(), LoaferDispatcher)

tests/test_manager.py:41: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
loafer/manager.py:43: in get_dispatcher
    return LoaferDispatcher(self.routes, self.consumers)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <loafer.dispatcher.LoaferDispatcher object at 0x7f5754fc3400>, routes = [], consumers = [<loafer.aws.consumer.Consumer object at 0x7f5754fc3550>], max_jobs = 10

    def __init__(self, routes, consumers, max_jobs=10):
        self.routes = routes
        self.consumers = consumers or []
>       self._semaphore = asyncio.Semaphore(self.max_jobs)
E       AttributeError: 'LoaferDispatcher' object has no attribute 'max_jobs'

loafer/dispatcher.py:16: AttributeError
=================================================================================== 14 failed, 64 passed in 1.12 seconds ===================================================================================
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/base_events.py", line 417, in __del__
    self.close()
  File "/usr/lib/python3.5/asyncio/unix_events.py", line 58, in close
    self.remove_signal_handler(sig)
  File "/usr/lib/python3.5/asyncio/unix_events.py", line 139, in remove_signal_handler
    signal.signal(sig, handler)
  File "/usr/lib/python3.5/signal.py", line 47, in signal
    handler = _signal.signal(_enum_to_int(signalnum), _enum_to_int(handler))
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object
Exception ignored in: <bound method BaseEventLoop.__del__ of <_UnixSelectorEventLoop running=False closed=True debug=False>>
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/base_events.py", line 417, in __del__
  File "/usr/lib/python3.5/asyncio/unix_events.py", line 58, in close
  File "/usr/lib/python3.5/asyncio/unix_events.py", line 139, in remove_signal_handler
  File "/usr/lib/python3.5/signal.py", line 47, in signal
TypeError: signal handler must be signal.SIG_IGN, signal.SIG_DFL, or a callable object

Fazendo as seguintes alterações:

diff --git a/loafer/dispatcher.py b/loafer/dispatcher.py
index 7714cea..490b5dc 100644
--- a/loafer/dispatcher.py
+++ b/loafer/dispatcher.py
@@ -10,10 +10,10 @@ logger = logging.getLogger(__name__)


 class LoaferDispatcher(object):
-    def __init__(self, routes, consumers, max_jobs=10):
+    def __init__(self, routes, consumers=[], max_jobs=10):
         self.routes = routes
-        self.consumers = consumers or []
-        self._semaphore = asyncio.Semaphore(self.max_jobs)
+        self.consumers = consumers
+        self._semaphore = asyncio.Semaphore(max_jobs)
         self._stop_consumers = True

         self.consumers_sources = dict((x.source, x) for x in self.consumers)

Os testes passam:

(loafer) allisson@allisson-olist:~/Projects/loafer$ py.test tests --cov loafer
=========================================================================================== test session starts ============================================================================================
platform linux -- Python 3.5.1+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/allisson/Projects/loafer, inifile: 
plugins: asyncio-0.4.1, env-0.6.0, cov-2.2.1
collected 78 items 

tests/test_dispatcher.py .............
tests/test_manager.py ......
tests/test_message_translator.py .
tests/test_route.py .......
tests/test_utils.py ........
tests/aws/test_consumer.py ......
tests/aws/test_message_translator.py ........................
tests/aws/test_publisher.py ....
tests/example/test_consumer.py ..
tests/example/test_jobs.py .....
tests/example/test_message_translator.py ..
------------------------------------------------------------------------------ coverage: platform linux, python 3.5.1-final-0 ------------------------------------------------------------------------------
Name                                   Stmts   Miss  Cover
----------------------------------------------------------
loafer/__init__.py                         1      0   100%
loafer/aws/__init__.py                     0      0   100%
loafer/aws/consumer.py                    42      0   100%
loafer/aws/message_translator.py          28      0   100%
loafer/aws/publisher.py                   20      0   100%
loafer/dispatcher.py                      65      5    92%
loafer/example/__init__.py                 0      0   100%
loafer/example/consumer.py                13      0   100%
loafer/example/jobs.py                    17      0   100%
loafer/example/message_translator.py      10      0   100%
loafer/exceptions.py                       8      0   100%
loafer/manager.py                         47     17    64%
loafer/message_translator.py               6      0   100%
loafer/route.py                           23      4    83%
loafer/utils.py                           28      0   100%
----------------------------------------------------------
TOTAL                                    308     26    92%

cleberzavadniak · 2016-06-29T01:03:39Z

@allisson : Os testes estão passando, sim, seu mentiroso. Dá um update no teu repositório e testa, aí, pra você ver.
(Que calúnia absurda!)
😀

allisson · 2016-06-29T11:52:58Z

Agora sim 👍

cleberzavadniak added 11 commits June 8, 2016 16:26

WIP. Very experimental.

bddcb58

Got rid of some prints

5d338b1

Got rid of lost NOQAs in the code

bbd1e8a

Dispatcher, manager, route and some tests

a0f020d

Reverted encoding header changes

c1c25d4

Got rid of a print in the tests

8e29577

Tests updated and passing

21567aa

Got rid of cli

dda3a6e

Changes in CHANGES and setup (no more CLI and no more prettyconf)

156687f

Bugfix: no more message_handler_name

0d31f5b

Improvements from last PR discussion

31c76a9

rhenter reviewed Jun 28, 2016
View reviewed changes

Fixed tests

d54012f

allisson merged commit 5661ba2 into olist:master Jun 29, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Run by paremeters #1

Run by paremeters #1

cleberzavadniak commented Jun 28, 2016

rhenter Jun 28, 2016

rhenter commented Jun 28, 2016

allisson commented Jun 28, 2016

cleberzavadniak commented Jun 29, 2016 •

edited

Loading

allisson commented Jun 29, 2016

Run by paremeters #1

Run by paremeters #1

Conversation

cleberzavadniak commented Jun 28, 2016

rhenter Jun 28, 2016

Choose a reason for hiding this comment

rhenter commented Jun 28, 2016

allisson commented Jun 28, 2016

cleberzavadniak commented Jun 29, 2016 • edited Loading

allisson commented Jun 29, 2016

cleberzavadniak commented Jun 29, 2016 •

edited

Loading