Skip to content

Conversation

pfreixes
Copy link
Contributor

@pfreixes pfreixes commented May 20, 2018

Please note that this PR sits on top of this one #138, so skip changes related to give support for Python 3.7.

Also, we could consider creating an ad-hoc branch to don't pollute the master brancgh, if you consider that we can move ahead and the iterations will be done in many small MR, that might leave the code in an inconsistent status.

This is the first PR that implements POC that gives to Uvloop the
capacity of emit traces that later can be collected by the upper layers.
What does this commit implement?

  • DNS trace events triggered by the calls to the getaddrinfo loop
    method.
  • The enabler and disabler context manager called tracing.
  • The collector interface used to gather the trace events called
    TracingCollector.

This POC assumes that the caller is the last responsible implementing
a context that isolates calling to traces that belong to different
tasks.

Missing pieces in this PoC:

  • Allow many collectors at a time.
  • Allow the collector filter which event traces are eligible to be traced. Not relying on the default do nothing of the default method.
  • The exceptions have to be propagated at tracing level? perhaps, what
    should we do when there is an exception and we have to call to
    dns_request_end? right now a None is passed as an argument.

Example of the code that uses the current POC

import asyncio
import contextvars
import uvloop
from types import SimpleNamespace

asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

trace_context = contextvars.ContextVar('trace_context')


class MyCollector(uvloop.TracingCollector):
    def __init__(self, loop=None):
        self._loop = loop if loop else asyncio.get_event_loop()

    @property
    def loop(self):
        return self._loop

    ## Collector methods
    def dns_request_begin(self, host, port, family, type, proto, flags):
        ctx = SimpleNamespace(dns_request_begin = self.loop.time())
        trace_context.set(ctx)

    def dns_request_end(self, data):
        delta = loop.time() - trace_context.get().dns_request_begin
        print(f"DNS request took {delta}")


async def main(loop):
    with uvloop.tracing(MyCollector(loop)):
        for i in range(100):
            host = await loop.getaddrinfo('localhost', 80)

loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

What I would like to implement in the next step:

  • Allow many collectors at a time.
  • Allow the collector filter which event traces are eligible to be traced. Not relying on the default do
    nothing
    of the default method.

As an example of the previous snippet with the missing features implemented:

async def main(loop):
    with uvloop.tracing(MyCollector(loop)):
        with uvloop.tracing(YourCollector()):
            for i in range(100):
                host = await loop.getaddrinfo('localhost', 80)

with uvloop.tracing(LoopStats(events_to_trace=[ .. loop stats events ..])):
    loop.run_until_complete(main(loop))

pfreixes added 30 commits April 10, 2018 06:44
This MR aligns the uvloop code to become compatible, not production
ready, with PY37 adding the following changes:

- Support for the new API used behind the `set_debug` function.
- Adds the `context=None` kw to those functions that will receive it
  when uvloop runs with PY37.
- Former asyncio.test_utils moved as an uvloop test resource.
pfreixes added 10 commits May 6, 2018 07:44
This is the first PR that implements POC that gives to uvloop the
capacity of emit traces that later can be collected by the upper layers.
What does this commit implement?

- DNS trace events triggered by the calls to the `getaddrinfo` loop
  method.
- The enabler and disabler context manager called `tracing`.
- The collector interface used to gather the trace events called
  `TracingCollector.

This POC assumes that the caller is the last responsible implementing
a context that isolates calling to traces that belong to different
tasks.

Missing pices in this PoC:

- Allow many collectors at a time.
- The exceptions have to be propagated at tracing level? perhaps, what
  should we do when there is an exception and we have to call to
`dns_request_end` ? right now a `None` is passed as argument
This trace will help us to measure the cost of having a tracing system
taking into account that this trace does not entail any IO operation
under the hood.
@pfreixes
Copy link
Contributor Author

Closing this PR, I'm more interested in open one the MR once #138 gets merged, so not having this full of useless commits. Also, I would prefer to make it in a specific branch for tracing.

@pfreixes pfreixes closed this May 22, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant