Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Black magic #46

Closed
wants to merge 40 commits into from
Closed

Black magic #46

wants to merge 40 commits into from

Conversation

coldfix
Copy link
Contributor

@coldfix coldfix commented Feb 24, 2014

I finally managed to create a reasonable version that uses black_magic.decorator to create nice function objects that preserve parameter names and default arguments even on python2.

Possible issues to take into consideration:

  • performance possibly goes down by a bit (don't know though)
  • usability with non-function callables: The obsub module was never thought to decorate anything other than basic functions, but I am not sure whether it actually worked anyway (2 lazy to try atm). With black_magic this issue is somewhat more contrieved. For some objects it will actually get easier, I believe. For others (like functools.partial) it will not be possible.
  • this introduces a non-standard dependency, on python2 even two (black-magic + funcsigs), so the usability as a drop-in decreases.
  • although I have tests for black-magic, there might of course still be some issues and its code quality isn't the best, currently

The boundevent is now cached as attribute of the instance and never gets
garbage collected before its owner object.

The list of event handlers becomes an instance variable of the
boundevent object. This is reasonable since it is (and always was) the
boundevent object's responsibility to manage the event handlers.

Furthermore, this makes the boundevent object more self-contained
(independent of the instance it is attached to), which will make it
usable on its own.
Together with the previous commit (making the boundevent class more
independent), this will allow the following useful reinterpretation:

- boundevent becomes an independent event class (the main event class
  even) that is fully capable exist without being an attribute within
  another class.

- the event class is just a helper to dynamically associate boundevent
  objects to objects. It can be viewed as a form of a '@cachedproperty'
  descriptor utility.
This enables boundevent to use any externally created list (but it will
still add and remove handlers to it!).
The caching of the boundevent instance will have the effect that it
the reference count of the object which it is tied to can never reach
zero. It can be cleaned up only by the garbage collector. This may be
very undesirable (and unexpected) in some circumstances.

Instead, we can use the event_handler argument of boundevent to create
boundevents that are specific to some instance.
Note, that this also makes the overload of __set__ unnecessary.
Conflicts:
	obsub.py
	test/test_core.py
	test/test_weakref.py
The new 'signal' can be used independently and deserves a better name
than 'boundevent'!
This makes the code much easier to understand. Furthermore, the closure
namespace that contains the function and event_handlers makes underscore
prefixes for "private" variables unnecessary.

Furthermore, this will make it easy to use 'black_magic.decorator' with
this module.
Hey, since we are writing decorator code here, why not do stuff that is
only for really cool people?
@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 7f3a13d on coldfix:black_magic into acf8d4f on aepsil0n:master.

The docstring of the wrapper closure is hidden from the outside world,
so it makes no sense.

Furthermore, the @wraps decorator now takes care to ensure the correct
call signature, contrary to what the removed comment wants to make us
believe.
For what it's worth this makes class-based access very slightly faster,
since black_magic.decorator.wraps has to be invoked only once per
attribute access. Huge use case... ;)
@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 73dd4d6 on coldfix:black_magic into acf8d4f on aepsil0n:master.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 3c32e60 on coldfix:black_magic into acf8d4f on aepsil0n:master.

Okay, there are two relevant changes:

- remove the partial() stuff
- remove doctests for memory management

These are just confusing to read and a user wouldn't even get the idea
that these fail unless explicitly stated - they are not documentation
but tests.
I forgot that every function is a descriptor, which makes it very
convenient to create bound member functions.
The event descriptor object is usually not inspected and doesn't need to
be updated. Even if it were inspected, it is questionable to update its
documentation.
@milibopp milibopp added this to the v0.3 milestone Feb 24, 2014
@milibopp
Copy link
Owner

Great work! This is a major rewrite, so it'll take some time to review.

Regarding the black_magic dependency: it does preserve function signatures, so this is probably important functionality beyond documentation, is it not? Otherwise, could you reasonably strip it out of this PR? I'm not saying that you should, just trying to assess our options here.

@milibopp milibopp added refactor and removed feature labels Feb 24, 2014
This allows the obsub module to be used as a drop-in more easily.

On the other hand this can have unexpected results when using functions
with default arguments without having black-magic installed:

You need to make sure, that all parameters that have default values in
your original event function have default values in all handlers as
well!
@coldfix
Copy link
Contributor Author

coldfix commented Feb 24, 2014

It is possible to exchange black_magic.decorator.wraps for functools.wraps. This makes the returned handlers less nice for help() and drops support for default parameters. This should allow obsub to be used as a drop-in more easily. I have provided a commit that implements this. Coverage will probably go down a bit, since the ImportError control path isn't executed.

@coldfix
Copy link
Contributor Author

coldfix commented Feb 24, 2014

Btw., c5b00ff (Use function.get instead of black_magic.partial) should restore approximately the original speed but makes it much harder to decorate anything but basic functions. So, if we care about covering many use cases, we should probably revert this.

@coveralls
Copy link

Coverage Status

Coverage decreased (-30.19%) when pulling a7a7d24 on coldfix:black_magic into acf8d4f on aepsil0n:master.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling d2af23d on coldfix:black_magic into 29c4714 on aepsil0n:master.

@coldfix
Copy link
Contributor Author

coldfix commented Mar 11, 2014

Finally, I managed to do some rudimentary speed tests.. Turns out, that black_magic.wraps indeed has a pretty strong performance hit.

The code in the latest change may seem a little confusing, but it mostly fixes the performance issue by moving the only wraps call to the contructor. Furthermore, the events are now practically indistinguishable from normal member functions. Also, it should be possible again to decorate some non-function objects.

If you are interested in the benchmark:

from obsub import event
import timeit

def create():
    class X(object):
        @event
        def foo(self, bar=3):
            return bar
    return X

X = create()
x = X()

def invoke():
    x.foo()
    x.foo(2)

def access():
    X.foo

def bench(func, number=10000):
    return timeit.timeit("%s()" % func,
                         setup="from __main__ import %s" % func,
                         number=number)


if __name__ == '__main__':
    print(bench('create'))
    print(bench('invoke'))
    print(bench('access'))

Result on CPython2.7

master a7a7d24 d2af23d
create 0.665 0.661 6.092
invoke 0.253 9.753 0.105
access 0.006 4.574 0.005

This looks similar onCPython2.6 and CPython3.3.

On pypy, however, the optimization doesn't make much difference:

master a7a7d24 d2af23d
create 0.264 0.266 0.3
invoke 0.164 0.164 0.083
access 0.063 0.063 0.072

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling 9baec0c on coldfix:black_magic into 29c4714 on aepsil0n:master.

@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling bfc2339 on coldfix:black_magic into 29c4714 on aepsil0n:master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage remained the same when pulling bfc2339 on coldfix:black_magic into 29c4714 on aepsil0n:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-16.25%) when pulling e707033 on coldfix:black_magic into 29c4714 on aepsil0n:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-19.12%) when pulling 78be84c on coldfix:black_magic into 29c4714 on aepsil0n:master.

In the interest of a more descriptive name that shows the relation to
the standard `event`. Possibly, `event` should be renamed as well. For
example to `instance_event` or `member_event`.
@coveralls
Copy link

Coverage Status

Coverage decreased (-19.12%) when pulling ad7329d on coldfix:black_magic into 29c4714 on aepsil0n:master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-14.44%) when pulling a10516d on coldfix:black_magic into 29c4714 on aepsil0n:master.

The following signatures are removed:

    cls.event.connect(instance, handler)

The following stays valid:

    cls.event.connect(handler)
    inst.event.connect(handler)

The rationale is simplicity: It is easier to remember a single calling
signature with only one argument. In particular the removed signature
might be confusing in terms of parameter order, since the *first*
parameter was optional.

Furthermore, this is easier to handle internally, especially with regard
to upcoming revival of a bound_event class.
@coldfix
Copy link
Contributor Author

coldfix commented Jun 12, 2014

There is a fatal flaw in this branch: Assuming a.foo is an event, a.foo == a.foo will always evaluate to false. This will break code like x.connect(a.foo) followed by x.disconnect(a.foo).

Reason: event.__get__ returns a new copy of a function each time and function comparison is done by identity. Furthermore, in python it is not possible to overload equality of functions (it is not possible to derive from FunctionType/MethodType).

The only possibility I can see to get the desired behaviour, is to revert back to a bound_event class. This in turn, will make automatic help() behave much less nicely, and therefore, the only reason left for the black_magic dependency is the support for default arguments in python2.

The previous approach was returning a new copy of a function from
`event.__get__` each time. Since function comparison is done by identity
this breaks code like `.connect(a.foo)` followed by `.disconnect(a.foo)`
(assuming `a.foo` is an event).

Furthermore, in python it is not possible to overload equality of
functions (it is not possible to derive from FunctionType/MethodType).
Therefore, the only possibility I can see to get the desired behaviour,
is to revert back to a bound_event class.
To enable later access of __get__ like in:

    - foo.bar.__get__(...)
    - Foo.bar.__get__(foo)
This is more flexible than a boolean option _decorate=True.
To show function signature in help strings
@coldfix coldfix closed this Jan 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants