Optimize init validators.

[Tinche]

OK, here's an idea for an optimization, since we're into that stuff now :)

Assume the attrs class attr.make_class('C', {'x': attr.ib(validator=validators.instance_of(int)), 'y': attr.ib(), 'z': attr.ib()}).

The master will generate an init like this:

def __init__(self, x, y, z):
    self.x = x
    self.y = y
    self.z = z
    validate(self)

so I was thinking how to improve this. Ideas in sequence:

1. validate() calls fields(). Can we avoid the function call by using the __attrs_attrs__ directly? validate() is a public facing API, so alas, no.
2. OK, so what if we don't even call validate(), and so avoid that call too? Just put the internal validation code in the generated __init__.
3. Now that the code is here, speed things up more by generating smart __init__ validation code. We don't have to skip attributes without validators every time, we can just not generate the code in the first place.
4. We can put things we need in globals directly, avoid some dict lookups.

So now, the generated __init__ code looks like this:

def __init__(self, x, y, z):
    self.x = x
    self.y = y
    self.z = z
    if _config._run_validators is False:
        return
    __attr_validator_x(self, __attr_x, self.x)

Before:

$ pyperf timeit --rigorous -g -s "import attr; from attr import validators; C = attr.make_class('C', {'x': attr.ib(validator=validators.instance_of(int)), 'y': attr.ib(), 'z': attr.ib()})" "C(1, 2, 3)"
........................................
2.73 us:  2 ########
2.76 us:  9 #####################################
2.78 us: 11 ##############################################
2.81 us: 13 ######################################################
2.83 us: 12 ##################################################
2.86 us: 19 ###############################################################################
2.88 us: 12 ##################################################
2.91 us: 11 ##############################################
2.93 us: 11 ##############################################
2.96 us:  7 #############################
2.98 us:  4 #################
3.01 us:  2 ########
3.03 us:  0 |
3.06 us:  2 ########
3.08 us:  1 ####
3.11 us:  0 |
3.13 us:  1 ####
3.16 us:  1 ####
3.18 us:  1 ####
3.21 us:  0 |
3.23 us:  1 ####

Median +- std dev: 2.87 us +- 0.09 us

After:

$ pyperf timeit --rigorous -g -s "import attr; from attr import validators; C = attr.make_class('C', {'x': attr.ib(validator=validators.instance_of(int)), 'y': attr.ib(), 'z': attr.ib()})" "C(1, 2, 3)"
........................................
1.36 us:  2 ###########
1.36 us:  2 ###########
1.37 us:  7 ########################################
1.38 us:  5 ############################
1.38 us:  6 ##################################
1.39 us:  8 #############################################
1.40 us: 10 ########################################################
1.40 us: 11 ##############################################################
1.41 us: 13 #########################################################################
1.42 us: 14 ###############################################################################
1.42 us:  7 ########################################
1.43 us:  7 ########################################
1.44 us:  9 ###################################################
1.45 us:  8 #############################################
1.45 us:  4 #######################
1.46 us:  0 |
1.47 us:  0 |
1.47 us:  4 #######################
1.48 us:  1 ######
1.49 us:  1 ######
1.49 us:  1 ######

Median +- std dev: 1.41 us +- 0.03 us

Even the case of no validators is faster since I omit the call to validate altogether. (~1%)

Complexity for performance. Worth it?

[codecov-io]

Current coverage is 100% (diff: 100%)

Merging #64 into master will not change coverage

@@           master   #64   diff @@
===================================
  Files           8     8          
  Lines         384   393     +9   
  Methods         0     0          
  Messages        0     0          
  Branches       89    90     +1   
===================================
+ Hits          384   393     +9   
  Misses          0     0          
  Partials        0     0          

Powered by Codecov. Last update cfa6d2e...c9da161

[Tinche]

If we're up for this, converters can be given the same treatment.

[hynek]

I’m not gonna lie, this looks scary. 😱 But 50% faster is kind of sweet. 🙈

How would a non-public _validate that doesn’t do the safety stuff work out performance-wise?

[Tinche]

I can check later today.

I really like the fact that at class definition time we actually know which attributes have validators, and we can generate an __init__ calling exactly those validators. It somehow feels it'd be a waste to not use this information, and it plays into our spiel of attrs generating classes better than or equal than you yourself can write.

(I'll fix the coverage too.)

[hynek]

Yeah you’re right. I’m just being lazy about rebasing immutable (IOW: don’t check; but do fix :)).

[Tinche]

Dammit, only now I see apparently limiting the number of generated attributes breaks tests!?!

Will take a closer look when I have some time.

[hynek]

It’s hypothesis being too slow again. :|

[hynek]

this needs a rebase :-P

[hynek]

also maybe the the average for the number of attributes to a low single digit please. :) that should help with the slownesses…

[Tinche]

Pingety ping.

[hynek]

We’re all going to hell but 16.1 is gonna be majestic.