New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Partition #2
Partition #2
Conversation
Conflicts: cytoolz/itertoolz/core.pyx cytoolz/itertoolz/tests/test_core.py
@eriknw do you get notifications on this repository? |
Yeah, I get notifications.
I do this for |
Thanks for the tip. That worked wonderfully. Unfortunately my custom code is slower than the In [1]: import cytoolz
In [2]: import toolz
In [3]: data = range(1000)
In [4]: timeit list(toolz.partition(5, data))
100000 loops, best of 3: 12.9 µs per loop
In [5]: timeit list(cytoolz.partition(5, data))
100000 loops, best of 3: 16.2 µs per loop I'll revert or (if I get adventurous), look into line profiling (I remember being impressed with cython tools for this). |
Yeah, I wasn't sure how to best go about def __next__(self):
cdef tuple result
cdef int i
cdef object val
result = PyTuple_New(self.n)
for i in range(self.n): # efficient loop if `i` is cdef integer type
val = next(self.seq)
Py_INCREF(val)
PyTuple_SET_ITEM(result, i, val)
return result I actually haven't done much with profile yet. I should probably try it out. Annotations-- |
No dice. I'll rebase out the recent commits and stick with the solution in mrocklin@4c29e14 In [1]: import toolz
In [2]: import cytoolz
In [3]: data = range(1000)
In [4]: timeit list(toolz.partition(5, data))
100000 loops, best of 3: 12.5 µs per loop
In [5]: timeit list(cytoolz.partition(5, data))
100000 loops, best of 3: 14.5 µs per loop |
Awesome (yes, I think it's awesome when Does Another pointer: Nice job getting your feet wet by the way. I can't wait to flatten the directory structure in |
I played around with this some more, and, wow, the We'll need to add I'm going to go ahead and flatten |
Yeah, I was confused the most recent time that I did timings, |
I wouldn't be surprised. I don't see where there is 30% to be gained. Heh, you chose a heck of a function with which to get your feet wet. |
Closing. I just added the stock version after flattening the directory structure. |
OK, here is a "getting the feet wet" pull request.
At first I just relied on
zip
andizip_longest
, as we do in toolz. This yielded a 30% speedup.Then I tried making an iterator class and ran into weird issues. I get a fair number of stackoverflows. In particular I get some strange integer wraparound at 256. Does Cython use one byte integers somehow? Anyway, if you have any thoughts on debugging this I'd be interested.
Fail case on my machine
oddly this works
but this doesn't