PERF/ENH: np.apply_along_axis -> reduce in moments.py/nanops.py #3185

Closed
jreback opened this Issue Mar 27, 2013 · 1 comment

Comments

Projects
None yet
3 participants
Contributor

jreback commented Mar 27, 2013

http://stackoverflow.com/questions/15652343/how-to-speed-up-pandas-rolling-sum
http://stackoverflow.com/questions/21058333/compute-rolling-maximum-drawdown-of-pandas-series

this numpy recipe might be of iterest to create a strided dimension:

numpy/numpy#31

prob easiest in these cases to create a algos.roll_generic_2d

./pandas/core/frame.py: (this is OK, just a fallback if reduce doesn't work)
 4109 :             result = np.apply_along_axis(func, axis, self.values)
./pandas/core/nanops.py:
  201 :         return np.apply_along_axis(get_median, axis, values)
  235 :             result = np.apply_along_axis(__builtin__.min, apply_ax, values)
  260 :             result = np.apply_along_axis(__builtin__.max, apply_ax, values)
./pandas/core/panel.py:
 1246 :         result = np.apply_along_axis(func, i, self.values)
./pandas/stats/moments.py:
  281 :         result = np.apply_along_axis(calc, axis, values)
  363 :     output = np.apply_along_axis(_ewma, 0, values)
  651 :     result = np.apply_along_axis(f, axis, values)
   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1000003    8.930    0.000    8.930    0.000 {method 'put' of 'numpy.ndarray' objects}
     1000    8.413    0.008   23.225    0.023 shape_base.py:11(apply_along_axis)
  1000000    3.612    0.000    3.612    0.000 {pandas.algos.roll_sum}
  1000000    0.833    0.000    5.277    0.000 moments.py:277(<lambda>)
  1000000    0.628    0.000    4.444    0.000 moments.py:499(call_cython)
  2000000    0.551    0.000    0.551    0.000 {method 'tolist' of 'numpy.ndarray' objects}
        2    0.517    0.259    0.517    0.259 {pandas.algos.take_2d_axis0_float64_float64}

@jreback jreback modified the milestone: 0.15.0, 0.14.0 Feb 15, 2014

jreback referenced this issue Oct 29, 2014

Open

API/ENH: master issue for pd.rolling_apply #8659

1 of 13 tasks complete

@jreback jreback modified the milestone: 0.16.0, 0.17.0 Jan 26, 2015

jreback added the Reshaping label Feb 5, 2015

wesm added the Won't Fix label Sep 29, 2016

Owner

wesm commented Sep 29, 2016

I'm putting this in the "won't fix" category as this will be addressed in low level code during the pandas 2.0 refactor

wesm closed this Sep 29, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment