Speed up columns slices #900
Conversation
|
🚀 Deployed on https://deploy-preview-900--etna-docs.netlify.app |
Codecov Report
@@ Coverage Diff @@
## master #900 +/- ##
==========================================
+ Coverage 84.95% 84.99% +0.03%
==========================================
Files 133 133
Lines 7593 7611 +18
==========================================
+ Hits 6451 6469 +18
Misses 1142 1142
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
.github/workflows/test.yml
Outdated
| virtualenvs-create: true | ||
| virtualenvs-in-project: true | ||
|
|
||
| - name: Load cached venv |
There was a problem hiding this comment.
We shouldn't use cache there -- it could be corrupted.
| path: .venv | ||
| key: venv-${{ runner.os }}-3.8-${{ hashFiles('**/poetry.lock') }} | ||
|
|
||
| - name: Install dependencies |
There was a problem hiding this comment.
As a result of caching we will run the same test case without pandas installing.
.github/workflows/test.yml
Outdated
| - name: Install dependencies | ||
| if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true' | ||
| run: | | ||
| poetry add "pandas${{ matrix.pandas-version }}" |
There was a problem hiding this comment.
I guess installing via poetry add will lead to long dependency resolving.
We should install it via pip after poetry install.
make_bug.py
Outdated
| @@ -0,0 +1,57 @@ | |||
| import numpy as np | |||
There was a problem hiding this comment.
Hm, what is the purpose of this script?
etna/transforms/math/statistics.py
Outdated
| @@ -73,7 +73,8 @@ def transform(self, df: pd.DataFrame) -> pd.DataFrame: | |||
| history = self.seasonality * self.window if self.window != -1 else len(df) | |||
| segments = sorted(df.columns.get_level_values("segment").unique()) | |||
|
|
|||
| x = df.loc[pd.IndexSlice[:], pd.IndexSlice[segments, self.in_column]].values[::-1] | |||
| df_slice = df.loc[pd.IndexSlice[:], pd.IndexSlice[:, self.in_column]].sort_index(axis=1) | |||
There was a problem hiding this comment.
I guess, we don't need pd.IndexSlice[:]
|
I added report with benchmark into |
Before submitting (must do checklist)
Proposed Changes
Look #775.
Closing issues
Closes #775.