New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix the problem #299 in terms of using tqdm in pandas #524
Conversation
Codecov Report
@@ Coverage Diff @@
## master #524 +/- ##
========================================
+ Coverage 99.1% 99.4% +0.3%
========================================
Files 8 8
Lines 668 674 +6
Branches 118 118
========================================
+ Hits 662 670 +8
+ Misses 4 3 -1
+ Partials 2 1 -1 |
I add new tests, so now, pandas.Series, pandas.Dataframe, and groupby cases are all covered. Not cover pandas.Panel because it is going to be removed. http://pandas.pydata.org/pandas-docs/version/0.20/whatsnew.html#whatsnew-0200-api-breaking-deprecate-panel |
Transmitted to `df.apply()`. | ||
""" | ||
|
||
# Precompute total iterations | ||
total = getattr(df, 'ngroups', None) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this ngroups
is an inner attribute of pandas. It may be changed in future, but seems not happen in near future or never. So let's keep it so far.
For perfection, I will suggest to use one of pandas public apis to get group size in future. So far, it is fine.
ff63a31
to
072c2fc
Compare
- closes tqdm#454 - TODO: might want to autodetect semi-unicode support and fallback to ascii?
Fix total computation for pandas apply
cdf3bc3
to
0c56c08
Compare
total
when usingprogress_apply
and other functions in pandas.#299 partially achieved its goal. With forbidding
*args
, one can get correct axis now, and use it to get thetotal
(as number of iterations inprogress_apply
)However, pandas sometimes runs several more iterations to optimise itself. For example, in
dataframe.progress_apply(func)
.func
is anumpy.ufunc
or is very slow, then it will be executedtotal
times.total+1
ortotal+shape[axis0]
times.I think there is no need to adjust such particular pandas implementation, so in the wrapper function, I write
if
total
is achieved,tbar
will not be updated.