Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature benchmarking #5

Merged
merged 48 commits into from Jun 12, 2019
Merged

Feature benchmarking #5

merged 48 commits into from Jun 12, 2019

Conversation

mansenfranzen
Copy link
Owner

Add benchmark utilities for pandas, dask and spark wranglers.

@mansenfranzen
Add sizeof pretty printer. Add align_values and align_width to
075070c 
`enumeration`.
@mansenfranzen
Add helper module.
4471020
@mansenfranzen
Add single pandas dataframe to exception of iterables.
c8ec9b8
@mansenfranzen
Replace + operator with add method following pandas/numpy warning.
8997f91
@mansenfranzen
Pass dict instead of items to print parameters correctly.
dc4c599
@mansenfranzen
Add exceptions module.
f064839
@mansenfranzen
Add benchmark module.
82929d1
@mansenfranzen
Comment src to avoid duplicated coverage.
24ae1c1
@mansenfranzen
Add memory_profiler to testing.
132ad48
@mansenfranzen
Add more expressive doc strings. Add tests.
f800ec6
@mansenfranzen
Add and adjust tests for enumeration and sizeof.
ebfb04e
@mansenfranzen
Add TimeProfiler and PandasTimeProfiler with tests to benchmark
e8a8a29 
module.
@mansenfranzen
Improve doc strings. Simplify attributes of PandasMemoryProfiler. Add
8d6c495 
`timings` property to `TimeProfiler.`
@mansenfranzen
Add dask base wrangler.
1ba767f
@mansenfranzen
Add spark base wrangler.
dea81bd
@mansenfranzen
Fix typo.
533685e
@mansenfranzen
Add custom report method for TimeProfiler. Add SparkTimeProfiler.
15d0952
@mansenfranzen
Add DaskTimeProfiler.
9576941
@mansenfranzen
Add dask setup to tox.
4c1bb91
@mansenfranzen
Add dask for TravisCI.
ff96663
@mansenfranzen
Add pretty_time_duration. Rename sizeof to pretty_file_size.
a725f58
@mansenfranzen
Move spark benchmark utility into spark subpackage.
a838161
@mansenfranzen
Reuse get_param_names helper function.
3e6b6ee
@mansenfranzen
Refactor BaseProfiler, MemoryProfiler and TimeProfiler to use
63723e1 
common interface for measurements. Remove wrangler specific benchmark
functions.
@mansenfranzen
Move spark fixture into spark subpackage.
07b4d10
@mansenfranzen
Move spark environment tests into spark subpackage.
c8da336
@mansenfranzen
Move SparkWrangler tests into spark subpackage.
cdfb47e
@mansenfranzen
Move pandas wrangler benchmark functions into pandas subpackage.
1703927
@mansenfranzen
Move dask benchmarks into dask subpackage.
fbcd573
@mansenfranzen
Add get_param_names helper function.
a05738f
@mansenfranzen
Add pytestmarks and remove wrong spark fixture.
bb0a7a1
@mansenfranzen
Refactor tests to use create_wrangler fixture. Add sleeps for memory
6e8b311 
profiling.
@mansenfranzen
Add sleeps for memory profiling.
a2b4dfc
@mansenfranzen
Allow memory usage tests to fail due to non deterministic memory
4bf9faa 
management.
@mansenfranzen
Add assertion for number of runs.
d14a5ff
@mansenfranzen
Remove note about memory consumption of child processes.
0927a2a
@mansenfranzen
Add DaskBaseProfiler and DaskMemoryProfiler.
b28f6db
@mansenfranzen
Doc string improvements.
01aa643
@mansenfranzen
Doc string improvement.
81a1aea
@mansenfranzen
Add SparkBaseProfiler. Add cache_input parameter. Add
347cb6a 
SparkMemoryProfiler template without current implementation.
@mansenfranzen
Remove unused cached_property.
358163d
@mansenfranzen
Remove obsolete commentaries.
1c910fb
@mansenfranzen
Allow failing of memory test due to non deterministic behaviour.
cd1ac61
@mansenfranzen
Remove SparkMemoryProfiler.
e9b803d
@mansenfranzen
Add test for caching/uncaching.
a838114
@mansenfranzen mansenfranzen added this to the 0.1.0 Release milestone Jun 5, 2019
@mansenfranzen mansenfranzen added the new feature Add a new feature label Jun 5, 2019
@mansenfranzen mansenfranzen merged commit 0cddd33 into master Jun 12, 2019
@mansenfranzen mansenfranzen deleted the feature_benchmarking branch June 22, 2019 13:34
@mansenfranzen mansenfranzen mentioned this pull request Jul 15, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
new feature Add a new feature
Projects
None yet
Milestone
0.1.0 Release
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@mansenfranzen