Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature benchmarking #5

Merged
merged 48 commits into from Jun 12, 2019
Merged

Feature benchmarking #5

merged 48 commits into from Jun 12, 2019

Commits on Mar 21, 2019

  1. Add sizeof pretty printer. Add align_values and align_width to 

    `enumeration`.
    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    075070c View commit details
    Browse the repository at this point in the history

  2. Add helper module.

    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    4471020 View commit details
    Browse the repository at this point in the history

  3. Add single pandas dataframe to exception of iterables.

    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    c8ec9b8 View commit details
    Browse the repository at this point in the history

  4. Replace + operator with add method following pandas/numpy warning.

    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    8997f91 View commit details
    Browse the repository at this point in the history

  5. Pass dict instead of items to print parameters correctly.

    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    dc4c599 View commit details
    Browse the repository at this point in the history

  6. Add exceptions module.

    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    f064839 View commit details
    Browse the repository at this point in the history

  7. Add benchmark module.

    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    82929d1 View commit details
    Browse the repository at this point in the history

  8. Comment src to avoid duplicated coverage.

    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    24ae1c1 View commit details
    Browse the repository at this point in the history

  9. Add memory_profiler to testing.

    @mansenfranzen
    mansenfranzen committed Mar 21, 2019
    Copy the full SHA
    132ad48 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2019

  1. Add more expressive doc strings. Add tests.

    @mansenfranzen
    mansenfranzen committed Mar 22, 2019
    Copy the full SHA
    f800ec6 View commit details
    Browse the repository at this point in the history

  2. Add and adjust tests for enumeration and sizeof.

    @mansenfranzen
    mansenfranzen committed Mar 22, 2019
    Copy the full SHA
    ebfb04e View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2019

  1. Add TimeProfiler and PandasTimeProfiler with tests to benchmark 

    module.
    @mansenfranzen
    mansenfranzen committed Mar 23, 2019
    Copy the full SHA
    e8a8a29 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2019

  1. Improve doc strings. Simplify attributes of PandasMemoryProfiler. Add 

    `timings` property to `TimeProfiler.`
    @mansenfranzen
    mansenfranzen committed Apr 3, 2019
    Copy the full SHA
    8d6c495 View commit details
    Browse the repository at this point in the history

  2. Add dask base wrangler.

    @mansenfranzen
    mansenfranzen committed Apr 3, 2019
    Copy the full SHA
    1ba767f View commit details
    Browse the repository at this point in the history

  3. Add spark base wrangler.

    @mansenfranzen
    mansenfranzen committed Apr 3, 2019
    Copy the full SHA
    dea81bd View commit details
    Browse the repository at this point in the history

  4. Fix typo.

    @mansenfranzen
    mansenfranzen committed Apr 3, 2019
    Copy the full SHA
    533685e View commit details
    Browse the repository at this point in the history

  5. Add custom report method for TimeProfiler. Add SparkTimeProfiler.

    @mansenfranzen
    mansenfranzen committed Apr 3, 2019
    Copy the full SHA
    15d0952 View commit details
    Browse the repository at this point in the history

  6. Add DaskTimeProfiler.

    @mansenfranzen
    mansenfranzen committed Apr 3, 2019
    Copy the full SHA
    9576941 View commit details
    Browse the repository at this point in the history

  7. Add dask setup to tox.

    @mansenfranzen
    mansenfranzen committed Apr 3, 2019
    Copy the full SHA
    4c1bb91 View commit details
    Browse the repository at this point in the history

  8. Add dask for TravisCI.

    @mansenfranzen
    mansenfranzen committed Apr 3, 2019
    Copy the full SHA
    ff96663 View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2019

  1. Add pretty_time_duration. Rename sizeof to pretty_file_size.

    @mansenfranzen
    mansenfranzen committed Apr 20, 2019
    Copy the full SHA
    a725f58 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2019

  1. Move spark benchmark utility into spark subpackage.

    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    a838161 View commit details
    Browse the repository at this point in the history

  2. Reuse get_param_names helper function.

    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    3e6b6ee View commit details
    Browse the repository at this point in the history

  3. Refactor BaseProfiler, MemoryProfiler and TimeProfiler to use 

    common interface for measurements. Remove wrangler specific benchmark
functions.
    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    63723e1 View commit details
    Browse the repository at this point in the history

  4. Move spark fixture into spark subpackage.

    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    07b4d10 View commit details
    Browse the repository at this point in the history

  5. Move spark environment tests into spark subpackage.

    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    c8da336 View commit details
    Browse the repository at this point in the history

  6. Move SparkWrangler tests into spark subpackage.

    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    cdfb47e View commit details
    Browse the repository at this point in the history

  7. Move pandas wrangler benchmark functions into pandas subpackage.

    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    1703927 View commit details
    Browse the repository at this point in the history

  8. Move dask benchmarks into dask subpackage.

    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    fbcd573 View commit details
    Browse the repository at this point in the history

  9. Add get_param_names helper function.

    @mansenfranzen
    mansenfranzen committed Apr 26, 2019
    Copy the full SHA
    a05738f View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2019

  1. Add pytestmarks and remove wrong spark fixture.

    @mansenfranzen
    mansenfranzen committed Apr 27, 2019
    Copy the full SHA
    bb0a7a1 View commit details
    Browse the repository at this point in the history

  2. Refactor tests to use create_wrangler fixture. Add sleeps for memory 

    profiling.
    @mansenfranzen
    mansenfranzen committed Apr 27, 2019
    Copy the full SHA
    6e8b311 View commit details
    Browse the repository at this point in the history

  3. Add sleeps for memory profiling.

    @mansenfranzen
    mansenfranzen committed Apr 27, 2019
    Copy the full SHA
    a2b4dfc View commit details
    Browse the repository at this point in the history

  4. Allow memory usage tests to fail due to non deterministic memory 

    management.
    @mansenfranzen
    mansenfranzen committed Apr 27, 2019
    Copy the full SHA
    4bf9faa View commit details
    Browse the repository at this point in the history

Commits on May 1, 2019

  1. Add assertion for number of runs.

    @mansenfranzen
    mansenfranzen committed May 1, 2019
    Copy the full SHA
    d14a5ff View commit details
    Browse the repository at this point in the history

  2. Remove note about memory consumption of child processes.

    @mansenfranzen
    mansenfranzen committed May 1, 2019
    Copy the full SHA
    0927a2a View commit details
    Browse the repository at this point in the history

  3. Add DaskBaseProfiler and DaskMemoryProfiler.

    @mansenfranzen
    mansenfranzen committed May 1, 2019
    Copy the full SHA
    b28f6db View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2019

  1. Doc string improvements.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    01aa643 View commit details
    Browse the repository at this point in the history

  2. Doc string improvement.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    81a1aea View commit details
    Browse the repository at this point in the history

  3. Add SparkBaseProfiler. Add cache_input parameter. Add 

    SparkMemoryProfiler template without current implementation.
    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    347cb6a View commit details
    Browse the repository at this point in the history

  4. Remove unused cached_property.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    358163d View commit details
    Browse the repository at this point in the history

  5. Remove obsolete commentaries.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    1c910fb View commit details
    Browse the repository at this point in the history

  6. Allow failing of memory test due to non deterministic behaviour.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    cd1ac61 View commit details
    Browse the repository at this point in the history

  7. Remove SparkMemoryProfiler.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    e9b803d View commit details
    Browse the repository at this point in the history

  8. Add test for caching/uncaching.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    a838114 View commit details
    Browse the repository at this point in the history

  9. Update changelog.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    e94fab2 View commit details
    Browse the repository at this point in the history

  10. Add args and kwargs to fit, transform and fit_transform to allow 

    subclasses to implement varying positional and keyword arguments
(linters).
    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    9d0d363 View commit details
    Browse the repository at this point in the history

  11. Remove unnecessary else clause.

    @mansenfranzen
    mansenfranzen committed Jun 5, 2019
    Copy the full SHA
    559fb46 View commit details
    Browse the repository at this point in the history