Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pandas rewrite Performance optimizations #136

Merged
merged 24 commits into from
Nov 11, 2020
Merged

Pandas rewrite Performance optimizations #136

merged 24 commits into from
Nov 11, 2020

Commits on Sep 14, 2020

  1. basic scatterplot experiments

    @dorisjlee
    dorisjlee committed Sep 14, 2020
    Configuration menu
    Copy the full SHA
    1db0846 View commit details
    Browse the repository at this point in the history

Commits on Sep 22, 2020

  1. Merge remote-tracking branch 'upstream/master'

    @dorisjlee
    dorisjlee committed Sep 22, 2020
    Configuration menu
    Copy the full SHA
    d26f378 View commit details
    Browse the repository at this point in the history

Commits on Sep 23, 2020

  1. experiment results with manually binned heatmaps

    @dorisjlee
    dorisjlee committed Sep 23, 2020
    Configuration menu
    Copy the full SHA
    4e8d9e7 View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2020

  1. experiment result

    @dorisjlee
    dorisjlee committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    15adb77 View commit details
    Browse the repository at this point in the history

  2. incorporated heatmap code into executor and renderer

    @dorisjlee
    dorisjlee committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    48633d4 View commit details
    Browse the repository at this point in the history

  3. additional experiments to evaluate scatter v.s. heatmap performance

    @dorisjlee
    dorisjlee committed Sep 24, 2020
    Configuration menu
    Copy the full SHA
    dcc8ece View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2020

  1. experiment based on real estate and airbnb data

    @dorisjlee
    dorisjlee committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    b7e0f60 View commit details
    Browse the repository at this point in the history

  2. modified general sampling criteria, suppress SettingWithCopyWarning s… 

    …temming from groupby .agg (lux-org#93)
    @dorisjlee
    dorisjlee committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    de07933 View commit details
    Browse the repository at this point in the history

  3. decrease sampling parameter

    @dorisjlee
    dorisjlee committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    3d59c5f View commit details
    Browse the repository at this point in the history

  4. change sampling strategy (above threshold keep 3/4 of data)

    @dorisjlee
    dorisjlee committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    8ed7010 View commit details
    Browse the repository at this point in the history

  5. remove experiment dir

    @dorisjlee
    dorisjlee committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    520885c View commit details
    Browse the repository at this point in the history

  6. modified performance param

    @dorisjlee
    dorisjlee committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    3de5bb7 View commit details
    Browse the repository at this point in the history

  7. merge

    @dorisjlee
    dorisjlee committed Sep 25, 2020
    Configuration menu
    Copy the full SHA
    7532b80 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2020

  1. Merge remote-tracking branch 'upstream/master'

    @dorisjlee
    dorisjlee committed Oct 16, 2020
    Configuration menu
    Copy the full SHA
    60bbed8 View commit details
    Browse the repository at this point in the history

  2. enforce lux-widget minimum version

    @dorisjlee
    dorisjlee committed Oct 16, 2020
    Configuration menu
    Copy the full SHA
    137abf5 View commit details
    Browse the repository at this point in the history

  3. update requirement.txt

    @dorisjlee
    dorisjlee committed Oct 16, 2020
    Configuration menu
    Copy the full SHA
    33bc23a View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2020

  1. merged upstream/master

    @dorisjlee
    dorisjlee committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    9138a5a View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/master'

    @dorisjlee
    dorisjlee committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    3545b50 View commit details
    Browse the repository at this point in the history

  3. testing out modin (Recursion error)

    @dorisjlee
    dorisjlee committed Oct 25, 2020
    Configuration menu
    Copy the full SHA
    593e03e View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2020

  1. merged fixed conflict in PandasExecutor (revert back to master type d… 

    …etection)
    @dorisjlee
    dorisjlee committed Nov 6, 2020
    Configuration menu
    Copy the full SHA
    098e7f3 View commit details
    Browse the repository at this point in the history

  2. create modin executor, all else in sync with master changes

    @dorisjlee
    dorisjlee committed Nov 6, 2020
    Configuration menu
    Copy the full SHA
    3ec1193 View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2020

  1. rewrote .loc with column reference, speed up by 100x

    @dorisjlee
    dorisjlee committed Nov 11, 2020
    Configuration menu
    Copy the full SHA
    b6a7dd6 View commit details
    Browse the repository at this point in the history

  2. replace agg("count") with .count() --> ~0.1ms speedup

    @dorisjlee
    dorisjlee committed Nov 11, 2020
    Configuration menu
    Copy the full SHA
    08d167e View commit details
    Browse the repository at this point in the history

  3. run black

    @dorisjlee
    dorisjlee committed Nov 11, 2020
    Configuration menu
    Copy the full SHA
    665be02 View commit details
    Browse the repository at this point in the history