Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parallel table proxy #186

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open

Parallel table proxy #186

wants to merge 15 commits into from

Commits on Feb 21, 2022

  1. Try figuring out how to integrate symlinks with daskms.

    @JSKenyon
    JSKenyon committed Feb 21, 2022
    Configuration menu
    Copy the full SHA
    81d8d08 View commit details
    Browse the repository at this point in the history

  2. Add WIP - seems to work for threaded case. Processes and distributed … 

    …need some investigation.
    @JSKenyon
    JSKenyon committed Feb 21, 2022
    Configuration menu
    Copy the full SHA
    2c6fb4c View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2022

  1. Commit changes - still not entirely satisfactory.

    @JSKenyon
    JSKenyon committed Feb 22, 2022
    Configuration menu
    Copy the full SHA
    9d5e9dd View commit details
    Browse the repository at this point in the history

  2. Change approach - this is more promising, non-invasive, more intuitive.

    @JSKenyon
    JSKenyon committed Feb 22, 2022
    Configuration menu
    Copy the full SHA
    bca8956 View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2022

  1. Add finalizer to tidy up when a parallel table is GCed.

    @JSKenyon
    JSKenyon committed Feb 23, 2022
    Configuration menu
    Copy the full SHA
    bf40c65 View commit details
    Browse the repository at this point in the history

  2. Basic sort of working (not in all cases) parallel table implementation.

    @JSKenyon
    JSKenyon committed Feb 23, 2022
    Configuration menu
    Copy the full SHA
    cdd2d3f View commit details
    Browse the repository at this point in the history

  3. Add reduce method to allow for pickling of Parallel table objects.

    @JSKenyon
    JSKenyon committed Feb 23, 2022
    3 Configuration menu
    Copy the full SHA
    5cad3da View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2022

  1. No need for symlinks with Ben's PR.

    @JSKenyon
    JSKenyon committed Feb 24, 2022
    Configuration menu
    Copy the full SHA
    c043dcd View commit details
    Browse the repository at this point in the history

  2. Remove internal thread pool using a Dummy. PoC.

    @JSKenyon
    JSKenyon committed Feb 24, 2022
    Configuration menu
    Copy the full SHA
    e174581 View commit details
    Browse the repository at this point in the history

  3. Minor changes before embarking on rework.

    @JSKenyon
    JSKenyon committed Feb 24, 2022
    Configuration menu
    Copy the full SHA
    3a2e60a View commit details
    Browse the repository at this point in the history

  4. Tidy up.

    @JSKenyon
    JSKenyon committed Feb 24, 2022
    Configuration menu
    Copy the full SHA
    c8ff7db View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2022

  1. First and very ugly stab at a ParallelTableProxy.

    @JSKenyon
    JSKenyon committed Feb 25, 2022
    Configuration menu
    Copy the full SHA
    612f349 View commit details
    Browse the repository at this point in the history

  2. Ensure futures block until they are finished executing. Simplify Tabl… 

    …eCache.
    @JSKenyon
    JSKenyon committed Feb 25, 2022
    Configuration menu
    Copy the full SHA
    d04b5f3 View commit details
    Browse the repository at this point in the history

  3. Fix __reduce__ and simplify.

    @JSKenyon
    JSKenyon committed Feb 25, 2022
    Configuration menu
    Copy the full SHA
    99c72fb View commit details
    Browse the repository at this point in the history

  4. Replace TableCache with TIDCache, equivalent but generic.

    @JSKenyon
    JSKenyon committed Feb 25, 2022
    Configuration menu
    Copy the full SHA
    45e5b66 View commit details
    Browse the repository at this point in the history