Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(feat): Support for pandas ExtensionArray #8723

Merged
merged 101 commits into from
Apr 18, 2024
Merged

(feat): Support for pandas ExtensionArray #8723

merged 101 commits into from
Apr 18, 2024

Commits on Feb 2, 2024

  1. (feat): first pass supporting extension arrays

    @ilan-gold
    ilan-gold committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    b2712f1 View commit details
    Browse the repository at this point in the history

  2. (feat): categorical tests + functionality

    @ilan-gold
    ilan-gold committed Feb 2, 2024
    Configuration menu
    Copy the full SHA
    47bddd2 View commit details
    Browse the repository at this point in the history

Commits on Feb 5, 2024

  1. (feat): use multiple dispatch for unimplemented ops

    @ilan-gold
    ilan-gold committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    dc8b788 View commit details
    Browse the repository at this point in the history

  2. (feat): implement (not really) broadcasting

    @ilan-gold
    ilan-gold committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    75524c8 View commit details
    Browse the repository at this point in the history

  3. (chore): add more groupby tests

    @ilan-gold
    ilan-gold committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    c9ab452 View commit details
    Browse the repository at this point in the history

  4. (fix): fix more groupby incompatibility

    @ilan-gold
    ilan-gold committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    1f3d0fa View commit details
    Browse the repository at this point in the history

  5. (bug): fix unused categories

    @ilan-gold
    ilan-gold committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    8a70e3c View commit details
    Browse the repository at this point in the history

  6. (chore): refactor dispatched methods + tests

    @ilan-gold
    ilan-gold committed Feb 5, 2024
    Configuration menu
    Copy the full SHA
    f5a6505 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. (fix): shared type should check for extension arrays first and then f… 

    …all back to numpy
    @ilan-gold
    ilan-gold committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    08a4feb View commit details
    Browse the repository at this point in the history

  2. (refactor): tests moved

    @ilan-gold
    ilan-gold committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    d5b218b View commit details
    Browse the repository at this point in the history

  3. (chore): more higher level tests

    @ilan-gold
    ilan-gold committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    00256fa View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2024

  1. (feat): to/from dataframe

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    b7ddbd6 View commit details
    Browse the repository at this point in the history

  2. (chore): check for plum import

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    a165851 View commit details
    Browse the repository at this point in the history

  3. (fix): __setitem__/__getitem__

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    a826edd View commit details
    Browse the repository at this point in the history

  4. (chore): disallow stacking

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    fde19ea View commit details
    Browse the repository at this point in the history

  5. (fix): pyproject.toml

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    4c55707 View commit details
    Browse the repository at this point in the history

  6. (fix): as_shared_type fix

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    58ba17d View commit details
    Browse the repository at this point in the history

  7. (chore): add variable tests

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    a255310 View commit details
    Browse the repository at this point in the history

  8. (fix): dask + categoricals

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    4e78b7e View commit details
    Browse the repository at this point in the history

  9. (chore): notes/docs

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    d9cedf5 View commit details
    Browse the repository at this point in the history

  10. (chore): remove old testing file

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    426664d View commit details
    Browse the repository at this point in the history

  11. (chore): remove ocmmented out code

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    22ca77d View commit details
    Browse the repository at this point in the history

  12. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    f32cfdf View commit details
    Browse the repository at this point in the history

  13. (fix): import plum dispatch

    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    60f8927 View commit details
    Browse the repository at this point in the history

  14. Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e… 

    …xtension_arrays
    @ilan-gold
    ilan-gold committed Feb 8, 2024
    Configuration menu
    Copy the full SHA
    ff22d76 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    2153e81 View commit details
    Browse the repository at this point in the history

  2. (refactor): use is_extension_array_dtype as much as possible

    @ilan-gold
    ilan-gold committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    b6d0b31 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e… 

    …xtension_arrays
    @ilan-gold
    ilan-gold committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    d285871 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 9, 2024
    Configuration menu
    Copy the full SHA
    d847277 View commit details
    Browse the repository at this point in the history

Commits on Feb 10, 2024

  1. (refactor): extension_array->array + move to indexing

    @ilan-gold
    ilan-gold committed Feb 10, 2024
    Configuration menu
    Copy the full SHA
    8238c64 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e… 

    …xtension_arrays
    @ilan-gold
    ilan-gold committed Feb 10, 2024
    Configuration menu
    Copy the full SHA
    1260cd4 View commit details
    Browse the repository at this point in the history

  3. (refactor): change order of classes

    @ilan-gold
    ilan-gold committed Feb 10, 2024
    Configuration menu
    Copy the full SHA
    b04ef98 View commit details
    Browse the repository at this point in the history

Commits on Feb 12, 2024

  1. (chore): add small pyarrow test

    @ilan-gold
    ilan-gold committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    b9937bf View commit details
    Browse the repository at this point in the history

  2. (fix): fix some mypy issues

    @ilan-gold
    ilan-gold committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    0bba03f View commit details
    Browse the repository at this point in the history

  3. (fix): don't register unregisterable method

    @ilan-gold
    ilan-gold committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    b714549 View commit details
    Browse the repository at this point in the history

  4. (fix): appease mypy

    @ilan-gold
    ilan-gold committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    a3a678c View commit details
    Browse the repository at this point in the history

  5. (fix): more sensible default implemetations allow most use without `p… 

    …lum`
    @ilan-gold
    ilan-gold committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    e521844 View commit details
    Browse the repository at this point in the history

  6. (fix): handling pyarrow tests

    @ilan-gold
    ilan-gold committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    2d3e930 View commit details
    Browse the repository at this point in the history

  7. (fix): actually do import correctly

    @ilan-gold
    ilan-gold committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    04c9969 View commit details
    Browse the repository at this point in the history

  8. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 12, 2024
    Configuration menu
    Copy the full SHA
    5514539 View commit details
    Browse the repository at this point in the history

Commits on Feb 13, 2024

  1. (fix): reduce condition

    @ilan-gold
    ilan-gold committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    bedfa5c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    e6c2690 View commit details
    Browse the repository at this point in the history

  3. (fix): column ordering for dataframes

    @ilan-gold
    ilan-gold committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    82dbda9 View commit details
    Browse the repository at this point in the history

  4. (refactor): remove encoding business

    @ilan-gold
    ilan-gold committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    12217ed View commit details
    Browse the repository at this point in the history

  5. (refactor): raise error for dask + extension array

    @ilan-gold
    ilan-gold committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    dd5b87d View commit details
    Browse the repository at this point in the history

  6. Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e… 

    …xtension_arrays
    @ilan-gold
    ilan-gold committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    761a874 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    52cabc8 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2024

  1. (fix): only wrap ExtensionDuckArray that has a .array which is a … 

    …pandas extension array
    @ilan-gold
    ilan-gold committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    e0d58fa View commit details
    Browse the repository at this point in the history

  2. (fix): use duck array equality method, not pandas

    @ilan-gold
    ilan-gold committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    c1e0e64 View commit details
    Browse the repository at this point in the history

  3. (refactor): bye plum!

    @ilan-gold
    ilan-gold committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    17e3390 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    dd2ef39 View commit details
    Browse the repository at this point in the history

  5. (fix): and to or for casting to ExtensionDuckArray

    @ilan-gold
    ilan-gold committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    c8e6bfe View commit details
    Browse the repository at this point in the history

Commits on Feb 16, 2024

  1. (fix): check for class, not type

    @ilan-gold
    ilan-gold committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    b2a9517 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 16, 2024
    Configuration menu
    Copy the full SHA
    f5e1bd0 View commit details
    Browse the repository at this point in the history

Commits on Feb 19, 2024

  1. (fix): only support native endianness

    @ilan-gold
    ilan-gold committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    407fad1 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e… 

    …xtension_arrays
    @ilan-gold
    ilan-gold committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    3a47f09 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 19, 2024
    Configuration menu
    Copy the full SHA
    fdd3de4 View commit details
    Browse the repository at this point in the history

Commits on Feb 20, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 20, 2024
    Configuration menu
    Copy the full SHA
    6b23629 View commit details
    Browse the repository at this point in the history

Commits on Feb 22, 2024

  1. (refactor): no need for superfluous checks in _maybe_wrap_data

    @ilan-gold
    ilan-gold committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    1c9047f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e… 

    …xtension_arrays
    @ilan-gold
    ilan-gold committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    9be6b03 View commit details
    Browse the repository at this point in the history

  3. (chore): clean up docs to no longer reference plum

    @ilan-gold
    ilan-gold committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    d9304f1 View commit details
    Browse the repository at this point in the history

  4. (fix): no longer allow ExtensionDuckArray to wrap ExtensionDuckArray

    @ilan-gold
    ilan-gold committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    6ec6725 View commit details
    Browse the repository at this point in the history

  5. (refactor): move implements logic to indexing

    @ilan-gold
    ilan-gold committed Feb 22, 2024
    Configuration menu
    Copy the full SHA
    bc9ac4c View commit details
    Browse the repository at this point in the history

Commits on Feb 29, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    1e906db View commit details
    Browse the repository at this point in the history

  2. (refactor): indexing.py -> extension_array.py

    @ilan-gold
    ilan-gold committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    6fb8668 View commit details
    Browse the repository at this point in the history

  3. (refactor): ExtensionDuckArray -> PandasExtensionArray

    @ilan-gold
    ilan-gold committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    8f034b4 View commit details
    Browse the repository at this point in the history

Commits on Mar 3, 2024

  1. Merge branch 'main' into extension_arrays

    @dcherian
    dcherian committed Mar 3, 2024
    Configuration menu
    Copy the full SHA
    90a6de6 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    2bd422a View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    ff67943 View commit details
    Browse the repository at this point in the history

  2. (fix): add writeable property

    @ilan-gold
    ilan-gold committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    661d9f2 View commit details
    Browse the repository at this point in the history

  3. (fix): don't check writeable for PandasExtensionArray

    @ilan-gold
    ilan-gold committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    caee1c6 View commit details
    Browse the repository at this point in the history

  4. (fix): move check eariler

    @ilan-gold
    ilan-gold committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    1d12f5e View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    31dfbb5 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    23b347f View commit details
    Browse the repository at this point in the history

  2. (refactor): correct guard clause

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    902c74b View commit details
    Browse the repository at this point in the history

  3. (chore): remove unnecessary AttributeError

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    0b64506 View commit details
    Browse the repository at this point in the history

  4. (feat): singleton wrapped as array

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    0c7e023 View commit details
    Browse the repository at this point in the history

  5. (feat): remove shared dtype casting

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    dd7fe98 View commit details
    Browse the repository at this point in the history

  6. (feat): loop once over dataframe.items

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    f0df768 View commit details
    Browse the repository at this point in the history

  7. (feat): add __len__ attribute

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    e2f0487 View commit details
    Browse the repository at this point in the history

  8. (fix): ensure constructor recieves pd.Categorical

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    1eb6741 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e… 

    …xtension_arrays
    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    2a7300a View commit details
    Browse the repository at this point in the history

  10. Update xarray/core/extension_array.py 

    Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>
    @ilan-gold @dcherian
    ilan-gold and dcherian committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    9cceadc View commit details
    Browse the repository at this point in the history

  11. Update xarray/core/extension_array.py 

    Co-authored-by: Deepak Cherian <dcherian@users.noreply.github.com>
    @ilan-gold @dcherian
    ilan-gold and dcherian committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    f2588c1 View commit details
    Browse the repository at this point in the history

  12. (fix): drop condition for categorical corrected

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    a0a63bd View commit details
    Browse the repository at this point in the history

  13. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    5bb2bde View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    f85f166 View commit details
    Browse the repository at this point in the history

Commits on Apr 4, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Apr 4, 2024
    Configuration menu
    Copy the full SHA
    7ecdeba View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    6bc40fc View commit details
    Browse the repository at this point in the history

Commits on Apr 13, 2024

  1. Apply suggestions from code review

    @dcherian
    dcherian committed Apr 13, 2024
    Configuration menu
    Copy the full SHA
    e9dc53f View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. (chore): test chunk behavior

    @ilan-gold
    ilan-gold committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    4791799 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'extension_arrays' of github.com:ilan-gold/xarray into e… 

    …xtension_arrays
    @ilan-gold
    ilan-gold committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    c649362 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    fc60dcf View commit details
    Browse the repository at this point in the history

  4. Update xarray/core/variable.py

    @dcherian
    dcherian committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    0374086 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into extension_arrays

    @dcherian
    dcherian committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    b9515a6 View commit details
    Browse the repository at this point in the history

  6. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    72bf807 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. (fix): bring back error

    @ilan-gold
    ilan-gold committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    63b6c42 View commit details
    Browse the repository at this point in the history

  2. (chore): add test for dropping cat for mean

    @ilan-gold
    ilan-gold committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    1d18439 View commit details
    Browse the repository at this point in the history

  3. Update whats-new.rst

    @dcherian
    dcherian committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    17f05da View commit details
    Browse the repository at this point in the history

  4. [pre-commit.ci] auto fixes from pre-commit.com hooks 

    for more information, see https://pre-commit.ci
    @pre-commit-ci
    pre-commit-ci[bot] committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    c906c81 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. Merge branch 'main' into extension_arrays

    @ilan-gold
    ilan-gold committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    e6db83b View commit details
    Browse the repository at this point in the history