Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ENH: Implement DataFrame interchange protocol #46141

Merged
merged 49 commits into from
Apr 27, 2022
Merged

ENH: Implement DataFrame interchange protocol #46141

merged 49 commits into from
Apr 27, 2022

Commits on Apr 19, 2022

  1. Vendor smoke tests from consortium 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    ac58967 View commit details
    Browse the repository at this point in the history

  2. Vendor dataframe_protocol spec 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    fce881e View commit details
    Browse the repository at this point in the history

  3. Copy over the prototype and polish it a bit 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    02946f8 View commit details
    Browse the repository at this point in the history

  4. Fix the protocol spec 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    14fd478 View commit details
    Browse the repository at this point in the history

  5. Enable pd.DataFrame.__dataframe__ 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    4515011 View commit details
    Browse the repository at this point in the history

  6. Align spec with existing implementations 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    7d6fd5b View commit details
    Browse the repository at this point in the history

  7. Fix protocol tests 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    5d64c4a View commit details
    Browse the repository at this point in the history

  8. Make DataFrame.__dataframe__ pass protocol tests 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    b36fd46 View commit details
    Browse the repository at this point in the history

  9. Explicitly mark abstract methods in spec 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    d334b20 View commit details
    Browse the repository at this point in the history

  10. Add more smoke tests 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    014165d View commit details
    Browse the repository at this point in the history

  11. Implement column chunking 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    def54ba View commit details
    Browse the repository at this point in the history

  12. Fix tests formatting 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    8e6b882 View commit details
    Browse the repository at this point in the history

  13. Start implementing chunk support in from_df 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    282c85d View commit details
    Browse the repository at this point in the history

  14. Test buffer contents if on CPU 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    9fbb58d View commit details
    Browse the repository at this point in the history

  15. Improve spec a bit 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    dd93625 View commit details
    Browse the repository at this point in the history

  16. Beautify spec whitespace 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    07c8fae View commit details
    Browse the repository at this point in the history

  17. Use constants from spec enums, beautify a bit 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    b74c06e View commit details
    Browse the repository at this point in the history

  18. Format by black 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    6637a29 View commit details
    Browse the repository at this point in the history

  19. Format exchange tests by black 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    0883406 View commit details
    Browse the repository at this point in the history

  20. Respond to review - move files around 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    49418d2 View commit details
    Browse the repository at this point in the history

  21. Separate buffer and column implementations 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    78aebaa View commit details
    Browse the repository at this point in the history

  22. Mimick what Modin did 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    1b64ae2 View commit details
    Browse the repository at this point in the history

  23. Make spec tests pass 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    870ad21 View commit details
    Browse the repository at this point in the history

  24. Add tests for dtype_to_arrow_c_fmt 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    edefc8f View commit details
    Browse the repository at this point in the history

  25. Fix test declarations, some impl bugs remain 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    7144cf2 View commit details
    Browse the repository at this point in the history

  26. Fix .describe_categoricals and some tests 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    0dc1e58 View commit details
    Browse the repository at this point in the history

  27. Auto-fix some pre-commit checks 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    0f7c654 View commit details
    Browse the repository at this point in the history

  28. Fix more issues found by commit checks 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    522a66a View commit details
    Browse the repository at this point in the history

  29. Fix categorical-related test failures 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    1525320 View commit details
    Browse the repository at this point in the history

  30. Add a whatsnew entry 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    0054c15 View commit details
    Browse the repository at this point in the history

  31. Fix rst linting 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    f8badc6 View commit details
    Browse the repository at this point in the history

  32. Fix DataFrame.__dataframe__ docstring 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    86005d4 View commit details
    Browse the repository at this point in the history

  33. Fix DataFrame.__dataframe__ docstring more 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    9ab797b View commit details
    Browse the repository at this point in the history

  34. Fix test_api::TestApi 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    7a54b20 View commit details
    Browse the repository at this point in the history

  35. Try to fix typecheck issues 

    Signed-off-by: Vasily Litvinov <vasilij.n.litvinov@intel.com>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    65a5370 View commit details
    Browse the repository at this point in the history

  36. Respond to review comments 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    594ac53 View commit details
    Browse the repository at this point in the history

  37. Fix mypy error 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    62c43af View commit details
    Browse the repository at this point in the history

  38. Change check for dlpack 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    cacc9f1 View commit details
    Browse the repository at this point in the history

  39. Address review comments 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    804aa89 View commit details
    Browse the repository at this point in the history

  40. Remove dead elif branch 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    d1c0d56 View commit details
    Browse the repository at this point in the history

  41. Fix tests broken by .column_names change 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 19, 2022
    Configuration menu
    Copy the full SHA
    5d98ebf View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2022

  1. Add tests for datetime dtype 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 20, 2022
    Configuration menu
    Copy the full SHA
    60379e5 View commit details
    Browse the repository at this point in the history

  2. Fix from_dataframe docstring 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 20, 2022
    Configuration menu
    Copy the full SHA
    497ca24 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2022

  1. Add tests for uint dtype 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 21, 2022
    Configuration menu
    Copy the full SHA
    39f5a5c View commit details
    Browse the repository at this point in the history

Commits on Apr 22, 2022

  1. Handle string dtype better 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    d73558a View commit details
    Browse the repository at this point in the history

  2. Add test for mixed object dtype 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 22, 2022
    Configuration menu
    Copy the full SHA
    4ed35bf View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2022

  1. Rename spec test for clarity 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 23, 2022
    Configuration menu
    Copy the full SHA
    2fca3c0 View commit details
    Browse the repository at this point in the history

Commits on Apr 24, 2022

  1. Add missing test cases in test_dtype_to_arrow_c_fmt 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 24, 2022
    Configuration menu
    Copy the full SHA
    f030d9f View commit details
    Browse the repository at this point in the history

  2. Add comments explaing magic dtype numbers 

    Signed-off-by: Vasily Litvinov <fam1ly.n4me@yandex.ru>
    @vnlitvinov
    vnlitvinov committed Apr 24, 2022
    Configuration menu
    Copy the full SHA
    cc94e57 View commit details
    Browse the repository at this point in the history