Skip to content

Feature Request for Polars library Support on Seaborn #3277

Closed as not planned

Description

@Killpit

Polars is one of the recent Python library that has all the functionalities in Pandas while being much faster. However, as I looked at the documentation from both Polars library and seaborn, I couldn't find anything other than having to use to_pandas() method.

If polars library support is added to seaborn, it becomes much easier and faster to create visualisations without losing time and the data frame becomes much tidier.

Activity

  1. thuiop commented on Feb 24, 2023

    @thuiop
    Contributor

    This would basically involve rewriting the whole library...

  2. jcmkk3 commented on Feb 24, 2023

    @jcmkk3

    This would basically involve rewriting the whole library...

    It depends on what "support" means. It seems like the best short-term solution would be to read dataframes with pandas api.interchange.from_dataframe function. This should allow seaborn to read data from any libraries that implement the dataframe interchange protocol (polars just being one example)

    This won't perform the compute with polars, but it will make things more seamless for the users. With newer versions of pandas and the right settings (I'm not sure exactly what those are), it should be possible to make them zero copy with the arrow format and since pandas is starting to perform more of its compute with arrow, there wouldn't even be much of a performance difference compared to polars.

  3. mwaskom commented on Feb 25, 2023

    @mwaskom
    Owner

    Yeah, this isn't a very clear request .

    The best short-term solution would be to read dataframes with pandas api.interchange.from_dataframe function.

    That is indeed on the roadmap for seaborn v0.13 now that it's landed in pandas stable.

    If polars library support is added to seaborn, it becomes much easier and faster to create visualisations without losing time and the data frame becomes much tidier.

    But I also take this to be essentially proposing a complete rewrite of the seaborn internals, and I'm going to need a far more compelling argument to take that even remotely seriously. I doubt that pandas is actually the bottleneck for performance with seaborn. Many of the statistical operations that I expect would be slow on large datasets (e.g. KDEs, histograms, bootstrapping) are implemented in numpy or scipy and operate on numpy arrays, so the pandas vs. polars question is moot. Also, "has all the functionalities in Pandas" is just not true, Polars is not feature complete with Pandas (and does not itself claim to be) and this has some important implications vis a vis seaborn, which uses pandas indexes in several ways.

    I am following the progress of the DataFrame API standard and am at least in principal open to that approach for taking advantage of distributed / out of core / GPU operations. But that progress seems pretty slow so I don't expect it to be actionable any time soon.

    I'm going to close as there's nothing specific to continue discussing about Polars itself. @jcmkk3 keep an eye out for a PR adding the dataframe interchange support sometime in the next month or two.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions