Polars support for Great Expectations

[Durga854]

Hello,
Could you please consider adding support for Polars in Great Expectations? It would be great to have functionality similar to what is available for Pandas and Spark. Thanks for your attention to this!

[matthiasgomolka]

I support this. Pandas is often quite slow.

Maybe it would be helpful to use narwhals in this process. Or maybe even ibis ...

[MarcoGorelli]

Narwhals is actually already a requirement of great-expectations, because great-expectations requires Altair which in turn requires Narwhals, so it could be used without introducing any extra dependencies 😉

Happy to help if the maintainers are open to this, feel free to reach out on Discord if you're interested

[Kilo59]

@MarcoGorelli

This is super interesting.
The first step might be to copy the PandasExecutionEngine and create a NarwhalsExecutionEngine. At that point, it should be trivial to create a new PolarsDatasource type that simply uses the NarwhalsExecutionEngine instead of the pandas one.

A specific PolarsExecutionEngine could also be created directly, but I suspect that might be a bit more work.

[Kilo59]

There seems to be a fair amount of interest in this.

• See Use Narwhals instead of Pandas #10702