Polars.NET

Blazingly Fast DataFrames & LINQ Engine for .NET

Powered by Rust Polars & Apache Arrow

Documentation Website | Quick Start | LINQ Guide | C# API | F# API | llms.txt | Cookbook

I really dreamed that we can play with DataFrames in .NET just like python, now we got a much more enjoyable and funny one.

• Native Pushdown LINQ: Compiles C# LINQ and F# query { } directly into Rust Polars physical plans then be optimized & executed natively. Verified with all 22 TPC-H benchmark queries implemented in native LINQ without client-side fallback. Includes .Explain(), .ToDataFrame(), and joins between in-memory collections and DataFrames..
• High Performance: Built on the Rust Polars engine and Apache Arrow columnar memory format. Delivers vectorized SIMD execution, query optimization (predicate and projection pushdown, common subplan elimination), and out-of-core streaming execution for larger-than-memory datasets.
• Deep .NET Ecosystem Integration: First-class bridges to standard .NET abstractions, including IAsyncEnumerable<T> streaming, ADO.NET IDataReader / DbDataReader ingestion, ML.NET IDataView pipelines, and direct extraction into System.Numerics.Tensors (ReadOnlyTensorSpan) for numerical computing and model inference.
• Universal I/O & Storage: Reads and writes Parquet, CSV, JSON/NDJSON, Arrow IPC, Excel, and Avro. Native integration with Arrow Database Connectivity (ADBC) and Delta Lake (ACID transactions, time travel, and SQL MERGE), targeting both local filesystems and cloud object storage (Amazon S3, Azure Blob, Google Cloud Storage).

Polars.NET targets .NET 8.0 and requires referencing both the managed API package and the matching native runtime package for your execution platform:

# C# API package
dotnet add package Polars.NET

# Platform native runtime (choose matching your deployment target):
dotnet add package Polars.NET.Native.win-x64
# dotnet add package Polars.NET.Native.linux-x64
# dotnet add package Polars.NET.Native.linux-arm64
# dotnet add package Polars.NET.Native.osx-arm64

# Optional: ML.NET integration
dotnet add package Polars.NET.ML

# F# API package
dotnet add package Polars.FSharp

# Platform native runtime:
dotnet add package Polars.NET.Native.win-x64

Write standard C# LINQ queries then execute them natively without intermediate managed objects:

using Polars.CSharp;
using Polars.CSharp.Linq;

public record Order(int Id, string Customer, string Region, double Revenue);

List<Order> orders = GetOrders();

// 1. Direct entry from in-memory collection, DataFrame, or ADO.NET IDataReader:
var query = from o in orders.AsPolarsQueryable()
            where o.Revenue >= 500.0 && o.Region != "Test"
            group o by o.Region into g
            select new {
                Region = g.Key,
                TotalRevenue = g.Sum(x => x.Revenue),
                OrderCount = g.Count()
            };

// 2. Materialize directly into a Polars DataFrame:
using DataFrame resultDf = query.ToDataFrame();
resultDf.Show();

// 3. Or inspect the execution plan:
query.InspectPlan();

Polars DSL for complex analytics, column expressions, and Selectors:

using Polars.CSharp;
using Pl = Polars.CSharp.Polars;
using Cs = Polars.CSharp.Polars.Selectors;

// Read Parquet lazily:
using var lf = LazyFrame.ScanParquet("orders/*.parquet");

using var summaryDf = lf
    .Filter(Pl.Col("order_date") >= new DateOnly(2025, 1, 1))
    .GroupBy("region")
    .Agg(
        Pl.Col("revenue").Sum().Alias("total_revenue"),
        Pl.Col("customer_id").NUnique().Alias("unique_customers"),
        Cs.Numeric().Mean().Name.Prefix("avg_")
    )
    .Sort("total_revenue", descending: true)
    .Collect();

summaryDf.Show();

F# strong typing CE, anonymous records, and pipeline combinators over native queries:

open Polars.FSharp
open Polars.FSharp.Query

// F# query { } compiles into polars execution plan:
let summaryQuery =
    query {
        for o in df.AsQueryable<Order>() do
        where (o.Revenue >= 500.0 && o.Region <> "Test")
        groupValBy o o.Region into g
        select {|
            Region = g.Key
            TotalRevenue = g.Sum(fun x -> x.Revenue)
            OrderCount = g.Count()
        |}
    }

use resultDf = summaryQuery.ToDataFrame()
resultDf |> DataFrame.show

Perform ANSI SQL MERGE upsert operations directly on DataFrames or in-memory collections:

List<ProductItem> inMemoryTarget = GetTargetProducts();
List<ProductItem> incomingUpdates = GetSourceProducts();

// Start merge from a List<T>:
using DataFrame mergedDf = inMemoryTarget.Merge(incomingUpdates, t => t.Id, s => s.Id)
    .WhenMatched(
        (t, s) => s.Price > t.Price,
        then => then.Update(set => set.Set(t => t.Price, (t, s) => s.Price))
    )
    .WhenNotMatchedByTarget(then => then.Insert())
    .WhenNotMatchedBySource(t => t.Category == "Seasonal", then => then.Delete())
    .ToDataFrame(); // Or materialize directly to List<ProductItem> via .ToList()

Comprehensive documentation, architecture references, and API catalogs are available on our documentation portal:

Polars.NET uses a modular three-tier architecture to prevent upstream breaking changes:

+-------------------------------------------------------------------+
|               Polars.NET User API Layer                           |
|   - Polars.CSharp                                                 |
|   - Polars.FSharp                                                 |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
|              Polars.NET.Core & Polars.NET.Linq                    |
|   - Common Infrastructure, LINQ Provider written in F#            |
|   - Apache Arrow C Data Interface                                 |
|   - P/Invoke bindings and unsafe wrappers                         |
+-------------------------------------------------------------------+
                                  |
                                  v
+-------------------------------------------------------------------+
|                 Rust polars & delta-rs Engine                     |
|   - Execution engine                                              |
|   - Physical query planner, optimizer                             |
|   - delta-rs integration                                          |
+-------------------------------------------------------------------+

• Benchmark suite: Polars.NET-Benchmark
• Recipe repository: Polars.NET-Cookbook

If Polars.NET is helpful to you, consider supporting the project via GitHub Sponsors to buy me a cup of black tea. Sláinte!

Polars.NET is built on:

• Polars (pola.rs) - The lightning-fast DataFrame library developed by Ritchie Vink and the Polars team.
• Apache Arrow - The columnar in-memory data standard powering zero-copy interop.

MIT License. See LICENSE for details.
Third-party licenses and notices are documented in THIRD_PARTY_LICENSES.html.