Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Avoid allocation in the hot path for struct JSON serialization #15449

Merged
merged 1 commit into from Apr 4, 2024

Conversation

ChayimFriedman2
Copy link
Contributor

This improves the following benchmark in 50%:

use criterion::{black_box, criterion_group, criterion_main, Criterion};
use polars::prelude::*;

fn to_json(df: &mut DataFrame) -> Vec<u8> {
    let mut json = Vec::new();
    JsonWriter::new(&mut json)
        .with_json_format(JsonFormat::Json)
        .finish(df)
        .unwrap();
    json
}

fn my_benchmark(c: &mut Criterion) {
    const SIZE: i32 = 10_000;
    let mut df = df![
        "a" => (0..SIZE).collect::<Vec<_>>(),
        "b" => (0..SIZE).map(|v| v.to_string()).collect::<Vec<_>>(),
    ]
    .unwrap();

    c.bench_function("to_json", |b| b.iter(|| to_json(black_box(&mut df))));
}

criterion_group!(benches, my_benchmark);
criterion_main!(benches);

Copy link

codecov bot commented Apr 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.13%. Comparing base (802d6c8) to head (4a9748f).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #15449      +/-   ##
==========================================
- Coverage   81.14%   81.13%   -0.02%     
==========================================
  Files        1362     1362              
  Lines      174846   174848       +2     
  Branches     2531     2531              
==========================================
- Hits       141881   141860      -21     
- Misses      32481    32504      +23     
  Partials      484      484              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@stinodego stinodego changed the title Avoid allocation in the hot path for struct JSON serialization perf: Avoid allocation in the hot path for struct JSON serialization Apr 2, 2024
@github-actions github-actions bot added performance Performance issues or improvements python Related to Python Polars rust Related to Rust Polars labels Apr 2, 2024
Copy link
Member

@ritchie46 ritchie46 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot. Painful to have that. :/

@ritchie46 ritchie46 merged commit 02f4d7f into pola-rs:main Apr 4, 2024
20 checks passed
@ChayimFriedman2 ChayimFriedman2 deleted the patch-1 branch April 4, 2024 17:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance issues or improvements python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants