Skip to content

Conversation

@rafapi
Copy link
Collaborator

@rafapi rafapi commented May 1, 2025

image

Samples: 10_000

  • New: 26.29 s (380.31 examples/s)
  • Old: 159.68 s (63.02 examples/s)

Speed-up factor: 6.03x

@rafapi rafapi changed the title Remove Pandas from data processing Optimising data preprocessing May 1, 2025
@AlexPiche AlexPiche changed the base branch from main to group_normalization May 2, 2025 16:24
@rafapi rafapi changed the title Optimising data preprocessing Optimise data preprocessing May 2, 2025
@rizar rizar changed the base branch from group_normalization to main May 6, 2025 12:49
@rizar
Copy link
Collaborator

rizar commented Jun 18, 2025

@rafapi in @AlexPiche 's PR #46 there are now some pretty have changed to preprocessing, you may want to restart you work here by benchmarking the speed there

@rizar rizar closed this Jun 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants