Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speedup dependencies dtypes #363

Closed
wants to merge 3 commits into from
Closed

Conversation

hagenw
Copy link
Member

@hagenw hagenw commented Feb 6, 2024

Speedup audb.Dependencies by using object dtype for strings instead of string.

This decision is based on the benchmark results comparing storing string as object or string, or using pyarrow dtypes (for discussion on benchmark see #365). One can see that we get slight improvments when strings as object and not as string, whereas using pyarrow dtypes slows down row based processing.

TODO: add a test what happens when we load existing db.pkl from cache that are using sting as dtype.

@hagenw
Copy link
Member Author

hagenw commented Feb 14, 2024

This is superseeded by #372

@hagenw hagenw closed this Feb 14, 2024
@hagenw hagenw deleted the speedup-dependencies-dtypes branch May 3, 2024 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant