Skip to content

Memory leak when processing large datasets #265

@stranske

Description

@stranske

Bug Report

Describe the bug
When processing datasets over 1GB, the application gradually consumes more memory until it crashes with OOM.

Steps to Reproduce

  1. Load a dataset larger than 1GB
  2. Run the data processing pipeline
  3. Monitor memory usage with top
  4. Observe memory climbing until OOM

Expected Behavior
Memory should stay stable during processing by properly releasing intermediate data.

Environment

  • OS: Ubuntu 22.04
  • Python: 3.11
  • Package version: 2.1.0

Metadata

Metadata

Labels

bugSomething isn't working

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions