Skip to content

improve pipeline garbage collection capabilities #208

@bstabler

Description

@bstabler

The current full example run (see #194 for example) is running into memory issues. @toliwaga guesses the intermediate tables for logsum samples (which are enormous) need to get flushed – currently they are taking up memory and space in the pipeline because the three components of the logsum models are separate steps and so the tables are saved as orca tables in memory and the pipeline. We need to more intelligently manage the pipeline memory.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions