Skip to content

Add Chain.get_draws(..., *, start, stop, step) kwargs to slice draws/stats upon retrieval #47

@michaelosthege

Description

@michaelosthege

For many use cases such as "only retrieve the last 100" this can give superior performance, because fewer data is moved around.

By slicing the draws directly at the backend implementation level, we can also remove the clen shortening which was introduced in c26a55e and modified in a5c1aae.

The workaround to truncate within a chain was simply because .to_inferencedata() iterates over the variables and retrieves them one by one.
If an insert happened in the meantime, the variables within a chain can have different lengths.

By introducing Chain.get_draws(name, *, ifrom, ito) (and the same for stats) the .to_inferencedata() implementation can determine the chain lengths once and retrieve only what it needs.

Bonus points for a start, stop, step kwarg that can be used for thinning. (Can ClickHouse do this?)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions