Skip to content

Should we handle asset_state cleanup when clear_on_success is set? #67228

@amoghrajesh

Description

@amoghrajesh

Ref: #66859 (comment)

What i think:
I was under the impression that "task states" only need to be cleared for clear_on_success because those are bound to tasks and asset states are intentionally not cleared on task success, server-side or worker-side cos asset state is designed to persist across task runs (e.g. watermarks, file counts that accumulate), so clearing it on success would not be beneficial by default. If there's a case for clearing asset state on success in the future, it should be a separate opt-in config (e.g. [state_store] clear_asset_state_on_success) as per me.

cc @kaxil

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

Status

Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions