You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Rate-limiting eager AutoMaterializations. Materialize as soon as conditions are met, but at most once per N seconds.
AutoMaterializePolicy.eager(cooldown_seconds=300)
If cooldown hasn't elapsed yet, Automaterialize decision is SKIP (rather than DISCARD)
Ideas of implementation
I thought this might have been possible by simply allowing floats instead of ints here...
max_materializations_per_minute (Optional[float])
but that rule is using DISCARD instead of SKIP, so I think it wouldn't work - rather than throttling, it would give up and require manual intervention.
What's the use case?
Rate-limiting eager AutoMaterializations. Materialize as soon as conditions are met, but at most once per N seconds.
AutoMaterializePolicy.eager(cooldown_seconds=300)
If cooldown hasn't elapsed yet, Automaterialize decision is SKIP (rather than DISCARD)
Ideas of implementation
I thought this might have been possible by simply allowing floats instead of ints here...
max_materializations_per_minute (Optional[float])
but that rule is using DISCARD instead of SKIP, so I think it wouldn't work - rather than throttling, it would give up and require manual intervention.
Additional information
Previous discussion:
https://discuss.dagster.io/t/12083303/for-auto-materialize-max-materializations-per-minute-i-d-pro
Message from the maintainers
Impacted by this issue? Give it a 👍! We factor engagement into prioritization.
The text was updated successfully, but these errors were encountered: