Skip to content

Eternal Durable function with multiple slots (multiple task hubs) #585

@dxynnez

Description

@dxynnez

Hi team,

We have a use case where we need to pull things indefinitely (hence long running) and persist state periodically.

Originally this was implemented with multiple timer functions (pull from different sources). But because of the nature of timer function & the underlying singleton lock implementation, most of the timers are running on the same worker instance most of the time, and hence causing imbalance on our workers.

We are now looking into using eternal durable function to replace the heavy-lifting work of the timer to load-balance the workers. But we have a few questions that need clarification:

  1. as this is eventually a durable function, the history is stored per taskhub. We use different taskHubs in different slots and during deployment (deploy to staging and swap to prod), looks like it's possible to have 2 eternal durable functions instances running on different slots even if we use the same instance Id (e.g., the old PROD is using taskhub A and is shutting down gracefully; and the new PROD is using a new taskhub and hence is unaware of any instanceId in the old taskhub and decides to enqueue & process the instance)?

  2. would the ContinueAsNew actually complete the durable function instance, or it would just wipe out the history without completing the instance? To make sure we never lose the eternal durable function, we will still have a timer to periodically schedule the eternal durable using the same instanceId (if it exists already, this would then be a no-op), but if the timer runs and tries to start a new durable function around the same time as the ContinueAsNew of the existing running eternal durable, what would happen? Is there any race that I would end up with 2 eternal durables with the same instanceId, even in the same taskhub?

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions