Idea
Add first-class support for async def actions and async workflow execution, enabling modern Python async patterns.
Motivation
Python's async ecosystem is mature and widely adopted. Many I/O-bound operations (HTTP calls, database queries, file operations) benefit from async execution. Dotflow should support async as a first-class citizen.
Proposed API
from dotflow import DotFlow, action
@action
async def fetch_data(context):
async with aiohttp.ClientSession() as session:
resp = await session.get("https://api.example.com/data")
return await resp.json()
@action
async def process_data(context):
return transform(context.previous_context.storage)
workflow = DotFlow()
workflow.task.add(fetch_data)
workflow.task.add(process_data)
await workflow.start_async(mode="sequential")
Implementation considerations
- Detect if action function is a coroutine (
asyncio.iscoroutinefunction)
- Add
start_async() method to DotFlow or make start() async-aware
Execution class should handle both sync and async actions
- Async parallel mode using
asyncio.gather() instead of multiprocessing
- Maintain backward compatibility — sync actions must continue to work unchanged
- Async storage providers (e.g.,
aioboto3 for S3)
Value
- Better performance for I/O-bound workflows
- Modern Python patterns
- Enables
asyncio.gather for true concurrent I/O without multiprocessing overhead
Idea
Add first-class support for
async defactions and async workflow execution, enabling modern Python async patterns.Motivation
Python's async ecosystem is mature and widely adopted. Many I/O-bound operations (HTTP calls, database queries, file operations) benefit from async execution. Dotflow should support async as a first-class citizen.
Proposed API
Implementation considerations
asyncio.iscoroutinefunction)start_async()method toDotFlowor makestart()async-awareExecutionclass should handle both sync and async actionsasyncio.gather()instead ofmultiprocessingaioboto3for S3)Value
asyncio.gatherfor true concurrent I/O without multiprocessing overhead