Andystaples/add functions support #75

andystaples · 2025-11-21T20:48:06Z

DRAFT PR - for collaboration
Requires at minimum the changes here
Azure/azure-functions-durable-extension#3260
in durable WebJobs extension allowing gRPC protocol for Python

Copilot

Pull request overview

This draft PR introduces Azure Functions support for the durabletask-python library by creating a new durabletask-azurefunctions package. This allows developers to use Durable Task patterns within Azure Functions using Python decorators and bindings that integrate with the Azure Functions worker.

Key Changes:

Added a new durabletask-azurefunctions package with decorators, client, and worker implementations for Azure Functions integration
Modified core durabletask/worker.py to support a new ProtoTaskHubSidecarServiceStub type alongside the existing gRPC stub
Introduced a base ProtoTaskHubSidecarServiceStub class that can be extended for different communication patterns

Reviewed changes

Copilot reviewed 12 out of 15 changed files in this pull request and generated 22 comments.

Show a summary per file

File	Description
`durabletask/worker.py`	Added import for new stub type, updated type hints to accept Union of stub types, added handling for `orchestratorCompleted` event
`durabletask/internal/ProtoTaskHubSidecarServiceStub.py`	New base stub class defining the protocol interface with callable attributes for all Task Hub operations
`durabletask-azurefunctions/pyproject.toml`	Package configuration for the new Azure Functions integration package with dependencies
`durabletask-azurefunctions/durabletask/azurefunctions/worker.py`	Worker implementation that extends `TaskHubGrpcWorker` without async worker loop for Functions execution model
`durabletask-azurefunctions/durabletask/azurefunctions/client.py`	Client implementation for Azure Functions that parses connection info from JSON and uses custom interceptors
`durabletask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_null_stub.py`	Null stub implementation that provides no-op lambdas for all stub operations
`durabletask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_grpc_interceptor.py`	Custom gRPC interceptor that adds Azure Functions-specific headers
`durabletask-azurefunctions/durabletask/azurefunctions/decorators/metadata.py`	Trigger and binding metadata classes for orchestration, activity, entity, and client bindings
`durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py`	Blueprint and DFApp classes providing decorators for registering Functions with Durable Task patterns
`durabletask-azurefunctions/durabletask/azurefunctions/decorators/__init__.py`	Package exports for decorator module
`durabletask-azurefunctions/durabletask/azurefunctions/constants.py`	Constants for trigger and binding type names
`durabletask-azurefunctions/CHANGELOG.md`	Initial changelog for the new package

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

durabletask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_null_stub.py

durabletask-azurefunctions/durabletask/azurefunctions/constants.py

durabletask-azurefunctions/durabletask/azurefunctions/worker.py

Copilot · 2025-11-21T20:55:36Z

durabletask-azurefunctions/durabletask/azurefunctions/client.py

+    creationUrls: dict[str, str]
+    managementUrls: dict[str, str]


[nitpick] Potential compatibility issue with type hint syntax. The use of dict[str, str] (PEP 585 style) requires Python 3.9+. While pyproject.toml specifies requires-python = ">=3.9", consider whether this is the intended minimum version or if Dict[str, str] from typing should be used for broader compatibility.

...letask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_grpc_interceptor.py

durabletask-azurefunctions/durabletask/azurefunctions/worker.py

durabletask/internal/ProtoTaskHubSidecarServiceStub.py

durabletask-azurefunctions/durabletask/azurefunctions/internal/azurefunctions_null_stub.py

durabletask-azurefunctions/durabletask/azurefunctions/worker.py

andystaples · 2025-11-21T20:49:40Z

durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py

+                if response is None:
+                    raise Exception("Orchestrator execution did not produce a response.")
+                # The Python worker returns the input as type "json", so double-encoding is necessary
+                return '"' + base64.b64encode(response.SerializeToString()).decode('utf-8') + '"'


Victoria - Currently, the return value from here is passed on to the host as type "json" so the host attempts to Newtonsoft deserialize it back into an object before handing back to the Durable middleware for final decoding. This breaks, unless I double-encode with quotes as above. Is there a way to communicate to the worker that this is a plain string instead?

Investigating this - will need a little more time to test on my end

This is actually coming from the OrchestrationTriggerConverter done here. The type is hard-coded to json. Changing the type to string works in this scenario, but I'm unsure if there are other cases where json was intended

@classmethod def encode(cls, obj: typing.Any, *, expected_type: typing.Optional[type]) -> meta.Datum: # Durable function context should be a json return meta.Datum(type='json', value=obj)

andystaples · 2025-11-21T21:00:29Z

durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py

+        # Obtain user-code and force type annotation on the client-binding parameter to be `str`.
+        # This ensures a passing type-check of that specific parameter,
+        # circumventing a limitation of the worker in type-checking rich DF Client objects.
+        # TODO: Once rich-binding type checking is possible, remove the annotation change.
+        user_code = fb._function._func
+        user_code.__annotations__[parameter_name] = str


Victoria - this is the same approach taken by the existing Durable Python SDK for the DurableClient binding - we force the annotation to be "str" so the worker takes a path that does not attempt to use the DurableClientConverter input parameter converter, which would throw NotImplementedError

https://github.com/Azure/azure-functions-python-library/blob/8bd30d300529cbda0526aa2d6606d11333c9c3aa/azure/functions/durable_functions.py#L130

Do you think it is worth moving the client_constructor logic in this PR into the DurableClientConverter in the -library, so that we don't have to do this type-hacking stuff?

We'd have to figure out how to detect which underlying provider for the durable_client_input binding is being used to know when to simply return the string for the old SDK vs parse it in the new

The main issue would be that we'd have something different to return based on the durable library.

Are the types going to be the same? (eg DurableClient for both packages) We could look at creating two separate converters - right now it's using the Generic converter, but it would be better to have our own

andystaples · 2025-11-21T21:02:15Z

durabletask-azurefunctions/pyproject.toml

+requires-python = ">=3.9"
+license = {file = "LICENSE"}
+readme = "README.md"
+dependencies = [
+    "durabletask>=0.5.0",
+    "azure-identity>=1.19.0",
+    "azure-functions>=1.11.0"
+]


TODO: Update python min version and rev durabletask dependency to 1.0.1/1.1.0
Also rev durabletask versions to the same based on size of changes needed

andystaples · 2025-11-21T21:02:40Z

durabletask-azurefunctions/pyproject.toml

+
+[project]
+name = "durabletask.azurefunctions"
+version = "0.1.0"


Victoria - what versioning strategy would you propose if the first version that goes to PyPi would be used for internal testing only?

dev versions work well - eg 1.0.0dev0 or 0.0.1dev0

andystaples · 2025-11-21T21:05:37Z

durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py

+        # TODO: Is there a better way to support retrieving the unwrapped user code?
+        df_client_middleware.client_function = fb._function._func  # type: ignore


Victoria - not sure if you remember this context from a while back, but this is also carryover from the previous SDK - I added this line to make retrieving the "unwrapped" user code possible for the unit testing scenario - see
https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-unit-testing-python#unit-testing-trigger-functions
If possible, I'd like to see a "better" solution for the new SDK. Hate to re-open a can of worms here, though

I vaguely remember context, but we can sync again over specific requirements.

andystaples · 2025-11-21T21:07:14Z

durabletask-azurefunctions/durabletask/azurefunctions/decorators/durable_app.py

+                stub = AzureFunctionsNullStub()
+                worker = DurableFunctionsWorker()
+                response: Optional[OrchestratorResponse] = None
+
+                def stub_complete(stub_response):
+                    nonlocal response
+                    response = stub_response
+                stub.CompleteOrchestratorTask = stub_complete


All of this is probably optimizable - do we really need to create a new stub and worker for each call? Can they be saved? Will look into this more at some point

berndverst · 2025-11-24T18:49:04Z

Can you add an overview / more detail to the PR description?

- Still needs eventSent and eventRecieved implementations

- Add new_uuid method to OrchestrationContext for deterministic replay-safe UUIDs - Fix entity locking behavior for Functions - Align _RuntimeOrchestrationContext param names with OrchestrationContext - Remap __init__.py files for new module - Update version to 0.0.1dev0 - Add docstrings to missing methods - Move code for executing orchestrators/entities to DurableFunctionsWorker - Add function metadata to triggers for detection by extension

.github/workflows/durabletask-azurefunctions.yml

+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python 3.14
+      uses: actions/setup-python@v5
+      with:
+        python-version: 3.14
+    - name: Install dependencies
+      working-directory: durabletask-azurefunctions
+      run: |
+        python -m pip install --upgrade pip
+        pip install setuptools wheel tox
+        pip install flake8
+    - name: Run flake8 Linter
+      working-directory: durabletask-azurefunctions
+      run: flake8 .
+    - name: Run flake8 Linter
+      working-directory: tests/durabletask-azurefunctions
+      run: flake8 .
+
+  run-docker-tests:


.github/workflows/durabletask-azurefunctions.yml

+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
+    env:
+      EMULATOR_VERSION: "latest"
+    needs: lint
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v4
+
+      - name: Pull Docker image
+        run: docker pull mcr.microsoft.com/dts/dts-emulator:$EMULATOR_VERSION
+
+      - name: Run Docker container
+        run: |
+          docker run --name dtsemulator -d -p 8080:8080 mcr.microsoft.com/dts/dts-emulator:$EMULATOR_VERSION
+
+      - name: Wait for container to be ready
+        run: sleep 10  # Adjust if your service needs more time to start
+
+      - name: Set environment variables
+        run: |
+          echo "TASKHUB=default" >> $GITHUB_ENV
+          echo "ENDPOINT=http://localhost:8080" >> $GITHUB_ENV
+
+      - name: Install durabletask dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install flake8 pytest
+          pip install -r requirements.txt
+
+      - name: Install durabletask-azurefunctions dependencies
+        working-directory: examples
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+
+      - name: Install durabletask-azurefunctions locally
+        working-directory: durabletask-azurefunctions
+        run: |
+          pip install . --no-deps --force-reinstall
+
+      - name: Install durabletask locally
+        run: |
+          pip install . --no-deps --force-reinstall
+
+      - name: Run the tests
+        working-directory: tests/durabletask-azurefunctions
+        run: |
+          pytest -m "dts" --verbose
+
+  publish:


To resolve the detected issue, you should add a permissions block to the workflow at the root level (before the jobs: key). This approach ensures that all jobs in the workflow inherit restrictive permissions unless overridden. The minimal starting point is contents: read, which is sufficient for most workflows that need to access the repository contents but do not need to write. If some jobs (such as a release/publish job) require more permissive access, you can further add job-level permissions blocks with additional scopes.

Steps to fix:

Edit .github/workflows/durabletask-azurefunctions.yml.

Insert a permissions: block directly after the workflow's name: key and before the on: key.

The block should specify contents: read.

This enforces least-privilege, and all jobs (unless overridden) now run with only read access to the repository contents.

No imports or code changes are required, as this is a configuration file. No other regions of code need to change.

.github/workflows/durabletask-azurefunctions.yml

+    if: startsWith(github.ref, 'refs/tags/azurefunctions-v')  # Only run if a matching tag is pushed
+    needs: run-docker-tests
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Extract version from tag
+        run: echo "VERSION=${GITHUB_REF#refs/tags/azurefunctions-v}" >> $GITHUB_ENV  # Extract version from the tag
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.14"  # Adjust Python version as needed
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install build twine
+
+      - name: Build package from directory durabletask-azurefunctions
+        working-directory: durabletask-azurefunctions
+        run: |
+          python -m build
+
+      - name: Check package
+        working-directory: durabletask-azurefunctions
+        run: |
+          twine check dist/* 
+
+      - name: Publish package to PyPI
+        env:
+          TWINE_USERNAME: __token__
+          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN_AZUREFUNCTIONS  }}  # Store your PyPI API token in GitHub Secrets
+        working-directory: durabletask-azurefunctions
+        run: | 
+          twine upload dist/* 


To fix this issue, explicitly set a permissions block for the whole workflow or for each job as needed. The ideal minimum is to set contents: read, unless the job requires other types of access. For the publish-release job, releasing to PyPI and not creating tags, releases, or manipulating pull requests, contents: read is sufficient (it only reads package files, not writing to the repo).

We should therefore add the following block near the top of the workflow (to affect all jobs, unless overridden):

permissions: contents: read

Alternatively, it could be added inside the publish-release job if only that job requires explicit restriction, but in almost all cases, setting it globally is clearer and safer (unless other jobs in the workflow require additional write privileges).

This change is made at the root of the YAML file, directly after the name: block and before the on: block.

No other code changes or external dependencies are required.

.github/workflows/durabletask-azurefunctions-dev.yml

+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Extract version from tag
+        run: echo "VERSION=${GITHUB_REF#refs/tags/azurefunctions-v}" >> $GITHUB_ENV  # Extract version from the tag
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.14"  # Adjust Python version as needed
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install build twine
+
+      - name: Append dev to version in pyproject.toml
+        working-directory: durabletask-azurefunctions
+        run: |
+          sed -i 's/^version = "\(.*\)"/version = "\1.dev${{ github.run_number }}"/' pyproject.toml
+
+      - name: Build package from directory durabletask-azurefunctions
+        working-directory: durabletask-azurefunctions
+        run: |
+          python -m build
+
+      - name: Check package
+        working-directory: durabletask-azurefunctions
+        run: |
+          twine check dist/* 
+
+      - name: Publish package to PyPI
+        env:
+          TWINE_USERNAME: __token__
+          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN_AZUREFUNCTIONS }}  # Store your PyPI API token in GitHub Secrets
+        working-directory: durabletask-azurefunctions
+        run: | 
+          twine upload dist/* 


The recommended fix is to explicitly add a permissions: block to the workflow. This block should be placed at the top level of the workflow (directly after the name: field), thereby applying to all jobs unless jobs override it. The safest minimum is permissions: contents: read, which will suffice unless more elevated privileges are required (not evidenced in any of the given steps). No additional imports, methods, or new dependencies are needed; this is a YAML configuration change. Edit .github/workflows/durabletask-azurefunctions-dev.yml to add:

permissions: contents: read

immediately after the name: field at line 1, before the on: trigger.

.github/workflows/durabletask-azurefunctions-experimental.yml

+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Extract version from tag
+        run: echo "VERSION=${GITHUB_REF#refs/tags/azurefunctions-v}" >> $GITHUB_ENV  # Extract version from the tag
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.14"  # Adjust Python version as needed
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install build twine
+
+      - name: Change the version in pyproject.toml to 0.0.0dev{github.run_number}
+        working-directory: durabletask-azurefunctions
+        run: |
+          sed -i 's/^version = ".*"/version = "0.0.0.dev${{ github.run_number }}"/' pyproject.toml
+
+      - name: Build package from directory durabletask-azurefunctions
+        working-directory: durabletask-azurefunctions
+        run: |
+          python -m build
+
+      - name: Check package
+        working-directory: durabletask-azurefunctions
+        run: |
+          twine check dist/* 
+
+      - name: Publish package to PyPI
+        env:
+          TWINE_USERNAME: __token__
+          TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN_AZUREFUNCTIONS  }}  # Store your PyPI API token in GitHub Secrets
+        working-directory: durabletask-azurefunctions
+        run: | 
+          twine upload dist/* 


To fix the problem, we should add a permissions block to the workflow YAML file. The recommended approach is to set it at the root of the workflow, so all jobs in the workflow will inherit the minimal permissions unless overridden. Unless the workflow requires write access to repository contents, the safest default is contents: read, as recommended by CodeQL. Review the workflow steps: Code checkout, extracting tag versions, Python actions, and publishing to PyPI via a secret token, none of which require GITHUB_TOKEN write access. Therefore, adding the following block to the top level of the YAML file (right after the name: block) will implement the principle of least privilege:

permissions: contents: read

This is to be inserted after the workflow name: and before the on: block for clarity and convention.

andystaples added 2 commits November 6, 2025 10:01

Storing changes commit

7ff1525

Working orchestrators + activities

552a2dd

andystaples requested a review from Copilot November 21, 2025 20:48

Copilot started reviewing on behalf of andystaples November 21, 2025 20:48 View session

Copilot finished reviewing on behalf of andystaples November 21, 2025 20:52

Copilot AI reviewed Nov 21, 2025

View reviewed changes

andystaples commented Nov 21, 2025

View reviewed changes

andystaples and others added 3 commits November 21, 2025 14:36

Nitpicks and cleanup

af0e3c2

Merge branch 'main' into andystaples/add-functions-support

86ee081

Save-all nits

3497148

andystaples added 5 commits November 24, 2025 11:59

Add entity support (needs extension change)

57de878

Refine entity support

18145f8

- Still needs eventSent and eventRecieved implementations

Finish entity support

9965ba4

Bump durabletask version, fix metadata

cc005ae

github-advanced-security bot found potential problems Dec 4, 2025

View reviewed changes

hallvictoria mentioned this pull request Dec 5, 2025

feat: support durabletask-azurefunctions Azure/azure-functions-python-library#309

Draft

andystaples and others added 3 commits December 5, 2025 12:14

Use Protocol for stubs

bf6d6f2

Merge branch 'main' into andystaples/add-functions-support

0a03bd1

Update to new workflow pattern

1176d03

github-advanced-security bot found potential problems Dec 5, 2025

View reviewed changes

andystaples added 4 commits December 5, 2025 13:35

Rename stub file

7bf763a

Fix import

811653e

Merge branch 'main' into andystaples/add-functions-support

068cb43

Experimental dependency revision

827d201

		# TODO: Is there a better way to support retrieving the unwrapped user code?
		df_client_middleware.client_function = fb._function._func # type: ignore

@@ -1,4 +1,6 @@
             name: Durable Task Scheduler SDK (durabletask-azurefunctions)
+            permissions:
+              contents: read
             on:
               push:

Andystaples/add functions support #75

Are you sure you want to change the base?

Andystaples/add functions support #75

Conversation

andystaples commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hallvictoria Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

berndverst commented Nov 24, 2025

Uh oh!

Check warning

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Check warning

Copilot Autofix

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

andystaples commented Nov 21, 2025 •

edited

Loading

hallvictoria Dec 1, 2025 •

edited

Loading