[BUG] Writing multiple messages to a single Eventhub

[Oliv4945]

Investigative information

• Timestamp: 2021-12-18T18:29:00
• Function App name: (debugging locally)
• Function name(s) (as appropriate):(debugging locally)
• Core Tools version: 4.0.3971 Commit hash: d0775d487c93ebd49e9c1166d5c3c01f3c76eaaf (64-bit)

Repro steps

The function.json file is

{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "type": "eventHubTrigger",
      "name": "inputEvent",
      "direction": "in",
      "eventHubName": "device.data.raw",
      "connection": "EVENT_HUB_CONNECTION_STRING",
      "cardinality": "one",
      "consumerGroup": "$Default",
      "dataType": "string"
    },
    {
      "type": "eventHub",
      "direction": "out",
      "name": "$return",
      "connection": "EVENT_HUB_CONNECTION_STRING",
      "eventHubName": "device.data.decoded"
    }
  ]
}

Test 1 - As per first proposition of 327

def main(inputEvent: func.EventHubEvent) -> typing.List[func.EventHubEvent]:
    return (inputEvent.get_body(), inputEvent.get_body())

Test 2 - As per first proposition of 327

def main(inputEvent: func.EventHubEvent) -> typing.List[func.EventHubEvent]:
    return (inputEvent.get_body(), inputEvent.get_body())

Test 3 - As per #192

def main(inputEvent: func.EventHubEvent) -> str:
    return (inputEvent.get_body(), inputEvent.get_body())

Expected behavior

One of the methods should be able to send several messages to Eventhub

Actual behavior

Test 1

Function can not be started

[2021-12-08T17:36:04.098Z] Executed 'Functions.decode' (Failed, Id=a20e6894-0641-48be-a974-98e13ba7517e, Duration=329ms)
[2021-12-08T17:36:04.098Z] System.Private.CoreLib: Exception while executing function: Functions.decode. System.Private.CoreLib: Result: Failure
Exception: FunctionLoadError: cannot load the decode function: has invalid non-type return annotation typing.List[azure.functions._eventhub.EventHubEvent]
Stack:   File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\dispatcher.py", line 311, in _handle__function_load_request
    self._functions.add_function(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\functions.py", line 240, in add_function
    raise FunctionLoadError(
.

Test 2

Function can not be started

[2021-12-08T17:40:06.521Z] Executed 'Functions.decode' (Failed, Id=f3a5d015-67b4-4b50-bdb4-a4d14a3625de, Duration=256ms)
[2021-12-08T17:40:06.527Z] System.Private.CoreLib: Exception while executing function: Functions.decode. System.Private.CoreLib: Result: Failure
Exception: FunctionLoadError: cannot load the decode function: has invalid non-type return annotation typing.List[str]
Stack:   File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\dispatcher.py", line 311, in _handle__function_load_request
    self._functions.add_function(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\functions.py", line 240, in add_function
    raise FunctionLoadError(
.

Test 3

Function is started but there is an execution error:

[2021-12-08T17:56:36.411Z] Executed 'Functions.decode' (Failed, Id=cc123efe-9b7a-419c-8e4a-a29b3be2e8cb, Duration=40ms)
[2021-12-08T17:56:36.412Z] System.Private.CoreLib: Exception while executing function: Functions.decode. System.Private.CoreLib: Result: Failure
Exception: NotImplementedError: unexpected Datum type: None
Stack:   File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\dispatcher.py", line 427, in _handle__invocation_request
    return_value = bindings.to_outgoing_proto(
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\bindings\meta.py", line 117, in to_outgoing_proto
    return datumdef.datum_as_proto(datum)
  File "C:\Program Files\Microsoft\Azure Functions Core Tools\workers\python\3.9/WINDOWS/X64\azure_functions_worker\bindings\datumdef.py", line 190, in datum_as_proto
    raise NotImplementedError(
.

Known workarounds

N/A

Contents of the requirements.txt file:

azure-functions==1.8.0
requests==2.26.0

[Oliv4945]

Hi, Microsoft team,

Did you had the opportunity to look at this? Thanks

[YunchuWang]

@Oliv4945 sorry for late reply! is the issue fixed for you now?

[YunchuWang]

I am able to reproduce this issue on coretool v4, py 3.9.

[Oliv4945]

Hi @YunchuWang, yes I still have the issue. Glad that you are able to reproduce

[YunchuWang]

@Oliv4945 thanks for the reply and raising the issue! I come up with a working solution writing multiple strings. (please see below)

Apology for missing examples on writing multiple messages to eventhub via output. Essentially when you return multiple messages in eventhub output, please ensure return a list of json-serializable objects (ref at how python worker encodes eventhub output value at https://github.com/Azure/azure-functions-python-library/blob/dev/azure/functions/eventhub.py#L90-L91)

Python worker accepts return type hint as str, bytes or class which is instance of type. If putting typing module class like List[XXX], the type of List is <class 'typing._SpecialGenericAlias'> which will cause error. (Please ref at how worker validates function return annotation at https://github.com/Azure/azure-functions-python-worker/blob/dev/azure_functions_worker/functions.py#L230-L248)

Funtime version: ~4
Python version: 3.9

Function file

from typing import List
import azure.functions as func


def main(events: List[func.EventHubEvent]) -> str:
    return [events[0].get_body().decode('utf-8'), events[1].get_body().decode('utf-8')]

function.json file

{
  "scriptFile": "__init__.py",
  "bindings": [
    {
      "type": "eventHubTrigger",
      "name": "events",
      "direction": "in",
      "eventHubName": "input",
      "connection": "EVENT_HUB_CONNECTION",
      "cardinality": "many",
      "consumerGroup": "$Default",
      "dataType": "binary"
    },
    {
      "type": "eventHub",
      "direction": "out",
      "name": "$return",
      "connection": "EVENT_HUB_CONNECTION",
      "eventHubName": "output_event"
    }
  ]
}

[YunchuWang]

Please let me know if you have any further questions.

[Oliv4945]

Thanks @YunchuWang for those explanations.