Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow ServiceBus AutoComplete to be configured #3179

Merged
merged 1 commit into from Jul 26, 2018

Conversation

Projects
None yet
7 participants
@mathewc
Copy link
Contributor

mathewc commented Jul 24, 2018

Addresses #2066.

Related SDK PR: Azure/azure-webjobs-sdk#1814. This change is required to enable the actual scenario of disabling AutoComplete and controlling message lifetime yourself.

Will make the same change in dev.

@mathewc mathewc force-pushed the sb-options branch from 43fc048 Jul 26, 2018

@mathewc mathewc force-pushed the sb-options branch to f41da7b Jul 26, 2018

@mathewc mathewc merged commit f41da7b into v1.x Jul 26, 2018

1 check passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Aug 17, 2018

@mathewc is there a pre-release of this we could try out at all? Checked nuget, but doesn't seem to be published?

It addresses the issues I was having in my StackOverflow post - https://stackoverflow.com/questions/51881103/calling-complete-or-deadletter-on-brokeredmessage-with-azure-functions-service/51889907

@braxler

This comment has been minimized.

Copy link

braxler commented Aug 23, 2018

Looks like it's working for v3.1.0, it didn't work for me for 1.x or 2.x

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Aug 23, 2018

@braxler - how can I try v3.1.0 ?
Is it pre-release?

@mathewc

This comment has been minimized.

Copy link
Contributor Author

mathewc commented Aug 24, 2018

This has been released as part of functions v1 release https://github.com/Azure/azure-functions-host/releases/tag/v1.0.11959. It is also part of our latest v2 beta releases.

@tomkuijsten

This comment has been minimized.

Copy link

tomkuijsten commented Aug 25, 2018

How do I use this in my function app? Is it just a matter of updating the nuget package or...?

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Aug 25, 2018

@tomkuijsten I have the same question!

@mathewc

This comment has been minimized.

Copy link
Contributor Author

mathewc commented Aug 27, 2018

If you're on the latest runtime version (v1 or v2) with the change, then you specify autoComplete: false in your host.json file, e.g.:

{
  "serviceBus": {
      "maxConcurrentCalls": 16,
      "prefetchCount": 100,
      "autoComplete": false
    }
}

If you have any problems with this, please log a new issue.

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Aug 28, 2018

@mathewc is this just for csx? Not pre-compiled functions?

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Aug 28, 2018

@mathewc
Happy for this to be moved to a new issue, but it feels like a duplicate of this, or I'm using the wrong runtime?

using runtime version 1.0.11913.0 this still happens:

The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue

If we have the following basic function:

[FunctionName("Function1")]
public static async Task Run(
    [ServiceBusTrigger("%Topic%", "%Subscription%", AccessRights.Manage, Connection = "ServiceBus.ConnectionString")]BrokeredMessage msg)
{
    await msg.DeadLetterAsync("reason", "description");
}
Listening on http://localhost:7071/
Hit CTRL-C to exit...
[28/08/2018 11:50:10] Reading host configuration file 'C:\****\bin\Debug\net471\host.json'
[28/08/2018 11:50:10] Host configuration file read:
[28/08/2018 11:50:10] {
[28/08/2018 11:50:10]   "serviceBus": {
[28/08/2018 11:50:10]     "autoComplete": false
[28/08/2018 11:50:10]   }
[28/08/2018 11:50:10] }
[28/08/2018 11:50:14] Starting Host (HostId=asglhwl19969-827647181, Version=1.0.11913.0, InstanceId=5aa426cb-bc7e-4162-ab2e-d6d2a7642bab, ProcessId=135424, AppDomainId=1, Debug=False, ConsecutiveErrors=0, StartupCount=1, FunctionsExtensionVersion=)
[28/08/2018 11:50:15] Loaded custom extension 'BotFrameworkConfiguration'
[28/08/2018 11:50:15] Loaded custom extension 'SendGridConfiguration'
[28/08/2018 11:50:15] Loaded custom extension 'EventGridExtensionConfig'
[28/08/2018 11:50:15] registered EventGrid Endpoint = http://localhost:7071/admin/extensions/EventGridExtensionConfig
[28/08/2018 11:50:15] Generating 2 job function(s)
[28/08/2018 11:50:15] Found the following functions:
[28/08/2018 11:50:15] ****.SubscriptionActivated.Run
[28/08/2018 11:50:15] ****.SubscriptionDecativated.Run
[28/08/2018 11:50:15]
[28/08/2018 11:50:15] Host initialized (6323ms)
[28/08/2018 11:50:16] Host lock lease acquired by instance ID '000000000000000000000000100BA934'.
Debugger listening on [::]:5858
[28/08/2018 11:50:17] Host started (8099ms)
[28/08/2018 11:50:17] Job host started
[28/08/2018 11:50:40] Function started (Id=b335df47-5c74-4638-a85a-8960f5f43b25)
[28/08/2018 11:50:41] Executing 'SubscriptionActivated' (Reason='New ServiceBus message detected on '***/Subscriptions/identity.subscriber'.', Id=b335df47-5c74-4638-a85a-8960f5f43b25)
[28/08/2018 11:50:42] Function completed (Success, Id=b335df47-5c74-4638-a85a-8960f5f43b25, Duration=1890ms)
[28/08/2018 11:50:42] Executed 'SubscriptionActivated' (Succeeded, Id=b335df47-5c74-4638-a85a-8960f5f43b25)
[28/08/2018 11:50:44] A ScriptHost error has occurred
[28/08/2018 11:50:44] Microsoft.ServiceBus: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue. TrackingId:ded0e336-c523-48c7-9952-d0d87bf8010b_G33_B10, SystemTracker:****:Topic:***|identity.subscriber, Timestamp:8/28/2018 11:50:43 AM.
[28/08/2018 11:50:44] MessageReceiver error (Action=Complete) : Microsoft.ServiceBus.Messaging.MessageLockLostException: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue. TrackingId:ded0e336-c523-48c7-9952-d0d87bf8010b_G33_B10, SystemTracker:****:Topic:***|identity.subscriber, Timestamp:8/28/2018 11:50:43 AM ---> System.ServiceModel.FaultException`1[System.ServiceModel.ExceptionDetail]: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue. TrackingId:ded0e336-c523-48c7-9952-d0d87bf8010b_G33_B10, SystemTracker:****:Topic:***|identity.subscriber, Timestamp:8/28/2018 11:50:43 AM
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.ThrowIfFaultMessage(Message wcfMessage)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.HandleMessageReceived(IAsyncResult result)
[28/08/2018 11:50:44] --- End of stack trace from previous location where exception was thrown ---
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Sbmp.DuplexRequestBindingElement.DuplexRequestSessionChannel.EndRequest(IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.RequestAsyncResult.<>c.<GetAsyncSteps>b__9_3(RequestAsyncResult thisPtr, IAsyncResult r)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
[28/08/2018 11:50:44] --- End of stack trace from previous location where exception was thrown ---
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.EndRequest(IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement.RedirectContainerChannelFactory`1.RedirectContainerSessionChannel.RequestAsyncResult.<>c__DisplayClass8_1.<GetAsyncSteps>b__4(RequestAsyncResult thisPtr, IAsyncResult r)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
[28/08/2018 11:50:44] --- End of stack trace from previous location where exception was thrown ---
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Sbmp.RedirectBindingElement.RedirectContainerChannelFactory`1.RedirectContainerSessionChannel.EndRequest(IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.RequestAsyncResult.<>c.<GetAsyncSteps>b__9_3(RequestAsyncResult thisPtr, IAsyncResult r)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
[28/08/2018 11:50:44] --- End of stack trace from previous location where exception was thrown ---
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Channels.ReconnectBindingElement.ReconnectChannelFactory`1.RequestSessionChannel.EndRequest(IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Sbmp.SbmpTransactionalAsyncResult`1.<>c.<GetAsyncSteps>b__18_3(TIteratorAsyncResult thisPtr, IAsyncResult a)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
[28/08/2018 11:50:44] --- End of stack trace from previous location where exception was thrown ---
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.Sbmp.SbmpMessageReceiver.EndUpdateCommand(IAsyncResult result)
[28/08/2018 11:50:44]    --- End of inner exception stack trace ---
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult.End[TAsyncResult](IAsyncResult result)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Common.AsyncResult`1.End(IAsyncResult asyncResult)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.<>c__22`1.<CallParallelAsync>b__22_1(TIteratorAsyncResult thisPtr, IAsyncResult r)
[28/08/2018 11:50:44]    at Microsoft.ServiceBus.Messaging.IteratorAsyncResult`1.StepCallback(IAsyncResult result)
[28/08/2018 11:50:44] Microsoft.ServiceBus: The lock supplied is invalid. Either the lock expired, or the message has already been removed from the queue. TrackingId:ded0e336-c523-48c7-9952-d0d87bf8010b_G33_B10, SystemTracker:****:Topic:***|identity.subscriber, Timestamp:8/28/2018 11:50:43 AM.
@tomkuijsten

This comment has been minimized.

Copy link

tomkuijsten commented Aug 30, 2018

@alexjamesbrown you should be on framework 1.0.11959.0 right? See release notes. Would explain why it's not yet working.

@tomkuijsten

This comment has been minimized.

Copy link

tomkuijsten commented Aug 30, 2018

@alexjamesbrown answer to my own question on forcing framework version (thanks to @keesschollaart81):

https://docs.microsoft.com/en-us/azure/azure-functions/set-runtime-version

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Aug 30, 2018

@tomkuijsten this only explains how to set it when deploying - not on how to set it for the local runtime
What I really want to be able to do is set FUNCTIONS_EXTENSION_VERSION in host.json (maybe??) - but this has no effect

@mathewc

This comment has been minimized.

Copy link
Contributor Author

mathewc commented Aug 30, 2018

You need to update your Azure Functions Tools to get the new version.

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Aug 30, 2018

@mathewc to which version please?

running func from the cmd line shows

Azure Functions Core Tools (1.0.14)
Function Runtime Version: 1.0.11913.0

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Aug 30, 2018

@mathewc ... which is weird, since 1.0.12 is listed as the 'latest version'
https://github.com/Azure/azure-functions-core-tools/releases/tag/1.0.12

I've installed via npm

@mathewc

This comment has been minimized.

Copy link
Contributor Author

mathewc commented Aug 30, 2018

Yes, it appears that we don't have CLI released with these changes, sorry. I'll start an internal thread on that and see why that is.

@mhoeger

This comment has been minimized.

Copy link
Contributor

mhoeger commented Aug 30, 2018

The CLI bits will be released shortly. Sorry about that @alexjamesbrown!

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Sep 21, 2018

@mhoeger @mathewc
sorry to drag this up again, but was this updated?

I've updated the cli, but VS still reports
ersion=1.0.11913.0

[21/09/2018 16:58:22] Host configuration file read:
[21/09/2018 16:58:22] {
[21/09/2018 16:58:22] "serviceBus": {
[21/09/2018 16:58:22] "autoComplete": false
[21/09/2018 16:58:22] }
[21/09/2018 16:58:22] }
[21/09/2018 16:58:26] Starting Host (HostId=asglhwl19969-827712717, Version=1.0.11913.0, InstanceId=01e2d5f1-96d1-47a2-8198-bd8a1e28eff7, ProcessId=557084, AppDomainId=1, Debug=False, ConsecutiveErrors=0, StartupCount=1, FunctionsExtensionVersion=)

@mhoeger

This comment has been minimized.

Copy link
Contributor

mhoeger commented Sep 21, 2018

@alexjamesbrown - The 1.0.11959 bits should have been released on August 30th. Let me double check!

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Sep 21, 2018

@mhoeger if I do
func

On cmd line, I get that version.... But visual studio (when hitting f5 in a function project) seems to still be running a previous version.

I've updated the Azure functions extension in vs

@mhoeger

This comment has been minimized.

Copy link
Contributor

mhoeger commented Sep 21, 2018

@alexjamesbrown - could you try restarting VS and trying again? It looks like those bits were moved to "pre-release" and never fully released. So sorry about the long delay in getting your fix!

@alexjamesbrown

This comment has been minimized.

Copy link

alexjamesbrown commented Sep 24, 2018

@mhoeger thanks, it looks like it's working now

@mpMelnikov

This comment has been minimized.

Copy link

mpMelnikov commented Oct 30, 2018

Should it work in v.2 functions?
Currently, I get "The lock supplied is invalid". I tried the following two host.json:

{
  "version": "2.0",
  "serviceBus": {
      "autoComplete": false
  }
}

and (as schema in master branch has messageHandlerOptions option)

{
  "version": "2.0",
  "serviceBus": {
    "messageHandlerOptions": {
      "autoComplete": false
    }
  }
}
@mpMelnikov

This comment has been minimized.

Copy link

mpMelnikov commented Oct 30, 2018

As it was explained here, "extensions" section is required.
Hosts.json which seems to work:

{
  "version": "2.0",
  "extensions": {
    "serviceBus": {
      "messageHandlerOptions": {
        "autoComplete": false
      }
    }
  }
}

@mathewc thank you again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.