-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make activation polling for blocking invocations configurable #4088
Conversation
@@ -232,6 +232,8 @@ | |||
|
|||
"CONFIG_whisk_transactions_header": "{{ transactions.header }}" | |||
|
|||
"CONFIG_whisk_controller_activation_polling": "{{ controller_activation_polling | default(true) }}" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add _from_db
perhaps?
a3a9cbc
to
a63f1ab
Compare
Codecov Report
@@ Coverage Diff @@
## master #4088 +/- ##
==========================================
- Coverage 85.86% 81.22% -4.64%
==========================================
Files 148 148
Lines 7249 7255 +6
Branches 445 439 -6
==========================================
- Hits 6224 5893 -331
- Misses 1025 1362 +337
Continue to review full report at Codecov.
|
Changed this back to a WIP state as the two active ack changes do not currently allow DB polling to be disabled. |
@@ -593,11 +596,15 @@ protected[actions] trait PrimitiveActions { | |||
// in case of an incomplete active-ack (record too large for example). | |||
activeAckResponse.foreach { | |||
case Right(activation) => result.trySuccess(Right(activation)) | |||
case _ => pollActivation(docid, context, result, i => 1.seconds + (2.seconds * i), maxRetries = 4) | |||
case _ if (controllerActivationConfig.pollingFromDb) => | |||
pollActivation(docid, context, result, i => 1.seconds + (2.seconds * i), maxRetries = 4) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm looking at a case right now where if this pollActivation is removed, the acks are received in the wrong order in some cases (completion before result) - have you seen anything similar? unfortunately, it is something weird where it only happens on first run of the controller - subsequent runs of same or different actions don't exhibit the problem. Restarting controller, and the first invocation always gets the acks in wrong order again.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I see the issue causing the out of order messages - the first call to AcknowledegmentMessage.parse(raw)
is slow, resulting in out of order messages IFF the result + completion are sent at approx same time from invoker. I'll try to figure out a way to force the order of these.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tysonnorris is this related to #4134, which supposedly has been fixed by now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tysonnorris, I was seeing the same problem with large activations responses. Let me rebase these changes with #4134 to see if the problem has been resolved.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After rebasing it looks the two active ack changes no longer required DB polling.
@dubee this lgtm - do you want to rebase to master? |
@rabbah, rebased and ready to go. |
…#4088) * Make activation polling for blocking invocations configurable * Update Tests
Description
Allows controller activation polling for blocking invocations to be enabled or disable via configuration.
Related issue and scope
My changes affect the following components
Types of changes
Checklist: