-
-
Notifications
You must be signed in to change notification settings - Fork 93
Enable new queueworker #3786
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Enable new queueworker #3786
Conversation
3e1f7b4
to
6a38259
Compare
@@ -16,7 +16,8 @@ | |||
<Compile Include="VendoredTablecloth.fs" /> | |||
<Compile Include="RuntimeTypes.fs" /> | |||
<Compile Include="RuntimeTypesAst.fs" /> | |||
<Compile Include="DvalReprInternal.fs" /> | |||
<Compile Include="DvalReprInternalDeprecated.fs" /> | |||
<Compile Include="DvalReprInternalNew.fs" /> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(nitpick) maybe without "New"? I'm not sure what value the suffix brings. Assumedly we're going to rip out the Deprecated one before long, at which point we'd strip the "new" anyway? Not a big deal either way.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, could remove that. It was initially to make fully sure that they had all been renamed (eg, get rid of the old name so that the compiler will tell you if you forgot to update something). but now that that's done, there probably isn't a value to the new name.
Actually, that's only true once this merges. Maybe I'll remove this on the next iteration.
This started as an attempt to enable the new queueworker, and morphed into replacing the queues entirely.
The old queue was (well, still is) quite janky. It's built on postgres and currently eats about 50% of our database's CPU capacity. It has long been a desire to replace it with Real Queues™️. As I started testing out the transition, it became clear that a direct port of the old queueworkers to the new one was untenable, as we used a single transaction around the entire job. That had been causing problems before, but we now do exceptions on timeouts and have connection pools and multi-threading and such, so as well as seeing actual problems, it seemed like there were enough unknown-unknowns that I wasn't confident switching over.
Hence, made a new plan, with a new design.
The remaining work is documented here. The next step is to manually deploy it to the QueueWorker and see what happens. If that works, we'll merge and get it deployed. Since the pubsub queues are empty at the moment until we enable it on a per-canvas basis, we can check configuration, metrics, etc, while slowly sending some events to it.
LaunchDarkly changes
I also changed how LaunchDarkly works, as it was a little error-prone. Now we add functions to the
LibService.LD
module, just like we do withLibService.Config
. The definition has name, defaults, and test defaults, and has a type ensuring we get the right kind of "user", so it's much hard to fuck up.Serialization changes
The format we used for storing values in the queue was bad and unsafe, so I made a new one. The new one is a direct serialization of a type, and the type is now separated from whatever
RT.Dval
happens to be at the time. The old format is still used in other places, but this removes one usage at least.Misc changes: