Enable new queueworker#3786
Merged
Merged
Conversation
3e1f7b4 to
6a38259
Compare
| <Compile Include="RuntimeTypesAst.fs" /> | ||
| <Compile Include="DvalReprInternal.fs" /> | ||
| <Compile Include="DvalReprInternalDeprecated.fs" /> | ||
| <Compile Include="DvalReprInternalNew.fs" /> |
Member
There was a problem hiding this comment.
(nitpick) maybe without "New"? I'm not sure what value the suffix brings. Assumedly we're going to rip out the Deprecated one before long, at which point we'd strip the "new" anyway? Not a big deal either way.
Member
Author
There was a problem hiding this comment.
Yeah, could remove that. It was initially to make fully sure that they had all been renamed (eg, get rid of the old name so that the compiler will tell you if you forgot to update something). but now that that's done, there probably isn't a value to the new name.
Actually, that's only true once this merges. Maybe I'll remove this on the next iteration.
StachuDotNet
approved these changes
May 13, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This started as an attempt to enable the new queueworker, and morphed into replacing the queues entirely.
The old queue was (well, still is) quite janky. It's built on postgres and currently eats about 50% of our database's CPU capacity. It has long been a desire to replace it with Real Queues™️. As I started testing out the transition, it became clear that a direct port of the old queueworkers to the new one was untenable, as we used a single transaction around the entire job. That had been causing problems before, but we now do exceptions on timeouts and have connection pools and multi-threading and such, so as well as seeing actual problems, it seemed like there were enough unknown-unknowns that I wasn't confident switching over.
Hence, made a new plan, with a new design.
The remaining work is documented here. The next step is to manually deploy it to the QueueWorker and see what happens. If that works, we'll merge and get it deployed. Since the pubsub queues are empty at the moment until we enable it on a per-canvas basis, we can check configuration, metrics, etc, while slowly sending some events to it.
LaunchDarkly changes
I also changed how LaunchDarkly works, as it was a little error-prone. Now we add functions to the
LibService.LDmodule, just like we do withLibService.Config. The definition has name, defaults, and test defaults, and has a type ensuring we get the right kind of "user", so it's much hard to fuck up.Serialization changes
The format we used for storing values in the queue was bad and unsafe, so I made a new one. The new one is a direct serialization of a type, and the type is now separated from whatever
RT.Dvalhappens to be at the time. The old format is still used in other places, but this removes one usage at least.Misc changes: