Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug (NetworkManger): The key already existed in the dictionary #925

Closed
wants to merge 2 commits into from

Conversation

Im5tu
Copy link
Contributor

@Im5tu Im5tu commented Feb 19, 2019

Recently we have been seeing the following under load:

ex_msg: The key already existed in the dictionary. | stacktrace: System.ArgumentException: The key already existed in the dictionary.
   at System.Collections.Concurrent.ConcurrentDictionary`2.System.Collections.Generic.IDictionary<TKey,TValue>.Add(TKey key, TValue value)
   at PuppeteerSharp.NetworkManager.<OnRequestWillBeSentAsync>d__44.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at PuppeteerSharp.NetworkManager.<Client_MessageReceived>d__36.MoveNext()

This PR looks at some of the causes for this. We are still testing the changes, but wanted to get this here for your visibility and thoughts also.

I've managed to:

  • Remove two of the dictionaries that were previously required
  • Optimise the way some of the dictionaries were used (ie: using TryRemove instead of Get/Remove)
  • Fixed up some event invocation handling
  • Fixed a few minor race conditions (ie: MultiMap.Add - note that there is still a problem when the list needs to grow the array, but from what I can see the default capacity should be enough to warrant not locking)

... all whilst hopefully kept the same semantics.

It'd be really nice if chrome fixed the mentioned bug and started to send back the request id, then the entire class could be handled with a single ConcurrentDictionary. I know this is not you though ;)

@kblok
Copy link
Member

kblok commented Feb 19, 2019

I'll go throw this tomorrow. But I get your point. Sometimes it sucks branching from Puppeteer's code but this makes sense.
Let's see what AppVeyor thinks about this :p

@Im5tu
Copy link
Contributor Author

Im5tu commented Feb 20, 2019

So it seems appveyor is having the same trouble I am running the test suite. Is this a known issue or more likely something that i've messed up?

@kblok
Copy link
Member

kblok commented Feb 20, 2019

@Im5tu can you pull from master?

@Im5tu
Copy link
Contributor Author

Im5tu commented Feb 21, 2019

@kblok - Done :)

{
var frame = await FrameManager?.GetFrameAsync(e.FrameId);

request = new Request(
request.RawRequest = new Request(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WorkerTests.ShouldEmitCreatedAndDestroyedEvents is failing around here. request is null, so we fail setting RawRequest

@kblok
Copy link
Member

kblok commented Feb 22, 2019

I think we still have something on this test ShouldNotCrashWhileRedirectingIfOriginalRequest

@Im5tu
Copy link
Contributor Author

Im5tu commented Feb 26, 2019

It appears that we get the redirect response prior to getting the request will send portion. I haven't managed to take a look further yet - should be able to get back to resolving this tomorrow :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants