PerformanceCounterCategory.Exists hangs #1158

Closed
brucechase opened this Issue Dec 18, 2012 · 14 comments

Comments

Projects
None yet
5 participants
@brucechase

Under the alpha version of SignalR, the Hub server hangs on the call to PerformanceCounterCategory.Exists (commented out in source below) within the Core.Infrastructure.PerformanceCounterManager.cs

private static IPerformanceCounter LoadCounter(string categoryName, string counterName, string instanceName)
        {
            try
            {
//                if (PerformanceCounterCategory.Exists(categoryName) && PerformanceCounterCategory.CounterExists(counterName, categoryName))
//                {
//                    return new PerformanceCounterWrapper(new PerformanceCounter(categoryName, counterName, instanceName, readOnly: false));
//                }
                return null;
            }
            catch (InvalidOperationException) { return null; }
            catch (UnauthorizedAccessException) { return null; }
        }

@davidfowl davidfowl closed this Dec 18, 2012

@ghost ghost assigned davidfowl Jan 6, 2013

@davidfowl davidfowl reopened this Jan 6, 2013

@davidfowl

This comment has been minimized.

Show comment Hide comment
@davidfowl

davidfowl Jan 6, 2013

Member

I just submitted a potential fix for this issue.

Member

davidfowl commented Jan 6, 2013

I just submitted a potential fix for this issue.

@davidfowl

This comment has been minimized.

Show comment Hide comment
@davidfowl

davidfowl Jan 6, 2013

Member

Fixed in 8adb268

Member

davidfowl commented Jan 6, 2013

Fixed in 8adb268

@Xiaohongt

This comment has been minimized.

Show comment Hide comment
@Xiaohongt

Xiaohongt Jan 7, 2013

Contributor

verified

Contributor

Xiaohongt commented Jan 7, 2013

verified

@Xiaohongt Xiaohongt closed this Jan 7, 2013

@khalidabuhakmeh

This comment has been minimized.

Show comment Hide comment
@khalidabuhakmeh

khalidabuhakmeh Feb 15, 2013

Hello, we are still seeing this issue. Whether the request for performance counter categories is done in a separate thread or not. I found this interesting stackoverflow post. The issue is really more of an environmental setup issue, than a SignalR issue.

http://stackoverflow.com/questions/2655067/performancecountercategory-exists-is-very-slow-if-category-doesnt-exists

We have isolated the issue to PerformanceCounterCategory. It does look like the implementation is sub-optimal, as it loops through EVERY performance counter to find the one you are talking about. We tried to do a PerformanceCounterCategory.GetCategories() but even that hangs (was hoping to do it in .NET memory rather than differing to Win32).

Two options to get around this issue:

  • Config setting specifying where you want performance counters or not.
  • Create performance counters when the SignalR app starts (overwrite if it already exists).

Thanks,

Khalid

Hello, we are still seeing this issue. Whether the request for performance counter categories is done in a separate thread or not. I found this interesting stackoverflow post. The issue is really more of an environmental setup issue, than a SignalR issue.

http://stackoverflow.com/questions/2655067/performancecountercategory-exists-is-very-slow-if-category-doesnt-exists

We have isolated the issue to PerformanceCounterCategory. It does look like the implementation is sub-optimal, as it loops through EVERY performance counter to find the one you are talking about. We tried to do a PerformanceCounterCategory.GetCategories() but even that hangs (was hoping to do it in .NET memory rather than differing to Win32).

Two options to get around this issue:

  • Config setting specifying where you want performance counters or not.
  • Create performance counters when the SignalR app starts (overwrite if it already exists).

Thanks,

Khalid

@davidfowl davidfowl reopened this Feb 15, 2013

@davidfowl

This comment has been minimized.

Show comment Hide comment
@davidfowl

davidfowl Feb 15, 2013

Member

Sub-optimal != hang forever. The latter is what is occurring and if we can avoid it by being smart (which to date we haven't been able to do), then I'd prefer going with that fix. I'm still puzzled as to why running it on another task would still hang. I attempted a fix by starting a new thread instead of a new task and aborting the thread if it failed to come back within 2 seconds.

If the first one fails then none of the rest should load.

Member

davidfowl commented Feb 15, 2013

Sub-optimal != hang forever. The latter is what is occurring and if we can avoid it by being smart (which to date we haven't been able to do), then I'd prefer going with that fix. I'm still puzzled as to why running it on another task would still hang. I attempted a fix by starting a new thread instead of a new task and aborting the thread if it failed to come back within 2 seconds.

If the first one fails then none of the rest should load.

@khalidabuhakmeh

This comment has been minimized.

Show comment Hide comment
@khalidabuhakmeh

khalidabuhakmeh Feb 15, 2013

Haha, yeah sub-optimal shouldn't hang forever. We are baffled too. We see the exception thrown from the thread, but it goes nowhere and nothing happens after that. I would also have assumed starting on a separate thread wouldn't hang the original thread.

We removed the threading (for test purposes), and just called the method directly. We saw a bunch of Win32 threads spawn up and just keep coming. It is one about every second.

The odd thing, is this only happens on my co-workers computer. My machine is fine.

Haha, yeah sub-optimal shouldn't hang forever. We are baffled too. We see the exception thrown from the thread, but it goes nowhere and nothing happens after that. I would also have assumed starting on a separate thread wouldn't hang the original thread.

We removed the threading (for test purposes), and just called the method directly. We saw a bunch of Win32 threads spawn up and just keep coming. It is one about every second.

The odd thing, is this only happens on my co-workers computer. My machine is fine.

@davidfowl

This comment has been minimized.

Show comment Hide comment
@davidfowl

davidfowl Feb 19, 2013

Member

@khalidabuhakmeh any updates on this? Can you try installing the performance counters and see if that makes it work? I'd like to have a broken machine around to see if we can work around it though.

Member

davidfowl commented Feb 19, 2013

@khalidabuhakmeh any updates on this? Can you try installing the performance counters and see if that makes it work? I'd like to have a broken machine around to see if we can work around it though.

@khalidabuhakmeh

This comment has been minimized.

Show comment Hide comment
@khalidabuhakmeh

khalidabuhakmeh Feb 19, 2013

@davidfowl we tried it both with performance counters installed and uninstalled, both yielding the same results on that machine. It does have something to do with the IIS Express process, because we to the breaking code and placed it in an ASP.Net MVC project and a Console application. The web project hung as suspected, while the console behaved fine.

@davidfowl we tried it both with performance counters installed and uninstalled, both yielding the same results on that machine. It does have something to do with the IIS Express process, because we to the breaking code and placed it in an ASP.Net MVC project and a Console application. The web project hung as suspected, while the console behaved fine.

@khalidabuhakmeh

This comment has been minimized.

Show comment Hide comment
@khalidabuhakmeh

khalidabuhakmeh Feb 19, 2013

@davidfowl There seems to be the same issue with RavenDB when running in IIS Express with the same symptoms that exist here. The user documents their solution. I'll try it and see if it solved the issue.

https://groups.google.com/forum/?fromgroups=#!topic/ravendb/Pp1SZtXuicM

To quote the solution from the discussion:

So then I knew IIS Express 8 was the issue, so I proceeded to uninstall it, then I deleted the IIS Express directories from "C:\Program Files" and "C:\Program Files (x86)", the I deleted the folder "IIS Express" under "C:\Users{Your Name}\My Documents", then I reinstalled IIS Express 8 using the Web platform installer.

@davidfowl There seems to be the same issue with RavenDB when running in IIS Express with the same symptoms that exist here. The user documents their solution. I'll try it and see if it solved the issue.

https://groups.google.com/forum/?fromgroups=#!topic/ravendb/Pp1SZtXuicM

To quote the solution from the discussion:

So then I knew IIS Express 8 was the issue, so I proceeded to uninstall it, then I deleted the IIS Express directories from "C:\Program Files" and "C:\Program Files (x86)", the I deleted the folder "IIS Express" under "C:\Users{Your Name}\My Documents", then I reinstalled IIS Express 8 using the Web platform installer.

davidfowl added a commit that referenced this issue Feb 22, 2013

Fix performance counter hanging issues.
- Always try to create the SignalR performance counters without checking
  for existence since that hangs.

#1158
@davidfowl

This comment has been minimized.

Show comment Hide comment
@davidfowl

davidfowl Feb 22, 2013

Member

@khalidabuhakmeh I have a pull request pending with a potential fix. Can you please try this out and see if it works?

Member

davidfowl commented Feb 22, 2013

@khalidabuhakmeh I have a pull request pending with a potential fix. Can you please try this out and see if it works?

@davidfowl

This comment has been minimized.

Show comment Hide comment
@davidfowl

davidfowl Feb 24, 2013

Member

I'm not convinced this will fix the problem since it'll likely end up in the same code path:

See the stack https://gist.github.com/davidfowl/4959507

Member

davidfowl commented Feb 24, 2013

I'm not convinced this will fix the problem since it'll likely end up in the same code path:

See the stack https://gist.github.com/davidfowl/4959507

@teelahti

This comment has been minimized.

Show comment Hide comment
@teelahti

teelahti Feb 25, 2013

I can also verify this problem. Win8 + IIS express 8, reinstalling IIS express did not help. SignalR hangs forever during MapHubs(). Slightly irrititating as I am preparing a SignalR talk for Finnish MS Techdays...

I can also verify this problem. Win8 + IIS express 8, reinstalling IIS express did not help. SignalR hangs forever during MapHubs(). Slightly irrititating as I am preparing a SignalR talk for Finnish MS Techdays...

@Xiaohongt

This comment has been minimized.

Show comment Hide comment
@Xiaohongt

Xiaohongt Feb 26, 2013

Contributor

verified

Contributor

Xiaohongt commented Feb 26, 2013

verified

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment