Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Frozen board #112

Closed
TurboPablos opened this issue Dec 7, 2017 · 6 comments
Closed

Frozen board #112

TurboPablos opened this issue Dec 7, 2017 · 6 comments

Comments

@TurboPablos
Copy link

Sometimes Sirene of Shame board is like frozen and then we need to click "Refresh all" button.

I wrote "sometimes" because unfortunetely we haven't found any pattern for it.

Have you ever encountered this kind of problem? If yes, it would be nice to fix it :)

@lprichar
Copy link
Member

lprichar commented Dec 8, 2017

I'm afraid I haven't seen that happen. Is there anything suspicious in the logs?

@bo-stig-christensen
Copy link

We had a look into the logs and found this:

2017-12-19 16:31:34,058 [CiWatcher] INFO SirenOfShame.Lib.ExceptionMessageBox [(null)] - StackTrace: System.AggregateException: One or more errors occurred. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at System.Threading.Tasks.Task1.get_Result()
at TfsRestServices.TfsRestWatcher.GetBuildStatus()
at SirenOfShame.Lib.Watcher.WatcherBase.GetBuildStatusAndFireEvents()
at SirenOfShame.Lib.Watcher.WatcherBase.StartWatching()
---> (Inner Exception #0) System.Threading.Tasks.TaskCanceledException: A task was canceled.<---

I had a quick look into the source code and correlated it with the fact that we sometimes may have a bit unstable internet connection. - If this happens to coincide with the TfsRestWatcher asking for build statuses, it probably gets a timeout through the TaskCancelled exception.

The watcher should probably have some resilience built in to harden it against timeouts and similar. - A good library for this could be Polly.

@lprichar
Copy link
Member

lprichar commented Jan 4, 2018

Nice, that's super-useful, thanks! I'm a huge fan of Polly, but there's already a server unavailable retry facility built in. I'll try to recreate tonight or this weekend and get in a fix shortly thereafter.

@bo-stig-christensen
Copy link

Did you find a way to implement resilience? ;-)

lprichar added a commit that referenced this issue Jan 24, 2018
lprichar added a commit that referenced this issue Jan 24, 2018
@lprichar lprichar mentioned this issue Jan 24, 2018
4 tasks
lprichar added a commit that referenced this issue Jan 24, 2018
@lprichar
Copy link
Member

So sorry for the long delay @bo-stig-christensen. I was able to reproduce by simulating 30 second+ network delays via Fiddler. I just published an update 2.4.12 that should contain the fix for you: https://sirenofshame.com/Downloads. Please re-open this issue if that fails to resolve the issue, and thanks again for finding that stack trace!

@bo-stig-christensen
Copy link

Thank you :)
We have installed the update and will now monitor if this fixes the issues we had :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants