Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hopac.Job.toAsync nullrefexn when try-with is used #153

Closed
haf opened this issue Mar 26, 2017 · 16 comments
Closed

Hopac.Job.toAsync nullrefexn when try-with is used #153

haf opened this issue Mar 26, 2017 · 16 comments

Comments

@haf
Copy link
Member

haf commented Mar 26, 2017

System.ArgumentNullException: key
Parameter name: Null key
  at System.Runtime.CompilerServices.ConditionalWeakTable`2[TKey,TValue].TryGetValue (TKey key, TValue& value) [0x0000b] in <f712f98eb8e445c8918edaf595bbe465>:0
  at Microsoft.FSharp.Control.AsyncBuilderImpl.MayLoseStackTrace (System.Exception exn) [0x00000] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:762
  at <StartupCode$FSharp-Core>.$Control+FromContinuations@1371-1.Invoke (System.Exception exn) [0x00000] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:1371
  at Hopac.Job+toAsync@1497-8[x].DoHandle (Hopac.Core.Worker& wr, System.Exception e) [0x0002f] in <57eeaba40b63217ba7450383a4abee57>:0
  at Hopac.Core.Handler.DoHandle (Hopac.Core.Handler hr, Hopac.Core.Worker& wr, System.Exception e) [0x00006] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.ContBind`2[X,Y].DoHandle (Hopac.Core.Worker& wr, System.Exception e) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.Handler.DoHandle (Hopac.Core.Handler hr, Hopac.Core.Worker& wr, System.Exception e) [0x00006] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.FailWork.DoWork (Hopac.Core.Worker& wr) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.Worker.Run (Hopac.Scheduler sr, System.Int32 me) [0x000aa] in <82aa62bc59cf45c1998a2363225b549c>:0

With code like this

screen shot 2017-03-26 at 18 59 49

e is null. The stack-trace above is if you comment out the try with. The code has the structure of;

job { try do! stanza .. with .. } |> Job.toAsync

Alternatively:

            do! Job.tryWith (stanza context) (fun e ->
              dt.oneFailed := true
              logger.debug (eventX "Stanza {stanza} raised an exception." >> setField "stanza" action)
              Job.raises e
            )

If I use |> run inside the async, this is the stack trace:

WARNING: You are making a blocking call to run a Hopac job from within a Hopac job, which means that your program may deadlock.
First occurrence (there may be others):
[23:30:26 DBG] Running stanza 0, action '(s) #handled'
  at System.Environment.get_StackTrace () [0x00000] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/corlib/System/Environment.cs:321
  at Hopac.Core.Condition.Wait (System.Object o, System.Int32& v) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Scheduler.run[x] (Hopac.Scheduler sr, Hopac.Job`1[T] xJ) [0x00000] in <57eeaba40b63217ba7450383a4abee57>:0
  at Hopac.Job+Global.run[x] (Hopac.Job`1[T] xJ) [0x00000] in <57eeaba40b63217ba7450383a4abee57>:0
  at EndToEnd+DSL+test@843-14[a,b,c,d,ctx].Invoke (Microsoft.FSharp.Core.Unit unitVar) [0x00000] in EndToEnd.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@839[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x0002b] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@839[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00065] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+tryWithDispatchInfoA@873[a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00038] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:878
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@839[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00065] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncBuilderImpl+resultA@805[a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x0002d] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:809
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncBuilderImpl+getCancellationToken@890.Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x00000] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:890
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@839[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00065] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@839[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00065] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@839[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00065] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncBuilderImpl+resultA@805[a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x0002d] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:809
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncBuilderImpl+resultA@805[a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x0002d] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:809
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncBuilderImpl+resultA@805[a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x0002d] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:809
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at <StartupCode$FSharp-Core>.$Control+ReifyResult@1785[T].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg7) [0x00000] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:1787
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncImpl+get_AwaitResult@1160[T].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x0005f] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:1176
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@839[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00065] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+bindA@817[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00075] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:834
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncBuilderImpl+resultA@805[a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x0002d] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:809
  at Microsoft.FSharp.Control.AsyncBuilderImpl+callA@839[b,a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] args) [0x00065] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:843
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncBuilderImpl+resultA@805[a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x0002d] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:809
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at Microsoft.FSharp.Control.AsyncBuilderImpl+resultA@805[a].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x0002d] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:809
  at Microsoft.FSharp.Control.AsyncBuilderImpl+args@823-1[a,b].Invoke (a a) [0x0002e] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:823
  at <StartupCode$FSharp-Core>.$Control+once@1369-3[a].Invoke (Microsoft.FSharp.Core.Unit unitVar0) [0x00000] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:1369
  at <StartupCode$FSharp-Core>.$Control.loop@425-51 (Microsoft.FSharp.Control.Trampoline this, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] action) [0x00000] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:426
  at Microsoft.FSharp.Control.Trampoline.ExecuteAction (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] firstAction) [0x00018] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:441
  at Microsoft.FSharp.Control.TrampolineHolder.Protect (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] firstAction) [0x00032] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:553
  at <StartupCode$FSharp-Core>.$Control.once@1361[a] (Microsoft.FSharp.Control.AsyncParamsAux aux, Microsoft.FSharp.Core.FSharpRef`1[T] underCurrentThreadStack, Microsoft.FSharp.Core.FSharpRef`1[T] contToTailCall, System.Threading.Thread thread, Microsoft.FSharp.Control.AsyncImpl+Latch latch, Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] cont, a x) [0x00081] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:1369
  at <StartupCode$FSharp-Core>.$Control+once@1361-4T[a].Invoke (Microsoft.FSharp.Core.FSharpFunc`2[T,TResult] cont, a x) [0x00000] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:1361
  at Microsoft.FSharp.Core.OptimizedClosures+Invoke@3253[T2,TResult,T1].Invoke (T2 u) [0x00000] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/prim-types.fs:3253
  at Hopac.Job+toAsync@1497-8[x].DoWork (Hopac.Core.Worker& wr) [0x00000] in <57eeaba40b63217ba7450383a4abee57>:0
  at Hopac.Job+xK'@1282-3[?,?].DoWork (Hopac.Core.Worker& wr) [0x00000] in <57eeaba40b63217ba7450383a4abee57>:0
  at Hopac.MoreUtil+DropCont`2[x,y].DoCont (Hopac.Core.Worker& wr, y _arg1) [0x00000] in <57eeaba40b63217ba7450383a4abee57>:0
  at Hopac.Core.Cont.Do[T] (Hopac.Core.Cont`1[T] tK, Hopac.Core.Worker& wr, T value) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Job+result@1022[x].DoJob (Hopac.Core.Worker& wr, Hopac.Core.Cont`1[T] aK) [0x00000] in <57eeaba40b63217ba7450383a4abee57>:0
  at Hopac.Infixes+op_GreaterGreaterMinusDot@204[y,?].DoJob (Hopac.Core.Worker& wr, Hopac.Core.Cont`1[T] aK) [0x00000] in <57eeaba40b63217ba7450383a4abee57>:0
  at Hopac.Core.ContBind`2[X,Y].DoCont (Hopac.Core.Worker& wr, X x) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.ContMap`2[X,Y].DoCont (Hopac.Core.Worker& wr, X x) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.Cont.Do[T] (Hopac.Core.Cont`1[T] tK, Hopac.Core.Worker& wr, T value) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Job+result@1022[x].DoJob (Hopac.Core.Worker& wr, Hopac.Core.Cont`1[T] aK) [0x00000] in <57eeaba40b63217ba7450383a4abee57>:0
  at Hopac.Core.ContBind`2[X,Y].DoCont (Hopac.Core.Worker& wr, X x) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.ContMap`2[X,Y].DoCont (Hopac.Core.Worker& wr, X x) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.ContMap`2[X,Y].DoCont (Hopac.Core.Worker& wr, X x) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.TaskToJobAwaiter`1[X].DoWork (Hopac.Core.Worker& wr) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.Worker.RunOnThisThread (Hopac.Scheduler sr, Hopac.Core.Work work) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.Worker.ContinueOnThisThread (Hopac.Scheduler sr, Hopac.Core.Work work) [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at Hopac.Core.TaskToJobAwaiter`1[X].Ready () [0x00000] in <82aa62bc59cf45c1998a2363225b549c>:0
  at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction (System.Action action, System.Boolean allowInlining, System.Threading.Tasks.Task& currentTask) [0x00025] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/Tasks/TaskContinuation.cs:821
  at System.Threading.Tasks.Task.FinishContinuations () [0x0007a] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:3624
  at System.Threading.Tasks.Task.FinishStageThree () [0x00045] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/Tasks/Task.cs:2366
  at System.Threading.Tasks.Task`1[TResult].TrySetResult (TResult result) [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0
  at System.Threading.Tasks.TaskCompletionSource`1[TResult].TrySetResult (TResult result) [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0
  at System.Threading.Tasks.TaskCompletionSource`1[TResult].SetResult (TResult result) [0x00000] in <f712f98eb8e445c8918edaf595bbe465>:0
  at EventStore.ClientAPI.Embedded.EmbeddedResponderBase`2[TResult,TResponse].Succeed (TResponse response) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at EventStore.ClientAPI.Embedded.EmbeddedResponders+ReadStreamForwardEvents.InspectResponse (EventStore.Core.Messages.ClientMessage+ReadStreamEventsForwardCompleted response) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at EventStore.ClientAPI.Embedded.EmbeddedResponderBase`2[TResult,TResponse].InspectMessage (EventStore.Core.Messaging.Message message) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at EventStore.ClientAPI.Embedded.EmbeddedResponseEnvelope.ReplyWith[T] (T message) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at EventStore.Core.Services.Storage.StorageReaderWorker.EventStore.Core.Bus.IHandle<EventStore.Core.Messages.ClientMessage.ReadStreamEventsForward>.Handle (EventStore.Core.Messages.ClientMessage+ReadStreamEventsForward msg) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at EventStore.Core.Bus.MessageHandler`1[T].TryHandle (EventStore.Core.Messaging.Message message) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at EventStore.Core.Bus.InMemoryBus.Publish (EventStore.Core.Messaging.Message message) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at EventStore.Core.Bus.InMemoryBus.Handle (EventStore.Core.Messaging.Message message) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at EventStore.Core.Bus.QueuedHandlerThreadPool.ReadFromQueue (System.Object o) [0x00000] in <2693ba61ad5441dc8e951ea5b1db3bdc>:0
  at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context (System.Object state) [0x0000e] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1304
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x0008d] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:957
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:904
  at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem () [0x0002a] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1281
  at System.Threading.ThreadPoolWorkQueue.Dispatch () [0x00096] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:854
  at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback () [0x00000] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/threadpool.cs:1209
@polytypic
Copy link
Member

Can you provide a complete example to reproduce this?

@haf
Copy link
Member Author

haf commented Mar 27, 2017

I can give you access to a private repository that can reproduce it repeatedly...? I'll try to create a sample too.

@haf
Copy link
Member Author

haf commented Mar 27, 2017

This is the function that initially passes along the null exception (I did null checks throughout Hopac to find it)


    internal static void Run(Scheduler sr, int me) {
      RunningWork = 1;

      var wr = new Worker();
      wr.Init(sr, 16000);
      wr.RandomHi = (ulong)DateTime.UtcNow.Ticks;

      var iK = new IdleCont();

      var wdm = (1L << 32) / sr.Events.Length;

      wr.Event = sr.Events[me];

      while (null != sr) {
        try {
        Restart:
          Work work = wr.WorkStack;
          if (null == work)
            goto EnterScheduler;

        WorkerLoop:
          wr.Handler = work;
          {
            var next = work.Next;
            if (null != next && null == sr.WorkStack) {
              Scheduler.PushAll(sr, next);
              next = null;
            }
            wr.WorkStack = next;
          }
          work.DoWork(ref wr);
          work = wr.WorkStack;
          if (null != work)
            goto WorkerLoop;

          wr.Handler = null;

        EnterScheduler:
          work = sr.WorkStack;
          if (null == work)
            goto TryIdle;

          Scheduler.Enter(sr);
        EnteredScheduler:
          work = sr.WorkStack;
          if (null == work)
            goto ExitAndTryIdle;

        SchedulerGotSome: {
            var last = work;
            int numWorkStack = sr.NumWorkStack;
            int n = (int)((numWorkStack - 1L) * wdm >> 32) + 1;
            sr.NumWorkStack = numWorkStack-n;
            n -= 1;
            while (n > 0) {
              last = last.Next;
              n -= 1;
            }
            var next = last.Next;
            last.Next = null;
            last = null;
            sr.WorkStack = next;
            if (null != next) {
              next = null;
              Scheduler.UnsafeSignal(sr);
            }
            Scheduler.Exit(sr);
            goto WorkerLoop;
          }

        ExitAndTryIdle:
          Scheduler.Exit(sr);

        TryIdle:
          iK.Value = Timeout.Infinite;

          var iJ = sr.IdleHandler;
          if (null != iJ) {
            wr.Handler = iK;
            iJ.DoJob(ref wr, iK);
          }

          if (0 == iK.Value)
            goto Restart;

          Scheduler.Enter(sr);
          work = sr.WorkStack;
          if (null != work)
            goto SchedulerGotSome;

          Scheduler.UnsafeWait(sr, iK.Value, wr.Event);
          goto EnteredScheduler;
        } catch (KillException) {
          Scheduler.Kill(sr);
          Scheduler.Dec(sr);
          sr = null;
        } catch (Exception e) {
          if (e == null) throw new InvalidOperationException("Caught null exception in Run");
          wr.WorkStack = new FailWork(wr.WorkStack, e, wr.Handler);
        }
      }
    }

With crash:

[ERROR] FATAL UNHANDLED EXCEPTION: System.InvalidOperationException: Caught null exception in Run
  at Hopac.Core.Worker.Run (Hopac.Scheduler sr, System.Int32 me) [0x0020c] in <5048adbc9bf049e5befa8ce1d7eb0639>:0
  at Hopac.Platform.Scheduler+thread@34.Invoke () [0x00001] in <58d9196bb26ceae6a74503836b19d958>:0
  at System.Threading.ThreadHelper.ThreadStart_Context (System.Object state) [0x00017] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/thread.cs:68
  at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x0008d] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:957
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state, System.Boolean preserveSyncCtx) [0x00000] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:904
  at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, System.Object state) [0x00031] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/executioncontext.cs:893
  at System.Threading.ThreadHelper.ThreadStart () [0x0000b] in /private/tmp/mono-20170301-46369-1aeloew/mono-4.8.0/mcs/class/referencesource/mscorlib/system/threading/thread.cs:105

@polytypic
Copy link
Member

Hmm... I wonder if this is CLR bug? See

http://stackoverflow.com/questions/5634417/caught-exception-is-null-itself

@haf
Copy link
Member Author

haf commented Mar 27, 2017

It's this line that's failing

screen shot 2017-03-27 at 16 56 05

Adding an else after it makes it not crash. But then it crashes in Hopac.Stream+mapFun@254-2 instead

@haf
Copy link
Member Author

haf commented Mar 27, 2017

Could it be this code?

screen shot 2017-03-27 at 17 19 46

...interacting with the goto-statements? It's implementation

namespace Hopac.Core {
  using Hopac.Core;
  using System.Runtime.CompilerServices;

  internal unsafe static class Unsafe {
    [MethodImpl(AggressiveInlining.Flag)]
    internal static byte *GetStackPtr() {
      byte dummy;
      return &dummy;
    }
  }
}

@neoeinstein
Copy link
Member

Having a failure depend on whether or not there is an else statement makes me want to consider if there is an issue with how zero ends up combined with the then case.

@haf
Copy link
Member Author

haf commented Mar 27, 2017

It's a stackoverflow in unmanaged code:

In TRAMPOLINE
TRAMPOLINE if branch
Calling Enter
Calling ev.Reset
goto EnteredScheduler
sr.WorkStack is being accessed
SchedulerGotSome running
SchedulerGotSome while loop starting
SchedulerGotSome setting WorkStack = next
SchedulerGotSome Scheduler.Exit
WorkerLoop entered
WorkerLoop Calling work.DoWork on EndToEnd+DSL+test@851-34 with worker Hopac.Core.Worker
[17:39:30 DBG] Running stanza 0, action '(s) #handled'
In TRAMPOLINE
TRAMPOLINE else branch on work EndToEnd+DSL+test@853-32, ReCQ.StreamProcessing.Tests.EndToEnd+DSL+test@853-32
Stack overflow in unmanaged: IP: 0x10bb47eae, fault addr: 0x70000444bff8
WorkerLoop entered
WorkerLoop Calling work.DoWork on Hopac.Core.FailWork with worker Hopac.Core.Worker
[17:39:31 DBG] Stanza (s) #handled raised an exception.
[ERROR] FATAL UNHANDLED EXCEPTION: System.StackOverflowException: The requested operation caused a stack overflow.

@haf
Copy link
Member Author

haf commented Mar 27, 2017

I think it is an interaction between Work implementations between Async wrappers and the Work-implementations that are the Job interface, that causes work.DoWork(ref wk) to be a mutually recursive call. The Work implementation changes between runs. Sometimes it's stream mapFun, sometimes the test job, sometimes MoreUtil+Handler, etc

@neoeinstein
Copy link
Member

I'm going to check to see if this is a platform-related issue. Building the tests, and then will run them on .NET Framework.

@neoeinstein
Copy link
Member

Getting all successful through Expecto, both DEBUG and RELEASE builds, when run on .NET Framework.

image

@neoeinstein
Copy link
Member

neoeinstein commented Mar 27, 2017

There is a platform-specific difference in how the static Alt<unit> is constructed. When in 32-bit mode, or on Mono, it uses AlwaysUnitNonTC, and when not on Mono and in 64-bit mode, it uses AlwaysUnitTC. The difference between these two Alts is subtle. The NonTC version uses Work.Do(uK, ref wr), while the TC version uses uK.DoWork(ref wr). Work.Do() provides an opportunity to "trampoline" to kill off some of the stack. It could be that we need to adjust the StackLimit to ensure we don't overflow. The default stack size is 16000 bytes, with 8000 used for RunOnThisThread.

@haf, can you try adjusting the StackLimit down here: https://github.com/Hopac/Hopac/blob/master/Libs/Hopac.Core/Engine/Worker.cs#L133 and then see if you can reproduce?

@haf
Copy link
Member Author

haf commented Mar 27, 2017

@neoeinstein Sure I can, but just to clarify; I haven't been able to create a simple repro yet, but I thought some generative tests could be useful to PR anyway. ;) Tomorrow I will scale down the complex (very complex) repro I have and try your suggestion.

@polytypic
Copy link
Member

Yes, the idea is that AlwaysUnitTC is used on platforms that actually implement tail-call optimization. It is quite possible that the optimizations are not always properly performed based on the simple criteria that Hopac uses. (E.g. if you are running a non-optimized build or under a debugger, you may not get proper tail-calls.) You might want to try and see what happens if you always use AlwaysUnitNonTC.

Note that the StackLimit doesn't actually select the stack size. It is just a limit after which the trampoline code will unwind back to the run loop. You could also try to modify Hopac to always unwind back to the run loop.

Note that neither of these changes should really have any effect on what exception, apparently null in this case, would be raised by the runtime in case of a stack overflow.

@haf
Copy link
Member Author

haf commented Mar 28, 2017

@polytypic I'm only running on mono and then it's my understanding that it doesn't use tail calls.

@neoeinstein I've tried adjusting the stack size, doesn't have an effect.

If I monkey around with the code, removing JobBuilder.Zero() calls and one debugWithBP(...)|>Job.toAsync, the stanza (a sub-test) passes, but then a later one fails:

WorkerLoop assigning work = wr.WorkStack on Hopac.Core.Taker`1[Hopac.Stream+Cons`1[StreamEvent]] with worker Hopac.Core.Worker
WorkerLoop setting wr.Handler = null
Calling Scheduler.Enter
Calling Scheduler.UnsafeWait
Calling UnsafeDec
Calling Exit
Calling ev.Wait
WorkerLoop assigning work = wr.WorkStack on Hopac.Core.Taker`1[Hopac.Stream+Cons`1[StreamEvent]] with worker Hopac.Core.Worker
WorkerLoop setting wr.Handler = null
Calling Scheduler.Enter
Calling Scheduler.UnsafeWait
Calling UnsafeDec
Calling Exit
Calling ev.Wait
toAsync, Cont_State calling passOn <null> <null> <fun:Invoke@3253>
System.InvalidOperationException: Caught null exception in RunOnThisThread<T>
  at Hopac.Core.Worker.RunOnThisThread[T] (Hopac.Scheduler sr, Hopac.Job`1[T] tJ, Hopac.Core.Cont`1[T] tK) [0x00046] in <1f1a348c87ce461f8e8975af0c181fb6>:0
  at Hopac.Job+toAsync@1495-7[x].Invoke (System.Tuple`3[T1,T2,T3] tupledArg) [0x00040] in <58da27390b63217ba74503833927da58>:0
  at <StartupCode$FSharp-Core>.$Control+FromContinuations@1353[T].Invoke (Microsoft.FSharp.Control.AsyncParams`1[T] _arg1) [0x0005c] in /private/tmp/mono--fsharp-20170301-46369-1vnanly/src/fsharp/FSharp.Core/control.fs:1371

The quest to add logging to Hopac continues.

@haf
Copy link
Member Author

haf commented Mar 28, 2017

It's failing when it's trying to run this function, downcast to a Job<_> in turn run through Job.toAsync:

      /// A function that takes a retry timeout (defaults to 50ms) and a time budget
      /// that it's allowed to spend, waiting for the passed alternative to
      /// become available. Returns an alternative, which, if cancelled, also cancels
      /// the passed alternative and the recursive checking after each retry timeout.
      let private eventually retryTimeout budget =
        fun (fn: Alt<'res>) cancelValue ->
          let rec inner budget' nack =
            // out of budget; return the cancel value
            if budget' < TimeSpan.Zero then Alt.always cancelValue else
            // otherwise, choose between the caller NACKing, retrying after the
            // timeout and the user-supplied function
            Alt.choose [
              nack ^->. cancelValue
              timeOut retryTimeout ^=>. inner (budget - retryTimeout) nack
              fn
            ]

          Alt.withNackFun (inner budget)

      /// A constructed 'eventually' function with the defaults.
      let private eval fn cancelValue =
        eventually defaultRetry defaultBudget fn cancelValue

So in the end it's my own fault :) Awesome! The inner function is recursively chewing through the stack.

@haf haf closed this as completed Mar 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants