internal errors during VM now produce a user code stacktrace

[timotheecour]

before PR

internal errors don't give much context, eg this code from nitely/nim-regex#4

import pkg/regex
proc main()=
  var m: RegexMatch
  doAssert "a".match(re"\w+", m)
static:
  main()

nim --version: latest devel c522a455df3817d98c2cd9132668c4eb9fbb0d88
nim c --skipUserCfg t0460.nim

gives (before my fix in nitely/nim-unicodedb#7) this:

Error: internal error: (filename: "vmgen.nim", line: 180, column: 16)
No stack traceback available
To create a stacktrace, rerun compilation with ./koch temp c <file>

which gives very little hint for how to debug

after PR

we get VM stacktrace pointing to user code (ie, not compiler code); this actually helped me for fixing nitely/nim-regex#4:

Error: internal error: (filename: "vmgen.nim", line: 180, column: 16)
stacktrace for rawExecute:
/Users/timothee/git_clone/nim/timn/tests/nim/all/t0460.nim(12, 9) t0460
/Users/timothee/git_clone/nim/timn/tests/nim/all/t0460.nim(10, 11) main
/Users/timothee/git_clone/nim/nim-regex/src/regex.nim(2417, 21) match
/Users/timothee/git_clone/nim/nim-regex/src/regex.nim(2390, 15) matchImpl
/Users/timothee/git_clone/nim/nim-regex/src/regex.nim(494, 6) match
/Users/timothee/git_clone/nim/nim-regex/src/regex.nim(371, 26) isWord
No stack traceback available
To create a stacktrace, rerun compilation with ./koch temp c <file>

after PR with nim compiled with --stacktrace:on

we get both user code VM stacktrace followed by compiler code RT stacktrace:

Error: internal error: (filename: "vmgen.nim", line: 180, column: 16)
stacktrace for rawExecute:
/Users/timothee/git_clone/nim/timn/tests/nim/all/t0460.nim(12, 9) t0460
/Users/timothee/git_clone/nim/timn/tests/nim/all/t0460.nim(10, 11) main
/Users/timothee/git_clone/nim/nim-regex/src/regex.nim(2417, 21) match
/Users/timothee/git_clone/nim/nim-regex/src/regex.nim(2390, 15) matchImpl
/Users/timothee/git_clone/nim/nim-regex/src/regex.nim(494, 6) match
/Users/timothee/git_clone/nim/nim-regex/src/regex.nim(371, 26) isWord
Traceback (most recent call last)
/Users/timothee/git_clone/nim/Nim_prs/compiler/nim.nim(98) nim
/Users/timothee/git_clone/nim/Nim_prs/compiler/nim.nim(75) handleCmdLine
/Users/timothee/git_clone/nim/Nim_prs/compiler/cmdlinehelper.nim(92) loadConfigsAndRunMainCommand
/Users/timothee/git_clone/nim/Nim_prs/compiler/main.nim(194) mainCommand
/Users/timothee/git_clone/nim/Nim_prs/compiler/main.nim(90) commandCompileToC
/Users/timothee/git_clone/nim/Nim_prs/compiler/modules.nim(147) compileProject
/Users/timothee/git_clone/nim/Nim_prs/compiler/modules.nim(88) compileModule
/Users/timothee/git_clone/nim/Nim_prs/compiler/passes.nim(197) processModule
/Users/timothee/git_clone/nim/Nim_prs/compiler/passes.nim(86) processTopLevelStmt
/Users/timothee/git_clone/nim/Nim_prs/compiler/sem.nim(603) myProcess
/Users/timothee/git_clone/nim/Nim_prs/compiler/sem.nim(571) semStmtAndGenerateGenerics
/Users/timothee/git_clone/nim/Nim_prs/compiler/semstmts.nim(2184) semStmt
/Users/timothee/git_clone/nim/Nim_prs/compiler/semexprs.nim(980) semExprNoType
/Users/timothee/git_clone/nim/Nim_prs/compiler/semexprs.nim(2712) semExpr
/Users/timothee/git_clone/nim/Nim_prs/compiler/semstmts.nim(2124) semStmtList
/Users/timothee/git_clone/nim/Nim_prs/compiler/semexprs.nim(2639) semExpr
/Users/timothee/git_clone/nim/Nim_prs/compiler/semexprs.nim(2228) semWhen
/Users/timothee/git_clone/nim/Nim_prs/compiler/semexprs.nim(2712) semExpr
/Users/timothee/git_clone/nim/Nim_prs/compiler/semstmts.nim(2124) semStmtList
/Users/timothee/git_clone/nim/Nim_prs/compiler/semexprs.nim(2764) semExpr
/Users/timothee/git_clone/nim/Nim_prs/compiler/semstmts.nim(2076) semStaticStmt
/Users/timothee/git_clone/nim/Nim_prs/compiler/vm.nim(2023) evalStaticStmt
/Users/timothee/git_clone/nim/Nim_prs/compiler/vm.nim(2013) evalConstExprAux
/Users/timothee/git_clone/nim/Nim_prs/compiler/vm.nim(1110) rawExecute
/Users/timothee/git_clone/nim/Nim_prs/compiler/vm.nim(450) compile
/Users/timothee/git_clone/nim/Nim_prs/compiler/vmgen.nim(2225) genProc
/Users/timothee/git_clone/nim/Nim_prs/compiler/msgs.nim(571) patch
/Users/timothee/git_clone/nim/Nim_prs/compiler/msgs.nim(564) internalError
/Users/timothee/git_clone/nim/Nim_prs/compiler/msgs.nim(442) rawMessage
/Users/timothee/git_clone/nim/Nim_prs/compiler/msgs.nim(439) rawMessage
/Users/timothee/git_clone/nim/Nim_prs/compiler/msgs.nim(350) handleError
/Users/timothee/git_clone/nim/Nim_prs/compiler/msgs.nim(340) quit

note

• generating this user code VM stacktrace doesn't incur additional cost (unlike --stacktrace:on which creates runtime frames) so IMO there's no need to hide it under a flag, it's always useful to show and gives more context pointing to user code

• this shouldn't give rise to duplicate stacktraces, eg a doAssert false occurring somewhere inside VM would just give the same error msg with user code stacktrace as before this PR

[Araq]

We should fix internal VM bugs, not hack around them via more global callbacks (not GC-safe!)

[timotheecour]

fixed with last commit, avoided adding the global callback and instead made it a field of ConfigRef which is already propagated everywhere
this should help fixing internal VM bugs (or help users that need to work around these); some such VM bugs are really not trivial and won't be fixed any time soon
if needed i can hide errorDiagnosticCallback behind a -d:nimErrorDiagnosticCallback but seems useful and harmless with that new commit

[Araq]

Hmm, why not create an internalErrorVM proc instead that is used in the VM and prints the stacktrace?

[narimiran]

why not create an internalErrorVM proc instead that is used in the VM and prints the stacktrace?

@timotheecour any progress on the above?

[Araq]

This solution is not clean enough and the suggested alternative wasn't explored. Closing.