Debugger improvements #2

[Jither]

As mentioned in #674, I've been working on improving the DebugMode functionality. This pull request should fix:

• Block-scoped variables (let/const) not being included in Globals/Locals
• Globals/Locals not being logically separated (Globals included local variables)
• Missing support for Source in breakpoints
• Most callables not being added to debug call stack (namely, only calls via literal Identifiers would be added)
• Some crashes when collecting values from Globals/Locals
• Erratic behavior of StepMode, particularly after a breakpoint

In addition, it:

• Implements handling of debugger statement (as an optional alternative to the existing VS debugging)
• Gets rid of basing StepMode changes and call stack on evaluation of AST nodes - this should simplify adding support for the remaining callables (getters/setters, constructors, generators...)

Expected major/minor pain points

• The CallStack no longer includes argument lists - as mentioned in Documentation on DebugMode #674, to make it work in the general case, something would need to parse and stringify all kinds of expressions that might make up an argument.
• The AllowDebuggerStatement boolean has been changed to an enum DebuggerStatementHandling to allow three choices. Not all that happy with the naming of the method or the values (Clr/Jint/Ignore) - maybe, for the latter, Native, Script, Ignore, would be better - or a completely different option construct. The enum probably shouldn't be in the root namespace either.
• Other than that, the only modifications to code outside the Debugger namespace at the moment are:
  • moving the call to AddToDebugCallStack closer to the call (we now need the ICallable in addition to - or, at the moment, instead of - the Expression).
  • amending GlobalEnvironmentRecord's GetAllBindingNames, in order to retrieve the declarative record in addition to the binding object. GetAllBindingNames is only referenced by DebugHandler in the current dev branch.
• The bulk of the code is tests - and yet they're far from exhaustive, and somewhat unorthodox (since they're mostly testing state machines, and are really more integration tests than unit tests). They're placed in their own namespace, which may be a style issue.
• The existing test of DebugInformation has been modified, since locals are no longer mixed into Globals, and the debugCallStack no longer includes argument lists.

Stuff to do/consider

There are still some missing features, but those require invasion into the main engine code:

• Because only JintCallExpression notifies the DebugHandler about the call stack changing, e.g. accessor properties and constructors aren't added to the stack, with all that entails.
• Would be nice to have the debugger hooked into the end of function calls, after the last statement, to allow inspection of return value, and to behave like well-known browser debuggers.
• The Globals/Locals split is inherited from the original DebugHandler code. Maybe the scopes should be the same as e.g. Chromium or Firefox - at least having a separate Block scope.
• Might want more info in CallStack than just the function name - e.g. location.

[Jither]

And sorry about the crazy amount of commits, including lots of "no, putting it there was a bad idea" followed almost immediately after by "no it wasn't - back it goes". ;-)

[lahma]

Great work overall! Kudos for code coverage from tests and I only found a few nits. W.r.t. Stuff to do/consider I think that can be another PR when/if so deemed.

[lahma]

Should this be in Runtime.Debugger namespace?

[Jither]

Completely agree - I don't know why I only (briefly) considered the Runtime.Interpreter.Statements namespace. :-)

[lahma]

as this is public, can we have short descriptions about modes?

[Jither]

I've added descriptions for the enum members and fixed XML comments in general - replacing <code> with inline <c>. Also renamed the Jint member to Script.

[lahma]

Awesome, thanks for tweaking. @sebastienros please take a look.

[lahma]

Thank you @Jither for this! I hope you find the time and interest to address any issues you might find later on 🚀

[Jither]

Thanks for the merge, @lahma and @sebastienros! As for issues later on - almost certainly. If nothing else, I'd like to do some more on the "Stuff to do/consider", but most of it requires diving much further into the Jint code and ES specs - even just the division into more scopes should probably be prepared for modules and other scopes (as far as I recall the Chromium devtools protocol has 9 or 10).

In the meantime, I'll probably be focusing on doing the example console UI implementation I mentioned in #674 - see if there are some glaring pain points in the minimal API when trying to do a full-featured debugger, and get something for testing future debugger functionality more easily than adding GUI in WPF XAML. I guess I'll use some minimal implementation of SynchronizationContext/Dispatcher for console apps - which would make it easier for others to adapt the code to WPF/WinForms/whatever, without getting too much of a headache handling Jint executing on a different thread from the UI.