Runtime parser result debugger with frontend support #219
Conversation
|
Critical roadblock at the moment is the usage of |
|
Added |
TODO
|
Currently implemented as a rose tree, with a mutable version and a frozen immutable version for safer analysis.
-- TODO -- > hook up the parse-time internals to the tree, so it can be populated when the parser runs > clean up the names of symbolic classes such as <|>
-- TODO -- > Fix residual check when parser being debugged fails. > Fix the lack of tree objects caused by reconstruct not fully traversing the tree of parsers.
-- TODO -- > Fix residual check when parser being debugged fails. > Fix the lack of tree objects caused by reconstruct not fully traversing the tree of parsers. > Remove handler from handler stack if parser being debugged fails.
-- TODO -- > Fix the lack of tree objects caused by something unknown?
children, broke entire parser stack flow. -- TODO -- > Investigate the failure chain and figure out a way to stop the check stack and debug context stack from being emptied out. > Fix the lack of tree objects caused by something unknown?
This reverts commit 84fb89c.
is built and can be mostly traversed -- TODO -- > Fix the fact that successful parses leave handlers on the handler stack. > Fix the lack of tree objects caused by lack of attached Debugged combinators.
tree is now fully populated -- TODO -- > Test deeper, more complex parsers to ensure that nothing actually gets lost. > Test recursive parsers to see if the infinite-recursion-stopping mechanics function as intended. > Make a simple printout view for the debug tree into the console for when the user does not want to use any of the GUI options.
order in the parser trees -- TODO -- > Test deeper, more complex parsers to ensure that nothing actually gets lost. > Test recursive parsers to see if the infinite-recursion-stopping mechanics function as intended. > Make a simple printout view for the debug tree into the console for when the user does not want to use any of the GUI options.
-- TODO -- > Test deeper, more complex parsers to ensure that nothing actually gets lost. > Test recursive parsers to see if the infinite-recursion-stopping mechanics function as intended. > Make a simple printout view for the debug tree into the console for when the user does not want to use any of the GUI options.
input for a parse attempt -- TODO -- > Make a simple printout view for the debug tree into the console for when the user does not want to use any of the GUI options.
to make looking at the tree simpler -- TODO -- > Make a simple printout view for the debug tree into the console for when the user does not want to use any of the GUI options.
finder just in case -- TODO -- > Re-use our debugged parsers when the parser is cyclic. > Make a simple printout view for the debug tree into the console for when the user does not want to use any of the GUI options.
cyclic to allow better renaming of parsers during debugger runtime -- TODO -- > Make a simple printout view for the debug tree into the console for when the user does not want to use any of the GUI options.
fix: GUI now executes correctly without redundant ops
- flattened packages to make the public API smoother - renamed some packages to remove pluralities for consistency - improved documentation for public API, especially for the debug combinators - cleared context with each parser run so debugged parsers can be re-used freely as long as the trees are saved or discarded appropriately between runs
attachDebuggerGUI now match - also fixed instruction generator to more closely match an iterative parser's generated instructions
to avoid reversing the ListMaps used
case it is needed -- TODO -- > Give the parse attempts the position info for renderers to take advantage of.
origin info instead of reconstructed info -- TODO -- > Give the parse attempts the position info for renderers to take advantage of.
-- TODO -- > Give the parse attempts the position info for renderers to take advantage of.
remaining issues: - `ParseAttempt` object public or private - `Lexer` private objects with parsers - possibly reintroduce `XMap` or similar mechanism
- prevents potential future binary incompatibilities
- Future versions of the JVM will restrict `setAccessible`'s abilities, and will render this collector eventually unusable.
- remove stale comment from AddAttemptAndLeave - remove types where obvious
- MapAddAll is internal only now - apply uses partial since code paths are the same - addNames now has a separate overload for IterableOnce of pairs of parsers and names
Scala 2.12 does not have IterableOnce ???
- Added zipWith and zipWith3 to `ContOps` to facilitate
If that's all you need, it is really only a few lines of code. import scala.scalajs.js
import scala.scalajs.js.annotation.JSGlobal
@js.native
@JSGlobal
class WeakMap[K, V] extends js.Object {
def delete(key: K): Boolean = js.native
def get(key: K): js.UndefOr[V] = js.native
def has(key: K): Boolean = js.native
def set(key: K, value: V): this.type = js.native
}On the other hand, if you need more APIs than that then it's more work. For example an |
|
@armanbilge raises another interesting point. Other than the lack of scala-js implementation, we could use the java API instead of the scala one and sidestep the cross-version collections API problem (and just suffer the java API instead)... |
For an implementation as simple as this is, you have to wonder why the ScalaJS team didn't decide to just add this to the standard library for it (even if it does not conform directly to (For what it's worth, this API is sufficient for the task at hand. The ability to iterate over the map isn't used in the internal logic that needs the weak map.) |
Because nobody got around to it yet? :) I don't see any issues open about it, and they accepted a similar PR recently in scala-js/scala-js#4366. |
I think I recall seeing this PR when looking for |
- Finally unified the Scala 2.12, 2.13 and 3.3 maps at last.
- same reasoning as in debugger, stops ephemeral parsers from flooding the heap
This is a possibility if the interfaces are exposed to ScalaJS (Scala Native probably has them, honestly). (At least they'd be stable, the one saving grace about Java.) |
|
When scala-js implements the java one, it'll just work out for us (and Arman notes this may be more efficient than the scala implementation if optimisations are applied per platform). In the mean time we just need a java API facade over the top of the actual java API and make XMap... Javaish |
It seems to compile and run (using |
|
For Java-ifying The direct translations of Java's There is also the issue that the |
|
There is one avenue to be explored, and that is manually defining a simpler Java API for some kind of "invariant map" in raw Java (that is then implemented within Scala instead). Now the issue is, would that be cross-platform? |
* feat: made `X...Exception`s package private, added `XUnsupportedOperationException` * style: add license headers
|
Ok are we ready to merge this for now? This will give us the debugger backends right, but we still need to figure out where to put the frontends (or rather, transfer them under my domain and get them ready for publishing and deploy) |
|
It should be ready to merge. It'll give just the backend (and whatever changes to Parsley were necessary for it), the frontends still live in a separate repository on my account. |
Summary
Adds a separate optional library to Parsley,
parsley-debug. This library contains a runtime and public APIs for attaching a higher-level debugger for Parsley, to complement its lower-level debugger found inparsley.debug.Public API
The public API is found (and documented) in
parsley.debugger.combinator, where the basis method of this debugger (attachDebugger) lives. This method traverses the internalLazyParsleytree of a parser and attaches debugger nodes to every single node (even arbitrary dynamically-produced nodes fromflatMap).This produces a pair of a generator function for a parse tree (a DebugTree; a tree representing the path of execution the parser took, along with relevant results and success statuses), and a debugged parser, which needs to be run before the generator function is called.
The tree itself has a fixed set of methods for querying information, found in
parsley.debugger.DebugTree.For the convenience of the user, utilities are also provided in
parsley.debugger.util.Collector, where parser names can be automatically collected from the field names found in a class (only for fields, not methods; useparsley.debugger.combinator.namedfor non-field parsers). However, that uses reflection viascala-reflect(!), which may need to be reworked (to use something else more cross-version compatible) or removed entirely.An interface,
parsley.debugger.frontend.DebugFrontendis also provided for defining custom processing frontends for the aforementionedDebugTree. One can traverse the tree there and do whatever they want to the tree (convert it to JSON, print it to console, display it in a GUI, etc.) as long as the entire tree is traversed. These can also be passed automatically (explicitly or via implicits) into the debugger attachment process, which can automate the processing of a tree from a parser as it runs.Known Issues or Missing Features
parsley.internal.deepembedding.frontend.debugger.DebugInjectingVisitorM.def-generated parsers must be manually renamed.parsley-debugas the artifact name, with the same version asparsley(perhaps this is not an issue).