Provide a command-line tool for linting and refactoring 007 code

[masak]

Call the command spy, to continue the 007 theme.

It would have two major parts: a linter, and a refactoring tool.

linter

Called with spy --lint script.007, the tool could detect various things that are smells rather than errors in a 007 program:

• Variable, sub, or macro not used
• Variable read from before being assigned to
• Variable assigned but value never used
• Variable assigned but value from that assignment never used
• Dead code (after return or throw, or code after conditions that is never reached)
• Conditionals that will always be true or never true
• Duplicated or overly complicated conditionals (like a ?? a !! b, which could just be a || b)
• while loops that will never run
• A type of value means evaluating a thing will certainly fail (like +none)
• On some execution paths, the type of a value in a variable means things will fail (like going through one leg of an if statement and assigning none)

refactoring tool

• Extract/inline/rename variable/constant
• Extract/inline/rename sub
• Extract/inline/rename macro
• Add/remove/reorder parameters
• Introduce parameter (turn expression in sub into a parameter)

Being refactors, the actions that affect parameters should also make sure to do the "corresponding" things to arguments at all statically detectable call sites.

what's needed to get there

The compiler already generates a Qtree for consumption by the runtime. The thing that needs to be built on top of that is a layer of primitives for locating stuff in the Qtree, and to manipulate it. Also a greater awareness of the connection between Qtree and original source text is needed.

[masak]

I think this one would be great to tackle, because it would shake out relatively common patterns of dealing with code.

Though linting and refactoring are still distinct from the kinds of analysis and manipulation a macro might want to do, they are in the same ballpark — chances are that making the API nicer for the former also makes it nicer for the latter.

[masak]

Though it's not a blocker by any means, it's intriguing to think that after we have #38, we might write this tool in 007 as well. That's not to say we shouldn't have a Perl 6 implementation first, of course.

[masak]

oh, and a

code tidy-fier

Like Perl::Tidy or gofmt.

spy --tidy script.007

[masak]

Though it's not a blocker by any means, it's intriguing to think that after we have #38, we might write this tool in 007 as well. That's not to say we shouldn't have a Perl 6 implementation first, of course.

Why wait until we have a 007 parser in 007? We can just parse using the Perl 6 parser, and then run the 007 script on the resulting Qtree.

I'm having some further thoughts on the API layer here between 007 and the linter tool, though. More to follow.

[masak]

• Variable read from before being assigned to

I wanted to say that the "before" here is interesting. It's not "source order", it's execution order. #160 is a good example of when the two differ:

repeat while my a = 5 {
    say(a);
}

In the above code, the a = 5 appears textually before the say(a), but the program will still do the assignment after the read, and so we should get a warning here.

Since #160 is a language-extending macro, this also indicates that macros (and similar) need to be "transparent" to control flow questions.

In the best case this is as simple to the linter working on the expanded code in this case (as opposed to when it checks for macro usages, in which case it needs to work on the unexpanded code), and then we just declare how control flow works for all our language primitives.

[masak]

I slapped a "needs more design" on this issue, because that's my explanation of why it's stalled since its creation back in 2015 (gasp).

Case in point, the previous comment points the way to the fact that even something as simple as use-before-assign relies on full Data flow analysis. The design must incorporate that.

A case of "this feels pretty easy", and then the innards fairly intricate, but the tool hides that and is still quite easy to use.

[masak]

Just driving by to say (again) that this issue is "CFG-compete", in the sense that compete information about the CFG is required to do a sufficient analysis for linting. Which is why #379 and similar are useful for this.