[core] Autofixes feature development

[MatiasComercio]

Autofixes

Hi to everyone! We are developing autofixes for PMD! 🎉
Together with @gibarsin, and with @jsotuyod as our mentor, we are implementing all the autofixes features listed here.

This issue will serve as our roadmap for all the changes that we are going to implement, linking the pull requests that implement each of the changes listed below. Note that we will be updating this list as ideas or suggestions arise.

Any feedback, suggestion or idea you may have will be really appreciated and of course welcomed.

We hope, and it is our intention, that autofixes get fully implemented by July, 2018.

That's all for now! Thanks in advance to everyone making this HUGE feature possible with feedback, bug reports (we hope not 😄), and any kind of help you may give us.

List of changes

• [core] - Add AST operation over nodes (addChild, setChild, removeChild, remove)
• [core] - Update tokens according to AST operations
• [core] - Store a Fix when reporting a new rule violation
• [core] - Apply stored Fixes after each rule finishes traversing the AST
  • Consider this comment for its development
• [core] - Update the source code file with the applied fixes
• [core] - Cache diff operations for fixed files when autofixes are disabled (and cache is enabled)
• [core] - Apply cached diff operations
• [core] - Add autofix as a command line parameter

Bare in mind issues

• Add a lock system to avoid node manipulation outside autofix rules
  • Consider this comment
  • Another option may be to have a common object with all nodes of the same AST pointing to it to implement the control of these modifications, but we have to discuss it yet.

Extras

• [java] Fix in Java grammar: Try with final resource node error #675: [java] Fix in Java grammar: Try with final resource node error
• [core] Token scheme generalization #679: [core] Token scheme generalization
• [core] Make MultiThreadProcessor more space efficient #1048: [core] Make MultiThreadProcessor more space efficient

Deprecated changes

• [core] Add text operations to manipulate a document #641, [core] Add operations to manipulate a document #828 [PR]: Add text operations to manipulate a document.
• [core] Add remove operation over nodes #696: Add remove operation over nodes
• [ ] [core] Add insert & replace child nodes #858: Add insert& replace operation over nodes

[MatiasComercio]

Hi to everyone!

We apologize for any inconvenience for those who expected the feature to be done by March.
We are still working really hard to implement autofixes with efficiency and practicality in mind, so that extending the autofixes functionality for any language can be as simple as possible.

After 6 months of research we have iterated through, at least, 10 different approaches, and we were implementing what we believed was the best one according to our goals.
In this last 3 months we've been doing some extra research and small experiments with several new architectures on how to translate AST changes into text changes, and we have finally chosen one in favor of the previous approach we were developing.
This new approach is still in alpha, and up to now has showed to be a very clean, efficient and easy way to implement and extend over the autofixes core functionality.

As this new approach of dealing with the translation from AST to text changes is completely different than the previous one, all the current open PRs related to any autofixes feature are being closed.
We are also updating the List of changes in the issue description to reflect the new work which needs to be done. Moreover, we are adding another list called Deprecated changes that will help anyone track what changes were implemented (or were still in PR) but have been discarded in favor of this new alternative.

Again we apologize, but we are confident that we are giving the best of ourselves to develop a great solution to this tremendous feature in the best possible way.

[dgroup]

Hi @MatiasComercio,
How is feature is going? Is there any task-roadmap in issues where we can help?

[oowekyala]

This project has not seen much development since the initial PRs. As part of the development of PMD 7 we actually removed part of those changes to cleanup the AST API.

FWIW I have a lot of local changes which are improvements of the AST API, designed to in the end enable edition of source files from the AST. I plan to open PRs for PMD 7 to gradually merge those changes. I think we could put that feature back on the table once that's done, but I'm not sure the feature will be finished and polished for PMD 7.

[dgroup]

@oowekyala, thank you for updates, checked #3692, it has 78+ files changed and that's impressive :)
Is there any small/routine tasks where regular java developers may help? I have few guys in my team who are really interested in this feature, thus we will be glad to join and contribute.

For example, you may decompose the routine self-explanatory tasks directly from the code using @todo and 0pdd.
Here an example how to format the @todo in order to get Github issues.

[blacelle]

For whoever is interested in autofixing PMD issues, the feature is available in https://github.com/solven-eu/cleanthat (which also covers various CheckStyle|SpotBugs issues, and various others patterns (e.g. to take advantage of jdk8 Streams API)).

Current set of covered rules is available at https://github.com/solven-eu/cleanthat/blob/master/MUTATORS.generated.MD

PMD rules can be covered through https://github.com/solven-eu/cleanthat/blob/master/java/src/main/java/eu/solven/cleanthat/engine/java/refactorer/mutators/composite/PMDMutators.java