RFP : a unified TreeDiff serialization format

[jrfaller]

Hi all,

I think it would be really nice to have a better machine processable output format for GumTree. Right now there is mainly the JSON and free text output that are not very useful (see here and here.

For instance, it is not even possible to use those diff outputs to build a GUI because it lack a huge number of information (such as filename, text positions, ...).

Therefore I think it's high time that we do something better. I think that using JSON or XML is the way to go, but I am not sure which information it would be interesting to have. Right now we have only matched nodes as well as actions. Only information on trees are node IDs in the JSON which I think is not enough, and textual type inside. Information about action is its type and parameters which I think is good.

What do you think we should include in GumTree's output? Don't hesitate to comment (you can even include snippets 😄).

Cheers!

[jrfaller]

One idea I had for instance was to include also the parsed ASTs in the diff output therefore all relevant information is in the same place?

[monperrus]

Here is a first proposal: https://docs.google.com/document/d/15ITrZ4c0qdb4rGLfaMt8K9VPG1CkNp5QSgfOY6jzpEk/edit?usp=sharing

Don’t hesitate to modify or comment the GDoc!

[Symbolk]

Why we diff? Because we want to understand the changes, so the format should serve for change comprehension. In fact, someone has modified Gumtree output format but getting a distingushed paper~ (See https://github.com/FudanSELab/CLDIFF)

I customized Gumtree for personal use, for meaningful diff summary, including:

Action Name
Parent Node Type: Target Node Type
Target Node Content Before Action
Line Range & Column Range Before Action
Target Node Content After Action (if exists)
Line Range & Column Range After Action (if exists)

For example:

Move
CompilationUnit::ImportDeclaration
import com.beust.jcommander.Parameter;
12, 273
import com.beust.jcommander.Parameter;
1, 0

So, basically you can read it as a phrase: "Move ImportDeclaration 'import com.beust.jcommander.Parameter;' in CompilationUnit from (12, 273) to (1, 0)".

[monperrus]

I customized Gumtree for personal use, for meaningful diff summary

That's interesting, thanks. Would you make a pull-request?

(Note that it's somehow off-track for this issue, we may open a new issue for this discussion)

[Symbolk]

I'd like to, but one major concern here.

As we know, Gumtree parses multple languages with various backends (parser), but for each of the supported languages, the definition of AST Node Type and the way to get ranges is different. Since I only use Gumtree to diff Java, so I only implement the Java version.

To generalize the output format, some refactoring must be done.

BTW, I made a pr earlier to fix a bug (#110 ), but it seems @jrfaller did not merge it.

[monperrus]

@Symbolk moved the discussion to #115

[chijianlei]

I have read the code of CLDiff, it deeply used JDT package. I'm now finding some methods to apply it on other parsers.