review: feature: add support for sniper mode #1927

pvojtechovsky · 2018-03-25T19:20:01Z

pvojtechovsky · 2018-04-07T20:58:03Z

Any idea how to test it?

There is actually PrintChangesTest#testPrintChanged, but it just renames a type member field and prints the modified class. I see that it behaves as expected, but the question is how to test more modifications (all possible modifications?) and how to assert about the printed result in some elegant way?

monperrus · 2018-04-12T18:26:07Z

super interesting. it seems to be simply the sniper mode? correct?

for testing, I propose the following contract:

for all tree x, if subtree y is changed to z, then 
  1) content_before_y_position is unchanged, 
  2) content_after_y_position is unchanged
  3) content_whole_x_before_modification != content_whole_x_after_modification
  4) content_whole_x_after_modification contains pretty_print(z)

For testing the forall, we can sample random subtrees in src/main and ideally we would have a tree generator, see #1960

pvojtechovsky · 2018-04-12T18:41:09Z

yes, is it sniper mode, but based on different concept then #1358. But similar like #1358 it is hard to implement it 100% correctly. Actually it supports printing of changes in type or method and in all other cases it falls back to normal printing. But even this is much better then nothing.

alloy tools

I never heard about that, but it sounds interesting. I will at least read some overview/tutorial to understand what is possible. Thanks for the tip.

for testing, I propose the following contract:

the problem I see in testing is that these tests must accept some degree of incorrectness - in case of unsupported element - when it will print correct java code, but with default formatting - not with origin formatting.

Also comments are not handled yet. But it looks it will be possible to handle them too. I read also the tests implemented for #1358, but there is no solution too.

pvojtechovsky · 2018-04-12T20:00:15Z

@monperrus, @surli , @tdurieux who would like to review that?

I would like to discuss next steps with reviewer. Note: this PR is somehow working and fully commented, but has still many todos (like comment handling, bug fixing, tests, etc.). I see these ways how to proceed with this PR:

A) to review and merge it as experimental nearly with the incomplete algorithms
B) to finish other features related to sniper mode

The case A) is nearly baby step. The case B) will produce another big PR and hard to review PR.

WDYT?

Note: first commit of this PR is #1950. So after that one is merged and after rebase, it will be little bit simpler too.

tdurieux · 2018-04-13T08:49:50Z

Personally, I think that this type of feature can only be tested by using it.
I think we will find out the problem just by writing some tests. At least I did not succeed when I was working on the sniper mode.

I think it is a good idea to merge like we did with the roles.

tdurieux · 2018-04-13T08:51:58Z

One question, how do you handle imports?

tdurieux · 2018-04-13T08:56:05Z

src/main/java/spoon/support/compiler/jdt/ContextBuilder.java

+				if (e instanceof CtPackage) {
+					/*
+					 * do not generate SourcePosition for package-info files, because
+					 * 1) JDT compiler actually delivers 0,0 as source position of type package-info. Comments and annotations are not included


Now, the package.info position contains the complete file, I don't know if it is more useful for you

ok, I will check it and may be remove that change. Thank you for information.

tdurieux · 2018-04-13T08:58:33Z

Personally, I struggled a lot to handle the CtCactch and CtSwitch that have really bad positions.
It is possible to add some test to add case and/or a catch

pvojtechovsky · 2018-04-13T09:06:18Z

One question, how do you handle imports?

I always generate them - never use origin source code. I want to focus on them later. I have not though about that deeper yet.

monperrus · 2018-04-16T19:20:49Z

This feature will be great to do automatic fixing of SonarQube violations.

pvojtechovsky · 2018-04-16T19:31:46Z

missing "sniper mode" is the show stopper for usage of spoon for refactoring. Whenever I speak with my colleagues about Spoon, they are first excited ... until they hear that any transformed and then printed code will produce many useless changes in source code. It is not acceptable for projects, where history of changes is daily needed to detect who is author of what and why it is done as it is done...

ashutosh1598 · 2018-05-28T09:56:25Z

src/test/java/spoon/test/prettyprinter/PrintChangesTest.java

+
+		new SourceFragmentsTreeCreatingChangeCollector().attachTo(f.getEnvironment());
+		//change the model
+		ctClass.getField("string").setSimpleName("modified");


Pavel, can you explain how can I add my own processor in sniper mode and how to use it on a directory with many files instead of a single class?

Hi Ashutosh,
Note: that this PR is experimental. I never tried it on real sources and I expect some problems which has to be solved first. So there is actually no easy way how to enable sniper mode in Spoon API. You have to do it manually in these steps:

register one change listener SourceFragmentsTreeCreatingChangeCollector in spoon environment AFTER the model is loaded and BEFORE you start any changes.

do changes on your model

print changes (type by type) using new printer

Alright. Thanks a lot.

ashutosh1598 · 2018-05-29T16:59:06Z

When I'm analysing this simple class(without making changes) :

public class A
{
    public static void pong(String args)
    {
        System.out.println("Hello World");
    }
}

everything is fine, but if I change the argument of pong to String args[] , it is showing the following error :

spoon.SpoonException: The SourcePosition of elements are not consistent
parentFragment: |45, 56|String args|
otherFragment: |45, 58|String args[]|
	at spoon.reflect.visitor.printer.change.SourceFragment.addChild(SourceFragment.java:265)

String []args works fine though.

pvojtechovsky · 2018-05-29T19:21:58Z

Thank You for reporting that. It looks like a problem in computation of source positions. I will may be have time to have a look at that next week.

I made PR #2015 with failing test for this problem.

pvojtechovsky · 2018-09-18T17:59:29Z

Rebase was needed. I hope MavenLauncherTest will pass - there were many conflicts in that file

monperrus · 2018-09-18T18:23:28Z

I would suggest to first merge ##2461 to get rid of the problem with MavenLauncherTest and have cleaner diffs for subsequent PRs incl. this one.

pvojtechovsky · 2018-09-18T18:32:10Z

ok, I agree. No hurry is needed here

spoon-bot · 2018-09-18T18:33:20Z

API changes: 3 (Detected by Revapi)

Old API: fr.inria.gforge.spoon:spoon-core:jar:7.1.0-20180918.153340-173 / New API: fr.inria.gforge.spoon:spoon-core:jar:7.1.0-SNAPSHOT

	Method was added to an interface.
Old	none
New	method Environment#createPrettyPrinter()
Breaking	binary: non_breaking,

	The return type changed from 'spoon.reflect.visitor.printer.internal.ElementSourceFragment' to 'spoon.support.sniper.internal.ElementSourceFragment'.
Old	method SourcePositionHolder#getOriginalSourceFragment()
New	method SourcePositionHolder#getOriginalSourceFragment()
Breaking	binary: breaking,

	Method was added to an interface.
Old	none
New	method Environment#setPrettyPrinterCreator(Supplier)
Breaking	binary: non_breaking,

monperrus · 2018-09-18T18:43:33Z

We don't want to rebase this one. Are you OK if I reset the last commit on master and push force?

pvojtechovsky · 2018-09-18T18:47:30Z

yes, I am OK with reset and push force. I have no new code there

monperrus · 2018-09-18T18:50:47Z

pushed-force.

I propose to merge this one and move on later with #2461 and the @zielint0's styling PRs

WDYT?

pvojtechovsky · 2018-09-18T19:03:30Z

propose to merge this one

it is ok for me too. I just cannot do it, because I am author. I guess we need somebody else who will merge it.
@surli @tdurieux ?

monperrus

Champagne for the sniper mode!

Thanks a lot to @GerardPaligot for the initial work on modelobs, @tdurieux for having kept it in the attic, as well as the awesome contributions on positions and @pvojtechovsky for the final heavy-lifting.

That's impressive team work over time.

stefanleh · 2018-10-11T18:57:08Z

missing "sniper mode" is the show stopper for usage of spoon for refactoring. Whenever I speak with my colleagues about Spoon, they are first excited ... until they hear that any transformed and then printed code will produce many useless changes in source code. It is not acceptable for projects, where history of changes is daily needed to detect who is author of what and why it is done as it is done...

I've used javaparser for my first try with automated code refactoring and then got the hint to check out spoon. Its really great but the many useless code changes and especially the comment handling, their position is really important in my current project, are a showstopper for real code transformation. Therefore i use it only for analysis and do the plumbing then with simple regex code manipulation based on the collected information. Would be really nice to have a "minimal change impact" mode.

monperrus · 2018-10-11T20:01:49Z

Would be really nice to have a "minimal change impact" mode.

This is exactly what we have now. Have you tried the sniper mode? See http://spoon.gforge.inria.fr/launcher.html ("sniper mode")

pvojtechovsky · 2018-10-12T17:46:12Z

@stefanleh and let us know your experience - you will be probably the first brave client who tries that new toy on real project ;-)

monperrus · 2019-11-06T09:16:39Z

FYI a real pull-request made with the sniper mode by @HarisAdzemovic

apache/sling-org-apache-sling-discovery-impl#1

tdurieux · 2019-11-06T12:22:23Z

Really cool!

tdurieux · 2019-11-07T21:37:30Z

C'est bluffant! Le 07/11/2019 à 18:36, Thomas Durieux a écrit :

…

Salut Martin (CC: Benoit: tu devrais aussi aimer ca), OpenAi a publié son générateur de fake news, et tu peux l’essayer ici: https://talktotransformer.com/. J’ai été curieux, et je voulais voir sa capacité a générer du text pour un PR (à partir du titre), le résultat m’a pas mal surpris... Titre: Strings and Boxed types should be compared using "equals()” Text: Strings and Boxed types should be compared using "equals()" rather than "==" like most of the other comparison operators. However, equality is more convenient when the type of both objects can be compared in constant time. For example, if String is an instance of the Boxed class and Integer is an instance of the String class then a String object that is compared with an Integer object using "==" would perform constant time operation, whereas the "==" operator does not perform a comparison on a String object in constant time. If you want to compare two primitive types in constant time, you can compare them with the "==" operator. However, this operator is rarely used for such a comparison. Here are some examples of comparison in constant time. double.compare(float.compare(double.compare(int.compare(Integer)))); double.compare(double.compare(int.compare(Double.compare(Integer, Integer WDYT? Thomas > On 6 Nov 2019, at 04:16, Martin Monperrus ***@***.*** > ***@***.***>> wrote: > > FYI a real pull-request made with the sniper mode by @HarisAdzemovic > <https://github.com/HarisAdzemovic> > > apache/sling-org-apache-sling-discovery-impl#1 > <apache/sling-org-apache-sling-discovery-impl#1> > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#1927?email_source=notifications&email_token=ABKRWYENBK2EIW7FSE4RHITQSKDPVA5CNFSM4EXJCCJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEDF22MQ#issuecomment-550219058>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABKRWYGWDE4R7PU6I4FNNMDQSKDPVANCNFSM4EXJCCJA>. >

-- Professor in Software Technology, KTH Royal Institute of Technology https://softwarediversity.eu Director of the CASTOR center for software research https://www.castor.kth.se/

pvojtechovsky force-pushed the refDJPP branch 3 times, most recently from fd61958 to 0ea3ea4 Compare April 2, 2018 20:48

pvojtechovsky force-pushed the refDJPP branch 2 times, most recently from b39d22a to 10e9a2f Compare April 5, 2018 20:00

pvojtechovsky changed the title ~~WIP: refactor: DJPP uses #printList(...)~~ WIP: feature: PrettyPrinter for changes only Apr 5, 2018

pvojtechovsky force-pushed the refDJPP branch from 10e9a2f to 2aee09f Compare April 7, 2018 20:54

This was referenced Apr 11, 2018

SourcePositions are incorrect in some cases #1953

Closed

refactor: printing of lists in DefaultJavaPrettyPrinter #1956

Merged

pvojtechovsky force-pushed the refDJPP branch from 2aee09f to e938998 Compare April 11, 2018 18:17

monperrus mentioned this pull request Apr 12, 2018

feature: add a random AST generator to be used at testing time #1960

Open

pvojtechovsky force-pushed the refDJPP branch from 09a32aa to ac3b54a Compare April 12, 2018 19:52

tdurieux reviewed Apr 13, 2018

View reviewed changes

monperrus changed the title ~~WIP: feature: PrettyPrinter for changes only~~ WIP: feature: add support for sniper mode Apr 15, 2018

pvojtechovsky force-pushed the refDJPP branch from ac3b54a to 5765da2 Compare April 16, 2018 19:15

pvojtechovsky force-pushed the refDJPP branch from 5765da2 to bb47731 Compare April 17, 2018 19:11

tdurieux mentioned this pull request Apr 26, 2018

fix(comment): Handle comments in the bottom of a file #1976

Merged

ashutosh1598 reviewed May 28, 2018

View reviewed changes

monperrus added 6 commits September 18, 2018 20:14

up

95351a2

up

fa9336b

up

dc10a60

up

fc3056c

up

e93d840

up

0ac1269

pvojtechovsky force-pushed the refDJPP branch from 3058c79 to 0ac1269 Compare September 18, 2018 18:14

monperrus mentioned this pull request Sep 18, 2018

refactor: remove unnecessary call to 'super' #2473

Merged

pvojtechovsky mentioned this pull request Sep 18, 2018

refactor: remove redundant initializer #2472

Closed

monperrus approved these changes Sep 18, 2018

View reviewed changes

monperrus merged commit fa462e2 into INRIA:master Sep 18, 2018

pvojtechovsky deleted the refDJPP branch September 18, 2018 19:12

monperrus mentioned this pull request Sep 19, 2018

feature: sniper mode (aka high fidelity code transformation) #1284

Closed

monperrus mentioned this pull request Oct 10, 2018

Release of Spoon 7.1.0 #2489

Closed

4 tasks

tdurieux mentioned this pull request Nov 3, 2018

differences between Spoon and JavaParser #1303

Open

INRIA deleted a comment from tdurieux Nov 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

review: feature: add support for sniper mode #1927

review: feature: add support for sniper mode #1927

pvojtechovsky commented Mar 25, 2018 •

edited by monperrus

pvojtechovsky commented Apr 7, 2018

monperrus commented Apr 12, 2018

pvojtechovsky commented Apr 12, 2018

pvojtechovsky commented Apr 12, 2018

tdurieux commented Apr 13, 2018

tdurieux commented Apr 13, 2018

tdurieux Apr 13, 2018

pvojtechovsky Apr 13, 2018

tdurieux commented Apr 13, 2018

pvojtechovsky commented Apr 13, 2018

monperrus commented Apr 16, 2018

pvojtechovsky commented Apr 16, 2018

ashutosh1598 May 28, 2018

pvojtechovsky May 28, 2018

ashutosh1598 May 28, 2018

ashutosh1598 commented May 29, 2018 •

edited by tdurieux

pvojtechovsky commented May 29, 2018 •

edited

pvojtechovsky commented Sep 18, 2018

monperrus commented Sep 18, 2018

pvojtechovsky commented Sep 18, 2018

spoon-bot commented Sep 18, 2018

monperrus commented Sep 18, 2018

pvojtechovsky commented Sep 18, 2018

monperrus commented Sep 18, 2018

pvojtechovsky commented Sep 18, 2018

monperrus left a comment

stefanleh commented Oct 11, 2018

monperrus commented Oct 11, 2018

pvojtechovsky commented Oct 12, 2018

monperrus commented Nov 6, 2019

tdurieux commented Nov 6, 2019

tdurieux commented Nov 7, 2019 via email

review: feature: add support for sniper mode #1927

review: feature: add support for sniper mode #1927

Conversation

pvojtechovsky commented Mar 25, 2018 • edited by monperrus

pvojtechovsky commented Apr 7, 2018

monperrus commented Apr 12, 2018

pvojtechovsky commented Apr 12, 2018

pvojtechovsky commented Apr 12, 2018

tdurieux commented Apr 13, 2018

tdurieux commented Apr 13, 2018

tdurieux Apr 13, 2018

Choose a reason for hiding this comment

pvojtechovsky Apr 13, 2018

Choose a reason for hiding this comment

tdurieux commented Apr 13, 2018

pvojtechovsky commented Apr 13, 2018

monperrus commented Apr 16, 2018

pvojtechovsky commented Apr 16, 2018

ashutosh1598 May 28, 2018

Choose a reason for hiding this comment

pvojtechovsky May 28, 2018

Choose a reason for hiding this comment

ashutosh1598 May 28, 2018

Choose a reason for hiding this comment

ashutosh1598 commented May 29, 2018 • edited by tdurieux

pvojtechovsky commented May 29, 2018 • edited

pvojtechovsky commented Sep 18, 2018

monperrus commented Sep 18, 2018

pvojtechovsky commented Sep 18, 2018

spoon-bot commented Sep 18, 2018

monperrus commented Sep 18, 2018

pvojtechovsky commented Sep 18, 2018

monperrus commented Sep 18, 2018

pvojtechovsky commented Sep 18, 2018

monperrus left a comment

Choose a reason for hiding this comment

stefanleh commented Oct 11, 2018

monperrus commented Oct 11, 2018

pvojtechovsky commented Oct 12, 2018

monperrus commented Nov 6, 2019

tdurieux commented Nov 6, 2019

tdurieux commented Nov 7, 2019 via email

pvojtechovsky commented Mar 25, 2018 •

edited by monperrus

ashutosh1598 commented May 29, 2018 •

edited by tdurieux

pvojtechovsky commented May 29, 2018 •

edited