Skip to content
This repository has been archived by the owner on Sep 3, 2020. It is now read-only.

Consider named parameters during rename refactorings #141

Conversation

mlangc
Copy link
Collaborator

@mlangc mlangc commented Feb 29, 2016

This PR implements support for named parameters when doing rename refactorings (see Ticket #1002501, as well as Ticket #1002572). To achieve this, this PR takes care of several things:

  1. Named parameters are desugared. Therefore we have to manually parse them in the original source code. Unfortunately, due to the compiler setting incorrect positions on some of the trees in the AST, this is harder than it should be. The parsing is done with the SourceWithMarker framework, that now supports backtracking (note that I've strong evidence for the fact that the newly added backtracking support should not cause performance regressions: https://github.com/mlangc/scala-refactoring-experiments/blob/master/src/test/scala/com/github/mlangc/experiments/refactoring/SourceWithMarkerBenchmark.scala).
  2. Named parameters are often used in connection with constructors and case classes. In these cases, there is a one-to-one mapping between certain named parameters and class values (think of constructors and generated copy and apply methods). This is what the updates related to SymbolExpander are about.
  3. Renaming a method parameter is no longer a local operation in the sense of Rename.PreparationResult.hasLocalScope if the method might be called from outside the file that defines it.
  4. The performance problems that led to support for named arguments being dropped in the past should be addressed by 168d91f. I did some benchmarking (see https://github.com/mlangc/scala-refactoring-experiments/blob/master/src/test/scala/com/github/mlangc/experiments/refactoring/RenameBenchmark.scala) and did not see any changes in the results, but there might be corner cases that are not covered by the benchmark.

Note that it is currently not possible to rename named parameters where they are used, but only where they are defined, that is you cannot initiate a rename refactoring at f(tryRenameMe = 22), but only at def f(tryRenameMe: Int). In my opinion, we should take care of this in a separate PR though because

  • this is not a regression
  • this PR is quite large already
  • fixing this issue would also require changes to Scala-IDE and possibly ENSIME

@misto
Copy link
Member

misto commented Mar 1, 2016

IIRC, one of the reasons the ApplyExtracted and BlockExtracter were commented out was that it had a huge impact on performance. Do you have any idea how the current situation is? We unfortunetly don't have tests for this.

@mlangc
Copy link
Collaborator Author

mlangc commented Mar 1, 2016

@misto The performance problems you are referring to should be addressed by a6a7f85.

@mlangc
Copy link
Collaborator Author

mlangc commented Mar 4, 2016

Darn, I forgot to test with Scala-2.10...

@mlangc mlangc force-pushed the 1002501-rename-does-not-consider-named-parameters branch 2 times, most recently from 511b1f7 to d5a150c Compare March 11, 2016 10:23
@mlangc mlangc changed the title WIP: 1002501 rename does not consider named parameters Consider named parameters during rename refactorings Mar 11, 2016
@@ -29,22 +29,73 @@ abstract class Rename extends MultiStageRefactoring with TreeAnalysis with analy

def prepare(s: Selection) = {

/*
* This heuristic decides if this refactoring is local to a single file or not. Note that
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a sensible trade off between safety and ease of use.

@misto
Copy link
Member

misto commented Mar 17, 2016

I played around with a few of the tests but couldn't break it. Nice job! Unfortunately, there's a minor conflict (with the recent imports removal Simon did) so we can't merge it automatically.

This commit contains some fixes that were necessary due to changes related to
our migration to SBT.
This patch reenables named parameter extraction, that was already implemented,
but deactivated for performance reasons (see
https://groups.google.com/forum/embed/#!topic/scala-internals/H_psZ3RV6-4).
We need this functionality so that named arguments can be handled properly by
rename refactorings.
This commit fixes a bug, which resulted in a serious performance problem, that
was the was the reason why support for named arguments was dropped in the past.
See https://groups.google.com/forum/embed/#!topic/scala-internals/H_psZ3RV6-4
for details.
Renaming class parameters is a global operation, unless the constructor is
explicitly marked as private.
This commit makes sure that a rename refactoring involving a class parameter
also affects locations where this parameter is used as a named argument of
the constructor, like `new Foo(bar = 42)`.
Scala-2.10 generates OffsetPositions at places where Scala-2.11 has
RangePositions. This commit adapts the code accordingly, and makes it simpler
along the way.
Up until now, the movements related to `SourceWithMarker` did not implement
backtracking. As a consequence, `'a'.zeroOrMore ~ 'a'` would not consume "a",
because `'a'.zeroOrMore` would always move over "a", leaving no room for the
following movement. While this wasn't a severe restriction for moving forward
since rules can be adapted accordingly, this becomes a real problem when moving
back, since `('a' ~ 'a'.zeroOrMore).backward` applies `'a'.zeroOrMore` before
`'a'`.
This commit makes sure that the parameters of compiler generated `apply` and
`copy` methods for case classes are associated with the related class vals.
This commit works around various limitations and bugs when extracting named
parameters.
This commit makes sure that case classes owned by methods are correctly handled
when doing rename refactorings.

Fix #1002501, Fix #1002572 together with the last commits
@mlangc mlangc force-pushed the 1002501-rename-does-not-consider-named-parameters branch from fefcd9d to c7c326c Compare March 17, 2016 16:45
@mlangc
Copy link
Collaborator Author

mlangc commented Mar 17, 2016

@misto I've just rebased.

Up until now, `Movements.until` stuck in an infinite loop for a `skipping`
movement that could be satisfied by an empty string, like `'a'.zeroOrMore`.
This isn't a severe limitation to someone who is well aware of this, but
I think that it can be very annoying otherwise.

{
val src = SourceWithMarker("aaaa")
assertTrue(src.moveMarker('a'.zeroOrMore.zeroOrMore).isDepleted)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's not a problem with this PR but maybe in future it's worth to consider renaming isDepleted to isConsumed. Currently it looks akin to PimpedTree.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree... isConsumed sounds better indeed. A rename refacoring might be in order ;-). However, this might cause some conflicts with PR #144, so maybe we should wait with this until both PRs are merged?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for a separate PR so we can merge this one

@kiritsuku
Copy link
Member

Impressive work. Worked very well in my tests. LGTM

@wpopielarski
Copy link
Member

Generally LGTM, but I have one general question about refactoring. There is ApplyExtractor and BlockExtractor and other and all refactoring actions should use them or every refactoring action should rather define its own well tailored extractors? I think it is a design question. For example I have a problem now in organize imports action concerning the discovering of usages stable identifiers from import global._. As I understand it is done by CompilationUnitDependencies and the big issue at least for novice is to add some snippet of code into it with fear that some other functionality can be broken.

@kiritsuku
Copy link
Member

I think both ApplyExtractor and BlockExtractor miss better Scaladoc that explains exactly what they are doing. These non-scalac tree nodes introduce additional complexity that potentially needs to be faced everywhere.
@wpopielarski I didn't understand what exactly your problem is but we can talk later about it.

@mlangc
Copy link
Collaborator Author

mlangc commented Mar 22, 2016

@wpopielarski I see your point: Both ApplyExtractor and BlockExtractor have been there before this PR, I just enhanced them. The only traverser that uses ApplyExtractor explicitly is TreeWithSymbolTraverse. Apart from that, there is also scala.tools.refactoring.common.EnrichedTrees.children, that nobody seems to use according to References -> Workspace (I'm not sure if this is reliable). BlockExtractor is not used in any traverser explicitly, but indirectly via TemplateExtractor. So you are right, the situation is quite confusing, and it would be great to bring some order into this.

As for CompilationUnitDependencies: This is certainly not the easiest part of the library, but it should not be affected by the changes to ApplyExtractor or BlockExtractor. Also, there are quite a few tests for this module. If you break it, the chances that one of them fails are high, so if you want to change something there, just do it and see what happens ;-).

@mlangc
Copy link
Collaborator Author

mlangc commented Mar 22, 2016

@sschaef I just added some Scaladocs for ApplyExtractor. I wanted to do the same for BlockExtractor but realized, that I don't understand it well enough myself (it wasn't added in this PR and wrong documentation is worse than no documentation).

What puzzles me for example, is that both ApplyExtractor and BlockExtractor concern themselves with named arguments. I tried to get rid of the (limited) named arguments handling in BlockExtractor but ran into various failing tests (mostly related to tree printing).

@misto, maybe you could shine some light on this?

@misto
Copy link
Member

misto commented Mar 23, 2016

What I attempted to do with the various XyExtractors was to sanitize or undo some of the desugaring the compiler does, so that during pattern matching over a tree, we get NamedArgument trees even though they are not really part of the AST. And IIRC, depending on how named arguments calls are ordered, they are sometimes desugared to Blocks and sometimes simply to Apply trees (I believe when the named parameters are passed in the same order as the defined parameters, it's an Apply tree). The story is similar with multiple-assignments (val (a, b) = ..), which is actually a pattern match that introduces a lot of additional trees and weird positions..

@misto
Copy link
Member

misto commented Mar 23, 2016

And yeah, luckily we have a very high test coverage, so if you manage to simplify something and get rid of code and no test breaks, then just go for it.

@mlangc
Copy link
Collaborator Author

mlangc commented Mar 25, 2016

@misto Thanks for the clarification; I've added a comment to BlockExtractor that highlights that it is mostly (in fact I think only - but didn't prove it) needed for tree printing.

@kiritsuku
Copy link
Member

Let's test this in production

@kiritsuku kiritsuku merged commit 7828e28 into scala-ide:master Mar 25, 2016
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants