Keep package structure in rootTreeOrProvider

[Duhemm]

rootTreeOrProvider is a property of ClassSymbol, and contain the
tree that defines this symbol (or a mean to get that tree).

When the tree came from unpickled TASTY files, we were able to see the
nested packages from which the tree comes, along with the imports.
However, when the symbol has been loaded from source, we were filling
the rootTreeOrProvider only with the TypeDef that introduced this
symbol.

This commit moves the assignment of rootTreeOrProvider out of the
typer and into its own phase. Before assigning, we recreate the package
structure so that the information that we get out of
rootTreeOrProvider is the same regardless of whether the symbol has
been loaded from source or unpickled from TASTY.

[Duhemm]

test performance please

[dottybot]

performance test scheduled: 1 job(s) in queue, 0 running.

[dottybot]

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/5276/ to see the changes.

Benchmarks is based on merging with master (a821035)

[Duhemm]

test performance please

[dottybot]

performance test scheduled: 1 job(s) in queue, 1 running.

[allanrenucci]

Can you clarify what problem this PR solves? What is rootTreeOrPeovider used for?

Is there another way to solve this rather than introducing a complete new tree traversal?

[nicolasstucki]

That would not be a complete tree traversal, it would be quite shallow. Traverse from the root package until you find the first TypeDef. The implementation is a bit awkward and makes it look like a full traversal.

[allanrenucci]

It looks like a complete tree traversal to me:

    override def traverse(tree: tpd.Tree)(implicit ctx: Context): Unit = {
      if (tree.isInstanceOf[tpd.TypeDef] && tree.symbol.isClass) {
        ...
      }
      traverseChildren(tree)
    }

traverseChildren is called unconditionally. Am I missing something?

[Duhemm]

rootTreeOrProvider is used to get the tree that introduced a given ClassSymbol. This tree can come either from source (for instance, if we have just compiled the file that defined it, or the file that defines it needs recompilation), or it can come from TASTy (we're unpickling).

Given a class symbol, the value of rootTreeOrProvider can come from 2 different paths:

• When the symbol comes from the source, rootTreeOrProvider used to be populated by the Typer, where it was set to the typed TypeDef that introduced the symbol. We have no information about the packages the TypeDef was nested in, nor about the imports.
• When the symbol comes from TASTy, rootTreeOrProvider is set to the unpickler by the symbol loader. When the tree is requested, the unpickler will unpickle the trees, which will include the nested package defs and imports.

In the IDE, we always want to see the top level imports, because this information is necessary to correctly do renamings.
With the IDE, we have 3 scenarios to consider:

1. The file is open in the IDE: We will have the complete tree without needing to look at rootTreeOrProvider, so we'll see the complete structure.
2. The file is not open in the IDE, and it doesn't need recompilation: We will unpickle the tree and see the complete structure.
3. The file is not open in the IDE, and it needs recompilation: We will use rootTreeOrProvider, and we won't be able to see top level imports.

This PR is only important in the 3rd scenario. When setting the rootTreeOrProvider, we take the typed compilation unit, keep the nested packages and imports and the class definition. This way, we can store in rootTreeOrProvider an information similar to what we'd if we were unpickling the tree. Doing this is moved out of the typer so that we have access to the typed compilation unit.

Now that we have IDE tests where the files are not "open in the IDE", I'll add tests where that matters.

[allanrenucci]

Also I don't think the benchmark set YretainTrees, so you would not be able to measure the impact of this change

[dottybot]

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/5276/ to see the changes.

Benchmarks is based on merging with master (193e064)

[nicolasstucki]

Also I don't think the benchmark set YretainTrees, so you would not be able to measure the impact of this change

No, they do not

[nicolasstucki]

@Duhemm what is the state of this PR

[Duhemm]

@nicolasstucki I've added a test that illustrates why this patch is needed. Without the addition of SetRootTree, the test fails as expected.

I've addressed your comments:

• The traversal is now very shallow
• I've moved from Phase to MacroTransform. I'm not convinced this is an improvement, since this phase doesn't change the tree.

[allanrenucci]

I think 6272e8d should be reverted. If it does not transform the tree then it doesn't have to be a macro transform

[nicolasstucki]

@Duhemm something failed in the tests

[nicolasstucki]

I've moved from Phase to MacroTransform. I'm not convinced this is an improvement, since this phase doesn't change the tree.

Yes you are right, it is better as a phase.

[nicolasstucki]

Otherwise looks good

[Duhemm]

@nicolasstucki I've fixed the test in 136b0ce, do you mind taking another look?