Deprecate Analysis.apis

[gkossakowski]

This is a proposal.

Analysis.apis

The Analysis.apis returns an instance of APIs. Methods in APIs allow one to inspect the public API of the compiled program. By the API we mean information like declared classes (and their method) in given source file.

One can think of APIs as a very basic reflection-like API.

The role of Analysis.apis

The main client of APIs is the incremental compiler itself. It uses information stored in APIs to find out what semantic changes were applied to Scala sources. Based on that knowledge, the incremental compiler algorithm determines which files should be recompiled. That's was the original use case for introduction of Analysis.apis.

Over time, some other use cases arose. In sbt 0.13.2 Analysis.apis is used for the following tasks performed outside of the incremental compiler:

• implementation of testQuick task uses information about compilation start time of each of source files to determine which tests got recompiled and should be rerun (see testQuickFilter in Defaults.scala)
• the sbt project definition itself uses information about compilation start time to determine when last recompilation has been performed. This information is used to decide whether xsbt.version.properties file should be regenerated (see generateVersionFile and lastCompilationTime methods in project/Util.scala)
• implementation of tests discovery (matching against Fingerprints) is implemented by using information stored in APIs in reflection-like manner (see the discover method in Tests.scala)

Given that Analysis.apis is public, and Analysis object is returned by the compile task then one can imagine other projects depending on Analysis.apis.

Why to touch Analysis.apis

The main motivation is anticipated work for #1078. In order to address performance problems of incremental compiler it's very likely we'll have to be more aggressive about it's internal data structures. The Analysis.apis is one of such data structures that is most likely to be redesigned as part of fixing #1078.

We should warn sbt users this API is going away. For details why we'll remove this API (or limit it to sbt-internal consumption) see the section on Pitfalls.

Migrating uses of Analysis.apis

The last bullet is the most advanced use of of information available through APIs API. The other two bullets are trivial to replace by dedicated APIs that Analysis should introduce.

Pitfalls with APIs

Originally, the APIs stored an index of entire Scala program sbt was compiling. It included information about public member in a source files including nested members. However, this turned to be very expensive both memory-wise and CPU-wise. In sbt 0.12.x hashing of extracted API has been introduced. As a result, almost all information about members in given source files is wiped out and replaced by hash sum of removed members.

In other words, for performance reasons APIs won't ever become a minimal reflection-like API.

Tasks related to deprecation

To make the proposal more specific, here's the list of tasks related to deprecation of Analysis.apis:

• deprecate Analysis.apis val, deprecate the entire APIs trait and provide documentation on how to migrate the most common use cases to other methods provided by sbt
• migrate from using Analysis.apis to Analysis.compilations for determining lastCompilationTime (Use Analysis.compilation for lastCompilationTime #1272)
• provide an direct API for discovering all classes that have main method defined (API for finding main methods in code compiled by sbt #1274)
• brainstorm reflection or ASM-based analysis of bytecode for tasks like tests discovery

[jsuereth]

So, do you have a way to still support the testQuick or test discovery use case?

I think testQuick (incremental running of tests based on detected change) is pretty compelling and needs to be supported via some mechanism. I'm ok not using apis for that, as apis exposes your internals, however I think that perhaps testQuick gives us a pretty good idea of the sorts of "external to compilation" incremental information we need to expose (that isn't just the whole API).

Related to tests, there's been a long-standing issue of "testOnly" selecting a specific test rather than a test class. related, we'd like to do the same with testQuick.

The write-up is great and compelling, but I think we need to talk about the testing case, as well as other "like-minded" tasks which I think we'll be trying to leverage more of in things like Activator...

[gkossakowski]

When it comes to testQuick I alluded this above but not clearly enough:

The last bullet is the most advanced use of of information available through APIs API. The other two bullets are trivial to replace by dedicated APIs that Analysis should introduce.

Specifically, for testQuick all we need is information about the last time each source file has been recompiled. I think we can provide a direct API for that in form of a new Relation in Relations. It would be very easy to implement and it would make a lot of sense to provide such API.

I don't follow this remark:

Related to tests, there's been a long-standing issue of "testOnly" selecting a specific test rather than a test class. related, we'd like to do the same with testQuick.

Could you elaborate or give me some pointers? Is this related to incremental compilation?

When it comes to test discovery (and generally, semantic analysis of compiled program) then I think the best bet is either Java reflection or Scala reflection (if full Scala semantics are needed). There will be some performance penalty of using reflection but I expect it to be lower than the 20% performance overhead described in #1078.

[jsuereth]

It's sort of related.

SO:

1. For testQuick, I'd like to try to isolate the "test needs to run" detection on a test itself, not the entire class/source file. Remember this needs to check if any dependencies of a given test have changed, so some way of flagging a particular portion of code as "give me the known dependencies" (possibly classfile based) so we can see if we need to run it. It's actually an incremental-ish thing, but I'm not sure how much it needs ot use the incremental algorithm vs. not
2. For test fingerprint/running, we may need to actually rework this functionality a bit to be regular/give sbt a bit more flexibility over what tests it can run and how it can associate a "test" with its compilation dependencies so we can re-run the minimum of tests on a change.

Maybe easier to do a G+ chat about it so we make sure we're in synch (not N'SYNC)

[gkossakowski]

Ok, I understand that now. I think you're talking about new functionality/improvements to testing infrastructure in sbt. I agree it's the best to have a chat on G+ on this and summarize our discussion here.

[gkossakowski]

I just checked activator's source code for uses of apis:

Greks-MBP:activator grek (master)$ ag -C 2 apis
project/integration.scala
27-    localRepoProjectsPublished <<= publishLocal,
28-    mains <<= compile in Compile map { a =>
29:      val defs = a.apis.internal.values.flatMap(_.api.definitions)
30-      val results = Discovery(Set("xsbti.Main"), Set())(defs.toSeq)
31-      results collect { 

project/Properties.scala
46-  
47-  def lastCompilationTime(analysis: sbt.inc.Analysis): Long = {
48:    val times = analysis.apis.internal.values map (_.compilation.startTime)
49-    if(times.isEmpty) 0L else times.max
50-  }

The first use is for discovering main methods that we'll have to find some alternative way of providing. The other one is determining last compilation time. It'll be easy to migrate to Analysis.compilations.