Allow incremental annotation processors to write resources

[oehme]

For instance a processor might want to generate a database schema based on annotated entity classes.

Supporting Filer.createResourceFile wouldn't be a big problem. But as far as I've seen, most processors that create resources don't use the Filer, because they don't want to mix generated resources with generated .java files. Instead they ask users to provide a processor argument, pointing the processor to a different location and then they use java.io directly. This means that Gradle will miss the creation of these files and thus would not clean them up when necessary.

Ideally the Filer API would provide a way for processors to define new output locations, which the user can then configure using compiler arguments. Right now only source output and class output are supported.

[cesar1000]

Incidentally, we have an annotation process that generates a database schema into the build directory, and we add it to the compiler task explicitly as an input.

We also use AutoService on our annotation processors, in order to publish the processor for discovery by the compiler. It looks like it uses the Filer API properly: https://github.com/google/auto/blob/master/service/src/main/java/com/google/auto/service/processor/AutoServiceProcessor.java

[oehme]

we add it to the compiler task explicitly as an input.

You mean as an output directory?

[cesar1000]

Yeah, sorry, we make it an explicit output in order to enable caching:

    variant.javaCompile.with {
        // Declare the schema output directory as a compilation output so that it's stored in the cache.
        outputs.dir(schemaOutputDirectory)
                .withPropertyName('schemaOutputDirectory')
    }

[ZacSweers]

Another example: we just open sourced this project for breadcrumbing metadata across compile-time boundaries. We do it by generating the data into resources and then consuming them in consumers of the library.

[oehme]

@hzsweers I'm afraid that processor currently won't work incrementally for deeper reasons: It's accessing internals of the compiler (e.g. the file manager) to get those resources. So we would have a hard time tracking its inputs and outputs. The casts it does to get those internals would also fail since we decorate the processing environment.

One way to make it work would be for Gradle to track resource changes and marking this processor as aggregating. It would then reprocess on any class or resource change. The only problem would be the casts then. You would have to add a workaround for Gradle and extract the original processing environment out of ours.

[ZacSweers]

Not being able to operate incrementally when reading data makes sense, but I don't see why producing data can't be incremental since it only writes data through normal file object APIs?

[cesar1000]

Also, would it make sense to use gradle variants for passing resources downstream instead of Java resources (when running in gradle)? You should be able to write a gradle plugin to set up the annotation processor parameters and add inputs and outputs to the compiler task.

…

On Sun, Apr 15, 2018, 16:14 Zac Sweers ***@***.***> wrote: Not being able to operate incrementally when reading data makes sense, but I don't see why producing data can't be incremental since it only writes data through normal file object APIs? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4702 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACZo0LW54gHVWGBJgEpGSjTbqZVl6Iweks5to1XcgaJpZM4SowMF> .

[oehme]

@hzsweers The writing part is not a big problem, but that's only half of your processor. I was just pointing out that fixing this won't be enough for your use case.

[ZacSweers]

Any update on this? This is actually going to break Dagger once Android's R8 tool is finished, as it generates R8 .pro files into resources. I was talking with Jerome from the android gradle plugin team at the android dev summit as well about this, and he confirmed this causes full recompilations with dagger in gradle 5.0. Would be kind of a 1 step forward 1 step back scenario :|

[isker]

@oehme would it be acceptable to start with just supporting Filer.createResourceFile? I am willing to take a crack at it if you think it's reasonable.

[oehme]

@isker yes, that would be a start. You'd have to add resource tracking to our incremental compile infrastructure, as we currently only track which classes to recompile.