New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow incremental annotation processors to write resources #4702

Open
oehme opened this Issue Mar 13, 2018 · 11 comments

Comments

Projects
None yet
4 participants
@oehme
Copy link
Member

oehme commented Mar 13, 2018

For instance a processor might want to generate a database schema based on annotated entity classes.

Supporting Filer.createResourceFile wouldn't be a big problem. But as far as I've seen, most processors that create resources don't use the Filer, because they don't want to mix generated resources with generated .java files. Instead they ask users to provide a processor argument, pointing the processor to a different location and then they use java.io directly. This means that Gradle will miss the creation of these files and thus would not clean them up when necessary.

Ideally the Filer API would provide a way for processors to define new output locations, which the user can then configure using compiler arguments. Right now only source output and class output are supported.

@cesar1000

This comment has been minimized.

Copy link

cesar1000 commented Apr 3, 2018

Incidentally, we have an annotation process that generates a database schema into the build directory, and we add it to the compiler task explicitly as an input.

We also use AutoService on our annotation processors, in order to publish the processor for discovery by the compiler. It looks like it uses the Filer API properly: https://github.com/google/auto/blob/master/service/src/main/java/com/google/auto/service/processor/AutoServiceProcessor.java

@oehme

This comment has been minimized.

Copy link
Member Author

oehme commented Apr 3, 2018

we add it to the compiler task explicitly as an input.

You mean as an output directory?

@cesar1000

This comment has been minimized.

Copy link

cesar1000 commented Apr 3, 2018

Yeah, sorry, we make it an explicit output in order to enable caching:

    variant.javaCompile.with {
        // Declare the schema output directory as a compilation output so that it's stored in the cache.
        outputs.dir(schemaOutputDirectory)
                .withPropertyName('schemaOutputDirectory')
    }
@ZacSweers

This comment has been minimized.

Copy link

ZacSweers commented Apr 14, 2018

Another example: we just open sourced this project for breadcrumbing metadata across compile-time boundaries. We do it by generating the data into resources and then consuming them in consumers of the library.

@oehme

This comment has been minimized.

Copy link
Member Author

oehme commented Apr 15, 2018

@hzsweers I'm afraid that processor currently won't work incrementally for deeper reasons: It's accessing internals of the compiler (e.g. the file manager) to get those resources. So we would have a hard time tracking its inputs and outputs. The casts it does to get those internals would also fail since we decorate the processing environment.

One way to make it work would be for Gradle to track resource changes and marking this processor as aggregating. It would then reprocess on any class or resource change. The only problem would be the casts then. You would have to add a workaround for Gradle and extract the original processing environment out of ours.

@ZacSweers

This comment has been minimized.

Copy link

ZacSweers commented Apr 15, 2018

Not being able to operate incrementally when reading data makes sense, but I don't see why producing data can't be incremental since it only writes data through normal file object APIs?

@cesar1000

This comment has been minimized.

Copy link

cesar1000 commented Apr 15, 2018

@oehme

This comment has been minimized.

Copy link
Member Author

oehme commented Apr 15, 2018

@hzsweers The writing part is not a big problem, but that's only half of your processor. I was just pointing out that fixing this won't be enough for your use case.

@ZacSweers

This comment has been minimized.

Copy link

ZacSweers commented Nov 17, 2018

Any update on this? This is actually going to break Dagger once Android's R8 tool is finished, as it generates R8 .pro files into resources. I was talking with Jerome from the android gradle plugin team at the android dev summit as well about this, and he confirmed this causes full recompilations with dagger in gradle 5.0. Would be kind of a 1 step forward 1 step back scenario :|

@isker

This comment has been minimized.

Copy link
Contributor

isker commented Feb 1, 2019

@oehme would it be acceptable to start with just supporting Filer.createResourceFile? I am willing to take a crack at it if you think it's reasonable.

@oehme

This comment has been minimized.

Copy link
Member Author

oehme commented Feb 2, 2019

@isker yes, that would be a start. You'd have to add resource tracking to our incremental compile infrastructure, as we currently only track which classes to recompile.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment