Extensibility API for AutoValue

[eamonnmcmanus]

Issues #87, #162, and #201 are examples of use cases that are a bit too specialized to put into the core of AutoValue, but which are still important when they apply. We should define an extensibility API so that users can build extensions that run at the same time as AutoValue. The general idea would be that there would be an interface, say AutoValueExtension, and that the AutoValue processor would use ServiceLoader to find implementations of this interface. So if you include on your -processorpath a jar that contains such an implementation, AutoValue will invoke it when it runs.

A possible sketch of the interface is this:

public interface AutoValueExtension {
  interface Context {
    ProcessingEnvironment processingEnvironment();
    String packageName();
    TypeElement autoValueClass();
    Map<String, ExecutableElement> properties();
  }

  boolean applicable(Context context);

  String generateClass(Context context, String classToExtend, String classToImplement);
}

The idea is that every extension gets its applicable method called, and those that return true get their generateClass method called. That method can return a String that is an implementation of the class called classToImplement that extends the class called classToExtend; or it can return null (for example if it has generated some related code but doesn't need to intervene in the AutoValue hierarchy). So if you have two extensions, @AutoValue class Foo might result in a hierarchy AutoValue_Foo extends $AutoValue_Foo extends $$AutoValue_Foo extends Foo, where AutoValue_Foo is generated by one of the extensions, $AutoValue_Foo is generated by the other, and $$AutoValue_Foo contains the logic generated by the AutoValue processor. This means that extensions can override the implementations of toString() etc.

[PhilippWendler]

As far as I understand, this would allow me to customize the object creation, too, because I can insert code into the constructor of the class that my extension generates, correct? This would be good.

The use case I have is that I want to automatically create an @AutoValue instance from something that wraps a Map<String, String>. We use something like this for configuration options (currently based on reflection instead of AutoValue).

So my constructor would not have parameters for all properties but only for the map.
In the constructor, I could retrieve the value for each property from the map, convert it to the expected type, and pass them to the super constructor. The down side would be that yet another class gets created that is actually not necessary if I could intercept the passing of values from the factory method to the AutoValue-generated constructor somehow without it (because my subclass would not add or override any methods, only the constructor).

[rharter]

I've started playing around with implementing this feature, and one thing that I've come across is this error:

@AutoValue classes cannot have abstract methods other than property getters and Builder converters

In my implementation, I'm creating an auto-extension that implements Android Parcelable, but the fact that my class implements an interface means there are extra abstract methods (from the interface).

It seems reasonable to me that other extensions might also want to add abstract methods that they can act on, so the question is, should this error simply go away? The problem with that is that you then lose the compile time error reporting in the case where there are any abstract methods that a generator didn't implement.

One possible solution I can see is to record all abstract methods on the annotated class, track which ones have been implemented in the processing step, and throw an error if there are any abstract methods not implemented by autovalue or it's extensions. Thoughts?

[eamonnmcmanus]

Good point. It would make sense for the extension API to include a way for extensions to see and claim abstract methods. Then the error above would be produced only after extensions have run if there are still methods that were not claimed by any of them.

[kevinb9n]

(Does it keep things simpler to just not care, and let the generated
concrete class just fail to compile if something is missing?)

On Mon, May 4, 2015 at 7:56 AM, Éamonn McManus notifications@github.com
wrote:

Good point. It would make sense for the extension API to include a way for
extensions to see and claim abstract methods. Then the error above would be
produced only after extensions have run if there are still methods that
were not claimed by any of them.

—
Reply to this email directly or view it on GitHub
#202 (comment).

Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb@google.com

[rharter]

That's also a fair point. Since the unclaimed methods are abstract, there
would still be a compile time error, so is this error message much more
helpful than that, or are we just duplicating compiler functionality there?

On Mon, May 4, 2015, 10:02 AM kevinb9n notifications@github.com wrote:

(Does it keep things simpler to just not care, and let the generated
concrete class just fail to compile if something is missing?)

On Mon, May 4, 2015 at 7:56 AM, Éamonn McManus notifications@github.com
wrote:

Good point. It would make sense for the extension API to include a way
for
extensions to see and claim abstract methods. Then the error above would
be
produced only after extensions have run if there are still methods that
were not claimed by any of them.

—
Reply to this email directly or view it on GitHub
#202 (comment).

Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb@google.com

—
Reply to this email directly or view it on GitHub
#202 (comment).

[eamonnmcmanus]

I should note that the message in question is actually a warning rather than an error, and does not in itself cause compilation to fail. The reason for that is that the unhelpful Eclipse compiler sometimes fails to realize that a concrete method overrides an abstract one.

The philosophy has been for AutoValue never to generate code that doesn't compile, unless it is accompanied by at least one error message that points to the area in the source code where the problem was. That's particularly important for IDE integration, because you really want an error message that is attached to the source you are looking at, so it can be redlined.

[kevinb9n]

Ah, got it.

On Mon, May 4, 2015 at 8:14 AM, Éamonn McManus notifications@github.com
wrote:

I should note that the message in question is actually a warning rather
than an error, and does not in itself cause compilation to fail. The reason
for that is that the unhelpful Eclipse compiler sometimes fails to realize
that a concrete method overrides an abstract one.

The philosophy has been for AutoValue never to generate code that doesn't
compile, unless it is accompanied by at least one error message that points
to the area in the source code where the problem was. That's particularly
important for IDE integration, because you really want an error message
that is attached to the source you are looking at, so it can be redlined.

—
Reply to this email directly or view it on GitHub
#202 (comment).

Kevin Bourrillion | Java Librarian | Google, Inc. | kevinb@google.com

[rharter]

Those are some really good points, @eamonnmcmanus.

I've got a start on the extensibility support. Would you mind taking a look at my branch and tell me if you think I'm on the right track. This is sort of in the POC stage right now and could use a little bit of cleanup (and still has problems with builders), but I just wanted to make sure I'm on the right track before moving too far.

I wasn't really sure how to test the extensions, so I added a constructor so that I could inject the extensions. I left it package private so it won't be a part of the public API, but perhaps there is a better way to do this.

[eamonnmcmanus]

@rharter, I had a quick look through your branch and it does look pretty much like what I had in mind. I don't have time right now to do a more thorough review but I hope to be able to get to it within the next week. Please nag me if you don't hear from me soon.

[rharter]

After discussion with @eamonnmcmanus, I think we've come up with a working final architecture. I'm documenting it here.

In the new architecture the AutoValue generated class will be the only direct subclass of the @AutoValue class Foo class, while each extension will have the opportunity to generate a subclass of that. The classes will have $ prepended to their name accordingly.

@AutoValue class Foo {
  public abstract String bar();
}

@Generated(AutoValue.class)
class $$AutoValue_Foo extends Foo {
  public String bar() {...}
  @Override public String toString() {...}
}

@Generated(CustomToStringExtension.class)
class $AutoValue_Foo extends $$AutoValue_Foo {
  @Override public String toString() {...}
}

@Generated(ParcelableExtension.class)
class AutoValue_Foo extends $AutoValue_Foo {
  // parcelable bits...
}

The special case here is that for any given @AutoValue class, there can only be one extends that requires itself be at the end of this chain. In the case of Android Parcelable, a static factory property is required on the implementing class, so it must remain at the end of the chain.

A couple of questions I have for this architecture:

• How to handle the Builder. Normally, a user can create a static builder() method that will return new AutoValue_Foo.Builder(), but we can no longer guarantee that the builder is available at that point in the chain, since it might be $$AutoValue.Builder().

[cgruber]

Ok - not going to lie, the million superclasses doesn't make me feel very
happy, but I think it's not the worst thing in the world.

Having only rudimentarily scanned the solution, has any consideration been
given to possible ordering needs of the extensions?

On Mon, 1 Jun 2015 at 15:43 Ryan Harter notifications@github.com wrote:

After discussion with @eamonnmcmanus https://github.com/eamonnmcmanus,
I think we've come up with a working final architecture. I'm documenting it
here.

In the new architecture the AutoValue generated class will be the only
direct subclass of the @autovalue class Foo class, while each extension
will have the opportunity to generate a subclass of that. The classes will
have $ prepended to their name accordingly.

@autovalue class Foo {
public abstract String bar();
}

@generated(AutoValue.class)
class $$AutoValue_Foo extends Foo {
public String bar() {...}
@OverRide public String toString() {...}
}

@generated(CustomToStringExtension.class)
class $AutoValue_Foo extends $$AutoValue_Foo {
@OverRide public String toString() {...}
}

@generated(ParcelableExtension.class)class AutoValue_Foo extends $AutoValue_Foo {
// parcelable bits...
}

The special case here is that for any given @autovalue class, there can
only be one extends that requires itself be at the end of this chain. In
the case of Android Parcelable, a static factory property is required on
the implementing class, so it must remain at the end of the chain.

A couple of questions I have for this architecture:

• How to handle the Builder. Normally, a user can create a static
  builder() method that will return new AutoValue_Foo.Builder(), but we
  can no longer guarantee that the builder is available at that point in the
  chain, since it might be $$AutoValue.Builder().

—
Reply to this email directly or view it on GitHub
#202 (comment).