New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for forcing Moshi to construct instances using a specified constructor #266
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very non-comprehensive pass.
Thanks for the PR and description!
This is very interesting.
Different default values for String, interfaces, and Enums is rather surprising behavior.
Definitely needs tests, but I'm hesitant to review/ask for changes because I'm not sure this is something Moshi will support in this way.
To clarify, the goal here is to support Kotlin's lazy
, right?
args[j] = Proxy.newProxyInstance(getClass().getClassLoader(), new Class[]{parameterType}, new InvocationHandler() { | ||
@Override | ||
public Object invoke(Object proxy, Method method, Object[] args) throws Throwable { | ||
throw new IllegalAccessException("Trying to call a method on a " + |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
UnsupportedOperationException, perhaps?
Needs tests, too.
@Override public String toString() { | ||
|
||
@Override | ||
public String toString() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Revert style change.
return (ClassFactory<T>) unsafeFactories.get(rawType); | ||
} | ||
|
||
// Try to find a constructor with the JsonConstructor annotation |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: full stop period in comments.
* This can't be a {@link LinkedHashTreeMap} since {@link Class} | ||
* does not implement {@link Comparable}. | ||
*/ | ||
private static Map<Class<?>, ClassFactory<?>> unsafeFactories = new HashMap<>(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LinkedHashMap
.
This should probably be cached somewhere more appropriate, like the Moshi instance.
But, even moreover, I don't think it actually needs to be cached. The Factory is supposed to do work to create the Adapter; the created Adapter is still fast. This is also an independent change. Revert?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The adapter may not always be fast without caching due to the recursive fallback call to ClassFactory.get(parameterType).newInstance()
. It's your call on this though, I don't really care too much about the caching since many real-world entities consist of primitives and String
s only anyway.
T newInstance() throws InvocationTargetException, IllegalAccessException, InstantiationException { | ||
Class<?>[] parameterTypes = constructor.getParameterTypes(); | ||
Object[] args = new Object[parameterTypes.length]; | ||
for (int j = 0; j < parameterTypes.length; j++) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: conventional i
instead of j
?
@Override public String toString() { | ||
|
||
@Override | ||
public String toString() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Revert style changes.
import java.lang.reflect.Field; | ||
import java.lang.reflect.InvocationTargetException; | ||
import java.lang.reflect.Method; | ||
import java.lang.reflect.*; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Qualify imports.
Thanks for your quick feedback!
With proper documentation, I don't see that as an issue. It's implied by the fact that only non-null values will be passed and of course String ∩ All Interfaces ∩ All Enums =
I've already added a simple test, but if you want more tests, sure, I can do that. If you're very concerned about integrating this into the library itself, there'd an alternative: The
That and other property delegates. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing test cases with multiple constructors where one and both are annotated.
} | ||
|
||
// Try to find a constructor with the JsonConstructor annotation | ||
for (int i = 0; i < rawType.getDeclaredConstructors().length; i++) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cache this call outside the loop or use for-each
} else if (parameterType == boolean.class || parameterType == Boolean.class) { | ||
args[j] = false; | ||
} else if (parameterType == String.class) { | ||
args[j] = ""; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should be null
} | ||
}); | ||
} else if (parameterType.isEnum() && parameterType.getEnumConstants().length > 0) { | ||
args[j] = parameterType.getEnumConstants()[0]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is this coming from? Reference types default to null
Class<?> parameterType = parameterTypes[j]; | ||
// Default values according to | ||
// https://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html | ||
if (parameterType == byte.class || parameterType == Byte.class) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Boxed primitives should all default to null, not the primitive default
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From the PR description:
The reason why parameters of reference types aren't simply set to null is that constructors may check for parameters to be non-null. Kotlin does this by default.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's tons of invariants constructors check. Maybe only one of two properties can be non-null or 0 is not in the valid range. This violates least-surprise and makes the objects serialize differently than what was actually deserialized. We cannot do this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. Perhaps my suggestion to make the ClassFactory
pluggable may be a better and more flexible option?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alternatively: Change this PR to use null
for reference types, but still make the ClassFactory
pluggable. The best of both worlds?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One final alternative: Null by default for reference types and non-null by setting a parameter of the JsonConstructor annotation to True.
|
||
private static <T> ClassFactory<T> createAnnotationClassFactory(final Class<?> rawType, final Constructor<?> constructor) { | ||
return new ClassFactory<T>() { | ||
@SuppressWarnings("unchecked") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit: too much indent
abstract T newInstance() throws | ||
InvocationTargetException, IllegalAccessException, InstantiationException; | ||
|
||
public static <T> ClassFactory<T> get(final Class<?> rawType) { | ||
if (unsafeFactories.containsKey(rawType)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just call get and check against null so you don't do hash and lookup steps twice.
|
||
@Test | ||
public void testJsonConstructorAnnotation() { | ||
try { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Delete try/catches. An exception always fails the test case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’m not sure we want this.
How’d you feel about adding a no-args constructor that calls this()
with default values? That’s so simple.
} | ||
} | ||
if (annotationFactory != null) { | ||
unsafeFactories.put(rawType, annotationFactory); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ick, this is particularly unsafe . . . static value without synchronization
} catch (Exception ignored) { | ||
} | ||
|
||
throw new IllegalArgumentException("cannot construct instances of " + rawType.getName()); | ||
} | ||
|
||
private static <T> ClassFactory<T> createAnnotationClassFactory(final Class<?> rawType, final Constructor<?> constructor) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I’m not very comfortable with this.
With many entities that have lots of properties, this would take more effort and in addition, I don't like the idea since it would lead to a lot of repetition. I understand you're concerned about the non-standard values passed to the annotated constructor and I've already suggested a solution:
I'd like avoid forking Moshi permanently to be able to use it with Kotlin, hence it would be great if we could compromise on one of the three alternatives. Personally, I think (1.) would be a safe and the best option to let users of the library customize the way instances are created. Currently it will fallback to |
I'm trying to limit how clever Moshi is. What about adding default parameters to your constructor? Would that make a no-args constructor less verbose? |
That's not a good option if I need actual default values for some parameters. Adding default values to other parameters just to generate a no-args constructor would make it really hard to distinguish between real default values and ones that are only there to generate a no-args constructor. Would option (1.) really make Moshi more clever? The code changes would be minimal and all of the logic of this PR would be moved to the user-side. And if the user doesn't register any |
Why wouldn't your actual default parameters be sufficient? Like, those are the parameters it can use, it doesn't need to be this |
@swankjesse Consider the following example:
I understand you wouldn't want the latter behavior in the library, but I hope I've made clear why letting users provide What potential harm do you see in letting users do that? |
I too feel like the defaults in the constructor should be sufficient. If
you can't rely on those then it sounds like you have a modeling problem
between your network and application domain and perhaps shouldn't be
sharing the same object between them.
…On Tue, Mar 7, 2017, 9:57 AM Christian Brüggemann ***@***.***> wrote:
@swankjesse <https://github.com/swankjesse> Consider the following
example:
data class Purchase(val amount: Int = 1, val price: Int = -1, val
customer: Customer = ..., val item: Item = ...)
- The default value for amount may make sense, because in our scenario
customers don't purchase an item more than once usually .
- The default value for price makes no sense and is only there to
generate a no-args constructor. I don't think it's immediately obvious to a
reader though, one may think that price == -1 simply means that there
is no price, the item was free or that it's a special value for something.
If I was new to a project, this would leave me confused.
- Default values for customer and item are hard. There are no sensible
defaults and constructing a *default* customer for example perhaps
isn't even possible or very cumbersome since it could be a framework class
or anything else the developer of Purchase has no control over. With
this PR, the ClassFactory would just go ahead and unsafely instantiate
a Customer, eventually replacing it with a valid value from the JSON
after instantiation of course.
I understand you wouldn't want the latter behavior in the library, but I
hope I've made clear why letting users provide ClassFactory instances
would be very useful.
What potential harm do you see in letting users do that?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#266 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEEEfJncn6BmaYYvwhQmT8fjMx22rRGks5rjXBUgaJpZM4MUqAZ>
.
|
So I should duplicate my entity classes and give the duplicate class a constructor with nonsensical defaults, probably marked as In my opinion, it's a huge issue that many serialization frameworks don't allow users to specificy how an object should be created and instead rely on |
By the way, Gson supports this very feature, but I prefer Moshi as it's based on Okio, prevents me from shooting myself in the foot by simply serializing everything I throw at it, including classes without a registered type adapter and because of other features.. |
I still don’t understand why a no-arguments constructor is bad. That way the “nonsensical defaults” live in your code rather than in Moshi. |
It's bad because it can be called from regular code and because it's boilerplate. Everytime a property is added, removed or changed, the no-args constructor that passes default values to the primary constructor needs to be updated to. If I instead specify defaults using the syntax
With a pluggable
Obviously not exactly like this, but you get the idea. In my code, I could then call |
Since I'm going to use this anyway, I've implemented the (small) necessary changes to support user-supplied |
Any news on this? |
Still contemplating options on this. At the moment burying defaults in Moshi feels unreasonable. Probably the right option for data classes is to use Kotlin’s reflection and actually call the constructor rather than assigning fields. I’m not sure if that’s symmetric, but it seems like a stronger design. |
Thanks for responding so quickly!
That definitely sounds like a better approach. Here's how it could be done: Alternative 1Let constructor parameters be annotated with Kotlin example with support for deserialization only: Kotlin example with support for deserialization and serialization: Java example with support for deserialization only:
Java example with support for deserialization and serialization:
Alternative 2Like you proposed, Kotlin reflection could be used to detect annotations like this: Alternative 3 (I think this would be the easiest to use)Call the constructor for deserialization iff it is annotated with Kotlin example with support for deserialization and serialization: Java example with support for deserialization and serialization:
|
Yeah, we’d want to do it as it's own module. It’d depend on kotlin.reflect and expose a JsonAdapter.Factory. |
So alternative 2 it is? That would add 2 MB and ~ 16k methods to apps using that module. Are you sure that'd be preferable over the other alternatives? |
Presumably people using Kotlin are already using Kotlin, so it’s not 2MiB in net new code. |
It is. For most applications, only the dependency on |
The Jackson integration requires kotlin-reflect. Seems sensible to do the
same here.
…On Mon, Mar 20, 2017 at 9:38 AM Christian Brüggemann < ***@***.***> wrote:
Presumably people using Kotlin are already using Kotlin, so it’s not 2MiB
in net new code.
It is. For most applications, only the dependency on kotlin-stdlib is
needed, but not kotlin-reflect. I don't use kotlin-reflect in any of my
applications, some of which rely heavily on Kotlin.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#266 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEEEXPjDM5Jklf9lyPL9Rcd56nIQmg_ks5rnoE2gaJpZM4MUqAZ>
.
|
That's one of the reasons why I avoid Jackson. Any reason why alternatives 1 and 3 are apparently out of question? |
3 doesn't work because the class file format doesn't hold parameters names
unless you target Java 8 and even then they're optional and opt-in (at
least from the Java side). 1 is some nasty duplication.
…On Mon, Mar 20, 2017 at 7:28 AM Christian Brüggemann < ***@***.***> wrote:
Thanks for responding so quickly!
Probably the right option for data classes is to use Kotlin’s reflection
and actually call the constructor rather than assigning fields. I’m not
sure if that’s symmetric, but it seems like a stronger design.
That definitely sounds like a better approach. Here's how it could be done:
Alternative 1
Let constructor parameters be annotated with @JSON. As a plus, this would
also work for Java classes. A drawback would be having to annotate
fields/properties *and* constructor parameters to support serialization
instead of only deserialization.
Kotlin example with support for deserialization only:
data class ***@***.***:Json(name = "bar") bar: String)
Kotlin example with support for deserialization and serialization:
data class ***@***.***:Json(name = "bar") @field:Json(name = "bar") bar:
String)
Java example with support for deserialization only:
class Foo {
String bar;
***@***.***(name = "bar") String bar) {
this.bar = bar;
}
}
Java example with support for deserialization and serialization:
class Foo {
@JSON(name = "bar") String bar;
***@***.***(name = "bar") String bar) {
this.bar = bar;
}
}
Alternative 2
Like you proposed, Kotlin reflection could be used to detect annotations
like this: @Property:Json(name = "bar"). Moshi would then know that the
constructor parameter name equals the property name and could call the
constructor for deserialization and use the field value for serialization.
Discovering these annotations would require a dependency on the heavy
kotlin-reflect library.
Alternative 3 (I think this would be the easiest to use)
Call the constructor for deserialization iff it is annotated with
@JsonConstructor, indicating that the library user would like Moshi to
call it. Require that constructor parameters are named the same as the
fields they are assigned to.
Kotlin example with support for deserialization and serialization:
data class Foo @JsonConstructor ***@***.***(name = "bar") bar:
String)
which due to the currently allowed targets of the @JSON annotation equals
data class Foo @JsonConstructor ***@***.***:Json(name = "bar") bar:
String)
Java example with support for deserialization and serialization:
class Foo {
@JSON(name = "bar") String bar;
@JsonConstructor Foo(String bar) {
// Moshi detects that the constructor parameter bar is named the same as the annotated field bar
this.bar = bar;
}
}
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#266 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAEEESLr1LOZby4jACq0h6LOvRlKbMqoks5rnmLSgaJpZM4MUqAZ>
.
|
I see, thanks. I guess this PR can be closed then. I'll try to implement a module for Kotlin and send you a PR when it's done, unless you already have an exact implementation in mind. |
I’m unsure whether you absolutely need kotlin.reflect. But I do think we win if we make a Kotlin-specific JsonAdapter. It can invoke the constructor, and potentially rely on the constructor to provide default values for fields that aren’t provided. |
This PR implements a method to force Moshi to construct instances of specific types using a constructor annotation. The main use case is Kotlin, but I imagine this could apply to Java code as well. Consider the following example:
Currently, Moshi supports deserializing a JSON into this type by unsafely creating the object, bypassing the constructor. A workaround is the Kotlin no-arg plugin introduced in Kotlin 1.0.6 which generates an empty constructor for annotated classes, but unfortunately both ways are problematic, as they don't initialize the hidden delegate instance for the
lazy
property.This problem is solved by the PR. The following modification is required in the code:
This change will tell Moshi to use the primary constructor to construct the object. The way Moshi generates values for the parameters of the constructor after this PR is pretty straight-forward:
0
forint
andInteger
,0.0
fordouble
andDouble
and so on.""
.values
field.ClassFactory.get(...).newInstance()
.To reduce cost,
ClassFactory
instances are cached. The reason why parameters of reference types aren't simply set tonull
is that constructors may check for parameters to be non-null. Kotlin does this by default.Afterwards, Moshi proceeds as normal by deserializing the JSON values into the appropriate fields of the newly created instance. However, since the real constructor was called, the delegate is initialized correctly.