Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Android projects codegeneration support #128

Open
ancientloregames opened this issue Dec 7, 2017 · 18 comments
Open

Android projects codegeneration support #128

ancientloregames opened this issue Dec 7, 2017 · 18 comments

Comments

@ancientloregames
Copy link

My team was impressed by the performance of your API in various benchmarks so we tried to replace Gson in our project with it, however we've faced some issues integrating it. At first, I tried to use Dynamic Codegen on a simple text project. The model is plain as desk:

public class Model
{
	public int int1;
	public int int2;
}

But I get exception:

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.bitrix.a01_test_jsoniter/com.bitrix.a01_test_jsoniter.MainActivity}: com.jsoniter.spi.JsonException: failed to generate decoder for: com.jsoniter.spi.ClassInfo@c4e768c with [], exception: javassist.NotFoundException: com.jsoniter.spi.Decoder
public static java.lang.Object decode_(com.jsoniter.JsonIterator iter) throws java.io.IOException { java.lang.Object existingObj = com.jsoniter.CodegenAccess.resetExistingObject(iter);
..........
    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2416)
    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2476)
    at android.app.ActivityThread.-wrap11(ActivityThread.java)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1344)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:148)
    at android.app.ActivityThread.main(ActivityThread.java:5417)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
 Caused by: com.jsoniter.spi.JsonException: failed to generate decoder for: com.jsoniter.spi.ClassInfo@c4e768c with [], exception: javassist.NotFoundException: com.jsoniter.spi.Decoder
public static java.lang.Object decode_(com.jsoniter.JsonIterator iter) throws java.io.IOException { java.lang.Object existingObj = com.jsoniter.CodegenAccess.resetExistingObject(iter);
..........

I take it, the reason is some compatibility issue with Javassist. So I tried to implement Static Codegen Wrapper.

public class ModelCodegenConfig implements StaticCodegenConfig
{
	@Override
	public void setup()
	{
		JsoniterSpi.registerPropertyDecoder(Model.class, "int1", new Decoder.IntDecoder()
		{
			@Override
			public int decodeInt(JsonIterator iter) throws IOException
			{
				return Integer.valueOf(iter.readString());
			}
		});
		JsoniterSpi.registerPropertyDecoder(Model.class, "int2", new Decoder.IntDecoder()
		{
			@Override
			public int decodeInt(JsonIterator iter) throws IOException
			{
				return Integer.valueOf(iter.readString());
			}
		});
	}
	@Override
	public TypeLiteral[] whatToCodegen()
	{
		return new TypeLiteral[] {
				TypeLiteral.create(int.class)
		};
	}
	public static void main(String[] args) throws Exception
	{
		StaticCodegen.main(new String[] { ModelCodegenConfig.class.getCanonicalName() });
	}
}

But I continue facing crashes with the following log:

 java.lang.RuntimeException: Unable to start activity ComponentInfo{com.bitrix.a01_test_jsoniter/com.bitrix.a01_test_jsoniter.MainActivity}: com.jsoniter.spi.JsonException: static gen should provide the decoder we need, but failed to create the decoder
  at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2416)
  at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2476)
  at android.app.ActivityThread.-wrap11(ActivityThread.java)
  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1344)
  at android.os.Handler.dispatchMessage(Handler.java:102)
  at android.os.Looper.loop(Looper.java:148)
  at android.app.ActivityThread.main(ActivityThread.java:5417)
  at java.lang.reflect.Method.invoke(Native Method)
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726)
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616)
Caused by: com.jsoniter.spi.JsonException: static gen should provide the decoder we need, but failed to create the decoder
  at com.jsoniter.Codegen.gen(Codegen.java:64)
  at com.jsoniter.Codegen.getDecoder(Codegen.java:25)
  at com.jsoniter.JsonIterator.read(JsonIterator.java:369)
  at com.jsoniter.JsonIterator.read(JsonIterator.java:359)
  at com.jsoniter.JsonIterator.deserialize(JsonIterator.java:425)
  at com.jsoniter.JsonIterator.deserialize(JsonIterator.java:395)
  at com.bitrix.a01_test_jsoniter.MainActivity.onCreate(MainActivity.java:29)
  at android.app.Activity.performCreate(Activity.java:6251)
  at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1107)
  at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2369)
  at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2476) 
  at android.app.ActivityThread.-wrap11(ActivityThread.java) 
  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1344) 
  at android.os.Handler.dispatchMessage(Handler.java:102) 
  at android.os.Looper.loop(Looper.java:148) 
  at android.app.ActivityThread.main(ActivityThread.java:5417) 
  at java.lang.reflect.Method.invoke(Native Method) 
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726) 
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616) 
Caused by: java.lang.ClassNotFoundException: jsoniter_codegen.cfg4.decoder.com.bitrix.a01_test_jsoniter.Model
  at java.lang.Class.classForName(Native Method)
  at java.lang.Class.forName(Class.java:324)
  at java.lang.Class.forName(Class.java:285)
  at com.jsoniter.Codegen.gen(Codegen.java:60)
  at com.jsoniter.Codegen.getDecoder(Codegen.java:25) 
  at com.jsoniter.JsonIterator.read(JsonIterator.java:369) 
  at com.jsoniter.JsonIterator.read(JsonIterator.java:359) 
  at com.jsoniter.JsonIterator.deserialize(JsonIterator.java:425) 
  at com.jsoniter.JsonIterator.deserialize(JsonIterator.java:395) 
  at com.bitrix.a01_test_jsoniter.MainActivity.onCreate(MainActivity.java:29) 
  at android.app.Activity.performCreate(Activity.java:6251) 
  at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1107) 
  at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2369) 
  at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2476) 
  at android.app.ActivityThread.-wrap11(ActivityThread.java) 
  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1344) 
  at android.os.Handler.dispatchMessage(Handler.java:102) 
  at android.os.Looper.loop(Looper.java:148) 
  at android.app.ActivityThread.main(ActivityThread.java:5417) 
  at java.lang.reflect.Method.invoke(Native Method) 
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726) 
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616) 
Caused by: java.lang.ClassNotFoundException: Didn't find class "jsoniter_codegen.cfg4.decoder.com.bitrix.a01_test_jsoniter.Model" on path: DexPathList[[zip file "/data/app/com.bitrix.a01_test_jsoniter-1/base.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_dependencies_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_0_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_1_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_2_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_3_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_4_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_5_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_6_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_7_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_8_apk.apk", zip file "/data/app/com.bitrix.a01_test_jsoniter-1/split_lib_slice_9_apk.apk"],nativeLibraryDirectories=[/data/app/com.bitrix.a01_test_jsoniter-1/lib/arm, /vendor/lib, /system/lib]]
  at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:56)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:511)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:469)
  at java.lang.Class.classForName(Native Method) 
  at java.lang.Class.forName(Class.java:324) 
  at java.lang.Class.forName(Class.java:285) 
  at com.jsoniter.Codegen.gen(Codegen.java:60) 
  at com.jsoniter.Codegen.getDecoder(Codegen.java:25) 
  at com.jsoniter.JsonIterator.read(JsonIterator.java:369) 
  at com.jsoniter.JsonIterator.read(JsonIterator.java:359) 
  at com.jsoniter.JsonIterator.deserialize(JsonIterator.java:425) 
  at com.jsoniter.JsonIterator.deserialize(JsonIterator.java:395) 
  at com.bitrix.a01_test_jsoniter.MainActivity.onCreate(MainActivity.java:29) 
  at android.app.Activity.performCreate(Activity.java:6251) 
  at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1107) 
  at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2369) 
  at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2476) 
  at android.app.ActivityThread.-wrap11(ActivityThread.java) 
  at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1344) 
  at android.os.Handler.dispatchMessage(Handler.java:102) 
  at android.os.Looper.loop(Looper.java:148) 
  at android.app.ActivityThread.main(ActivityThread.java:5417) 
  at java.lang.reflect.Method.invoke(Native Method) 
  at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:726) 
  at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:616) 
Suppressed: java.lang.ClassNotFoundException: jsoniter_codegen.cfg4.decoder.com.bitrix.a01_test_jsoniter.Model
  at java.lang.Class.classForName(Native Method)
  at java.lang.BootClassLoader.findClass(ClassLoader.java:781)
  at java.lang.BootClassLoader.loadClass(ClassLoader.java:841)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:504)
  		... 23 more
Caused by: java.lang.NoClassDefFoundError: Class not found using the boot class loader; no stack trace available

Am I doing something wrong, or your project is simply not compatible with Android? Will it be in near future?
@taowen
Copy link
Contributor

taowen commented Dec 8, 2017 via email

@taowen
Copy link
Contributor

taowen commented Dec 9, 2017

http://jsoniter.com/java-features.html#reflection mode is still faster than gson, you can start from there. I will provide a full example on Android + static code generation once I am back home. Currently on the road, do not have time to work on this.

@ancientloregames
Copy link
Author

Thanks for your response! I've tried both codegen methods, and haven't tried reflection. So, is dynamic codegen confirmed not to be an option for Android?

@taowen
Copy link
Contributor

taowen commented Dec 11, 2017 via email

@ancientloregames
Copy link
Author

I've managed to replace Gson with your library (Reflective mode). It does work faster (up to three times)! Nicely done! Can't wait for you explanation of how to work with the static codegen properly.

@taowen
Copy link
Contributor

taowen commented Dec 12, 2017

please refer to https://github.com/json-iterator/java/blob/master/android-demo/build.gradle

this is the generated code: https://github.com/json-iterator/java/tree/master/android-demo/src/main/java/jsoniter_codegen/cfg1173796797

@taowen taowen closed this as completed Dec 12, 2017
@ancientloregames
Copy link
Author

Hi. I've tried to build your sample project for android, but keep getting following exception in the Gradle Console during the assemly:

Exception in thread "main" java.lang.ClassNotFoundException: com.example.myapplication.DemoCodegenConfig
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at com.jsoniter.static_codegen.StaticCodegen.main(StaticCodegen.java:31)

 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':generateSources'.
> Process 'command '/home/colfair/android-studio/jre/bin/java'' finished with non-zero exit value 1

@taowen
Copy link
Contributor

taowen commented Dec 13, 2017

try comment out

afterEvaluate {
    android.applicationVariants.all { variant ->
        variant.javaCompiler.dependsOn(generateSources)
    }
}

and build, then build again. It referenced the classes output, which is a circular dependency.

@taowen
Copy link
Contributor

taowen commented Dec 13, 2017

see the latest https://github.com/json-iterator/java/blob/master/android-demo/build.gradle

it should be fixed now

@ancientloregames
Copy link
Author

ancientloregames commented Dec 14, 2017 •
edited

Thanks for your fast replies! Your fix works fine in the sample project. However, I have some complications in implementing it into my team's project. The dependency on your library itself must be in our library module, witch is, in its turn, to be included to the application module, so I tried to put the task execution into the application module, but an exception, similar to the previous one, occurs:
Execution failed for task ':jsoniterStaticCodgen'.
> Process 'command '/home/colfair/android-studio/jre/bin/java'' finished with non-zero exit value 1
Without any further explanation.
Tried to move task to the library module and change the collection of build variants to the libraryVariants. Same exception.

@nedtwigg
Copy link

If you'd like another approach, you could put these files into your buildSrc, and then this into your build.gradle.

@taowen
Copy link
Contributor

taowen commented Dec 14, 2017

@nedtwigg thanks! if want to merge your plugin into jsoniter, just give me a pull request.

@taowen taowen reopened this Dec 14, 2017
@nedtwigg
Copy link

nedtwigg commented Dec 14, 2017 •
edited

I'd love to donate the plugin! The hard part is integrating it into your build / release process, and putting it on the Gradle Plugin Portal, which is hard as third party. The plugin itself is just a few lines, feel free to just nab it if you're up for taking over deployment. I'd be happy to help answer questions & maintain it.

@taowen
Copy link
Contributor

taowen commented Dec 14, 2017

I am unfamiliar with gradle build system. Would like to keep this for community support :)

@nedtwigg
Copy link

Ha, and I know nothing about maven ;-) Looks like there's a gradle-maven-plugin for calling gradle from maven. If you make a folder with that plugin inside that supports publish, test, I'd be happy to stick the appropriate gradle build into it :)

@taowen
Copy link
Contributor

taowen commented Dec 14, 2017

what is the process to put things into Gradle Plugin Portal?

@nedtwigg
Copy link

You create an account here: https://login.gradle.org/user/register

If you publish interactively (at the console), you login via the console each time you publish. Or you can get an API key and do headless publish in CI. You shouldn't have to make changes to the plugin very often (it doesn't care what version of json-iterator the user is using), so manual publish would be fine.

Then you will be in these search results: https://plugins.gradle.org/search?term=json

@ancientloregames
Copy link
Author

ancientloregames commented Mar 5, 2018 •
edited

Hi! Sorry for no participating in resolving the issue for so long time. Was busy with some other stuff, so I sticked with reflective approach for a while. Finally, I've got some time. I figured out that the reason your build task not working in my project is the gradle version I use (4+). It uses api and implementation instead of compile for dependencies, so Compile Configuration will not provide the local JVM with the needed classpath. Unfortunately api and implementation Configurations are unresolvable.
I've found another approach, which involves setting the classpath on afterEvaluate as followed:

afterEvaluate {
    android.libraryVariants.each {
        Task task = jsoniterCodegen
        task.classpath += it.compileConfiguration
        it.javaCompiler.dependsOn task
    }
}

All looked fine on the simple test project, but on the actual one, I've faced new problem. This approach leads to an exception, if the project depends on another local project:

Error:Could not determine the dependencies of task ':jsoniter_lib:jsoniterCodegen'.
> Could not resolve all task dependencies for configuration ':jsoniter_lib:releaseCompileClasspath'.
   > More than one variant of project :sublib matches the consumer attributes:
       - Configuration ':sublib:releaseApiElements' variant android-aidl:
           - Found artifactType 'android-aidl' but wasn't required.
           - Required com.android.build.api.attributes.BuildTypeAttr 'release' and found compatible value 'release'.
           - Required com.android.build.gradle.internal.dependency.AndroidTypeAttr 'Aar' and found compatible value 'Aar'.
           - Found com.android.build.gradle.internal.dependency.VariantAttr 'release' but wasn't required.
           - Required org.gradle.api.attributes.Usage 'java-api' and found compatible value 'java-api'.

As the last resort, I managed to create custom Configuration and provide it with the required dependencies.

afterEvaluate {
    Configuration jsonConf = configurations.create("jsonConf")
    jsonConf.defaultDependencies {
        it.add(owner.project.dependencies.create("com.jsoniter:jsoniter:0.9.21"))
    }
    android.libraryVariants.each {
        Task task = jsoniterCodgen
        task.classpath += jsonConf
        it.javaCompiler.dependsOn task
    }
}

But this code looks really nasty to my taste. Any thoughts on how to improve it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@taowen @nedtwigg @ancientloregames