Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Java] Create sequential serializer failed #1176

Open
qinzhikui opened this issue Nov 30, 2023 · 6 comments
Open

[Java] Create sequential serializer failed #1176

qinzhikui opened this issue Nov 30, 2023 · 6 comments
Labels
bug Something isn't working java

Comments

@qinzhikui
Copy link

qinzhikui commented Nov 30, 2023

Describe the bug

java.lang.RuntimeException: Create sequential serializer failed
at io.fury.serializer.CodegenSerializer.loadCodegenSerializer(CodegenSerializer.java:48)
	at io.fury.resolver.ClassResolver.lambda$getObjectSerializerClass$2(ClassResolver.java:949)
	at io.fury.builder.JITContext.registerSerializerJITCallback(JITContext.java:132)
	at io.fury.resolver.ClassResolver.getObjectSerializerClass(ClassResolver.java:947)
	at io.fury.resolver.ClassResolver.getSerializerClass(ClassResolver.java:902)
	at io.fury.resolver.ClassResolver.getSerializerClass(ClassResolver.java:805)
	at io.fury.resolver.ClassResolver.createSerializer(ClassResolver.java:1161)
	at io.fury.resolver.ClassResolver.getOrUpdateClassInfo(ClassResolver.java:1101)
...
Caused by: java.lang.IllegalStateException: Impossible because we just compiled class
	at io.fury.builder.CodecUtils.loadOrGenCodecClass(CodecUtils.java:95)
	at io.fury.builder.CodecUtils.loadOrGenObjectCodecClass(CodecUtils.java:40)
	at io.fury.serializer.CodegenSerializer.loadCodegenSerializer(CodegenSerializer.java:45)
	... 65 more
Caused by: java.lang.ClassNotFoundException: 
	at io.fury.util.ClassLoaderUtils$ByteArrayClassLoader.findClass(ClassLoaderUtils.java:209)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:589)
	at io.fury.util.ClassLoaderUtils$ByteArrayClassLoader.loadClass(ClassLoaderUtils.java:225)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	at io.fury.builder.CodecUtils.loadOrGenCodecClass(CodecUtils.java:93)
	... 67 more

Expected behavior
A clear and concise description of what you expected to happen.

To Reproduce
Steps to reproduce the behavior.
Please try to provide a reproducible script.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):
Please specify the versions of Fury, OS, and others such as JDK/Python/Gcc/Clang/Go/NodeJS if they are used.

@qinzhikui qinzhikui added the bug Something isn't working label Nov 30, 2023
@chaokunyang
Copy link
Collaborator

chaokunyang commented Nov 30, 2023

@qinzhikui thanks for creating this issue, could you provide the reproduction code and fury version?

@qinzhikui
Copy link
Author

@qinzhikui thanks for creating this issue, could you provide the reproduction code and fury version?

Fury version 0.3.0.
I cannot reproduce it locally.
I use ThreadLocalFury in production. Most threads are fine, only a few threads will have this problem.
I find this log 'Compile [class] take xxx ms'
I found that one thread will print 'Compile [class] take xxx ms', but other threads will generate the above exception

`private ThreadSafeFury fury;

public FuryOHCacheSerializer(Class clazz) {
    fury = new ThreadLocalFury(classLoader -> {
        Fury f = Fury
                .builder()
                .withLanguage(Language.JAVA)
                .withClassLoader(classLoader)
                .build();
        f.register(clazz);
        return f;
    });
}`

@chaokunyang chaokunyang changed the title Create sequential serializer failed [Java] Create sequential serializer failed Nov 30, 2023
@chaokunyang
Copy link
Collaborator

Seems this is a multi-thread related issue, but it's not easy to figure out what happend without the reproduction code. I checked the implementation, this exception should not happen.

@qinzhikui
Copy link
Author

close it first and see if anyone else is encountering this problem

@chaokunyang
Copy link
Collaborator

I'll try to reproduce it, and give you feedbacks if I make it

@andyczerwonka
Copy link

Looks like we're seeing the same issue. #1325

chaokunyang added a commit that referenced this issue May 6, 2024
## What does this PR do?
Some collectionSerializer may overwrite write/read method, then clear
element serializer may not got invoked.

This PR clears serializer for collection/map to avoid container use
wrong serializer for nested elements.

## Related issues

#1558
#1455,
#1325 and
#1176.

## Does this PR introduce any user-facing change?

<!--
If any user-facing interface changes, please [open an
issue](https://github.com/apache/incubator-fury/issues/new/choose)
describing the need to do so and update the document if necessary.
-->

- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?


## Benchmark

<!--
When the PR has an impact on performance (if you don't know whether the
PR will have an impact on performance, you can submit the PR first, and
if it will have impact on performance, the code reviewer will explain
it), be sure to attach a benchmark data here.
-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working java
Projects
None yet
Development

No branches or pull requests

3 participants