[Bugfix] Fix fastjson classloader leaked #260
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this:
Which issues of this PR fixes :
Fixes #
Problem Summary(Required) :
Problem
If a job fails many times, flink can warn there is a classloader leak when fastjson serializes/deserializes data
Caused by: java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.
Reason
Fastjson uses two global/static configs by default to serialize and deserialize data (SerializeConfig#globalInstance and ParserConfig#global). Flink's classloader mechanism will treat them as classloader leak if a JVM instance run a job in multiple times, such as failover or in session mode. See flink classloader.check-leaked-classloader for details.
Solution
Don't use fastjson's global configs. Instead each sink instance creates and uses it's own configs, and do not shared with other instance
Checklist: