Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TiStore: add implement Serializable #251

Merged
merged 1 commit into from
Aug 9, 2021

Conversation

liangyouze
Copy link
Contributor

What problem does this PR solve?

When I use it in Tispark, the following exception will be thrown

Caused by: java.io.NotSerializableException: org.tikv.common.region.TiStore
Serialization stack:
	- object not serializable (class: org.tikv.common.region.TiStore, value: org.tikv.common.region.TiStore@1f547af8)
	- writeObject data (class: java.util.ArrayList)
	- object (class java.util.ArrayList, [org.tikv.common.region.TiStore@1f547af8, org.tikv.common.region.TiStore@701c482e, org.tikv.common.region.TiStore@4738131e])
	- field (class: org.tikv.common.region.TiRegion, name: stores, type: interface java.util.List)
	- object (class org.tikv.common.region.TiRegion, {Region[2009] ConfVer[5] Version[8] Store[4] KeyRange[]:[bNmuyJQt]})
	- writeObject data (class: scala.collection.immutable.List$SerializationProxy)
	- object (class scala.collection.immutable.List$SerializationProxy, scala.collection.immutable.List$SerializationProxy@416d90f0)
	- writeReplace data (class: scala.collection.immutable.List$SerializationProxy)
	- object (class scala.collection.immutable.$colon$colon, List({Region[2009] ConfVer[5] Version[8] Store[4] KeyRange[]:[bNmuyJQt]}, {Region[2] ConfVer[5] Version[8] Store[1] KeyRange[bNmuyJQt]:[]}))
	- field (class: org.apache.spark.rawkvbulkload.TiReginSplitPartitionerV2, name: orderedRegions, type: class scala.collection.immutable.List)
	- object (class org.apache.spark.rawkvbulkload.TiReginSplitPartitionerV2, org.apache.spark.rawkvbulkload.TiReginSplitPartitionerV2@3c98981e)
	- field (class: org.apache.spark.rawkvbulkload.RawKVBulkLoad, name: partitioner, type: class org.apache.spark.rawkvbulkload.TiReginSplitPartitionerV2)
	- object (class org.apache.spark.rawkvbulkload.RawKVBulkLoad, org.apache.spark.rawkvbulkload.RawKVBulkLoad@6a38e3d1)
	- element of array (index: 0)
	- array (class [Ljava.lang.Object;, size 1)
	- field (class: java.lang.invoke.SerializedLambda, name: capturedArgs, type: class [Ljava.lang.Object;)
	- object (class java.lang.invoke.SerializedLambda, SerializedLambda[capturingClass=class org.apache.spark.rawkvbulkload.RawKVBulkLoad, functionalInterfaceMethod=scala/Function1.apply:(Ljava/lang/Object;)Ljava/lang/Object;, implementation=invokeStatic org/apache/spark/rawkvbulkload/RawKVBulkLoad.$anonfun$bulkLoad$5$adapted:(Lorg/apache/spark/rawkvbulkload/RawKVBulkLoad;Lscala/collection/Iterator;)Ljava/lang/Object;, instantiatedMethodType=(Lscala/collection/Iterator;)Ljava/lang/Object;, numCaptured=1])
	- writeReplace data (class: java.lang.invoke.SerializedLambda)
	- object (class org.apache.spark.rawkvbulkload.RawKVBulkLoad$$Lambda$2335/456897159, org.apache.spark.rawkvbulkload.RawKVBulkLoad$$Lambda$2335/456897159@7dd7ec56)
	at org.apache.spark.serializer.SerializationDebugger$.improveException(SerializationDebugger.scala:41)
	at org.apache.spark.serializer.JavaSerializationStream.writeObject(JavaSerializer.scala:47)
	at org.apache.spark.serializer.JavaSerializerInstance.serialize(JavaSerializer.scala:101)
	at org.apache.spark.util.ClosureCleaner$.ensureSerializable(ClosureCleaner.scala:413)
	... 23 more

What is changed and how it works?

The Class TiStore should implement Serializable

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity
  • Breaking backward compatibility

Related changes

  • Need to cherry-pick to the release branch
  • Need to update the documentation
  • Need to be included in the release note

@marsishandsome
Copy link
Collaborator

please add sign-off

@marsishandsome
Copy link
Collaborator

/run-all-tests

Signed-off-by: liangyouze <liangyouze876@gmail.com>
Copy link
Collaborator

@marsishandsome marsishandsome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@marsishandsome
Copy link
Collaborator

/merge

@ti-srebot
Copy link
Collaborator

/run-all-tests

@ti-srebot
Copy link
Collaborator

@liangyouze merge failed.

@marsishandsome
Copy link
Collaborator

/run-all-tests

@marsishandsome marsishandsome merged commit 68e097d into tikv:master Aug 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants