-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-13098][SQL] remove GenericInternalRowWithSchema #10992
Conversation
Test build #50418 has finished for PR 10992 at commit
|
retest this please |
Test build #50423 has finished for PR 10992 at commit
|
retest this please |
Test build #50428 has finished for PR 10992 at commit
|
retest this please. |
Test build #50434 has finished for PR 10992 at commit
|
cc @davies |
The difference between Row and InternalRow is that the objects inside InternalRow should be internal types. For this case, we pickle the objects in internal types, so it should be InternalRow (it's more clear even with some duplicated codes). |
in |
@cloud-fan I see, that make sense. LGTM, merging this into master, thanks! |
@cloud-fan btw, it's not external type either, for example, it's Int for DateType, not java.sql.Date. |
This class is only used for serialization of Python DataFrame. However, we don't require internal row there, so
GenericRowWithSchema
can also do the job.