Skip to content

Missing function to compare DataFrameSchemas with order #1502

@Jolanrensen

Description

@Jolanrensen

df1.schema() == df2.schema() (or df1.schema().compare(df2.schema()).isEqual()) returns true,
even if their columns are ordered like "a, b, c" and "b, c, a", respectively.

This is useful from the Kotlin perspective, as it's safe to cast df1 to the schema of df2; they have the same columns, accessors and could be converted to the same class.

However, in DataFrame, columns have an order. This can be seen when converted to JSON, rendered to HTML, or viewed as a table.
Also, df1 == df2 returns false if there is a different order, which is very inconsistent with the schema equality check above.

We have the move operation to change the order. However, to test move or for a user to check that two DataFrame's schemas match exactly, there's no easy way to do so at present.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions