Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-43380][SQL][FOLLOWUP] Deprecate toSqlType(avroSchema: Schema, …useStableIdForUnionType: Boolean): SchemaType #43557

Closed
wants to merge 1 commit into from

Conversation

beliefer
Copy link
Contributor

@beliefer beliefer commented Oct 27, 2023

What changes were proposed in this pull request?

#43530 provides a new method:

  /**
   * Converts an Avro schema to a corresponding Spark SQL schema.
   *
   * @since 4.0.0
   */
  def toSqlType(avroSchema: Schema, useStableIdForUnionType: Boolean): SchemaType = {
    toSqlTypeHelper(avroSchema, Set.empty, useStableIdForUnionType)
  }

Because take AvroOptions as parameter causes the performance regression, the old toSqlType looks very useless.

This PR also improve some caller of toSqlType by pass useStableIdForUnionType directly.

Why are the changes needed?

Deprecate toSqlType(avroSchema: Schema, …useStableIdForUnionType: Boolean): SchemaType

Does this PR introduce any user-facing change?

'No'.

How was this patch tested?

Exists test cases.

Was this patch authored or co-authored using generative AI tooling?

'No'.

…useStableIdForUnionType: Boolean): SchemaType
@beliefer
Copy link
Contributor Author

ping @cloud-fan cc @zeruibao

Copy link
Contributor

@zeruibao zeruibao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@beliefer beliefer closed this in dc02991 Oct 30, 2023
@beliefer
Copy link
Contributor Author

Merged to master. @cloud-fan @zeruibao Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants