Skip to content

Spark 3.2: RewriteDataFiles - Escape special characters in table identifiers#5112

Merged
danielcweeks merged 4 commits intoapache:masterfrom
bijanhoule:bug/rewrite-data-files-table-identifier-escaping
Jun 23, 2022
Merged

Spark 3.2: RewriteDataFiles - Escape special characters in table identifiers#5112
danielcweeks merged 4 commits intoapache:masterfrom
bijanhoule:bug/rewrite-data-files-table-identifier-escaping

Conversation

@bijanhoule
Copy link
Contributor

Allows e.g. db.special-chars.table.special-chars when calling the RewriteDataFiles action or procedure, e.g.:

CALL system.rewrite_data_files(table => '`db.with-special`.`table.with-special`')

Currently this causes an exception as the full identifier (<catalog>.<database>.<table>) is stringified without any quoting.

IllegalArgumentException: Cannot parse path or identifier: dev.db.with-special.table.with-special

Allows e.g. `db.special-chars`.`table.special-chars`
@danielcweeks danielcweeks merged commit 6ef656b into apache:master Jun 23, 2022
namrathamyske pushed a commit to namrathamyske/iceberg that referenced this pull request Jul 10, 2022
…tifiers (apache#5112)

* Spark 3.2: Escape table identifier in RewriteDataFiles Procedure/Action

Allows e.g. `db.special-chars`.`table.special-chars`

* mark original method as deprecated

* @deprecated tag in javadoc as well

* keep existing interface
namrathamyske pushed a commit to namrathamyske/iceberg that referenced this pull request Jul 10, 2022
…tifiers (apache#5112)

* Spark 3.2: Escape table identifier in RewriteDataFiles Procedure/Action

Allows e.g. `db.special-chars`.`table.special-chars`

* mark original method as deprecated

* @deprecated tag in javadoc as well

* keep existing interface
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments