Skip to content

HIVE-28903: Skip deleting archived path when drop partition/table #5769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 6, 2025

Conversation

wecharyu
Copy link
Contributor

@wecharyu wecharyu commented Apr 12, 2025

What changes were proposed in this pull request?

When drop archived partition/table, only the original path of partition should be considered as a deletion candidate

Why are the changes needed?

Hadoop archived path is not supported to be deleted, in current implement the drop_partition/table calls would drop metadata successfully but throw exception when deleting the archived paths.

Does this PR introduce any user-facing change?

No.

Is the change a dependency upgrade?

No.

How was this patch tested?

  • Add a qtest
mvn test -pl itests/qtest -Pitests -Dtest=org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver -Dqfile=archive_drop.q
  • Test locally by sql:
-- create a table
create table tbl (id int) partitioned by (dt date, region string);

SET hive.execution.engine=mr;
insert overwrite table tbl partition (dt='2025-01-01', region='us') select 1;
insert overwrite table tbl partition (dt='2025-01-01', region='sg') select 2;

-- change to managed table
ALTER TABLE tbl SET TBLPROPERTIES ('EXTERNAL'='FALSE');

-- archive partitions
SET hive.archive.enabled=true;
ALTER TABLE tbl ARCHIVE PARTITION (dt='2025-01-01');

-- drop partition
alter table tbl drop partition (dt='2025-01-01', region='us');

-- drop table
drop table tbl;

Copy link
Contributor

@SourabhBadhya SourabhBadhya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please consider adding tests for this.

Copy link
Contributor

@SourabhBadhya SourabhBadhya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1 (pending tests)

Copy link

sonarqubecloud bot commented May 6, 2025

@SourabhBadhya SourabhBadhya merged commit ffefb7d into apache:master May 6, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants