PHOENIX-5592 MapReduce job to asynchronously delete rows where the VI… #664
Conversation
…EW_TTL has expired
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/ViewTtlTool.java
Outdated
Show resolved
Hide resolved
yanxinyi
force-pushed
the
PHOENIX-5592-view-ttl
branch
from
85c79e3
to
e1a0ca4
Compare
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/ViewTTLDeleteJobMapper.java
Show resolved
Hide resolved
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/ViewTTLDeleteJobMapper.java
Outdated
Show resolved
Hide resolved
| String deleteIfExpiredStatement = "SELECT /*+ NO_INDEX */ count(*) FROM " + value.getViewName(); | ||
| deletingExpiredRows(connection, view, Long.valueOf(value.getViewTtl()), | ||
| deleteIfExpiredStatement, config); | ||
| List<PTable> allIndexesOnView = view.getIndexes(); |
There was a problem hiding this comment.
AFAIK, this also gets indexes on the base table. Is it expected?
There was a problem hiding this comment.
that's expected. We delete expired rows for the base table, as well as the index table.
There was a problem hiding this comment.
@yanxinyi We are only deleting from view indexes right?
There was a problem hiding this comment.
yes. that's correct.
| } | ||
|
|
||
| @Override public float getProgress() throws IOException, InterruptedException { | ||
| return 0; |
| deletingExpiredRows(tenantConnection, value, config); | ||
| } | ||
| } else { | ||
| deletingExpiredRows(connection, value, config); |
There was a problem hiding this comment.
I am assuming these are for global views? And have been validated that they do not have any child views. It would be useful to pass that info (global/tenant view) in the value (ViewInfoTracker) so that additional validation can be done here.
There was a problem hiding this comment.
We already did check before passing to the mapper. In other words, only leaf view can be processed and deleted
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/ViewTtlTool.java
Outdated
Show resolved
Hide resolved
| } | ||
|
|
||
| public String getJobName() { | ||
| if (this.jobName == null) { |
There was a problem hiding this comment.
nit: Can we define a job name format and build that consistently?
There was a problem hiding this comment.
all jobs start with prefix ViewTTLTool-
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/ViewTtlTool.java
Outdated
Show resolved
Hide resolved
...e/src/main/java/org/apache/phoenix/mapreduce/util/PhoenixMultiViewDeletionInputStrategy.java
Outdated
Show resolved
Hide resolved
...e/src/main/java/org/apache/phoenix/mapreduce/util/PhoenixMultiViewDeletionInputStrategy.java
Outdated
Show resolved
Hide resolved
|
close this PR since implementation changes. the new PR is here: #762 |
Co-authored-by: Gokcen Iskender <47044859+gokceni@users.noreply.github.com>
…EW_TTL has expired