HIVE-27916: Increase tez.am.resource.memory.mb for TestIcebergCliDrver to 512MB #4907
Conversation
| <property> | ||
| <name>tez.am.resource.memory.mb</name> | ||
| <value>256</value> | ||
| <value>512</value> |
There was a problem hiding this comment.
not sure if that is required, as there were no issues upstream so far, but it won't harm
There was a problem hiding this comment.
btw, should we drop hive.tez.container.size from tez and llap tez-site.xml? the same thing is set in hive-site.xml but with a lower limit - 128 vs 512
There was a problem hiding this comment.
hive.tez.container.size is for task containers in particular (it's used to create the Resource object which is provided to yarn), whereas tez.am.resource.memory.mb is for the Application Master, so basically, it would make sense to review it in (only) case of llap drivers' config, as it's confusing: LLAP is not Tez container mode
regarding duplicated stuff in hive-site.xml and tez-site.xml...yeah, it's confusing too, let me check
regarding upstream failures: yeah, I cannot recall precommit issue in particular, however there were some in downstream, so as you said: it won't harm
|
Kudos, SonarCloud Quality Gate passed!
|
|
closing this, please find comment in jira: https://issues.apache.org/jira/browse/HIVE-27916?focusedCommentId=17796287&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17796287 |
|
other confusing stuff regarding hive.tez.container.size will be handled in HIVE-27937 |
|
looks like we need this change: downstream I was able to reproduce AM OOM flakiness which disappeared with this fix |
abstractdog
force-pushed
the
HIVE-27916
branch
from
d66f0d0 to
212093e
Compare
|
abstractdog
changed the title
apache#4907) (Laszlo Bodor reviewed by Ayush Saxena)
apache#4907) (Laszlo Bodor reviewed by Ayush Saxena)
5 participants
What changes were proposed in this pull request?
Increasing tez.am.resource.memory.mb for tez based iceberg cli driver.
Why are the changes needed?
To improve test stability.
Does this PR introduce any user-facing change?
No.
Is the change a dependency upgrade?
No.
How was this patch tested?
Precommit.