Skip to content

Ephemeral storage is respected from the overlod for peon tasks#14201

Merged
gianm merged 1 commit intoapache:masterfrom
churromorales:ephemeral-storage-upstream
May 5, 2023
Merged

Ephemeral storage is respected from the overlod for peon tasks#14201
gianm merged 1 commit intoapache:masterfrom
churromorales:ephemeral-storage-upstream

Conversation

@churromorales
Copy link
Contributor

I noticed during testing that the ephemeral storage resource was not inherited from the overlord. We should only be overriding the specific "cpu" and "memory" resources, currently we are calculating cpu/memory and only setting those. We should inherit all the other resources and only override cpu and memory

@gianm
Copy link
Contributor

gianm commented May 3, 2023

I am not a k8s expert so this is perhaps a basic question, but: is the effect of this that task pods will end up with the same ephemeral storage requirement as the Overlord pod that launched them? If so I don't understand why this is necessarily good -- wouldn't tasks generally need different amount of storage than the Overlord?

@churromorales
Copy link
Contributor Author

@gianm this is a good point, i want to inherit properly from the overlord, then have another PR to override. But first I should be doing the right thing, unless you think both can go in this PR. If you don't inherit, you use the default value, which is immutable....you can't change it.

@gianm
Copy link
Contributor

gianm commented May 5, 2023

Got it, that makes sense. It's ok to do the next step as a follow-up IMO, so I'll merge this one.

@gianm gianm merged commit 123c490 into apache:master May 5, 2023
@abhishekagarwal87 abhishekagarwal87 added this to the 27.0 milestone Jul 19, 2023
churromorales added a commit to churromorales/druid that referenced this pull request Sep 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants