Skip to content
This repository has been archived by the owner on Mar 3, 2023. It is now read-only.

terrible resource is needed when submitting to aurora #3504

Open
dttlgotv opened this issue Apr 2, 2020 · 2 comments
Open

terrible resource is needed when submitting to aurora #3504

dttlgotv opened this issue Apr 2, 2020 · 2 comments

Comments

@dttlgotv
Copy link

dttlgotv commented Apr 2, 2020

I write a simple example to test resource requirement to submit to aurora. It is very terrible that too more resource is needed. Please check below:

my example:
builder.newSource(() -> ThreadLocalRandom.current().nextInt(100))
.setName("random-sentences-source");

      Config config = Config.newBuilder()
                  .setNumContainers(1)
                  .setPerContainerCpu(2)
                  .setPerContainerRamInGigabytes(3)
                  .build();

When I submit this topo to aurora, 4.5 cpu and 6.3G memory are allocated to this topology. It seems that Tmaster used 2.5G cpu and 3.3G memory.

My question is how to limit tmaster resource requirement. It seems that Tmaster task using more resource.

@nwangtw
Copy link
Contributor

nwangtw commented Apr 2, 2020

In aurora, all containers need to have the same size, and container 0 is mainly for tmaster. Therefore the key is to keep the container size under control. In your example, your cpu and memory usage could be lower if you have 2 containers and 1 core/1.5G per container.

@dttlgotv
Copy link
Author

dttlgotv commented Apr 2, 2020

In aurora, all containers need to have the same size, and container 0 is mainly for tmaster. Therefore the key is to keep the container size under control. In your example, your cpu and memory usage could be lower if you have 2 containers and 1 core/1.5G per container.

Let me try, thanks a lot

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants