Skip to content
This repository has been archived by the owner on Dec 4, 2023. It is now read-only.

Agent fails to control more than 10 routers #3092

Closed
k-wall opened this issue Aug 15, 2019 · 0 comments
Closed

Agent fails to control more than 10 routers #3092

k-wall opened this issue Aug 15, 2019 · 0 comments

Comments

@k-wall
Copy link
Member

k-wall commented Aug 15, 2019

If the number of routers in a standard address space grows beyond 10, agent cannot establish a management dialogue with the router properly. Node rejects the addition of the 11th listener on the connection object, so management request/response will fail.

node:10) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 receiver_open listeners added. Use emitter.setMaxListeners() to increase limit (node:10) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 disconnected listeners added. Use emitter.setMaxListeners() to increase limit (node:10) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 sender_error listeners added. Use emitter.setMaxListeners() to increase limit
https://nodejs.org/api/events.html#events_emitter_setmaxlisteners_n exists as a guard to trap programming errors involving leaked listeners. In our case, this number of routers is possible (not terribly useful) depending on plan configuration.

k-wall added a commit to k-wall/enmasse that referenced this issue Aug 15, 2019
@k-wall k-wall closed this as completed in df1df83 Aug 15, 2019
lulf pushed a commit to lulf/enmasse that referenced this issue Aug 20, 2019
lulf pushed a commit that referenced this issue Aug 21, 2019
* Backport address allocation equality fix

Backport of commit 228fa4e

* Turn down noisy per reconcilation per address logging (#3085)

* Restrict the number of active promises when creating/deleting entities on the routers (#3095)

Fix #3090

* Fix #3086: Adjust node max_old_space_size to a percentage of available container memory (#3099)

* Correct address status equality check (#3094)

* Fix #3089: Correct address status equality check

* address review comments
* made same_allocation same implementation symmetric

* Fix #3092: Turn off EventEmitter#maxListeners check for router management connection (#3093)

* Restrict the number of active promises when creating/deleting entities on the brokers

Fix #3090

* Ensure that addresses are not synced until addresses have been defined

As proposed in #3101

* Fix unit tests

* Start the watcher after the listeners are in place

Signed-off-by: Vanessa <vbusch@redhat.com>

* Fix test

Signed-off-by: Vanessa <vbusch@redhat.com>
lulf pushed a commit that referenced this issue Aug 23, 2019
lulf pushed a commit that referenced this issue Aug 23, 2019
lulf pushed a commit that referenced this issue Aug 23, 2019
* Turn down noisy per reconcilation per address logging (#3085)

* Restrict the number of active promises when creating/deleting entities on the brokers

Fix #3090

* Restrict the number of active promises when creating/deleting entities on the routers (#3095)

Fix #3090

* Fix #3086: Adjust node max_old_space_size to a percentage of available container memory (#3099)

* Correct address status equality check (#3094)

* Fix #3089: Correct address status equality check

* address review comments
* made same_allocation same implementation symmetric

* Fix #3092: Turn off EventEmitter#maxListeners check for router management connection (#3093)

* Ensure that addresses are not synced until addresses have been defined

As proposed in #3101

* Fix unit tests

* Start the watcher after the listeners are in place

Signed-off-by: Vanessa <vbusch@redhat.com>

* Improve api server address create performance (#3114)

This change improves api server address create performance by almost an order of magnitude when 1000 addresses are defined.
The changes moves the validation of spec.address to standard-controller for the standard address space. For the brokered address space, the validation remains in the api-server, as it would require adding write-back capability to the agent. Future refactoring/consolidation of agent/standard-controller should incorporate this validation.

Fixes #3111
lulf pushed a commit that referenced this issue Aug 23, 2019
* Correct address allocation equality. (#3083)

This defect meant that all existing addresses were considered modified even if they weren't.
This lead to unnecessary reconciliations against all routers and brokers, consuming extra resources across the system.

Fix: #3081

* Turn down noisy per reconcilation per address logging (#3085)

* Restrict the number of active promises when creating/deleting entities on the brokers

Fix #3090

* Restrict the number of active promises when creating/deleting entities on the routers (#3095)

Fix #3090

* Fix #3086: Adjust node max_old_space_size to a percentage of available container memory (#3099)

* Correct address status equality check (#3094)

* Fix #3089: Correct address status equality check

* address review comments
* made same_allocation same implementation symmetric

* Fix #3092: Turn off EventEmitter#maxListeners check for router management connection (#3093)

* Ensure that addresses are not synced until addresses have been defined

As proposed in #3101

* Fix unit tests

* Start the watcher after the listeners are in place

Signed-off-by: Vanessa <vbusch@redhat.com>

* Improve api server address create performance (#3114)

This change improves api server address create performance by almost an order of magnitude when 1000 addresses are defined.
The changes moves the validation of spec.address to standard-controller for the standard address space. For the brokered address space, the validation remains in the api-server, as it would require adding write-back capability to the agent. Future refactoring/consolidation of agent/standard-controller should incorporate this validation.

Fixes #3111
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant