Usage of Elastic causes high

[giancolli]

ELK-Solution version

4.1.0

Elastic-Stack version

7.16.3

API-Management version

7.7 August21 Update

API-Management deployment option

No response

Question

Hi @cwiechmann, this issue is a follow-up of issue #180 .

After your analysis we applied suggested cache configurations on the envSettings, acting on the CACHE_API_PATHS parameter.

Nevertheless, we still face a very high amount of Cassandra Mismatch blocking when the API Builder is running.

Other interesting point is the following:
Considering this configuration (EMEA)
ADMIN_NODE_MANAGER=EMEA|https://node01.***.net:8090,EMEA|https://node04.***.net:8090

We noticed that when the API Builder is on, ONLY node04 faces a high increase in:

• CPU Usage
  

• Internet traffic received (normal amount is about 5 GB/hour, with the builder goes over 50GB hour)
  

Note: API Builder was on from around 10 to around 18

Can you gently help us to:

• Understand why the API Builder impacts so hard only node04
• Find a fix to reduce the stress API Builder puts on nodes, which will probably be beneficial for the Cassandra Mismatches issue.

Thanks
Gianmario

[cwiechmann]

Is this a new environment? Can you some details about your environment. Especially when you talking about node04, etc.

And what exactly is internet traffic? Is it a dedicated network interface?

[giancolli]

Hi @cwiechmann, can you gently provide your Axway business mail so that we can communicate more details on that channel?

Thanks
Gianmario

[cwiechmann]

Hi @giancolli, Please check the behavior of Logstash and if it caches API-Details properly. Please check this video if you haven't done yet to understand what to look for:
https://youtu.be/rLT84dkgGAE?t=290

Let us know, the metrics you see in Stack-Monitoring.

[lucasbuschlinger]

Hi @cwiechmann,
Logstash seems to be fine, especially after starting the second node at around 11:00

FYI, we also have opened a corresponding Axway Case (01384086) and a meeting is scheduled for Monday, 14:00 CEST.

[cwiechmann]

Yes, that looks perfect! And what about the APM Details for the API-Builder API requests?

[lucasbuschlinger]

Sorry, I missed that part

If I get this right, this indicated that we're not caching enough paths again?

[lucasbuschlinger]

I just checked the APM dependencies and found that the above mentioned node04 is having lots of requests:

This should also explain the higher load (its basically the admin node manager of the secondary cluster) - interestingly node01 is not listed as a dependency...

Edit: the second listed one is a different hub altogether

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 90 days. It will be closed in the next 30 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.