Restcomm1147 rms resources leak on conf by gvagenas · Pull Request #2564 · RestComm/Restcomm-Connect

gvagenas · 2017-10-09T13:04:27Z

PR also includes MGCP resources metrics and updated tests to assert MGCP resources are properly cleaned up

This refer to RESTCOMM-1147

This refer to RESTCOMM-1155

…resources_leak_on_conf

This refer to RESTCOMM-1155

This close RESTCOMM-1155

This refer to RESTCOMM-1158

…restcomm1147_rms_resources_leak_on_conf

This close RESTCOMM-1158 This refer to RESTCOMM-1155

…test This refer to RESTCOMM-1155

This close RESTCOMM-1163

…rt proper MGCP resources at the end of each test This refer to RESTCOMM-1155 This refer to RESTCOMM-1147

jaimecasero · 2017-10-09T13:13:23Z

@@ -0,0 +1,26 @@
+<?xml version="1.0" encoding="UTF-8" ?>


i guess we need to find a way to configure how this is used in JenkinsCI env...

Agree @jaimecasero . I did this to quickly enable logs in the testsuite since they are essential when working locally with the testsuite

jaimecasero · 2017-10-09T13:17:20Z

-    public static Map<MediaSession, ActorRef> getLinks() {
-        return links;
+        connectionToEndpointMap = new ConcurrentHashMap<ActorRef, ActorRef>();
+        connEndpointMap = new ConcurrentHashMap<String, String>();


considering this is Actor state, and concurrency should be already solved by Actor model, is there any particular intention on using ConcurrentHashMap??

@jaimecasero no particular reason, just a habit

jaimecasero · 2017-10-09T13:20:59Z

-
-    public static Map<MediaSession, ActorRef> getLinks() {
-        return links;
+        connectionToEndpointMap = new ConcurrentHashMap<ActorRef, ActorRef>();


i dont seem to find where connectionToEndpointMap is used? could it be this slid into final code after different design/refactorings?

@jaimecasero I forgot to clean up this one. I will clean this now

jaimecasero · 2017-10-09T13:29:23Z

        final ActorRef self = self();
        final DeleteConnection dlcx = (DeleteConnection) message;
+        if (dlcx.getConnectionIdentifier() == null) {
+            connEndpointMap.values().removeAll(Collections.singleton(dlcx.getEndpointIdentifier().getLocalEndpointName()));


can we log the result of this operation, i think it could be worth to print it, whether we deleted something or not...

@jaimecasero you mean the operation to remove from the Map? Yes I can add this at the log statement later

jaimecasero · 2017-10-09T13:30:12Z

+                logger.info(msg);
+            }
+        } else {
+            connEndpointMap.remove(dlcx.getConnectionIdentifier().toString());


same here, can we save the result of this operation for logging later?

Sure as above

jaimecasero · 2017-10-09T13:32:17Z

+        } else {
+            connEndpointMap.remove(dlcx.getConnectionIdentifier().toString());
+            monitoringService.tell(new MgcpConnectionDeleted(dlcx.getConnectionIdentifier().toString(), null), self());
+            if (!connEndpointMap.values().contains(dlcx.getEndpointIdentifier().getLocalEndpointName())) {


could you add some comment to this logic? im not sure im following the intention. It seems we already removed the map entry using the connId, which is the key to the map. Then we check if the endpointId is still in the map as value, if not we tell the service that is was removed. Is there an scenario where connId was removed, but endpointId is still present, what would be the use case there? Are the endpointId shared among different connIds?

@jaimecasero RC only deletes connections, on the RMS side when there is no connection for a given endpoint, the endpoint is released.
This is what the method does here.
First remove endpoint-connection, then check if there are more connections for the give endpoint, IF NOT, consider the endpoint released and tell the MonitoringService.

I added comments for that.

jaimecasero · 2017-10-09T13:33:54Z

+        mgcpEndpointsConference = new AtomicInteger(0);
+        mgcpEndpointsPacketRelay = new AtomicInteger(0);
+
+        mgcpEndpointMap = new ConcurrentHashMap<String, String>();


again, do we need concurrent collections as Actor state?

Its a just a habit

jaimecasero · 2017-10-09T13:36:51Z

+        mgcpEndpointsConference = new AtomicInteger(0);
+        mgcpEndpointsPacketRelay = new AtomicInteger(0);
+
+        mgcpEndpointMap = new ConcurrentHashMap<String, String>();


seems we only need the size of these maps to return as part of onStatistics logic... Do we really need to save all this info into maps? it seems a potential place for unnecesary leaks, when the size of these maps can be already calculated by using other counters already present... I would try to reduce state to simple counters, that maybe be wrong but do not risk to introduce more leaks...

@jaimecasero I am thinking that in the future we can provide MGCP resource details by call, similar to the live call details, so in that case the Map is needed.
Agree with your point but I don't see a risk of leaks here.

jaimecasero · 2017-10-09T13:39:23Z

-                fsm.transition(message, completed);
-            }
+            //Don't wait for ever for the CallController response
+            context().setReceiveTimeout(Duration.create(2000, TimeUnit.SECONDS));


this kind of timeouts should come from configuration, so we can tweak then (or customers) depending on envirnoment... where is 2 secs coming from (for me is regular call timeout, just wondering if this is iline with some other system in the cloud or something)=

@jaimecasero this is a safety net for Conference in a cluster configuration where notification from media server might never arrive. It doesn't need to be on configuration since this is an internal, very specific timer to be able to cleanup call no matter what error condition might occur. No particular reason for for the 2 sec duration, I can reduce it if you think its too much

2 seconds seems ok

jaimecasero · 2017-10-09T13:42:02Z

        assertTrue(maxConcurrentIncomingCalls==0);
        assertTrue(maxConcurrentOutgoingCalls==0);
+
+        Map<String, Integer> mgcpResources = MonitoringServiceTool.getInstance().getMgcpResources(metrics);


this will be a fairly common construct, could we isolate this in a Assert util class, where access to monitoring services, and literal to specific counters are protected

@jaimecasero I thought about that but let this task for later as it will take significant time. I did half of the work (MonitoringServiceTool.getInstance().getMgcpResources(metrics)) but the rest of the enhancement can be done in another sprint with proper estimation

This refer to RESTCOMM-1147

gvagenas added 20 commits October 1, 2017 09:55

Work in progress for RESTCOMM-1147, RMS resources leak

6ab2108

Fixes for SmsTest

7a988ed

Merge branch 'master' into restcomm1147_rms_resources_leak_on_conf

e9d3923

Code cleanup.

110a6a2

This refer to RESTCOMM-1147

Work in progress for MgcpMonitoringService

2cb0698

This refer to RESTCOMM-1155

Merge branch 'issue1155_MgcpMonitoringService' into restcomm1147_rms_…

644664b

…resources_leak_on_conf

Work in progress for MgcpMonitoringService

0355b20

This refer to RESTCOMM-1155

Work in progress for MgcpMonitoring service

e89b53e

This refer to RESTCOMM-1155

Patch to monitor MGCP resources and be able to track MGCP resources leak

e3734ae

This close RESTCOMM-1155

Merge branch 'master' into restcomm1147_rms_resources_leak_on_conf

e17ded6

Work in progress to remove endpoints on connection destroy

fe49c02

This refer to RESTCOMM-1158

Merge branch 'restcomm1158_removeEndpoints_onConnectionDestroy' into …

aaebb11

…restcomm1147_rms_resources_leak_on_conf

Patch to remove endpoints on connection destroy

b2b3ae5

This close RESTCOMM-1158 This refer to RESTCOMM-1155

Patch for DialConferenceTest

804ba65

Addition to DialConferenceTest to assert MOH is properly stopped.

3164114

Testsuite enhancements for checking MGCP resources at the end of the …

b03a9e7

…test This refer to RESTCOMM-1155

Patch to properly cleanup MGCP resources for Failed Calls

fba78b9

This close RESTCOMM-1163

Enhancements to testsuite for DialActionTest and DialForkTest to asse…

2c69abf

…rt proper MGCP resources at the end of each test This refer to RESTCOMM-1155 This refer to RESTCOMM-1147

Merge branch 'master' of github.com:RestComm/Restcomm-Connect

5255baa

Merge branch 'master' into restcomm1147_rms_resources_leak_on_conf

5345768

gvagenas requested review from jaimecasero and maria-farooq October 9, 2017 13:04

gvagenas added the Peer Review label Oct 9, 2017

jaimecasero reviewed Oct 9, 2017

View reviewed changes

Code cleanup and added comments

0e7dc36

This refer to RESTCOMM-1147

gvagenas merged commit 0e7dc36 into master Oct 10, 2017

gvagenas removed the Peer Review label Oct 10, 2017

jaimecasero deleted the restcomm1147_rms_resources_leak_on_conf branch April 11, 2018 20:12

Conversation

gvagenas commented Oct 9, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaimecasero Oct 9, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jaimecasero Oct 9, 2017 •

edited

Loading