Invalid use of SingleClientConnManager - exception

[lukeis]

Originally reported on Google Code with ID 2526

Hi,
if RemoteWebDriver is used, the 'java.lang.IllegalStateException: Invalid use of SingleClientConnManager:
connection still allocated.' exception can be raised when a method from RemoteWebDriver
is called and it is interrupted (due to SIGTERM, CTRL+C, etc.) or it is executed in
parallel. There are several opened issues for this topic.

This problem becomes more complex when the server is in a grid, due to the browser
remains opened, and, sometimes, the server becomes unstable (or slow).

I have worked on this issue, and I guess I found a solution.

At 'HttpCommandExecutor' I have added this method (this code is attached):

<code>
    public void cleanClientConnection() {
        // Shutdown current connection and open another one
        client.getConnectionManager().shutdown();

        HttpParams params = new BasicHttpParams();
        // Use the JRE default for the socket linger timeout.
        params.setParameter(CoreConnectionPNames.SO_LINGER, -1);
        HttpClientParams.setRedirecting(params, false);
        client = new DefaultHttpClient(getClientConnectionManager(params),
                params);
        if (remoteServer.getUserInfo() != null) {
            // Use HTTP Basic auth
            UsernamePasswordCredentials credentials = new UsernamePasswordCredentials(
                    remoteServer.getUserInfo());
            ((DefaultHttpClient) client).getCredentialsProvider()
                    .setCredentials(AuthScope.ANY, credentials);
        }

        // Some machines claim "localhost.localdomain" is the same as
        // "localhost".
        // This assumption is not always true.

        String host = remoteServer.getHost().replace(".localdomain", "");

        targetHost = new HttpHost(host, remoteServer.getPort(),
                remoteServer.getProtocol());
    }
</code>

And, at 'RemoteWebDriver.quit' method, before perform any action, previous method 'cleanClientConnection'
is called.

Summarizing, this method closes any opened connections and creates another client.

Reported by mpuchol@netquest.com on 2011-09-22 13:27:20

---

- _Attachment: [NQHttpCommandExecutor.java](https://storage.googleapis.com/google-code-attachments/selenium/issue-2526/comment-0/NQHttpCommandExecutor.java)_

[lukeis]

Sorry, but previous solution only works for servers allocated at localhost (at same
machine in which tests are executed).
I'm working to solve it. If I will find out any solution, I will tell you.

Thank you, and I'm sorry.

Reported by mpuchol@netquest.com on 2011-09-23 07:16:09

[lukeis]

Thank you for a very good bug report. I have been waiting for this one to come ;)

AFIK the reason we're getting in trouble is because we're getting an InterruptedException
or some odd exception from the bowels of httpcomponents, and it's coming in the wrong
place ;)

The technically correct solution to this issue is to always consume or abort the entity
for the http request, not the patch you're trying to do.

I would be *very* interested if you could categorize which exceptions you're able to
create and where they happen (you could attach a debugger and set breakpoint on throw)
and start producing errors.

I'm more interested in the *type* of error you're able to produce than the patch itself,
since we have a code-wide problem of flaky error handling on the http-client stuff.
We only recently managed to centralize the construction of the clients, and I expect
to fix this issue by making all clients use a common piece of code. Now the reason
this needs to be standardized in the code-base is /only/ because of the error handling,
so that's why it hasn't been done before. So stack-traces/evidence is appreciated just
as much as actual patches on this one.

This potentially affects stability of grid and server, so I'll be putting priority
on this one for the next release or so.

Reported by kristian.rosenvold on 2011-09-23 20:42:59

• Status changed: Accepted

[lukeis]

Btw; parallel execution in itself should not be the problem. But once you get that exception
is thrown in one thread, there is little knowing which thread will run into trouble.

Reported by kristian.rosenvold on 2011-09-25 19:06:48

[lukeis]

Ok.

I'm working on it.

I will reproduce the errors and will take the throws produces to attach here.

I have obtained some conclusions from my previous research. You are using 'SingleClientConnManager'
as client connection manager, which opens one (only one) connection to a server and
this connection is used in further communications. As you said in previous comments,
until it is consumed or aborted, this connection can not be reused. So, imagine this
scenario:
- One thread calls the 'HttpCommandExecutor.execute' method, 'HttpCommadnExecutor.fallBackExecute'
is called, and then...
- Another thread calls the 'HttpCommandExecutor.execute', when this new thread tries
to get the connection to the same server (it is normal in a grid), the connection is
not aborted neither consumed, and the new thread get the 'java.lang.IllegalStateException:
Invalid use of SingleClientConnManager: connection still allocated.' exception.

To solve this issue, I have tried to use the 'ThreadSafeClientConnManager' class, insted
the 'SingleClientConnManager'. ThreadSafeClientConnManager allows several connections
to the same server using several threads.

In my tests, I have used 'ThreadSafeClientConnManager' and it works worrectly. If you
are interested on using 'SingleClientConnManager' insted 'ThreadSafeClientConnManager',
please, tell me and I will follow researching the issue.

Regards. 

Reported by mpuchol@netquest.com on 2011-09-26 08:49:45

[lukeis]

Your comments make me think you are using an old version. There is no usage of SingleClientConnection
manager on 2.7 or trunk. Please do not track a problem like this on older versions.

Reported by kristian.rosenvold on 2011-09-26 09:11:31

[lukeis]

I'm using 2.7, but when I treated this problem, I was using version 2.5 (and 'NQHttpCommandExecutor.java'
is based on this version). Anyway, at 2.7, in file 'HttpCommandExecutor.java', line
193 there is a call to 'SingleClientConnection', but there is no call to this private
static function. Sorry, it confused me. I'm analyzing the issue for version 2.7. Thanks.

Reported by mpuchol@netquest.com on 2011-09-26 10:10:08

[lukeis]

Using 2.7, it works perfectly. Sorry for the issue track.

I recommend remove the SingleClientConnection call located at HttpCommandExecutor.java
to avoid further confusions.

Regards.

Reported by mpuchol@netquest.com on 2011-09-26 10:43:54

[lukeis]

Have removing that piece of dead code lined up in the stack 

Reported by kristian.rosenvold on 2011-09-26 11:09:05

• Status changed: Invalid

[lukeis]

Reported by luke.semerau on 2015-09-17 18:13:46

• Labels added: Restrict-AddIssueComment-Commit