CloudAppender failed to report: java.net.SocketException: Too many open files

[yegor256]

migrated from Trac, where originally posted by yegor256 on 6-Feb-2012 8:50pm

I think that we should somehow avoid such a situation:

CloudAppender failed to report: java.net.SocketException: Too many open files
    at java.net.Socket.createImpl(Socket.java:397)
    at java.net.Socket.connect(Socket.java:527)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:559)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:158)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:394)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:529)
    at sun.net.www.protocol.https.HttpsClient.<init>(HttpsClient.java:272)
    at sun.net.www.protocol.https.HttpsClient.New(HttpsClient.java:329)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.getNewHttpClient(AbstractDelegateHttpsURLConnection.java:172)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:916)
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:158)
    at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1019)
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:230)
    at com.rexsl.log.HttpFeeder.post(HttpFeeder.java:112)
    at com.rexsl.log.HttpFeeder.feed(HttpFeeder.java:83)
    at com.rexsl.log.CloudAppender.run(CloudAppender.java:168)
    at java.lang.Thread.run(Thread.java:662)

[yegor256]

migrated from Trac, where originally posted by yegor256 on 6-Feb-2012 8:56pm

looks like com.netbout.log.HttpFeeder doesn't close URLConnection. let's reproduce the problem with a unit test first, and then fix it. the budget is 1 hour.

[yegor256]

migrated from Trac, where originally posted by yegor256 on 6-Feb-2012 8:57pm

I think that powermock will be required to reproduce the problem in a test...

[yegor256]

migrated from Trac, where originally posted by yegor256 on 6-Feb-2012 8:57pm

Evgeniy, can you do this?

[yegor256]

migrated from Trac, where originally posted by e.nyavro on 7-Feb-2012 4:32pm

I have reproduced this problem in test. Currently I'm unable to commit to svn, I'm getting svn: Access denied message. So here is the code:

    @Test
    public void feedExcessiveTest() {
        final ContainerMocker container = new ContainerMocker()
            .expectMethod(Matchers.equalTo("POST"))
            .expectHeader(
                HttpHeaders.CONTENT_TYPE,
                Matchers.equalTo(MediaType.TEXT_PLAIN)
            )
            .returnBody("posted")
            .mock();
        final HttpFeeder feeder = new HttpFeeder();
        feeder.setUrl(container.home().toString());
        feeder.activateOptions();
        try {
            for(int i=0;i<1500;i++) {
                feeder.feed("some text");
            }
        } catch (IOException e) {
            e.printStackTrace();
            //fail();
        }
    }

On my Ubuntu system 1500 tries to open HttpURLConnection crash with exception.
Is it really neccessary to open separate connection to url in HttpFeeder->post(...) for each line of message? conn.disconnect() at the end of post method fixes this problem.

[yegor256]

migrated from Trac, where originally posted by yegor256 on 7-Feb-2012 7:00pm

Looks very good. In order to add your code to the project you should start a new branch, commit there, test locally, and let me know when it's ready. I will merge to trunk. Read more here: http://www.tpc2.com/dev/branches

Don't forget that in order to test everything locally you should run mvn -Dci clean install (mind the -Dci flag)

[yegor256]

migrated from Trac, where originally posted by yegor256 on 7-Feb-2012 7:01pm

Actually I also don't think that it's a the best way to behave - new connection per every new line of log. But I didn't find any better way. We don't know when the next line will arrive (inside the HttpFeeder). Thus, we should proceed lines in isolated manner - one at a time.

If you still think that there is a better way - submit a new ticket about it. For every ticket you submit (and if it gets accepted) you get 1/2hrs on your balance.

[yegor256]

migrated from Trac, where originally posted by e.nyavro on 14-Feb-2012 12:44pm

In [848/rexsl]:

#CommitTicketReference repository="rexsl" revision="848"
refs #143 - new branch

[yegor256]

migrated from Trac, where originally posted by e.nyavro on 14-Feb-2012 12:55pm

In [849/rexsl]:

#CommitTicketReference repository="rexsl" revision="849"
refs #143 - CloudAppender failed to report: java.net.SocketException: Too many open files

[yegor256]

migrated from Trac, where originally posted by yegor256 on 15-Feb-2012 2:54am

What is the reason of HttpFeederTest#BODY constant? With this constant you're showing to the reader (myself) that there is some connection between two tests in the file, since they use the same contstant. In fact, there is absolutely no connection. So, this constant is "misleading". Let's use just two different strings in these two tests.

As for POST constant, you can use com.rexsl.test.RestTester#POST instead.

[yegor256]

migrated from Trac, where originally posted by e.nyavro on 15-Feb-2012 4:11pm

In [852/rexsl]:

#CommitTicketReference repository="rexsl" revision="852"
refs #143 - CloudAppender failed to report: java.net.SocketException: Too many open files

[yegor256]

migrated from Trac, where originally posted by yegor256 on 15-Feb-2012 5:17pm

looks cool now, let us try to merge...

[yegor256]

migrated from Trac, where originally posted by guard on 15-Feb-2012 5:26pm

In [853/rexsl]:

#CommitTicketReference repository="rexsl" revision="853"
refs #143 - /branches/ticket143 merged into /trunk, 1 hours spent

[yegor256]

migrated from Trac, where originally posted by guard on 15-Feb-2012 5:26pm

In [854/rexsl]:

#CommitTicketReference repository="rexsl" revision="854"
refs #143 - /branches/ticket143 successfully closed

[yegor256]

migrated from Trac, where originally posted by yegor256 on 20-Feb-2012 9:52pm

Thanks!