Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http too many open file #20

Closed
at15 opened this issue Mar 20, 2017 · 3 comments
Closed

http too many open file #20

at15 opened this issue Mar 20, 2017 · 3 comments
Assignees
Projects
Milestone

Comments

@at15
Copy link
Member

at15 commented Mar 20, 2017

When benchmarking in localhost, got the error, could because the loader and the server is in the same machine, need to containerize xephon-k to confirm this.

Seems related to traefik/traefik#157 and traefik/traefik#127 Though I am not sure if the MaxIdleConnection per host applies to server, I know it works for client.

@at15 at15 added the bug label Mar 20, 2017
@at15 at15 self-assigned this Mar 20, 2017
@at15
Copy link
Member Author

at15 commented Mar 20, 2017

Also from the client part, it seems the maxidle connection (not per host) can also be tuned http://tleyden.github.io/blog/2016/11/21/tuning-the-go-http-client-library-for-load-testing/

  // Customize the Transport to have larger connection pool
  defaultRoundTripper := http.DefaultTransport
  defaultTransportPointer, ok := defaultRoundTripper.(*http.Transport)
  if !ok {
      panic(fmt.Sprintf("defaultRoundTripper not an *http.Transport"))
  }
  defaultTransport := *defaultTransportPointer // dereference it to get a copy of the struct that the pointer points to
  defaultTransport.MaxIdleConns = 100
  defaultTransport.MaxIdleConnsPerHost = 100

@at15
Copy link
Member Author

at15 commented Mar 20, 2017

seems to be deal with ulimit ? my default hard limit is 4096 based on /proc/pid/limits

ulimit -Sn 4096

The file-max can be found on /proc/sys/fs and my default is 3257324, which is much larger .... need to change it I think

at15 added a commit that referenced this issue Mar 20, 2017
- #20, need to run it in docker container to see if the problem is
  caused by running both the loader and Xephon-K on the host
at15 added a commit that referenced this issue Mar 20, 2017
- didn't see the accept:4 error on server side
at15 added a commit that referenced this issue Mar 20, 2017
- when the limit is changed from 1024 -> 4096, 3000 is the best worker
  number, after that, the performance decrease a lot
- add by putting server in the container, it no longer have the accept
  error
@at15 at15 added the bench label Mar 24, 2017
@at15 at15 added this to the 0.2.0 milestone Mar 25, 2017
@at15 at15 added this to TODO in Benchmark Apr 20, 2017
@at15
Copy link
Member Author

at15 commented Feb 15, 2018

Close in favor of #60

Adjust ulimit

@at15 at15 closed this as completed Feb 15, 2018
Benchmark automation moved this from TODO to DONE Feb 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Benchmark
  
DONE
Development

No branches or pull requests

1 participant