any advice on improve performance on AWS

[royxue]

I just started to use fasthttp, I checked the CPU usage is only around 20%, I want to improve performance.

is there any advice on how to improve performance on AWS?

[royxue]

@valyala I added some parts of my code, sorry, it's very messy...

[valyala]

@royxue , it would be great if you'll provide the following details:

• Source code for the RequestHandler you use.
• How many requests per second the server currently handles?
• How long does it take to process each request?

The universal advice is to profile your program, detect and eliminate bottlenecks.

[royxue]

@valyala
there is almost complete source code, except the matrix computation, and gcd function.
As now, the throughput is 2xxxx.

Oh, ok, I will try to profile my project.

However, is there any suggestion about achieve more CPU usage, I think this way would improve the performance. I tried ReusePort, but it seems not working very well.(May I did it wrong way, as Im just a beginner of Go)

[valyala]

Below are comments regarding your code:

• Always check for returned errors and do one of the following things on non-nil error:

  • Log the error and stop the program if the program cannot recover from the error. See log.Fatalf and log.Panicf.
  • Log the error and recover from it if this expected, but infrequent error. For instance, use default value if parsing fails. Such log messages may help in debugging and fixing the program in the future. See log.Printf.
  • Increment error counter and recover from the error if this is expected and frequent error. If error occurs in almost every request, then there is no need in spamming logs with such error messages. It would be better just updating error counter, which may help investigating the problem in the future. See expvar for error counters.
  • Prepend the error with the information about the current execution context and return it to the upper function if the error cannot be recovered at this level, but could be recovered in upper levels. See fmt.Errorf.

• Move the following line outside the request handler:

  l, _ := time.LoadLocation("EST")

  It doesn't depend on the request and always remains constant, so there is no need in spending CPU time on it during every request.

• There is no need in [:] after Peek calls. The following line may be rewritten:

  keyY.SetString(string(ctx.QueryArgs().Peek("key")[:]), 10)

  into

  keyY.SetString(string(ctx.QueryArgs().Peek("key")), 10)

• math/big functions used in the request handler may require a lot of CPU.

By the way, which program do you use for performance testing? Probably it sends requests over a small number of connections, so they load only a few CPU cores while other cores are idle. This may be fixed either by increasing the number of connections to the server or by moving expensive computations (I think this may be the code working with math/big) into a separate goroutines. Something like this:

func requestHandler(ctx *fasthttp.RequestCtx) {
    // move expensive computations to a separate goroutine and wait for the completion.
    ch := make(chan struct{})
    go func() {
        defer close(ch)
        requestHandlerSync(ctx)
    }()
    <-ch
}

func requestHandlerSync(ctx *fasthttp.RequestCtx) {
    l, _ := time.LoadLocation("EST")
    t := time.Now().In(l).String()[:19]
    keyX := new(big.Int)
    keyX.SetString("64266330917908644872330635228106713310880186591609208114244758680898150367880703152525200743234420230", 10)
    keyY := new(big.Int)
    keyY.SetString(string(ctx.QueryArgs().Peek("key")[:]), 10)
    keyZ := gcd(keyX, keyY)
    var k = 1 + keyZ % 25
    message := ctx.QueryArgs().Peek("message")
    n := int(math.Sqrt(float64(len(message))))
    final := string(matrix(n, message, int(k)))

    fmt.Fprintln(ctx, "Awesomething,3879-5761-4082")
    fmt.Fprintln(ctx, t)
    fmt.Fprintln(ctx, final)
}

[royxue]

@valyala
Thx a lot, I will try what you said.
btw I saw there are three ways to create server, ListenAndServe, Server, and Custom Server, is there any differences between these methods?

[valyala]

I saw there are three ways to create server, ListenAndServe, Server, and Custom Server, is there any differences between these methods?

They differ in the level of details you can control on the server:

• The most detailed is custom server - you can tune a lot of parameters defined in Server struct.
• Serve accepts custom net.Listener, so you can control how the server accepts incoming connections, wrap accepted connections for gathering various stats or for custom encryption / authorization, etc.
• ListenAndServe uses default server settings and default listener for accepting incoming connections from the given address.