Skip to content

Grpc keep alive ping frames are not forwarded to the client #5181

@anjmao

Description

@anjmao

Do you want to request a feature or report a bug?

Bug

Did you try using a 1.7.x configuration for the version 2.0?

  • Yes
  • No

What did you do?

I have a grpc backend with bidi streams and keep alive server parameters

keepaliveParams = keepalive.ServerParameters{
	MaxConnectionIdle:     20 * time.Minute, // If a client is idle for given duration, send a GOAWAY.
	MaxConnectionAge:      20 * time.Minute, // If any connection is alive for more than given duration, send a GOAWAY.
	MaxConnectionAgeGrace: 10 * time.Second, // Allow given duration for pending RPCs to complete before forcibly closing connections
	Time:                  10 * time.Second,  // Ping the client if it is idle for given duration to ensure the connection is still active.
	Timeout:               2 * time.Second,  // Wait given duration for the ping ack before assuming the connection is dead.
}

// ...

srv := grpc.NewServer(grpc.KeepaliveParams(keepaliveParams))
srv.Serve()

Also I have grpc client which connects to the server and calls some bidi stream.
Since I have grpc keep alive server parameters I want my client to be pinged in given time interval.

What did you expect to see?

I expect client to receive ping frames from the grpc server.

Framer 0xc0001ca000: read PING flags=ACK len=8 ping="\x02\x04\x10\x10\t\x0e\a\a"
2019/08/07 10:48:30 http2: Framer 0xc0001ca000: wrote PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2019/08/07 10:48:30 http2: Framer 0xc0001ca000: read PING flags=ACK len=8 

What did you see instead?

2019/08/07 10:51:34 frame.go:388: http2: Framer 0xc0009c6000: wrote PING flags=ACK len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2019/08/07 10:51:38 frame.go:514: http2: Framer 0xc0009c6000: read PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2019/08/07 10:51:38 transport.go:2391: http2: Transport received PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2019/08/07 10:51:38 frame.go:388: http2: Framer 0xc0009c6000: wrote PING flags=ACK len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2019/08/07 10:51:42 frame.go:514: http2: Framer 0xc0009c6000: read PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
2019/08/07 10:51:42 transport.go:2391: http2: Transport received PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"

I see that only fraefik receives ping frames and doesn't forwards them to original client. This is a problem for production environments when you have mobile clients with long lived RPC streams and in case when client lost it's internet connection grpc server should be able to detect that client connection was lost.

Output of traefik version: (What version of Traefik are you using?)

Version:      2.0.0-beta1
Codename:     faisselle
Go version:   go1.12.7
Built:        2019-07-19T15:50:09Z
OS/Arch:      darwin/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?

config.toml

[entryPoints]
  [entryPoints.web]
    address = ":80"
  [entryPoints.traefik]
    address = ":8100"

[api]

[log]
  level = "DEBUG"

[providers.file]
  filename = "dynamic_conf.toml"

dynamic_conf.toml

[http]

  [http.routers]
    [http.routers.routerTest]
      service = "srv-grpc"
      rule = "Host(`frontend.local`)"

  [http.services]
    [http.services.srv-grpc]
      [http.services.srv-grpc.loadBalancer]
        [[http.services.srv-grpc.loadBalancer.servers]]
          url = "h2c://backend.local:10000"

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions