Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http.Server.Serve: "protocol not available" (OpenBSD 6.5) #10

Closed
peterljung opened this issue Jun 2, 2019 · 8 comments

Comments

2 participants
@peterljung
Copy link

commented Jun 2, 2019

I have problem running the test on OpenBSD 6.5.

$ uname -a
OpenBSD nuc.lounge.se 6.5 GENERIC.MP#3 amd64
$ go version
go version go1.12.1 openbsd/amd64

This is probably related to something in OpenBSD. I have a very similar issue with another web framework. Could ephemeral port ranges have something to do with it?

$ doas ./aerotest                                                                                                        
Server running on: http://localhost:4000
--------------------------------------------------------------------------------
panic: set tcp 127.0.0.1:4000->127.0.0.1:38930: protocol not available

goroutine 21 [running]:
github.com/aerogo/aero.(*Application).serveHTTP(0xc000102000, 0xc0000a0d88)
	/home/peter/go/pkg/mod/github.com/aerogo/aero@v1.3.7/Application.go:418 +0x29f
created by github.com/aerogo/aero.(*Application).ListenAndServe
	/home/peter/go/pkg/mod/github.com/aerogo/aero@v1.3.7/Application.go:197 +0xe3

But maybe you have seen something similar on another platform?

I tried a simple example using http on the same machine and that works fine.

package main

import (
	"fmt"
	"log"
	"net/http"
)

type helloHandler struct{}

func (h helloHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "hello, you've hit %s\n", r.URL.Path)
}

func main() {
	err := http.ListenAndServe(":9999", helloHandler{})
	log.Fatal(err)
}

Any idea what this could be related to?

@akyoto

This comment has been minimized.

Copy link
Member

commented Jun 2, 2019

Have never seen this panic before using Arch Linux on x86_64.

λ uname -a
Linux home 5.1.5-arch1-2-ARCH #1 SMP Mon May 27 03:37:39 UTC 2019 x86_64 GNU/Linux

I don't have an OpenBSD installation to test with, but if you're seeing the same problems in another framework then it might be something related to your system.

I don't want to be so quick on dropping responsibility onto someone else, though, so let's dig a little deeper.

The line referenced in the panic is this one:

// This will block the calling goroutine until the server shuts down.
// The returned error is never nil and in case of a normal shutdown
// it will be `http.ErrServerClosed`.
err := server.Serve(listener)

if err != http.ErrServerClosed {
	panic(err)
}

Via some google searches I found these threads, all returning the same error message:

The message itself is coming from a syscall code 42 on OpenBSD in go/src/syscall/zerrors_openbsd_amd64.go:

This leads me to searching again and finding those:

It looks like the problem is related to the TCP_USER_TIMEOUT socket option.

@akyoto akyoto changed the title Problem running test on OpenBSD 6.5 http.Serve: "protocol not available" (OpenBSD 6.5) Jun 2, 2019

@peterljung

This comment has been minimized.

Copy link
Author

commented Jun 3, 2019

Yes, you may be on to the problem ...

TCP socket options in linux include TCP_USER_TIMEOUT: http://man7.org/linux/man-pages/man7/tcp.7.html

TCP_USER_TIMEOUT (since Linux 2.6.37)

OpenBSD does not: https://man.openbsd.org/tcp

Is this a generic problem with the go port or http package? Or can this option be controlled in aero?

I minimal example which trigger the problem would be good to be able to send a bug to OpenBSD mailing lists if it is a general go port problem. I tried with the small HTTP example above but that did not trigger the issue ...

@peterljung

This comment has been minimized.

@akyoto

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

depth isn't showing it:

λ depth github.com/aerogo/aero
github.com/aerogo/aero
  ├ compress/gzip
  ├ context
  ├ crypto/tls
  ├ errors
  ├ fmt
  ├ io
  ├ io/ioutil
  ├ mime
  ├ net
  ├ net/http
  ├ net/http/httptest
  ├ os
  ├ os/signal
  ├ path/filepath
  ├ sort
  ├ strconv
  ├ strings
  ├ sync
  ├ syscall
  ├ time
  ├ github.com/aerogo/csp
    └ strings
  ├ github.com/aerogo/http/ciphers
    └ crypto/tls
  ├ github.com/aerogo/http/client
    ├ bytes
    ├ compress/gzip
    ├ crypto/tls
    ├ fmt
    ├ io
    ├ io/ioutil
    ├ log
    ├ net
    ├ net/url
    ├ strconv
    ├ sync
    ├ unicode
    ├ github.com/aerogo/http/ciphers
    ├ github.com/akyoto/stringutils/convert
    └ github.com/json-iterator/go
      ├ bytes
      ├ encoding
      ├ encoding/base64
      ├ encoding/json
      ├ errors
      ├ fmt
      ├ io
      ├ math
      ├ math/big
      ├ reflect
      ├ sort
      ├ strconv
      ├ strings
      ├ sync
      ├ unicode
      ├ unicode/utf16
      ├ unicode/utf8
      ├ unsafe
      ├ github.com/modern-go/concurrent (unresolved)
      └ github.com/modern-go/reflect2 (unresolved)
  ├ github.com/aerogo/linter-performance
    ├ fmt
    ├ time
    ├ github.com/aerogo/http/client
    └ github.com/akyoto/color
      ├ fmt
      ├ io
      ├ os
      ├ strconv
      ├ strings
      ├ sync
      ├ github.com/mattn/go-colorable
        ├ bytes
        ├ io
        ├ os
        └ github.com/mattn/go-isatty
          └ golang.org/x/sys/unix
            ├ bytes
            ├ encoding/binary
            ├ net
            ├ runtime
            ├ sort
            ├ strings
            ├ sync
            ├ syscall
            ├ time
            └ unsafe
      └ github.com/mattn/go-isatty
  ├ github.com/aerogo/session
    ├ fmt
    ├ sync
    ├ time
    └ github.com/akyoto/uuid
      ├ bytes
      ├ crypto/md5
      ├ crypto/rand
      ├ crypto/sha1
      ├ encoding/binary
      ├ encoding/hex
      ├ errors
      ├ fmt
      ├ hash
      ├ io
      ├ net
      ├ os
      ├ strings
      ├ sync
      └ time
  ├ github.com/aerogo/session-store-memory
    ├ errors
    ├ sync
    └ github.com/aerogo/session
  ├ github.com/akyoto/color
  ├ github.com/akyoto/hash
    └ github.com/zeebo/xxh3
      ├ math/bits
      ├ unsafe
      └ golang.org/x/sys/cpu
        ├ encoding/binary
        └ runtime
  ├ github.com/akyoto/stringutils/unsafe
    ├ reflect
    └ unsafe
  └ github.com/json-iterator/go
61 dependencies (42 internal, 19 external, 0 testing).

The biggest problem for me in debugging this is not having a physical system to test with. I'll try to download Virtualbox.

In the meantime, could you try downloading the git repo, using the repo via replace and removing the following lines from the source:

  1. https://github.com/aerogo/aero/blob/v1.3.10/Application.go#L378-L380
  2. https://github.com/aerogo/aero/blob/v1.3.10/Listener.go#L24-L36

Only remove the second if the first didn't work. If removing the timeouts from the first link works, I'm honestly going to be surprised. These timeouts are important and exist for a reason. Without these timeouts, you will have a lot of idle connections and your app might eventually crash. This might be circumvented by putting something like nginx in front of the server, but if the server is the main server / actual endpoint of the client then you definitely would want to have these timeouts.

@akyoto akyoto added the unclear label Jun 3, 2019

@akyoto akyoto changed the title http.Serve: "protocol not available" (OpenBSD 6.5) http.Server.Serve: "protocol not available" (OpenBSD 6.5) Jun 4, 2019

@akyoto

This comment has been minimized.

Copy link
Member

commented Jun 4, 2019

Found the culprit, it's https://github.com/aerogo/aero/blob/v1.3.11/Listener.go#L32

connection.SetKeepAlivePeriod(keepAlivePeriod)

Aero on OpenBSD

@akyoto

This comment has been minimized.

Copy link
Member

commented Jun 4, 2019

Fixed in 1.3.12 via 0795717:

Aero 1.3.12 on OpenBSD

@akyoto akyoto closed this Jun 4, 2019

@peterljung

This comment has been minimized.

Copy link
Author

commented Jun 4, 2019

Super thanks!

I suppose you need to handle timeouts in the application code to avoid triggering the TCP timeout.

@akyoto akyoto added bug and removed unclear labels Jun 4, 2019

@akyoto

This comment has been minimized.

Copy link
Member

commented Jun 5, 2019

I might be wrong, but I believe there is a default TCP keep-alive interval considering that SetKeepAlive(true) is returning no errors on OpenBSD.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.