integration_test failures, listener test okay.

[aaronjohnson]

Hi,

I'm am having trouble running the integration tests. I tried both go 1.1.2 and go1.1 Here is the output for the latter.

bash-3.2# go version
go version go1.1 darwin/386
bash-3.2# gor
Version: 0.3.3
Usage:
gor listen -h
gor replay -h
bash-3.2# go test
Listening for HTTP traffic on 127.0.0.1:50000
Forwarding requests to replay server: 127.0.0.1:50001 Limit: 0
2013/08/16 00:30:50 Starting replay server at: 127.0.0.1:50001
2013/08/16 00:30:50 Forwarding requests to: http://127.0.0.1:50002 limit: 0
--- FAIL: TestReplay (1.50 seconds)
integration_test.go:132: Timeout error
Listening for HTTP traffic on 127.0.0.1:50010
Forwarding requests to replay server: 127.0.0.1:50011 Limit: 0
2013/08/16 00:30:51 Starting replay server at: 127.0.0.1:50011
2013/08/16 00:30:51 Forwarding requests to: http://127.0.0.1:50012 limit: 0
--- FAIL: TestWithoutReplayRateLimit (0.60 seconds)
integration_test.go:173: It should forward all requests without rate-limiting 0
Listening for HTTP traffic on 127.0.0.1:50020
Forwarding requests to replay server: 127.0.0.1:50021 Limit: 0
2013/08/16 00:30:52 Starting replay server at: 127.0.0.1:50021
2013/08/16 00:30:52 Forwarding requests to: http://127.0.0.1:50022 limit: 5
--- FAIL: TestReplayRateLimit (0.60 seconds)
integration_test.go:181: It should forward only 5 requests with rate-limiting 0
Listening for HTTP traffic on 127.0.0.1:50030
Forwarding requests to replay server: 127.0.0.1:50031 Limit: 3
2013/08/16 00:30:52 Starting replay server at: 127.0.0.1:50031
2013/08/16 00:30:52 Forwarding requests to: http://127.0.0.1:50032 limit: 0
--- FAIL: TestListenerRateLimit (0.60 seconds)
integration_test.go:189: It should forward only 3 requests with rate-limiting 0
FAIL
exit status 1
FAIL github.com/buger/gor 3.339s
bash-3.2# uname -a
Darwin laptop.local 12.4.0 Darwin Kernel Version 12.4.0: Wed May 1 17:57:12 PDT 2013; root:xnu-2050.24.15~1/RELEASE_X86_64 x86_64
bash-3.2#

I am able to run the listener test,

bash-3.2# go test
127.0.0.1:49165
2013/08/16 00:35:57 Forwarding request: &{GET /pub/WWW/ HTTP/1.1 1 1 map[] 0x1149d0c0 0 [] false www.w3.org map[] map[] map[] /pub/WWW/ }
PASS
ok github.com/buger/gor/listener 0.022s
bash-3.2#

[buger]

@aaronjohnson hi, i added more logging, try to get new version and run tests again.

Check for: "Error while starting http server" string.

Also try to increase timeout here:

    select {
    case <-received:
    case <-time.After(time.Second):
        t.Error("Timeout error")
    }

[aaronjohnson]

I am seeing the same error with the latest code. I also tried increasing the timeout to "3*time.Second". It took longer but same error.

I also added a log for the request response in the first test and see that it is a 404 status.

bash-3.2# go test
Listening for HTTP traffic on 127.0.0.1:50000
Forwarding requests to replay server: 127.0.0.1:50001 Limit: 0
2013/08/16 08:03:51 Starting replay server at: 127.0.0.1:50001
2013/08/16 08:03:51 Forwarding requests to: http://127.0.0.1:50002 limit: 0
--- FAIL: TestReplay (3.10 seconds)
integration_test.go:133: Response: &{404 Not Found 404 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8] Content-Length:[3] Date:[Fri, 16 Aug 2013 15:03:51 GMT]] 0x114b2840 3 [] false map[] 0x11461af0}

diff --git a/integration_test.go b/integration_test.go
index e2538e6..7bca077 100644
--- a/integration_test.go
+++ b/integration_test.go
@@ -124,15 +124,17 @@ func TestReplay(t *testing.T) {

        request = getRequest(p)

-       _, err := http.DefaultClient.Do(request)
+       response, err := http.DefaultClient.Do(request)

        if err != nil {
                t.Error("Can't make request", err)
        }

+        t.Log("Response: ", response)
+
        select {
        case <-received:
-       case <-time.After(time.Second):
+       case <-time.After(3*time.Second):
                t.Error("Timeout error")
        }
 }

[buger]

I finally could replicate it and found the problem.

Thing is that Mac OS is BSD based system, and it not support TCP and UDP traffic capture using RAW sockets.

http://stackoverflow.com/questions/6878603/strange-raw-socket-on-mac-os-x

This will not work on *BSD (including OSX/Darwin). See the investigation here for more details:

b. FreeBSD
**********

FreeBSD takes another approach. It *never* passes TCP or UDP packets to raw
sockets. Such packets need to be read directly at the datalink layer by using
libraries like libpcap or the bpf API. It also *never* passes any fragmented 
datagram. Each datagram has to be completeley reassembled before it is passed
to a raw socket.
FreeBSD passes to a raw socket:
    a) every IP datagram with a protocol field that is not registered in
    the kernel
    b) all IGMP packets after kernel finishes processing them
    c) all ICMP packets (except echo request, timestamp request and address
    mask request) after kernel finishes processes them

I tried libpcap for traffic capture, but found current binding pretty buggy, maybe something changed from that time #8

For now i'll close this issue, and update pull-request.

Also I created new issue #22

[buger]

Also i suggest to use http://www.vagrantup.com/ for local development. Gor developed on MacOS using Vagrant with Ubuntu inside.

[denji]

scan, capture and hexdump incoming tcp/udp/icmp traffic without libcap
https://github.com/kala13x/scap

[buger]

@denji it use RAW sockets, exactly what currently used by Gor