reverse resolutions for remote side of connections. #404

inercia · 2015-08-27T12:41:02Z

Async reverse dns lookups in probe that creates nodes for the remote side of connections.

It's part of the solution for #364

TODO:

unit test
update detailed_node.go to display the remote name

probe/endpoint/resolver.go

+		if err == gcache.NotFoundKeyError {
+			// we trigger a asynchronous reverse resolution when not cached, but we return
+			// immediately so the client does not block...
+			r.addresses <- address


tomwilkie · 2015-09-01T11:19:07Z

#411 is merged, so you should now be able to put the reverse dns lookups on pseudo nodes and plumb this all the way through to the UI.

probe/endpoint/reporter.go

+				Addr: c.RemoteAddress.String(),
+			})
+		}
+	}


tomwilkie · 2015-09-01T12:09:11Z

We'll also need unit tests for the resolver.

probe/main.go

@@ -104,7 +104,8 @@ func main() {
 	}

 	var (
-		endpointReporter = endpoint.NewReporter(hostID, hostName, *spyProcs, *useConntrack)
+		revResolver      = endpoint.NewReverseResolver(endpoint.RAddrCacheLen)
+		endpointReporter = endpoint.NewReporter(hostID, hostName, *spyProcs, *useConntrack, revResolver)


peterbourgon · 2015-09-02T08:20:11Z

Just waiting for the WIP prefix to go away, and those two checkboxes to be checked :)

probe/endpoint/resolver.go

+	throttle := time.Tick(time.Second / 10)
+	for address := range r.addresses {
+		<-throttle // rate limit our DNS resolutions
+		names, err := net.LookupAddr(address)


inercia · 2015-09-03T07:36:32Z

@tomwilkie what should I do with the reverse address in detailed_node.go? should I replace IPs by names when available? or maybe should I show those names in some other way?

tomwilkie · 2015-09-03T08:11:01Z

@inercia yes I'd try and replace IP addresses in the connections table, in connectionDetailsRows()

https://github.com/weaveworks/scope/blob/master/render/detailed_node.go#L206

inercia · 2015-09-03T09:18:21Z

@tomwilkie the problem with doing a s/IP/name/ is that some names are rather long (eg, db3msgr5011705.gateway.messenger.live.com., c2-54-163-163-128.compute-1.amazonaws.com.) so I'm concerned about the result in the UI...

tomwilkie · 2015-09-03T09:26:05Z

The UI will truncate them and offer tooltips (recent change just went in, you'll need to rebase).

I wouldn't be too worries about it.

probe/endpoint/resolver.go

+	for address := range r.addresses {
+		<-throttle                     // rate limit our DNS resolutions
+		_, err := r.cache.Get(address) // and check if the answer is already in the cache
+		if err == nil {


Add nodes for the remote side of connections iff we have a DNS reverse resolution for the IP. Unit test for the resolver

probe/endpoint/resolver.go

+		// we trigger a asynchronous reverse resolution when not cached
+		select {
+		case r.addresses <- request:
+			if wait {


tomwilkie · 2015-09-05T19:40:48Z

I just put in a few small changes to address some of my own review feedback...

probe/endpoint/resolver_test.go

+func TestReverseResolver(t *testing.T) {
+	tests := map[string]string{
+		"1.2.3.4": "test.domain.name",
+		"4.3.2.1": "im.a.little.tea.pot",


peterbourgon · 2015-09-07T08:36:52Z

LGTM!

reverse resolutions for remote side of connections.

inercia force-pushed the scope-364 branch from 171e978 to d9a8633 Compare August 27, 2015 12:42

tomwilkie reviewed Aug 27, 2015
View reviewed changes

inercia force-pushed the scope-364 branch 4 times, most recently from b1852fc to 917a48a Compare August 28, 2015 10:17

inercia self-assigned this Aug 28, 2015

tomwilkie reviewed Sep 1, 2015
View reviewed changes

probe/endpoint/reporter.go

Addr: c.RemoteAddress.String(),

})

}

}

This comment was marked as abuse.

Sign in to view

This comment was marked as abuse.

Sign in to view

This comment was marked as abuse.

Sign in to view

inercia force-pushed the scope-364 branch from 917a48a to f560ea7 Compare September 1, 2015 16:43

inercia mentioned this pull request Sep 1, 2015

Probe is using 40% cpu #284

Closed

inercia force-pushed the scope-364 branch from f560ea7 to 160e0e7 Compare September 1, 2015 16:55

tomwilkie reviewed Sep 1, 2015
View reviewed changes

inercia force-pushed the scope-364 branch from 160e0e7 to 5bcceb2 Compare September 1, 2015 20:43

inercia force-pushed the scope-364 branch from 5bcceb2 to 460bedd Compare September 2, 2015 09:39

tomwilkie reviewed Sep 2, 2015
View reviewed changes

peterbourgon reviewed Sep 3, 2015
View reviewed changes

peterbourgon assigned inercia and unassigned peterbourgon Sep 3, 2015

inercia force-pushed the scope-364 branch 3 times, most recently from 5509503 to 8c817d1 Compare September 4, 2015 15:00

New asynchronous, caching DNS resolver for reverse resolutions

15e25ed

Add nodes for the remote side of connections iff we have a DNS reverse resolution for the IP. Unit test for the resolver

inercia force-pushed the scope-364 branch from 8c817d1 to 15e25ed Compare September 4, 2015 15:22

inercia assigned peterbourgon and unassigned inercia Sep 4, 2015

tomwilkie reviewed Sep 4, 2015
View reviewed changes

probe/endpoint/resolver.go

// we trigger a asynchronous reverse resolution when not cached

select {

case r.addresses <- request:

if wait {

This comment was marked as abuse.

Sign in to view

Some review feedback.

ad6702a

peterbourgon reviewed Sep 7, 2015
View reviewed changes

probe/endpoint/resolver_test.go

func TestReverseResolver(t *testing.T) {

tests := map[string]string{

"1.2.3.4": "test.domain.name",

"4.3.2.1": "im.a.little.tea.pot",

This comment was marked as abuse.

Sign in to view

Wrap comments at 80col throughout the fileset

7513b4e

peterbourgon assigned inercia and unassigned peterbourgon Sep 7, 2015

inercia added a commit that referenced this pull request Sep 7, 2015

Merge pull request #404 from weaveworks/scope-364

d3c7138

reverse resolutions for remote side of connections.

inercia merged commit d3c7138 into master Sep 7, 2015

peterbourgon deleted the scope-364 branch September 7, 2015 08:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

reverse resolutions for remote side of connections. #404

reverse resolutions for remote side of connections. #404

inercia commented Aug 27, 2015

This comment was marked as abuse.

This comment was marked as abuse.

tomwilkie commented Sep 1, 2015

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

tomwilkie commented Sep 1, 2015

This comment was marked as abuse.

This comment was marked as abuse.

peterbourgon commented Sep 2, 2015

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

inercia commented Sep 3, 2015

tomwilkie commented Sep 3, 2015

inercia commented Sep 3, 2015

tomwilkie commented Sep 3, 2015

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

tomwilkie commented Sep 5, 2015

This comment was marked as abuse.

peterbourgon commented Sep 7, 2015

reverse resolutions for remote side of connections. #404

reverse resolutions for remote side of connections. #404

Conversation

inercia commented Aug 27, 2015

This comment was marked as abuse.

This comment was marked as abuse.

tomwilkie commented Sep 1, 2015

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

tomwilkie commented Sep 1, 2015

This comment was marked as abuse.

This comment was marked as abuse.

peterbourgon commented Sep 2, 2015

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

inercia commented Sep 3, 2015

tomwilkie commented Sep 3, 2015

inercia commented Sep 3, 2015

tomwilkie commented Sep 3, 2015

This comment was marked as abuse.

This comment was marked as abuse.

This comment was marked as abuse.

tomwilkie commented Sep 5, 2015

This comment was marked as abuse.

peterbourgon commented Sep 7, 2015