Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(router): implement router #10

Merged
merged 1 commit into from
Nov 10, 2017
Merged

feat(router): implement router #10

merged 1 commit into from
Nov 10, 2017

Conversation

kdada
Copy link
Collaborator

@kdada kdada commented Oct 20, 2017

Implement regexp router.

  • Support regexp router.
  • Support Middleware.
Path examples:
/segments/segment/resources/resource
/segments/{segment}/resources/{resource}
/segments/{segment:[a-z]{1,2}}.log/resources/{resource}
/segments/{segment:[a-z]{1,2}}.log/resources/{resource}/{path:*}
/segments/{segment:[a-z]{1,2}}.log{temp:.*}sss/paths/{path:*}  

Performance:

goos: linux
goarch: amd64
pkg: github.com/kdada/go-http-routing-benchmark
BenchmarkNirvana_GithubStatic     	10000000	       119 ns/op	       0 B/op	       0 allocs/op
BenchmarkAce_GithubStatic         	20000000	        95.7 ns/op	       0 B/op	       0 allocs/op
BenchmarkBear_GithubStatic        	 5000000	       328 ns/op	     120 B/op	       3 allocs/op
BenchmarkBeego_GithubStatic       	 1000000	      1130 ns/op	     352 B/op	       3 allocs/op
BenchmarkBone_GithubStatic        	  200000	      9363 ns/op	    2880 B/op	      60 allocs/op
BenchmarkDenco_GithubStatic       	50000000	        29.4 ns/op	       0 B/op	       0 allocs/op
BenchmarkEcho_GithubStatic        	10000000	       171 ns/op	      32 B/op	       1 allocs/op
BenchmarkGin_GithubStatic         	20000000	        63.4 ns/op	       0 B/op	       0 allocs/op
BenchmarkGocraftWeb_GithubStatic  	 3000000	       566 ns/op	     296 B/op	       5 allocs/op
BenchmarkGoji_GithubStatic        	10000000	       152 ns/op	       0 B/op	       0 allocs/op
BenchmarkGojiv2_GithubStatic      	 1000000	      2201 ns/op	    1312 B/op	      10 allocs/op
BenchmarkGoRestful_GithubStatic   	  100000	     11422 ns/op	    4392 B/op	      24 allocs/op
BenchmarkGoJsonRest_GithubStatic  	 2000000	       845 ns/op	     329 B/op	      11 allocs/op
BenchmarkGorillaMux_GithubStatic  	  200000	     11018 ns/op	     976 B/op	       9 allocs/op
BenchmarkHttpRouter_GithubStatic  	50000000	        34.4 ns/op	       0 B/op	       0 allocs/op
BenchmarkHttpTreeMux_GithubStatic 	30000000	        43.0 ns/op	       0 B/op	       0 allocs/op
BenchmarkKocha_GithubStatic       	30000000	        48.4 ns/op	       0 B/op	       0 allocs/op
BenchmarkLARS_GithubStatic        	20000000	        61.8 ns/op	       0 B/op	       0 allocs/op
BenchmarkMacaron_GithubStatic     	 1000000	      1431 ns/op	     720 B/op	       8 allocs/op
BenchmarkMartini_GithubStatic     	  200000	     10475 ns/op	     768 B/op	       9 allocs/op
BenchmarkPat_GithubStatic         	  200000	     10474 ns/op	    3648 B/op	      76 allocs/op
BenchmarkPossum_GithubStatic      	 2000000	       671 ns/op	     416 B/op	       3 allocs/op
BenchmarkR2router_GithubStatic    	 5000000	       351 ns/op	     144 B/op	       4 allocs/op
BenchmarkRevel_GithubStatic       	 1000000	      3066 ns/op	    1408 B/op	      26 allocs/op
BenchmarkRivet_GithubStatic       	20000000	        74.9 ns/op	       0 B/op	       0 allocs/op
BenchmarkTango_GithubStatic       	 2000000	       788 ns/op	     248 B/op	       8 allocs/op
BenchmarkTigerTonic_GithubStatic  	10000000	       184 ns/op	      48 B/op	       1 allocs/op
BenchmarkTraffic_GithubStatic     	   50000	     33813 ns/op	   22776 B/op	     166 allocs/op
BenchmarkVulcan_GithubStatic      	 2000000	       590 ns/op	      98 B/op	       3 allocs/op


BenchmarkNirvana_GithubParam      	10000000	       206 ns/op	       0 B/op	       0 allocs/op
BenchmarkAce_GithubParam          	 5000000	       239 ns/op	      96 B/op	       1 allocs/op
BenchmarkBear_GithubParam         	 2000000	       780 ns/op	     496 B/op	       5 allocs/op
BenchmarkBeego_GithubParam        	 2000000	       984 ns/op	     352 B/op	       3 allocs/op
BenchmarkBone_GithubParam         	  300000	      5095 ns/op	    1888 B/op	      19 allocs/op
BenchmarkDenco_GithubParam        	10000000	       231 ns/op	     128 B/op	       1 allocs/op
BenchmarkEcho_GithubParam         	10000000	       237 ns/op	      32 B/op	       1 allocs/op
BenchmarkGin_GithubParam          	20000000	       106 ns/op	       0 B/op	       0 allocs/op
BenchmarkGocraftWeb_GithubParam   	 1000000	      1091 ns/op	     712 B/op	       9 allocs/op
BenchmarkGoji_GithubParam         	 2000000	       740 ns/op	     336 B/op	       2 allocs/op
BenchmarkGojiv2_GithubParam       	 1000000	      1839 ns/op	    1408 B/op	      13 allocs/op
BenchmarkGoJsonRest_GithubParam   	 1000000	      1406 ns/op	     713 B/op	      14 allocs/op
BenchmarkGoRestful_GithubParam    	  200000	     11469 ns/op	    2824 B/op	      22 allocs/op
BenchmarkGorillaMux_GithubParam   	  200000	      6669 ns/op	    1296 B/op	      10 allocs/op
BenchmarkHttpRouter_GithubParam   	10000000	       172 ns/op	      96 B/op	       1 allocs/op
BenchmarkHttpTreeMux_GithubParam  	 2000000	       648 ns/op	     384 B/op	       4 allocs/op
BenchmarkKocha_GithubParam        	 5000000	       382 ns/op	     128 B/op	       5 allocs/op
BenchmarkLARS_GithubParam         	20000000	        96.4 ns/op	       0 B/op	       0 allocs/op
BenchmarkMacaron_GithubParam      	 1000000	      1828 ns/op	    1056 B/op	      10 allocs/op
BenchmarkMartini_GithubParam      	  200000	      7998 ns/op	    1152 B/op	      11 allocs/op
BenchmarkPat_GithubParam          	  200000	      7300 ns/op	    2464 B/op	      48 allocs/op
BenchmarkPossum_GithubParam       	 2000000	       875 ns/op	     544 B/op	       5 allocs/op
BenchmarkR2router_GithubParam     	 2000000	       623 ns/op	     432 B/op	       5 allocs/op
BenchmarkRevel_GithubParam        	  500000	      3871 ns/op	    1904 B/op	      31 allocs/op
BenchmarkRivet_GithubParam        	 5000000	       277 ns/op	      96 B/op	       1 allocs/op
BenchmarkTango_GithubParam        	 1000000	      1173 ns/op	     472 B/op	      11 allocs/op
BenchmarkTigerTonic_GithubParam   	  500000	      2983 ns/op	    1440 B/op	      24 allocs/op
BenchmarkTraffic_GithubParam      	  200000	     11505 ns/op	    6952 B/op	      56 allocs/op
BenchmarkVulcan_GithubParam       	 2000000	       902 ns/op	      98 B/op	       3 allocs/op


BenchmarkNirvana_GithubAll        	   30000	     43178 ns/op	       0 B/op	       0 allocs/op
BenchmarkAce_GithubAll            	   30000	     47501 ns/op	   13792 B/op	     167 allocs/op
BenchmarkBear_GithubAll           	   10000	    155288 ns/op	   86448 B/op	     943 allocs/op
BenchmarkBeego_GithubAll          	   10000	    198875 ns/op	   71456 B/op	     609 allocs/op
BenchmarkBone_GithubAll           	    1000	   2007746 ns/op	  720160 B/op	    8620 allocs/op
BenchmarkDenco_GithubAll          	   30000	     44171 ns/op	   20224 B/op	     167 allocs/op
BenchmarkEcho_GithubAll           	   30000	     48986 ns/op	    6496 B/op	     203 allocs/op
BenchmarkGin_GithubAll            	  100000	     20937 ns/op	       0 B/op	       0 allocs/op
BenchmarkGocraftWeb_GithubAll     	   10000	    212242 ns/op	  131656 B/op	    1686 allocs/op
BenchmarkGoji_GithubAll           	    5000	    316694 ns/op	   56112 B/op	     334 allocs/op
BenchmarkGojiv2_GithubAll         	    3000	    555556 ns/op	  352720 B/op	    4321 allocs/op
BenchmarkGoJsonRest_GithubAll     	   10000	    280764 ns/op	  134371 B/op	    2737 allocs/op
BenchmarkGoRestful_GithubAll      	    1000	   2469835 ns/op	  912696 B/op	    4868 allocs/op
BenchmarkGorillaMux_GithubAll     	     300	   4007698 ns/op	  251648 B/op	    1994 allocs/op
BenchmarkHttpRouter_GithubAll     	   50000	     31013 ns/op	   13792 B/op	     167 allocs/op
BenchmarkHttpTreeMux_GithubAll    	   10000	    118069 ns/op	   65856 B/op	     671 allocs/op
BenchmarkKocha_GithubAll          	   20000	     77520 ns/op	   23304 B/op	     843 allocs/op
BenchmarkLARS_GithubAll           	  100000	     19292 ns/op	       0 B/op	       0 allocs/op
BenchmarkMacaron_GithubAll        	   10000	    287076 ns/op	  146161 B/op	    1624 allocs/op
BenchmarkMartini_GithubAll        	     500	   3335191 ns/op	  226547 B/op	    2325 allocs/op
BenchmarkPat_GithubAll            	     500	   3283946 ns/op	 1499568 B/op	   27435 allocs/op
BenchmarkPossum_GithubAll         	   10000	    129200 ns/op	   84448 B/op	     609 allocs/op
BenchmarkR2router_GithubAll       	   10000	    117080 ns/op	   77328 B/op	     979 allocs/op
BenchmarkRevel_GithubAll          	    2000	    720247 ns/op	  369920 B/op	    6121 allocs/op
BenchmarkRivet_GithubAll          	   30000	     53120 ns/op	   16272 B/op	     167 allocs/op
BenchmarkTango_GithubAll          	   10000	    202611 ns/op	   85451 B/op	    2064 allocs/op
BenchmarkTigerTonic_GithubAll     	    2000	    560224 ns/op	  239104 B/op	    5374 allocs/op
BenchmarkTraffic_GithubAll        	     300	   5195233 ns/op	 3097761 B/op	   23742 allocs/op
BenchmarkVulcan_GithubAll         	   10000	    151446 ns/op	   19894 B/op	     609 allocs/op

@jimexist
Copy link
Contributor

考虑一下 edge case:emoji 🍆 in path


// AddExecutor adds executor to the router node.
// A router can hold many executors, but there is only one executor
// is selected for a match.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove is

// FindStringRouter find a router by first char.
func (p *Progeny) FindStringRouter(char byte) Router {
length := len(p.StringRouters)
if length <= 3 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 is not a good heuristic. maybe try 20-50? linear search is not that slow, you got space locality and line cache.

Copy link
Collaborator Author

@kdada kdada Oct 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested it with:

package main

import (
	"fmt"
	"sort"
	"time"
)

type CharRouter struct {
	Char    byte
	Pointer *int
}
type Progeny struct {
	StringRouters []CharRouter
}

func (p *Progeny) FindStringRouter(char byte) bool {
	length := len(p.StringRouters)
	if length <= 0 {
		return false
	}
	index := sort.Search(len(p.StringRouters), func(i int) bool {
		return char <= p.StringRouters[i].Char
	})
	return index < length && char == p.StringRouters[index].Char
}

func (p *Progeny) Find(char byte) bool {
	for _, cr := range p.StringRouters {
		if cr.Char == char {
			return true
		}
	}
	return false
}

func main() {
	p := &Progeny{
		StringRouters: make([]CharRouter, 0, 256),
	}
	for i := 0; i < 256; i++ {
		p.StringRouters = append(p.StringRouters, CharRouter{byte(i), nil})
		if i >= 0 {
			Test(p, 1000000)
		}
	}
}

func Test(p *Progeny, count int) {
	length := len(p.StringRouters)
	counter := 0
	totalBSR := int64(0)
	totalLSR := int64(0)
	for i := 0; i < length; i++ {
		bsr := TestBS(p, byte(i), count)
		lsr := TestLS(p, byte(i), count)
		if bsr < lsr {
			counter++
		}
		totalBSR += bsr
		totalLSR += lsr
	}
	fmt.Printf("Len: %d  \t BS:%d, %.2f%%  \t Total: BS(%d), LS(%d)\n", length, counter, float64(counter)*100/float64(length), totalBSR, totalLSR)
}

func TestBS(p *Progeny, char byte, count int) int64 {
	start := time.Now().UnixNano()
	for i := 0; i < count; i++ {
		p.FindStringRouter(char)
	}
	return time.Now().UnixNano() - start
}

func TestLS(p *Progeny, char byte, count int) int64 {
	start := time.Now().UnixNano()
	for i := 0; i < count; i++ {
		p.Find(char)
	}
	return time.Now().UnixNano() - start
}

The result is:

...
Len: 101         BS:44, 43.56%           Total: BS(2081710163), LS(1990178100)
Len: 102         BS:46, 45.10%           Total: BS(2103198775), LS(2023960297)
Len: 103         BS:46, 44.66%           Total: BS(2118603745), LS(2056275622)
Len: 104         BS:47, 45.19%           Total: BS(2148520686), LS(2104218474)
Len: 105         BS:48, 45.71%           Total: BS(2180732642), LS(2159296633)
Len: 106         BS:49, 46.23%           Total: BS(2208165772), LS(2206370126)
Len: 107         BS:50, 46.73%           Total: BS(2272162642), LS(2313611472)
Len: 108         BS:51, 47.22%           Total: BS(2309499809), LS(2336858570)
Len: 109         BS:51, 46.79%           Total: BS(2306542632), LS(2342357262)
Len: 110         BS:54, 49.09%           Total: BS(2322731078), LS(2388230653)
Len: 111         BS:53, 47.75%           Total: BS(2315820134), LS(2392378395)
Len: 112         BS:54, 48.21%           Total: BS(2331587726), LS(2437614218)
Len: 113         BS:55, 48.67%           Total: BS(2413778245), LS(2549104883)
Len: 114         BS:57, 50.00%           Total: BS(2457508030), LS(2589879197)
Len: 115         BS:57, 49.57%           Total: BS(2446028708), LS(2591940396)
Len: 116         BS:60, 51.72%           Total: BS(2517156756), LS(2704493175)
...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that 106 is the boundary.

index := 0
if length > 0 {
index = sort.Search(length, func(i int) bool {
return c < p.StringRouters[i].Char
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

where's the discrepancy between

c < p.StringRouters[i].Char

and

char <= p.StringRouters[i].Char

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An array:

[a, b, c, d, e, f]

if c is 'd', c < p.StringRouters[i].Char takes index 4, char <= p.StringRouters[i].Char is 3.

if index >= length {
p.StringRouters = append(p.StringRouters, cr)
} else {
p.StringRouters = append(p.StringRouters[:index+1], p.StringRouters[index:]...)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why flip?

Copy link
Collaborator Author

@kdada kdada Oct 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not flip. It just moves [index:end] to [index+1:end+1]. Then insert cr to [index].

"reflect"
"testing"
)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this part needs to be more heavily tested:

  1. unions like: {name:(aaa|bbb)}
  2. regex validity pre-checking: invalid regex like: {name:(a} should not pass
  3. overlaps checking: /path/{name:[a-z]*} and /path/ddy are overlapping

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also does it support inner catch group for url? i hope not:

{name:some-format-(\d+)-caught}

Copy link
Collaborator Author

@kdada kdada Oct 26, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. All valid regular expression is allowed here.
  2. The path {name:(a} will get an error after Parse()
  3. Priority: String Node > Regexp Node > Path Node. So /path/ddy is prior to /path/{name:[a-z]*}.

It supports {name:some-format-(\d+)-caught}, but its value won't be extracted.

@jimexist
Copy link
Contributor

what is the behavior between canonical url and non-canonical urls regarding trailing slash?

/path/to/api v.s. /path/to/api/? which one to use? does it issue HTTP 307 redirect?

@kdada
Copy link
Collaborator Author

kdada commented Oct 26, 2017

考虑一下 edge case:emoji 🍆 in path

@jimexist Supported. Emoji is same as other chars.

@kdada
Copy link
Collaborator Author

kdada commented Oct 26, 2017

what is the behavior between canonical url and non-canonical urls regarding trailing slash?

/path/to/api v.s. /path/to/api/? which one to use? does it issue HTTP 307 redirect?

The two paths is different for router. But we can handle trailing slash in framework.

@jimexist
Copy link
Contributor

jimexist commented Nov 3, 2017

progeny 这个名字比较深奥,可以用一个简单的名字,比如 children?

Copy link
Contributor

@jimexist jimexist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approve to move things along

@walktall
Copy link
Contributor

walktall commented Nov 3, 2017

有很多error可以从fmt.Errorf换成errors.New

// Handler contains middlewares and executor.
type Handler struct {
Middlewares []Middleware
Executor Executor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这里为什么不直接是Executors

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

return &FullMatchRegexpNode{
Key: seg.Keys[0],
}, nil
} else {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这个else没什么意义了,上面都return了

@caitong93
Copy link
Contributor

caitong93 commented Nov 7, 2017

progeny 这个名字比较深奥,可以用一个简单的名字,比如 children?

+1

@kdada

String RouterKind = "String"
// Regexp means the router has a regular expression.
Regexp RouterKind = "Regexp"
// Path means the router matches the rest. Path router only can
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the rest (of what) ?

}

// Merge merges r to the current router. The type of r should be same
// as the current one or it panics.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

没有panic啊?

@walktall
Copy link
Contributor

有些错误处理测试没有覆盖到。

ok github.com/caicloud/nirvana/router 0.009s coverage: 76.5% of statements

This was referenced Nov 10, 2017
@ddysher
Copy link
Member

ddysher commented Nov 10, 2017

LGTM. Discussed offline, this PR is ready to go.

@ddysher ddysher merged commit 2f2339b into caicloud:master Nov 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants