Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ftr: Use invoker with same ip as client first. #1023

Merged
merged 22 commits into from Feb 25, 2021
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
b6022bb
fix ctx linter error
AlexStocks Jan 4, 2021
d305b67
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin Jan 10, 2021
5919b11
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin Jan 13, 2021
3dc2f17
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin Jan 17, 2021
55737aa
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin Jan 22, 2021
17c4a9e
Merge pull request #1015 from apache/dependabot/go_modules/develop/gi…
AlexStocks Jan 25, 2021
65cd856
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin Jan 26, 2021
8414d85
feat: add ip level self discovery
LaurenceLiZhixin Jan 27, 2021
512b6c6
fix: change comment
LaurenceLiZhixin Jan 27, 2021
6a6e805
fix: change model to self
LaurenceLiZhixin Jan 27, 2021
1353c33
fix: delete unused gost version
LaurenceLiZhixin Jan 29, 2021
c698ef2
Merge remote-tracking branch 'upstream/develop' into develop
LaurenceLiZhixin Jan 29, 2021
e65c046
Merge branch 'develop' into Ftr/selfDiscovery
LaurenceLiZhixin Jan 29, 2021
b04e7c1
fix: fmt project
LaurenceLiZhixin Jan 29, 2021
17bfc3a
fix: go mod tidy
LaurenceLiZhixin Jan 29, 2021
a5b3ed4
fix: change directory name
LaurenceLiZhixin Jan 29, 2021
5399c8e
fix: delete item in go sun
LaurenceLiZhixin Jan 29, 2021
d826f34
fix
LaurenceLiZhixin Jan 29, 2021
58a3fba
fix: change name to self priority router
LaurenceLiZhixin Jan 31, 2021
81b5d08
fix: change name to self priority router
LaurenceLiZhixin Jan 31, 2021
152d4dd
fix: change name to self priority router
LaurenceLiZhixin Jan 31, 2021
e1b52d0
fix: change name
LaurenceLiZhixin Feb 1, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
43 changes: 43 additions & 0 deletions cluster/router/selfDiscovery/factory.go
@@ -0,0 +1,43 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package self_disc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dont use _ in package name. I think can change to self

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think consistency for directory name and package name will not lead to misunderstanding.

And directory name should not be ‘aB’

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think consistency for directory name and package name will not lead to misunderstanding.

And directory name should not be ‘aB’

fixed


import (
"github.com/apache/dubbo-go/cluster/router"
"github.com/apache/dubbo-go/common"
"github.com/apache/dubbo-go/common/constant"
"github.com/apache/dubbo-go/common/extension"
)

func init() {
extension.SetRouterFactory(constant.SelfDiscoveryRouterName, newSelfDiscRouteFactory)
}

// SelfDiscRouteFactory
type SelfDiscRouteFactory struct {
}

// newSelfDiscRouteFactory construct a new SelfDiscRouteFactory
func newSelfDiscRouteFactory() router.PriorityRouterFactory {
return &SelfDiscRouteFactory{}
}

// NewPriorityRouter construct a new NewSelfDiscRouter via url
func (f *SelfDiscRouteFactory) NewPriorityRouter(url *common.URL) (router.PriorityRouter, error) {
return NewSelfDiscRouter(url)
}
74 changes: 74 additions & 0 deletions cluster/router/selfDiscovery/factory_test.go
@@ -0,0 +1,74 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package self_disc

import (
"context"
"testing"
)

import (
"github.com/stretchr/testify/assert"
)

import (
"github.com/apache/dubbo-go/common"
"github.com/apache/dubbo-go/protocol"
)

// nolint
type MockInvoker struct {
url *common.URL
}

// nolint
func NewMockInvoker(url *common.URL) *MockInvoker {
return &MockInvoker{
url: url,
}
}

// nolint
func (bi *MockInvoker) GetUrl() *common.URL {
return bi.url
}

// nolint
func (bi *MockInvoker) IsAvailable() bool {
return true
}

// nolint
func (bi *MockInvoker) IsDestroyed() bool {
return true
}

// nolint
func (bi *MockInvoker) Invoke(_ context.Context, _ protocol.Invocation) protocol.Result {
return nil
}

// nolint
func (bi *MockInvoker) Destroy() {
}

// nolint
func TestSelfDiscRouteFactory(t *testing.T) {
factory := newSelfDiscRouteFactory()
assert.NotNil(t, factory)
}
106 changes: 106 additions & 0 deletions cluster/router/selfDiscovery/self_disc_route.go
@@ -0,0 +1,106 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package self_disc

import (
"github.com/RoaringBitmap/roaring"
)

import (
"github.com/apache/dubbo-go/cluster/router"
"github.com/apache/dubbo-go/cluster/router/utils"
"github.com/apache/dubbo-go/common"
"github.com/apache/dubbo-go/common/logger"
"github.com/apache/dubbo-go/protocol"
)

const (
selfDesc = "self-desc"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use common.SelfDiscoveryRouterName ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

common.SelfDiscoveryRouterName is to find out registered router.
selfDesc is to find the bitmap of target router in cache.
In order to be consistent with the code design of other routers, I think your suggesting way can be confusing.

name = "self-desc-router"
)

// SelfDiscRouter provides a ip-same-first routing logic
// if there is not provider with same ip as consumer, it would not filter any invoker
// if exists same ip invoker, it would retains this invoker only
type SelfDiscRouter struct {
url *common.URL
localIP string
}

// NewSelfDiscRouter construct an SelfDiscRouter via url
func NewSelfDiscRouter(url *common.URL) (router.PriorityRouter, error) {
r := &SelfDiscRouter{
url: url,
localIP: url.Ip,
}
return r, nil
}

// Route gets a list of match-logic invoker
func (r *SelfDiscRouter) Route(invokers *roaring.Bitmap, cache router.Cache, url *common.URL, invocation protocol.Invocation) *roaring.Bitmap {
addrPool := cache.FindAddrPool(r)
// Add selfDesc invoker to the list
selectedInvokers := utils.JoinIfNotEqual(addrPool[selfDesc], invokers)
// If all invokers are considered not match, downgrade to all invoker
if selectedInvokers.IsEmpty() {
logger.Warnf(" Now all invokers are not match, so downgraded to all! Service: [%s]", url.ServiceKey())
return invokers
}
return selectedInvokers
}

// Pool separates same ip invoker from others.
func (r *SelfDiscRouter) Pool(invokers []protocol.Invoker) (router.AddrPool, router.AddrMetadata) {
rb := make(router.AddrPool, 8)
rb[selfDesc] = roaring.NewBitmap()
selfDescFound := false
for i, invoker := range invokers {
if invoker.GetUrl().Ip == r.localIP {
rb[selfDesc].Add(uint32(i))
selfDescFound = true
}
}
if selfDescFound {
// found self desc
logger.Debug("found self desc ")
return rb, nil
}
for i, _ := range invokers {
rb[selfDesc].Add(uint32(i))
}
return rb, nil
}

// ShouldPool will always return true to make sure self call logic constantly.
func (r *SelfDiscRouter) ShouldPool() bool {
return true
}

func (r *SelfDiscRouter) Name() string {
return name
}

// Priority
func (r *SelfDiscRouter) Priority() int64 {
return 0
}

// URL Return URL in router
func (r *SelfDiscRouter) URL() *common.URL {
return r.url
}
76 changes: 76 additions & 0 deletions cluster/router/selfDiscovery/self_disc_route_test.go
@@ -0,0 +1,76 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

package self_disc

import (
"fmt"
"testing"
)

import (
"github.com/stretchr/testify/assert"
)

import (
"github.com/apache/dubbo-go/cluster/router"
"github.com/apache/dubbo-go/cluster/router/chain"
"github.com/apache/dubbo-go/cluster/router/utils"
"github.com/apache/dubbo-go/common"
"github.com/apache/dubbo-go/protocol"
"github.com/apache/dubbo-go/protocol/invocation"
)

const (
selfDiscRoute1010IP = "192.168.10.10"
selfDiscRoute1011IP = "192.168.10.11"
selfDiscRoute1012IP = "192.168.10.12"
selfDiscRouteMethodNameTest = "test"
selfDiscRouteUrlFormat = "dubbo://%s:20000/com.ikurento.user.UserProvider"
)

func TestSelfDiscRouterRoute(t *testing.T) {
defer protocol.CleanAllStatus()
consumerURL, _ := common.NewURL(fmt.Sprintf(selfDiscRouteUrlFormat, selfDiscRoute1010IP))
url1, _ := common.NewURL(fmt.Sprintf(selfDiscRouteUrlFormat, selfDiscRoute1010IP))
url2, _ := common.NewURL(fmt.Sprintf(selfDiscRouteUrlFormat, selfDiscRoute1011IP))
url3, _ := common.NewURL(fmt.Sprintf(selfDiscRouteUrlFormat, selfDiscRoute1012IP))
hcr, _ := NewSelfDiscRouter(consumerURL)

var invokers []protocol.Invoker
invoker1 := NewMockInvoker(url1)
invoker2 := NewMockInvoker(url2)
invoker3 := NewMockInvoker(url3)
invokers = append(invokers, invoker1, invoker2, invoker3)
inv := invocation.NewRPCInvocation(selfDiscRouteMethodNameTest, nil, nil)
res := hcr.Route(utils.ToBitmap(invokers), setUpAddrCache(hcr.(*SelfDiscRouter), invokers), consumerURL, inv)
// now only same ip invoker is selected
assert.True(t, len(res.ToArray()) == 1)

// now all invoker with ip that not match client are selected
invokers = invokers[1:]
res = hcr.Route(utils.ToBitmap(invokers), setUpAddrCache(hcr.(*SelfDiscRouter), invokers), consumerURL, inv)
assert.True(t, len(res.ToArray()) == 2)
}

func setUpAddrCache(r router.Poolable, addrs []protocol.Invoker) router.Cache {
pool, info := r.Pool(addrs)
cache := chain.BuildCache(addrs)
cache.SetAddrMeta(r.Name(), info)
cache.SetAddrPool(r.Name(), pool)
return cache
}
2 changes: 2 additions & 0 deletions common/constant/key.go
Expand Up @@ -214,6 +214,8 @@ const (
ListenableRouterName = "listenable"
// HealthCheckRouterName Specify the name of HealthCheckRouter
HealthCheckRouterName = "health_check"
// SelfDiscoveryRouterName Specify the name of HealthCheckRouter
SelfDiscoveryRouterName = "self_disc"
// TagRouterName Specify the name of TagRouter
TagRouterName = "tag"
// TagRouterRuleSuffix Specify tag router suffix
Expand Down
2 changes: 1 addition & 1 deletion go.mod
Expand Up @@ -13,7 +13,7 @@ require (
github.com/coreos/etcd v3.3.25+incompatible
github.com/creasty/defaults v1.5.1
github.com/dubbogo/go-zookeeper v1.0.2
github.com/dubbogo/gost v1.10.1
github.com/dubbogo/gost v1.11.0
github.com/elazarl/go-bindata-assetfs v1.0.0 // indirect
github.com/emicklei/go-restful/v3 v3.4.0
github.com/frankban/quicktest v1.4.1 // indirect
Expand Down
4 changes: 4 additions & 0 deletions go.sum
Expand Up @@ -176,6 +176,8 @@ github.com/dubbogo/go-zookeeper v1.0.2/go.mod h1:fn6n2CAEer3novYgk9ULLwAjuV8/g4D
github.com/dubbogo/gost v1.9.0/go.mod h1:pPTjVyoJan3aPxBPNUX0ADkXjPibLo+/Ib0/fADXSG8=
github.com/dubbogo/gost v1.10.1 h1:39kF9Cd5JOiMpmwG6dX1/aLWNFqFv9gHp8HrhzMmjLY=
github.com/dubbogo/gost v1.10.1/go.mod h1:+mQGS51XQEUWZP2JeGZTxJwipjRKtJO7Tr+FOg+72rI=
github.com/dubbogo/gost v1.11.0 h1:9KtyWQz1gMlAfwzen5iyhMdoe08SPBBUVhco4rdgJ9I=
github.com/dubbogo/gost v1.11.0/go.mod h1:w8Yw29eDWtRVo3tx9nPpHkNZnOi4SRx1fZf7eVlAAU4=
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using go mod tidy to delete the useless gost version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

github.com/dubbogo/jsonparser v1.0.1/go.mod h1:tYAtpctvSP/tWw4MeelsowSPgXQRVHHWbqL6ynps8jU=
github.com/dustin/go-humanize v0.0.0-20171111073723-bb3d318650d4/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
Expand Down Expand Up @@ -474,7 +476,9 @@ github.com/jtolds/gls v4.20.0+incompatible h1:xdiiI2gbIgH/gLH7ADydsJ1uDOEzR8yvV7
github.com/jtolds/gls v4.20.0+incompatible/go.mod h1:QJZ7F/aHp+rZTRtaJ1ow/lLfFfVYBRgL+9YlvaHOwJU=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88 h1:uC1QfSlInpQF+M0ao65imhwqKnz3Q2z/d8PWZRMQvDM=
github.com/k0kubun/colorstring v0.0.0-20150214042306-9440f1994b88/go.mod h1:3w7q1U84EfirKl04SVQ/s7nPm1ZPhiXd34z40TNz36k=
github.com/k0kubun/pp v3.0.1+incompatible h1:3tqvf7QgUnZ5tXO6pNAZlrvHgl6DvifjDrd9g2S9Z40=
github.com/k0kubun/pp v3.0.1+incompatible/go.mod h1:GWse8YhT0p8pT4ir3ZgBbfZild3tgzSScAn6HmfYukg=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00=
Expand Down