namespace isolation: http interface and classifier #747

dantin · 2017-09-12T07:21:45Z

Configure namespace related info through pd-ctl/api
Add table_namespace_classifier to return namespace name for region/store based on the namespace configuration

sre-bot · 2017-09-12T07:21:47Z

Hi contributor, thanks for your PR.

This patch needs to be approved by someone of admins. They should reply with "/ok-to-test" to accept this PR for running test automatically.

disksing · 2017-09-19T11:38:24Z

pdctl/command/namespace_command.go

+	"github.com/spf13/cobra"
+	"net/http"
+	"strconv"
+)


Please 3rd-part packages from standard packages and leave a blank line between them. We prefer the default output of goimports tool.

disksing · 2017-09-19T11:38:51Z

pdctl/command/namespace_command.go

+func NewAppendTableIDCommand() *cobra.Command {
+	c := &cobra.Command{
+		Use:   "append <name> <table_id>",
+		Short: "add table id in namespace",


disksing · 2017-09-19T11:39:34Z

pdctl/command/namespace_command.go

+		return
+	}
+	if _, err := strconv.Atoi(args[1]); err != nil {
+		fmt.Println("table_id shoud be  a number")


Remove extra space.

disksing · 2017-09-19T11:41:14Z

server/api/namespace.go

+
+// Get list namespace mapping
+func (h *namespaceHandler) Get(w http.ResponseWriter, r *http.Request) {
+	//TODO


Remove redundant line.

disksing · 2017-09-19T11:41:34Z

server/api/namespace.go

+	}
+}
+
+// Get list namespace mapping


list -> lists.

disksing · 2017-09-19T11:57:15Z

server/table_namespace_classifier_test.go

@@ -0,0 +1,101 @@
+// Copyright 2016 PingCAP, Inc.


2016->2017.

disksing · 2017-09-19T12:01:44Z

server/table_namespace_classifier.go

+func (c tableNamespaceClassifier) GetRegionNamespace(regionInfo *core.RegionInfo) string {
+	for name, ns := range c.nsInfo.namespaces {
+		startTable := c.tableIDDecoder.DecodeTableID(regionInfo.StartKey)
+		endTable := c.tableIDDecoder.DecodeTableID(regionInfo.EndKey)


We can move the 2 lines out of for loop. Also we can check if start == end before the loop, if they are not equal, we can return defaultNamespace directly.

disksing · 2017-09-19T12:02:47Z

server/table_namespace_classifier.go

+}
+
+func (c tableNamespaceClassifier) GetAllNamespaces() []string {
+	nsList := []string{}


nsList := make([]string, 0, len(c.nsInfo.namespaces))

disksing · 2017-09-19T12:05:35Z

server/core/tablecodec.go

+	"encoding/binary"
+	"github.com/juju/errors"
+)
+


I guess we can add some references to the table codec implementation on tidb.

@siddontang any comments on this?

disksing · 2017-09-19T12:06:49Z

server/core/namespaces.go

+// 1. relation between a name and several tables
+// 2. relation between a name and several stores
+// It is used to bind tables with stores
+type Namespace struct {


How about moving it to server/table_namespace_classifier.go? Since it is only used by server and server/api package.

siddontang · 2017-09-20T10:59:11Z

server/table_namespace_classifier.go

+// 1. relation between a Name and several tables
+// 2. relation between a Name and several stores
+// It is used to bind tables with stores
+type Namespace struct {


Seem it is TableNamespace, not a Namespace.

Will there be other namespace? How about change table_namespace_classifier.go to namespace_classifier.go?

We may have other namespace implementation later. So we can't assume there is only one table namespace.

siddontang · 2017-09-20T11:00:50Z

server/table_namespace_classifier.go

+	}
+}
+
+func (c tableNamespaceClassifier) GetAllNamespaces() []string {


Will we call this function frequently?
/cc @disksing

Now it is called every time we call scheduler.Schedule().

Seem this is not effective.

siddontang · 2017-09-20T11:01:59Z

server/core/tablecodec.go

+
+type defaultTableIDDecoder struct{}
+
+var tablePrefix = []byte{'t'}


seem we should also consider table index.

@disksing

Just table prefix is ok. We have no plan to separate rows and indexes into different namespaces.

* new GetRegionNamespace Method, checking both StartKey and EndKey * go fmt server/table_namespace_classifier.go * go fmt server/core/tablecodec.go

disksing · 2017-09-25T01:56:49Z

LGTM.

disksing · 2017-09-25T02:03:33Z

PTAL @nolouch @siddontang

nolouch

LGTM

ming535 force-pushed the dantin/namespace branch from 0fa7d11 to e6fb6f8 Compare September 13, 2017 08:40

add basic structure

1199eda

dantin force-pushed the dantin/namespace branch from a8efc47 to 1199eda Compare September 14, 2017 04:17

dantin added 8 commits September 14, 2017 13:58

refactoring using classifer interface.

b772a0d

refactoring logic in HTTP api

2c69c66

implement create/list namespace

4a55ea9

rename functions

6312a0b

rename functions

63d85c1

implements append table id to namespace

0d005bb

implement add store to namespace

c03fea0

Merge branch 'namespace' into dantin/namespace

e5117de

dantin changed the title ~~[WIP] resource isolation~~ namespace isolation: http interface demo Sep 15, 2017

dantin added 2 commits September 15, 2017 18:11

remove checker from http interface.

4ea37d4

revert limit logic

7b85640

dantin requested review from nolouch and disksing September 18, 2017 02:50

ming535 mentioned this pull request Sep 19, 2017

Add table namespace classifier #761

Merged

ming535 changed the title ~~namespace isolation: http interface demo~~ namespace isolation: http interface and classifier Sep 19, 2017

disksing reviewed Sep 19, 2017

View reviewed changes

Add table namespace classifier (#761)

ab84e88

ming535 force-pushed the dantin/namespace branch from 8b018e2 to ab84e88 Compare September 20, 2017 06:26

ming535 added 6 commits September 20, 2017 14:32

Fix goimport for tablecodec.go

4e2cda5

Merge remote-tracking branch 'upstream/namespace' into dantin/namespace

8116e73

Rename checker to replicaChecker

be00ebc

Move namespaceInfo to table_namespace_classifier.go

c034d8c

Some refacotring

1b0c1a4

Add namespace name validation

b87db27

ming535 force-pushed the dantin/namespace branch 2 times, most recently from ede8762 to ae256aa Compare September 20, 2017 10:05

ming535 force-pushed the dantin/namespace branch from ba0b634 to acdf062 Compare September 20, 2017 10:16

Use map for tableIDs and storeIDs in namespace

fb96f63

ming535 force-pushed the dantin/namespace branch from acdf062 to fb96f63 Compare September 20, 2017 10:20

Change tableIDs and storeIDs map value to bool

8a8e1ac

siddontang reviewed Sep 20, 2017

View reviewed changes

dantin and others added 2 commits September 20, 2017 19:47

Only check startKey to avoid empty region

522cc87

rename cache to cached for gocheck

7d855fb

LiangShang force-pushed the dantin/namespace branch from d7622b1 to 7d855fb Compare September 22, 2017 06:50

new GetRegionNamespace (#766)

1ae6402

* new GetRegionNamespace Method, checking both StartKey and EndKey * go fmt server/table_namespace_classifier.go * go fmt server/core/tablecodec.go

nolouch approved these changes Sep 25, 2017

View reviewed changes

disksing merged commit d5f6a6f into namespace Sep 25, 2017

disksing deleted the dantin/namespace branch September 25, 2017 05:28

sre-bot added the contribution Indicates that the PR was contributed by an external member. label Dec 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

namespace isolation: http interface and classifier #747

namespace isolation: http interface and classifier #747

dantin commented Sep 12, 2017 •

edited by ming535

Loading

sre-bot commented Sep 12, 2017

disksing Sep 19, 2017

disksing Sep 19, 2017

disksing Sep 19, 2017

disksing Sep 19, 2017

disksing Sep 19, 2017

disksing Sep 19, 2017

disksing Sep 19, 2017

disksing Sep 19, 2017

disksing Sep 19, 2017

ming535 Sep 20, 2017

disksing Sep 19, 2017

siddontang Sep 20, 2017

ming535 Sep 21, 2017

siddontang Sep 21, 2017

siddontang Sep 20, 2017

disksing Sep 20, 2017

siddontang Sep 21, 2017

siddontang Sep 20, 2017

disksing Sep 20, 2017

disksing commented Sep 25, 2017

disksing commented Sep 25, 2017

nolouch left a comment


		type defaultTableIDDecoder struct{}

		var tablePrefix = []byte{'t'}

namespace isolation: http interface and classifier #747

namespace isolation: http interface and classifier #747

Conversation

dantin commented Sep 12, 2017 • edited by ming535 Loading

sre-bot commented Sep 12, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

disksing commented Sep 25, 2017

disksing commented Sep 25, 2017

nolouch left a comment

Choose a reason for hiding this comment

dantin commented Sep 12, 2017 •

edited by ming535

Loading