Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问如何将namespace的json配置数据导入到etcd呢? #51

Closed
flappyink opened this issue Sep 17, 2019 · 5 comments
Closed

请问如何将namespace的json配置数据导入到etcd呢? #51

flappyink opened this issue Sep 17, 2019 · 5 comments

Comments

@flappyink
Copy link

flappyink commented Sep 17, 2019

对etcd不是很熟悉,用这样的命令将etc/namespaces下的json串导入到etcd后

./etcdctl put /gaea/file/namespace/test_namespace_1.json "{"name":"test_namespace_1","online":true,"read_only":false,"allowed_dbs":{"test_db":true},"slow_sql_time":"1000","black_sql":[""],"allowed_ip":null,"slices":[{"name":"slice-0","user_name":"root","password":null,"master":"127.0.0.1:3306","slaves":null,"statistic_slaves":null,"capacity":12,"max_capacity":24,"idle_timeout":60}],"shard_rules":[{"db":"test_db","table":"log","type":"date_month","key":"created_at","slices":["slice-0"],"date_range":["201804-201910"]}],"users":[{"user_name":"root","password":"root","namespace":"test_namespace_1","rw_flag":2,"rw_split":1,"other_property":0}],"default_slice":"slice-0","global_sequences":null}"

启动Gaea服务

bogon:Gaea user$ ./bin/gaea -config etc/gaea.ini
Build Version Information:Version: 666d59e9f7ea563733299d97260ee43307881d70
GitRevision: 666d59e9f7ea563733299d97260ee43307881d70
User: user@MacBook-Pro-7.local
GolangVersion: go1.13
BuildStatus: Clean
BuildTime: 2019-09-16--14:08:20

服务无法启动,也没有相关的报错信息,请问应该如何正确的把文本模式下正常工具的namespace配置导入etcd呢?

@teckick
Copy link
Contributor

teckick commented Sep 17, 2019

配置里面的"需要转义?

报错信息可以在日志中查看

@flappyink
Copy link
Author

日志里头显示如下的错误:

[2019-09-17 13:44:40] [gaea_proxy] [MacBook-Pro-7.local] [WARN] [900000001] [github.com/XiaoMi/Gaea/proxy/server.loadAllNamespace:manager.go:69] list namespace failed, err: client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.
[2019-09-17 13:44:40] [gaea_proxy] [MacBook-Pro-7.local] [WARN] [900000001] [github.com/XiaoMi/Gaea/proxy/server.LoadAndCreateManager:manager.go:41] init namespace manager failed, client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.
[2019-09-17 13:44:40] [gaea_proxy] [MacBook-Pro-7.local] [FATAL] [900000001] [main.main:main.go:60] init manager failed, error: client: response is invalid json. The endpoint is probably not valid etcd cluster endpoint.

gaea的配置如下

; config type, etcd/file, you can test gaea with file type, you shoud use etcd in production
config_type=etcd
;file config path, 具体配置放到file_config_path的namespace目录下,该下级目录为固定目录
file_config_path=./etc/file
;coordinator addr
coordinator_addr=http://localhost:2379
;远程配置(当前为etcd)根目录
coordinator_root=/gaea
;etcd user config
username=root
password=root
;environ
environ=local
;service name
service_name=gaea_proxy
;log config
log_path=./logs
log_level=Notice
log_filename=gaea
log_output=file

;admin addr
admin_addr=0.0.0.0:13307
; basic auth
admin_user=admin
admin_password=admin

;proxy addr
proto_type=tcp4
proxy_addr=0.0.0.0:13306
proxy_charset=utf8

;slow sql time, when execute time is higher than this, log it, unit: ms
slow_sql_time=100

;close session after session timeout, unit: seconds
session_timeout=3600

;stats conf
stats_enabled=true

;encrypt key
encrypt_key=1234abcd5678efg*

本地的etcd服务,是最新的3.4.0,跑起来的状态如下所示

MacBook-Pro-7:etcd user$ ./etcdctl --write-out=table endpoint status
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|    ENDPOINT    |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 127.0.0.1:2379 | 8e9e05c52164694d |   3.4.0 |   45 kB |      true |      false |         2 |         20 |                 20 |        |
+----------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+

@flappyink
Copy link
Author

JSON配置的数据是这样导入的,不知道有没有问题

MacBook-Pro-7:etcd user$ ./etcdctl put /gaea/namespace/test_namespace_1.json "{\"name\":\"test_namespace_1\",\"online\":true,\"read_only\":false,\"allowed_dbs\":{\"test\":true},\"slow_sql_time\":\"1000\",\"black_sql\":[\"\"],\"allowed_ip\":null,\"slices\":[{\"name\":\"slice-0\",\"user_name\":\"root\",\"password\":null,\"master\":\"127.0.0.1:3306\",\"slaves\":null,\"statistic_slaves\":null,\"capacity\":12,\"max_capacity\":24,\"idle_timeout\":60}],\"shard_rules\":[{\"db\":\"test\",\"table\":\"log\",\"type\":\"date_month\",\"key\":\"created_at\",\"slices\":[\"slice-0\"],\"date_range\":[\"201804-201910\"]}],\"users\":[{\"user_name\":\"root\",\"password\":\"root\",\"namespace\":\"test_namespace_1\",\"rw_flag\":2,\"rw_split\":1,\"other_property\":0}],\"default_slice\":\"slice-0\",\"global_sequences\":null}"
OK
MacBook-Pro-7:etcd user$ ./etcdctl get /gaea/namespace/test_namespace_1.json
/gaea/namespace/test_namespace_1.json
{"name":"test_namespace_1","online":true,"read_only":false,"allowed_dbs":{"test":true},"slow_sql_time":"1000","black_sql":[""],"allowed_ip":null,"slices":[{"name":"slice-0","user_name":"root","password":null,"master":"127.0.0.1:3306","slaves":null,"statistic_slaves":null,"capacity":12,"max_capacity":24,"idle_timeout":60}],"shard_rules":[{"db":"test","table":"log","type":"date_month","key":"created_at","slices":["slice-0"],"date_range":["201804-201910"]}],"users":[{"user_name":"root","password":"root","namespace":"test_namespace_1","rw_flag":2,"rw_split":1,"other_property":0}],"default_slice":"slice-0","global_sequences":null}

@flappyink
Copy link
Author

flappyink commented Sep 17, 2019

Etcd版本退回到3.3.15解决问题,还有个注意的地方就是往etcd写入namespace配置数据的命令如下

使用V2版本的API,V3的有点小问题

ETCDCTL_API=2 ./etcdctl set /gaea/namespace/test_namespace_1 "{\"name\":\"test_namespace_1\",\"online\":true,\"read_only\":false,\"allowed_dbs\":{\"test\":true},\"slow_sql_time\":\"1000\",\"black_sql\":[\"\"],\"allowed_ip\":null,\"slices\":[{\"name\":\"slice-0\",\"user_name\":\"root\",\"password\":null,\"master\":\"127.0.0.1:3306\",\"slaves\":null,\"statistic_slaves\":null,\"capacity\":12,\"max_capacity\":24,\"idle_timeout\":60}],\"shard_rules\":[{\"db\":\"test\",\"table\":\"log\",\"type\":\"date_month\",\"key\":\"created_at\",\"slices\":[\"slice-0\"],\"date_range\":[\"201804-201910\"]}],\"users\":[{\"user_name\":\"root\",\"password\":\"root\",\"namespace\":\"test_namespace_1\",\"rw_flag\":2,\"rw_split\":1,\"other_property\":0}],\"default_slice\":\"slice-0\",\"global_sequences\":null}"

chicliz pushed a commit that referenced this issue May 26, 2020
panhongrainbow pushed a commit to panhongrainbow/Gaea that referenced this issue Jun 27, 2021
@cxl0704
Copy link

cxl0704 commented Jun 15, 2022

V3版本etcd部署Gaea参考指导:https://blog.caixinglong.cn/archives/hnixyl

需部署gaea-cc,通过api添加namespace,不能通过etcdctl进行添加,因为Gaea存储namespace到etcd中,用户名和密码是加密过的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants