No codis-config binary in 3.0.3

[Roycohen]

Hi Guys,

I'm installing codis with GO 1.7 and I've successfully compiled it with all dependencies.
First there are issues resolving something in the Makefile but I could managed to resolve it.

Now after everything is compiled I have the follwoing files under bin directory:
drwxr-xr-x. 4 root root 4096 Jul 4 16:10 assets
-rwxr-xr-x. 1 root root 15041943 Jul 4 16:10 codis-admin
-rwxr-xr-x. 1 root root 16271124 Jul 4 16:10 codis-dashboard
-rwxr-xr-x. 1 root root 8843404 Jul 4 16:10 codis-fe
-rwxr-xr-x. 1 root root 8620289 Jul 4 16:10 codis-ha
-rwxr-xr-x. 1 root root 10079739 Jul 4 16:10 codis-proxy
-rwxr-xr-x. 1 root root 6312957 Jul 4 16:08 codis-server
-rw-r--r--. 1 root root 151 Jul 4 16:08 version

Where and how I can compile the codis-config.
Do I need it in this version?

[spinlock]

codis-config has been removed from codis3.x. Please use codis-dashboard and codis-fe instead.
codis-dashboard is equivalent to codis-config --dashboard and codis-fe is the frond end of codis-dashboad.

You can find a bash script named demo.sh under folder scripts. It will setup a full featured codis-cluster for demo purpose.

[Roycohen]

Thanks you very much for the detailed answer.
I will go over it and Let You Know how it is working for me.

Where I can find English Documentation for the new release?
I'm eager to use codis for a long time and now is the time for me to make the switch :)

[Roycohen]

Do I have to install ZooKeeper and etcd?

[spinlock]

You can choose either one of them.

I prefer etcd for demo purpose. You can clone it from github, and compile it with script build.sh (can be found under its repo). Then the binary files etcd and etcdctl will be generated to bin folder. You can copy them to any where you want (e.g. $GOPATH/bin).

[Roycohen]

Great.
I already manage to solve this.
Now, What is the dashboard port.
I can log in to several ports but I only see JSON.

In addition, what the demo config?
And what is the use of etcd in this case? To manage to configurations?

[spinlock]

sorry, these's no english document for this release yet. so I wrote some comments for you.

#!/bin/bash

PATH=$PATH:`realpath ../bin`

# preapre step 1.
# choose etcd as default coordinator
for x in etcd codis-dashboard codis-proxy codis-admin codis-server; do
    which $x >/dev/null
    if [ $? -ne 0 ]; then
        echo "missing $x"
        exit 1
    fi
done

# prepare step 2.
# create a temp folder for demo (logs & configs)
rm -rf tmp; mkdir -p tmp && pushd tmp
if [ $? -ne 0 ]; then
    echo "pushd failed"
    exit 1
fi

trap "kill 0" EXIT SIGQUIT SIGKILL SIGTERM

# setup 1.
# run etcd with default configuration (listen on port 2379)
nohup etcd --name=codis-test &>etcd.log &
lastpid=$!
pidlist=$lastpid
echo "etcd.pid=$lastpid"

# setup 2.
# create 8 codis-server instances
#   + listen on port 16379 ~ 16386
for ((i=0;i<8;i++)); do
    let p="16379+i"
    nohup codis-server --port ${p} &>redis-${p}.log &
    lastpid=$!
    pidlist="$pidlist $lastpid"
    echo "codis-server-${p}.pid=$lastpid"
done

# setup 3.
# create 4 codis-proxy instances with default config
#   + listen on port 19000 ~ 19003 for redis-client
#   + listen on port 11080 ~ 11083 for codis-cluster (restful api)
#       - can be visited by codis-dashboard or codis-admin
#   + run `codis-proxy -h` for help
#       - default config can be generated by `codis-proxy --default-config`
for ((i=0;i<4;i++)); do
    let p1="11080+i"
    let p2="19000+i"
    cat > ${p1}.toml <<EOF
product_name = "codis-test"
product_auth = ""
proto_type = "tcp4"
admin_addr = "0.0.0.0:${p1}"
proxy_addr = "0.0.0.0:${p2}"
EOF
    nohup codis-proxy -c ${p1}.toml &>${p1}.log &
    lastpid=$!
    pidlist="$pidlist $lastpid"
    echo "proxy-${p1}x${p2}.pid=$lastpid"
done

# setup 4.
# create 1 codis-dashboard instance with default config (with etcd)
#   + listen on port 18080 for codis-fe
cat > dashboard.toml <<EOF
coordinator_name = "etcd"
coordinator_addr = "127.0.0.1:2379"
product_name = "codis-test"
product_auth = ""
admin_addr = "0.0.0.0:18080"
EOF

nohup codis-dashboard -c dashboard.toml &> dashboard.log &
lastpid=$!
pidlist="$pidlist $lastpid"
echo "dashboard.pid=$lastpid"

# setup 5.
# setup codis-fe for codis-dashboard
cat > codis.json <<EOF
[
    {
        "name": "codis-test",
        "dashboard": "127.0.0.1:18080"
    }
]
EOF

nohup ../../bin/codis-fe -d codis.json --listen 0.0.0.0:8080 &> fe.log &
lastpid=$!
pidlist="$pidlist $lastpid"
echo "fe.pid=$lastpid"

sleep 3

for pid in $pidlist; do
    ps -p $pid >/dev/null
    if [ $? -ne 0 ]; then
        echo "pid=$pid not found"
        exit 1
    fi
done

codis_admin() {
    codis-admin --dashboard=127.0.0.1:18080 $@
    if [ $? -ne 0 ]; then
        echo "codis-admin error: $@"
        exit 1
    fi
}

# setup 6.
# create 4 groups (group-1 ~ group-4)
for ((i=0;i<4;i++)); do
    let g="i+1"
    codis_admin --create-group --gid $g
done

# setup 7.
# 2 codis-server for each group (1master + 1slave)
for ((i=0;i<8;i++)); do
    let p="16379+i"
    let g="i/2+1"
    codis_admin --group-add --gid $g -x 127.0.0.1:${p}
done

# setup 8.
# add codis-proxy instances to codis-cluster
for ((i=0;i<4;i++)); do
    let p1="11080+i"
    codis_admin --create-proxy -x 127.0.0.1:${p1}
done

# setup 9.
# setup slots (auto-rebalance 1024 slots into 4 groups)
codis_admin --slot-action --interval=100
codis_admin --rebalance --confirm

echo done

# setup 10.
# all done!!
# please visit codis-fe (localhost:8080) with your browser
#    + once the auto-rebalace phase has been completed, you can run redis-benchmark or anything else you want

while true; do
    date
    sleep 60
done

[spinlock]

here's the architecture of codis3, https://raw.githubusercontent.com/CodisLabs/codis/release3.0/doc/pictures/architecture.png

we use zookeeper & etcd as external storage to save the configs of codis-clusters (such as slot mappings, proxy instances and group servers, etc .,)

[Roycohen]

@spinlock THANKS YOU VERY MUCH!!!
You are much appreciated.
I will continue with my tests and report if I have questions or issues.

Questions that pops out are:

1. Redis Client is the application layer client? PHP client or JAVA client or any client?
2. listen on port 11080 ~ 11083 for codis-cluster (restful api)?? Whta does that means? What is codis-cluster? Whta is the purpose of that.
3. Probably i need to connect to the proxy, so what is codis-cluster? in addition, if i have several proxies i need to make round-robin or any other method myself to decide to which proxy to connect? or there is any other method?

NOTE:
I've just tried to use rename method from PHP client and I'm getting an error. "read error on connection"
Is rename method is supported?

[Roycohen]

I'm getting this error when trying to use:
codis-admin --dashboard=127.0.0.1:18080 --create-group --gid 1
2016/07/06 17:50:12 dashboard.go:394: [PANIC] call rpc create-group to dashboard 127.0.0.1:18080 failed

It seems like everything is fine but the codis-admin command doesn't work.

When trying to log in to web I get 403 forbidden.
netstat -nap | grep 8080
tcp 0 0 :::18080 :::* LISTEN 21134/codis-dashboa
tcp 0 0 :::8080 :::* LISTEN 21159/codis-fe
tcp 0 0 ::ffff:192.168.56.4:8080 ::ffff:192.168.56.2:49978 ESTABLISHED 21159/codis-fe

[spinlock]

Hi @Roycohen

Q1: Yep, you can choose any client that implements the RESP protocol. And here is the list of the recommended clients from the official website.
Please note that, codis only supports a subset of redis commands, here is the blacklist.

Q2: codis-proxy will open 2 different tcp ports, you can find their definitions from its config file: one for your client program; and the second one, namely admin_addr, can be used to communicate with codis-dashboard or codis-admin to share proxy's status or sync slot mappings, group servers, etc.,

Q3: codis-cluster means the codis-dashbard + codis-proxy(s) + codis-server(s). If you want to use multiple proxies, jodis may be your best choice. And we'll publish a go version of jodis as soon as possible.

Q4: please check the log of codis-dashboard and the detail of the panic message, it'll tell you some reasons. I think it was probably caused by 'group-1 already exists'.

[spinlock]

@Roycohen
codis supports most of the redis commands, including pipeline. but it doesn't support transactions or any other command which will block the tcp connection.
once the codis-proxy detects any invalid commands from client or any failure from backend codis-server, it'll reset the client's connection. please check your proxy's log file, it will tell you more information.

[Roycohen]

I have issues reloading dashboard if I don't delete the old data in etcd.

2016/07/10 11:23:35 topom.go:121: [ERROR] store: acquire lock of aniview-realtime-analytics failed
[error]: 105: Key already exists (/codis3/aniview-realtime-analytics/topom) [8]
    4   /usr/src/go/src/github.com/CodisLabs/codis/pkg/models/etcd/etcdclient.go:133
            github.com/CodisLabs/codis/pkg/models/etcd.(*EtcdClient).Create
    3   /usr/src/go/src/github.com/CodisLabs/codis/pkg/models/store.go:63
            github.com/CodisLabs/codis/pkg/models.(*Store).Acquire
    2   /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:120
            github.com/CodisLabs/codis/pkg/topom.(*Topom).setup
    1   /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:95
            github.com/CodisLabs/codis/pkg/topom.New
    0   /usr/src/go/src/github.com/CodisLabs/codis/cmd/dashboard/main.go:116
            main.main
        ... ...
[stack]:
    2   /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:121
            github.com/CodisLabs/codis/pkg/topom.(*Topom).setup
    1   /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:95
            github.com/CodisLabs/codis/pkg/topom.New
    0   /usr/src/go/src/github.com/CodisLabs/codis/cmd/dashboard/main.go:116
            main.main
        ... ...
2016/07/10 11:23:35 main.go:118: [PANIC] create topom with config file failed
coordinator_name = "etcd"
coordinator_addr = "127.0.0.1:2379"
admin_addr = "0.0.0.0:18080"
product_name = "aniview-realtime-analytics"
product_auth = ""
[error]: store: acquire lock of aniview-realtime-analytics failed
    2   /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:122
            github.com/CodisLabs/codis/pkg/topom.(*Topom).setup
    1   /usr/src/go/src/github.com/CodisLabs/codis/pkg/topom/topom.go:95
            github.com/CodisLabs/codis/pkg/topom.New
    0   /usr/src/go/src/github.com/CodisLabs/codis/cmd/dashboard/main.go:116
            main.main
        ... ...
[stack]:
    0   /usr/src/go/src/github.com/CodisLabs/codis/cmd/dashboard/main.go:118
            main.main
        ... ...

[spinlock]

@Roycohen you can remove the lock node manually with command codis-admin --remove-lock.

[Roycohen]

@spinlock thank you very much.
Do you know why I might have 403 forbidden?
From the logs it seems like everything was initialized as it should.

[spinlock]

@Roycohen what's your error message?

It may be caused by some failures while loading the css resources.
Please make sure that codis-fe and the folder assets are under the same directory and try again.