idea about k8s crd & skywalkingcli

[wayilau]

Please answer these questions before submitting your issue.

• Why do you submit this issue?
• Question or discussion
• Bug
• Requirement
• Feature or performance improvement

Question

As we know, k8s is very popular.
when we are working cli for skywalking. From k8s crd, my idea is that we can use k8s crds to achieve this goal. we can do this directly when skywalking deployed in k8s. we also can achieve this by service catalog when skywalking deployed outside of k8s. From the crd we defined, we can do a lot of things. it is also not conflict with the cli command.

[wu-sheng]

CLI is a stand alone idea. It doesn't need to link to cry.

At the same time, crd is a good goal too. If you want to work on that, @hanahmily and @innerpeacez are the right ones to talk with. We could discuss here or in mail list.

[wayilau]

OK. FYI @hanahmily @innerpeacez
I will give my detail idea later here or in mail list. I also need your help.

[innerpeacez]

@hanahmily seems to have begun to conceive operator. Looking forward to your idea. @wayilau

[wayilau]

SW CRDS & Skywalking

detail idea.

As Skywalking can be deployed on Kubernetes or outside of Kubernetes. Now i am not sure whether we can use union crds to manage both of them. If we don't consider so much, we only use the oap address so we can achieve this. otherwise, we may use service-catalog when Skywalking deployed outside of Kubernetes.
SW crds are easy to defined, same as sw-cli, we also can add other options & parameters, all the same goal they are.
the differences are that sw cli is stand alone tool used in pc directory. but SW crds are working on Kubernetes, which is tool interacted with Kubernetes tightly.

we can use crds to:

• list instances like :

kubectl get swinstance -n ${namespace}

• list instance endpoints list:

kubectl get swendpoints -n ${namespace}

• get single instance

kubectl get swinstance ${name} -n ${namespace}

• list service like:

kubectl  get swservice -n ${namespace}

• list single service like:

kubectl get swservice ${svc-name} -n ${namespace}

• ...

Also we can add a lot option, like 'start' & 'end'.

As sw cli needs a configuration. we can add configuration when we install the crds. from describe command we can see the events of conn to oap. like logs to watch the oap server. Also we can control the event what we want know.

FYI @wu-sheng @innerpeacez @hanahmily

[wu-sheng]

Do you know watching this subproject? https://github.com/apache/skywalking-cli
Could you define what is the difference between these two?

[wayilau]

Yes, I am watching this subproject. @wu-sheng
But CRD is another choice as cli does, i think the crds tool is familiar with Kubernetes users.
Another way If we don't need another tool, As mentioned above, we can do our operator to manage skywalking on k8s.

[wu-sheng]

In my mind, the operator should focus on the operation side, rather than like CLI to just query and interact with SkyWalking inside APIs. And, you need to notice, there is no operation related GraphQL API today. If you want to do, you need a more detailed document.

Also, as the CLI project has not released yet, and @innerpeacez is working on the helm release project, also Istio CI is WIP, which has a much higher priority.

@wayilau Do you consider about the workload? Do you want to do this alone? Or having other contributors to work with you? Normally, the subproject requires at least one committer to lead, and the PMC to watch, otherwise, the contributor team may break the Apache rules.

[innerpeacez]

But CRD is another choice as cli does, i think the crds tool is familiar with Kubernetes users.
Another way If we don't need another tool, As mentioned above, we can do our operator to manage skywalking on k8s.

I think skywalking-operator is a good way to manage skywalking deployment. And the skywalking-operator can also have a corresponding chart.After the CLI, I don't think SW need to use CRD to do some queries.Because the CLI should eventually be included in the oap image.

[wu-sheng]

Because the CLI should eventually be included in the oap image.

CLI even could has its own image.

[wu-sheng]

I think skywalking-operator is a good way to manage skywalking deployment.

This is the CRD in my mind.

[innerpeacez]

CLI even could has its own image.

Yes, because swctl config has been designed

[wayilau]

Yes, they are all crd, but do different jobs. operator used to manage skywalking cluster status. but crds could do things like swcli.
Another question, does cli could sperate the cluster dynamicly when i deployed multiple Skywalking cluster on k8s? or we need to give the oap url for every query?

[wu-sheng]

Another question, does cli could sperate the cluster dynamicly when i deployed multiple Skywalking cluster? or we need to give the oap url for every query?

Today, there is no alias name or cluster name config. But this could be added. CLI is a new project, even 0.1.0 isn't released yet.

[hanahmily]

Thanks for your proposal @wayilau. It's a great idea to operate SW backend(OAP/UI) in both VM and Kubernetes with the same standard.
I want to pick up CRDs as our final solution of CLI interaction, which is an alternative to Web UI. Because it's the future, and I desire to follow it closely. But the concern is also the "future", well, it didn't come though. Most developers/operators aren't familiar with Kubernetes, not to CRDs.
Another reason is SkyWalking is being used by operators in most cases, this user group prefer an easier way to figure out their jobs. A K8S way will bring much more complex and chaos to them.
So my suggestion is hiding K8S and VM behind SWCLI, then users don't need to care about on which platform they're working. It is not some new idea, and some successful project has followed this way, for instance, Istio.

And for CRDs, I would like to put it into operator, controller, and sidecar injection. The Kubernetes community of SkyWalking focus on making SkyWalking in Kubernetes to work better than other platforms instead of replacing them, which empower the Kubernetes to attract more and more users to work on it.

[wayilau]

Thanks for your reply. @hanahmily
Now, it's time for swcli. I want join you when it's time for operator or crds in the future.
Yes, you are right, we just want to empower, not replacing. And Is there an repo for Skywalking Operator ?

[wu-sheng]

Is there an repo for Skywalking Operator ?

Not yet. We usually require you to propose the detailed design, and ask for review that in the mail list. If other committers join, then we could open a repo for that.

[wayilau]

I see. @wu-sheng
BTW, @hanahmily have you begun to do this already ? If you have not, I will .

[hanahmily]

I'm already working on it. It's arch of Kubernetes native, operator is one of its components though. We can review it next week(I hope). Then everyone interested in it would pick up a piece to detail and polish it. Alan Lau <notifications@github.com> 于2019年11月20日周三 上午9:17写道：

…

I see. @wu-sheng <https://github.com/wu-sheng> BTW, @hanahmily <https://github.com/hanahmily> have you begun to do this already ? If you have not, I will . — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3889?email_source=notifications&email_token=AAO6UWOFI2NFOFHRSDA3IFDQUSF4FA5CNFSM4JO4YKD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEQLEDI#issuecomment-555790861>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAO6UWLR3MJJAZ77ILAPNVLQUSF4FANCNFSM4JO4YKDQ> .

-- Gao Hongtao Apache SkyWalking & ShardingSphere

[wu-sheng]

@hanahmily Notice, @innerpeacez has changed the skywalking-k8s repo to release mode, so basically, in the master branch, it only includes 6.5.0 k8s helm. Are you going to need another repo to host this? Is there a separated release process?

[wayilau]

@hanahmily
OK, I'm waiting for your good news in the mail list.

[hanahmily]

Absolutely. I'm also think how to name this repo. And the deployment arch will follow micro service/multiple processes way since Kubernetes seems expert at handling it.

…

On Wed, Nov 20, 2019, 10:01 AM 吴晟 Wu Sheng ***@***.***> wrote: @hanahmily <https://github.com/hanahmily> Notice, @innerpeacez <https://github.com/innerpeacez> has changed the skywalking-k8s repo to release mode, so basically, in the master branch, it only includes 6.5.0 k8s helm. Are you going to need another repo to host this? Is there a separated release process? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3889?email_source=notifications&email_token=AAO6UWOZGFLCWJTZPMGX2VLQUSK75A5CNFSM4JO4YKD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEQNSHA#issuecomment-555800860>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAO6UWJR7V7R655PT4K53ZLQUSK75ANCNFSM4JO4YKDQ> .

[wu-sheng]

OK, as it is planned to be a separated repo, it is good for me.

[innerpeacez]

OK, I'm waiting for your good news in the mail list.

The same to me

[hanahmily]

@wayilau maybe you interested in https://docs.google.com/document/d/1tPv3ppjNEMa5frX2qRzUXVNky4U4TuTr8qzaiU2wpso/edit?usp=sharing

[wayilau]

Yes, I am reading the doc. @hanahmily

[wu-sheng]

CLI subproject has been created and released for a long time.