Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial codes of sedna #1

Merged
merged 51 commits into from
Jan 29, 2021
Merged

Add initial codes of sedna #1

merged 51 commits into from
Jan 29, 2021

Conversation

llhuii
Copy link

@llhuii llhuii commented Jan 28, 2021

Original project: https://github.com/edgeai-neptune/neptune/
These are:

  1. Add docs:
  • readme with english and chinese version
  • installation doc
  • proposals of framework and three ai features: federeated learning, incremental learning, joint inference
  1. Add codes:
  • build, hack scripts
  • gm/lc codes
  • lib code
  1. three examples, each one in each ai feature

jaypume and others added 30 commits January 29, 2021 00:35
Signed-off-by: Jie Pu <pujie2@huawei.com>
1. add the design document of the basic framework.
2. add the roadmap.
3. add the proposal of joint inference.

Signed-off-by: Jie Pu <pujie2@huawei.com>
Signed-off-by: Jie Pu <pujie2@huawei.com>
Signed-off-by: Jie Pu <pujie2@huawei.com>
Signed-off-by: khalid-davis <huangqinkai1@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
1. basic hack scripts.
2. dataset/model/joint-inference API.
3. commom go files: pkg/version, pkg/util, go.mod, go.sum.
4. client-gen generated files for API.
5. golangci.yml

Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: JimmyYang <yangjin39@huawei.com>
Signed-off-by: JimmyYang <yangjin39@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
1. comment unreachable links
2. fix small typo

Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
1.rename HardExampleAlgorithm to HardExampleMining
2.regenerate zz_generated.deepcopy.go

Signed-off-by: TymonXie <xiedaqi1@huawei.com>
1. remove unused code
2. fix bin path
3. merge image rules into one

Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: bryson-davis <qinkai1994@qq.com>
Signed-off-by: bryson-davis <qinkai1994@qq.com>
Signed-off-by: JimmyYang <yangjin39@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
1.add crd and crd sample for federatedlearning job
2.add federatedlearningjob_types.go, and update register.go

Signed-off-by: TymonXie <xiedaqi1@huawei.com>
1. update zz_generated.deepcopy.go
2. update pkg/client

Signed-off-by: TymonXie <xiedaqi1@huawei.com>
Signed-off-by: TymonXie <xiedaqi1@huawei.com>
khalid-huang and others added 15 commits January 29, 2021 01:03
Signed-off-by: khalid-davis <huangqinkai1@huawei.com>
Signed-off-by: khalid-davis <huangqinkai1@huawei.com>
Signed-off-by: JimmyYang <yangjin39@huawei.com>
…mple for incrementallearning job 2.add incrementallearningjob_types.go, and update register.go 3.update zz_generated.deepcopy.go and client dir

Signed-off-by: TymonXie <xiedaqi1@huawei.com>
1. add independent namespace, service account.
2. deploy gm deployment instead of pod, service instead of host
namespace.
3. add resource limits for gm/lc.
4. add alternative command to set gm_ip.
5. unset 'Always' imagePullPolicy.

Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: TymonXie <xiedaqi1@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
Developers can run `hack/local-up.sh` to setup up a local environment
including:
1. a local k8s cluster with a master node.
2. a kubeedge node.
3. our gm/lc.

Based on the kubeedge-local-up script which builds a local k8s cluster
and kubeedge, our local-up script installs our package locally for
simply developing and preparing for e3e tests.

It does:
1. build the gm/lc/worker images.
2. download kubeedge source code and run its localup script.
3. prepare our k8s env.
4. config gm config and start gm.
5. start lc.
6. add cleanup

For cleanup, it needs to do our cleanups before kubeedge cleanup
otherwise lc cleanup (via kubectl delete) is stuck and lc container is
kept running.

Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: khalid-davis <huangqinkai1@huawei.com>
Signed-off-by: khalid-davis <huangqinkai1@huawei.com>
Signed-off-by: khalid-davis <huangqinkai1@huawei.com>
Signed-off-by: khalid-davis <huangqinkai1@huawei.com>
1. reuse CR definition of pkg/apis in all managers.
2. cleanup server.
3. fix db getClient bug, and optimize it.
4. rename wsclient to gmclient.
5. fix GetUniqueIdentifier bug.

Signed-off-by: JimmyYang <yangjin39@huawei.com>
Signed-off-by: JimmyYang <yangjin39@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
@llhuii
Copy link
Author

llhuii commented Jan 28, 2021

Hi @kevin-wangzefeng, pls take a look, thank you.

@llhuii
Copy link
Author

llhuii commented Jan 28, 2021

FYI, I have removed fossa related status. I will add fossa badge after this project is in fossa.

@llhuii llhuii changed the title Merge initial codes of sedna Add initial codes of sedna Jan 28, 2021
llhuii added 2 commits January 29, 2021 09:37
1. remove preview description
2. update framework of english/chinese

Signed-off-by: llhuii <liulinghui@huawei.com>
also add a python script used to generate the copyright

Signed-off-by: llhuii <liulinghui@huawei.com>
Signed-off-by: llhuii <liulinghui@huawei.com>
@kevin-wangzefeng
Copy link
Member

Looks good to me, thanks for initializing the code.

@kevin-wangzefeng
Copy link
Member

I'm going to manually merge this PR as bot is not yet enabled on this repo

@kevin-wangzefeng kevin-wangzefeng merged commit 2cfb6c7 into kubeedge:main Jan 29, 2021
jaypume referenced this pull request in jaypume/sedna May 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants