Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KubeRay: Relocate files to enable controller extension with Kubebuilder #268

Merged
merged 5 commits into from
May 19, 2022
Merged

KubeRay: Relocate files to enable controller extension with Kubebuilder #268

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
b988ea7
KubeRay: Relocate files to enable controller extension with Kubebuilder
brucez-anyscale May 18, 2022
40bc1e7
update suite test
brucez-anyscale May 18, 2022
3c738eb
address comments
brucez-anyscale May 18, 2022
6e994dd
Fix apply issue
brucez-anyscale May 19, 2022
03db534
make manifests
brucez-anyscale May 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apiserver/pkg/manager/resource_manager.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (

"github.com/ray-project/kuberay/apiserver/pkg/util"
api "github.com/ray-project/kuberay/proto/go_client"
"github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/pkg/client/clientset/versioned/typed/raycluster/v1alpha1"
v1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
Expand Down
2 changes: 1 addition & 1 deletion apiserver/pkg/model/converter.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"github.com/golang/protobuf/ptypes/timestamp"
"github.com/ray-project/kuberay/apiserver/pkg/util"
api "github.com/ray-project/kuberay/proto/go_client"
"github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
v1 "k8s.io/api/core/v1"
)

Expand Down
2 changes: 1 addition & 1 deletion apiserver/pkg/util/cluster.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import (
"strconv"

api "github.com/ray-project/kuberay/proto/go_client"
rayclusterapi "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
rayclusterapi "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
Expand Down
2 changes: 1 addition & 1 deletion ray-operator/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ RUN go mod download

# Copy the go source
COPY main.go main.go
COPY api/ api/
COPY apis/ apis/
COPY controllers/ controllers/

# Build
Expand Down
4 changes: 2 additions & 2 deletions ray-operator/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,14 +82,14 @@ docker-push: ## Push docker image with the manager.
##@ Deployment

install: manifests kustomize ## Install CRDs into the K8s cluster specified in ~/.kube/config.
$(KUSTOMIZE) build config/crd | kubectl apply -f -
($(KUSTOMIZE) build config/crd | kubectl create -f -) || ($(KUSTOMIZE) build config/crd | kubectl replace -f -)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use create and replace instead of apply due to
The CustomResourceDefinition "rayclusters.ray.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes


uninstall: manifests kustomize ## Uninstall CRDs from the K8s cluster specified in ~/.kube/config.
$(KUSTOMIZE) build config/crd | kubectl delete -f -

deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config.
cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG}
$(KUSTOMIZE) build config/default | kubectl apply -f -
($(KUSTOMIZE) build config/default | kubectl create -f -) || ($(KUSTOMIZE) build config/default | kubectl replace -f -)

undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config.
$(KUSTOMIZE) build config/default | kubectl delete -f -
Expand Down
17 changes: 17 additions & 0 deletions ray-operator/PROJECT
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
domain: io
layout:
- go.kubebuilder.io/v3
multigroup: true
projectName: ray-operator
repo: github.com/ray-project/kuberay/ray-operator
resources:
- api:
crdVersion: v1
namespaced: true
controller: true
domain: io
group: ray
kind: RayCluster
path: github.com/ray-project/kuberay/ray-operator/apis/ray/v1alpha1
version: v1alpha1
version: "3"
0 ray-operator/api/raycluster/v1alpha1/doc.go → ray-operator/apis/raycluster/v1alpha1/doc.go
View file
Open in desktop
File renamed without changes.
0 .../raycluster/v1alpha1/groupversion_info.go → .../raycluster/v1alpha1/groupversion_info.go
View file
Open in desktop
File renamed without changes.
0 ...i/raycluster/v1alpha1/raycluster_types.go → ...s/raycluster/v1alpha1/raycluster_types.go
View file
Open in desktop
File renamed without changes.
0 ...cluster/v1alpha1/raycluster_types_test.go → ...cluster/v1alpha1/raycluster_types_test.go
View file
Open in desktop
File renamed without changes.
0 ...rator/api/raycluster/v1alpha1/register.go → ...ator/apis/raycluster/v1alpha1/register.go
View file
Open in desktop
File renamed without changes.
0 ...cluster/v1alpha1/zz_generated.deepcopy.go → ...cluster/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop
File renamed without changes.
1,184 changes: 993 additions & 191 deletions ray-operator/config/crd/bases/ray.io_rayclusters.yaml
View file
Open in desktop

Large diffs are not rendered by default.

0 ray-operator/controllers/common/constant.go → ...controllers/raycluster/common/constant.go
View file
Open in desktop
File renamed without changes.
5 changes: 3 additions & 2 deletions ray-operator/controllers/common/ingress.go → .../controllers/raycluster/common/ingress.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,9 @@ package common
import (
"fmt"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/utils"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/utils"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
"github.com/sirupsen/logrus"
networkingv1 "k8s.io/api/networking/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
Expand Down
4 changes: 2 additions & 2 deletions ...erator/controllers/common/ingress_test.go → ...rollers/raycluster/common/ingress_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ import (
"reflect"
"testing"

"github.com/ray-project/kuberay/ray-operator/controllers/utils"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/utils"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"

"github.com/stretchr/testify/assert"

Expand Down
0 ray-operator/controllers/common/metrics.go → .../controllers/raycluster/common/metrics.go
View file
Open in desktop
File renamed without changes.
4 changes: 2 additions & 2 deletions ray-operator/controllers/common/pod.go → ...ator/controllers/raycluster/common/pod.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ import (
"strconv"
"strings"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/utils"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/utils"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
"k8s.io/apimachinery/pkg/api/resource"

logf "sigs.k8s.io/controller-runtime/pkg/log"
Expand Down
6 changes: 3 additions & 3 deletions ray-operator/controllers/common/pod_test.go → ...controllers/raycluster/common/pod_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,12 @@ import (
"strings"
"testing"

"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/utils"

v1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/resource"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/utils"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/utils/pointer"
)
Expand Down
4 changes: 2 additions & 2 deletions ray-operator/controllers/common/rbac.go → ...tor/controllers/raycluster/common/rbac.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package common

import (
"github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/utils"
"github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/utils"
v1 "k8s.io/api/core/v1"
rbacv1 "k8s.io/api/rbac/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
Expand Down
4 changes: 2 additions & 2 deletions ray-operator/controllers/common/service.go → .../controllers/raycluster/common/service.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
package common

import (
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/utils"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/utils"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)
Expand Down
2 changes: 1 addition & 1 deletion ...erator/controllers/common/service_test.go → ...rollers/raycluster/common/service_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import (
"reflect"
"testing"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"

"github.com/stretchr/testify/assert"

Expand Down
12 changes: 6 additions & 6 deletions ...ator/controllers/raycluster_controller.go → ...llers/raycluster/raycluster_controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
package controllers
package raycluster

import (
"context"
"fmt"
"strings"
"time"

rbacv1 "k8s.io/api/rbac/v1"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/common"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/utils"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/common"
_ "github.com/ray-project/kuberay/ray-operator/controllers/common"
"github.com/ray-project/kuberay/ray-operator/controllers/utils"
rbacv1 "k8s.io/api/rbac/v1"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
_ "github.com/ray-project/kuberay/ray-operator/controllers/raycluster/common"
"k8s.io/client-go/tools/record"

"github.com/go-logr/logr"
Expand Down
7 changes: 4 additions & 3 deletions ...ollers/raycluster_controller_fake_test.go → ...luster/raycluster_controller_fake_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,16 @@ See the License for the specific language governing permissions and
limitations under the License.
*/

package controllers
package raycluster

import (
"context"
"testing"

"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/common"

. "github.com/onsi/ginkgo"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/common"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/pkg/client/clientset/versioned/scheme"
"github.com/stretchr/testify/assert"
"k8s.io/apimachinery/pkg/api/errors"
Expand Down
8 changes: 4 additions & 4 deletions ...controllers/raycluster_controller_test.go → .../raycluster/raycluster_controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,20 +13,20 @@ See the License for the specific language governing permissions and
limitations under the License.
*/

package controllers
package raycluster

import (
"context"
"fmt"
"reflect"
"time"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
"github.com/ray-project/kuberay/ray-operator/controllers/common"
"github.com/ray-project/kuberay/ray-operator/controllers/utils"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/common"
"github.com/ray-project/kuberay/ray-operator/controllers/raycluster/utils"

. "github.com/onsi/ginkgo"
. "github.com/onsi/gomega"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"

corev1 "k8s.io/api/core/v1"
v1 "k8s.io/api/core/v1"
Expand Down
6 changes: 3 additions & 3 deletions ray-operator/controllers/suite_test.go → ...ator/controllers/raycluster/suite_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@ See the License for the specific language governing permissions and
limitations under the License.
*/

package controllers
package raycluster

import (
"path/filepath"
"testing"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
"sigs.k8s.io/controller-runtime/pkg/envtest/printer"

. "github.com/onsi/ginkgo"
Expand Down Expand Up @@ -57,7 +57,7 @@ var _ = BeforeSuite(func(done Done) {

By("bootstrapping test environment")
testEnv = &envtest.Environment{
CRDDirectoryPaths: []string{filepath.Join("..", "config", "crd", "bases")},
CRDDirectoryPaths: []string{filepath.Join("..", "..", "config", "crd", "bases")},
ErrorIfCRDPathMissing: true,
}

Expand Down
2 changes: 1 addition & 1 deletion ray-operator/controllers/utils/util.go → ...ator/controllers/raycluster/utils/util.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import (
"strings"
"unicode"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
"github.com/sirupsen/logrus"

corev1 "k8s.io/api/core/v1"
Expand Down
2 changes: 1 addition & 1 deletion ray-operator/controllers/utils/util_test.go → ...controllers/raycluster/utils/util_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
corev1 "k8s.io/api/core/v1"
v1 "k8s.io/api/core/v1"
)
Expand Down
10 changes: 5 additions & 5 deletions ray-operator/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,18 @@ import (
"fmt"
"os"

"github.com/ray-project/kuberay/ray-operator/controllers/raycluster"

"go.uber.org/zap/zapcore"
"sigs.k8s.io/controller-runtime/pkg/healthz"

"github.com/ray-project/kuberay/ray-operator/controllers"

"k8s.io/apimachinery/pkg/runtime"
utilruntime "k8s.io/apimachinery/pkg/util/runtime"
clientgoscheme "k8s.io/client-go/kubernetes/scheme"
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/log/zap"

rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/api/raycluster/v1alpha1"
rayiov1alpha1 "github.com/ray-project/kuberay/ray-operator/apis/raycluster/v1alpha1"
// +kubebuilder:scaffold:imports
)

Expand Down Expand Up @@ -52,7 +52,7 @@ func main() {
"watch-namespace",
"",
"Watch custom resources in the namespace, ignore other namespaces. If empty, all namespaces will be watched.")
flag.BoolVar(&controllers.PrioritizeWorkersToDelete, "prioritize-workers-to-delete", false,
flag.BoolVar(&raycluster.PrioritizeWorkersToDelete, "prioritize-workers-to-delete", false,
"Temporary feature flag - to be deleted after testing")
opts := zap.Options{
Development: true,
Expand Down Expand Up @@ -85,7 +85,7 @@ func main() {
os.Exit(1)
}

if err = controllers.NewReconciler(mgr).SetupWithManager(mgr, reconcileConcurrency); err != nil {
if err = raycluster.NewReconciler(mgr).SetupWithManager(mgr, reconcileConcurrency); err != nil {
setupLog.Error(err, "unable to create controller", "controller", "RayCluster")
os.Exit(1)
}
Expand Down
2 changes: 1 addition & 1 deletion ray-operator/pkg/client/clientset/versioned/fake/register.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ray-operator/pkg/client/clientset/versioned/scheme/register.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ...operator/pkg/client/clientset/versioned/typed/raycluster/v1alpha1/fake/fake_raycluster.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ray-operator/pkg/client/clientset/versioned/typed/raycluster/v1alpha1/raycluster.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ray-operator/pkg/client/clientset/versioned/typed/raycluster/v1alpha1/raycluster_client.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ray-operator/pkg/client/informers/externalversions/generic.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ray-operator/pkg/client/informers/externalversions/raycluster/v1alpha1/raycluster.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion ray-operator/pkg/client/listers/raycluster/v1alpha1/raycluster.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions tests/compatibility-test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ def apply_kuberay_resources():
shell_assert_success('kind load docker-image kuberay/operator:{}'.format(kuberay_sha))
shell_assert_success('kind load docker-image kuberay/apiserver:{}'.format(kuberay_sha))
shell_assert_success(
'kubectl apply -k manifests/cluster-scope-resources')
'kubectl create -k manifests/cluster-scope-resources')
shell_assert_success(
'kubectl apply -k manifests/base')
'kubectl create -k manifests/base')


def create_kuberay_cluster():
Expand Down