-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Golang profile controller #2533
Changes from 11 commits
e45a6c2
d363b12
e3d48c5
c64db59
3d6f665
e6406d9
c6dc79b
c99152a
a1facd6
61fa470
bfc8bb2
7492845
d91b4f8
d5dda9d
d6bc2d7
1a80265
b929f25
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
|
||
# Binaries for programs and plugins | ||
*.exe | ||
*.exe~ | ||
*.dll | ||
*.so | ||
*.dylib | ||
bin | ||
|
||
# Test binary, build with `go test -c` | ||
*.test | ||
|
||
# Output of the go coverage tool, specifically when used with LiteIDE | ||
*.out | ||
|
||
# Kubernetes Generated files - skip generated files, except for vendored files | ||
|
||
!vendor/**/zz_generated.* | ||
|
||
# editor and IDE paraphernalia | ||
.idea | ||
*.swp | ||
*.swo | ||
*~ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,20 @@ | ||
# Build the manager binary | ||
ARG GOLANG_VERSION=1.11.5 | ||
FROM golang:${GOLANG_VERSION} as builder | ||
|
||
# Copy in the go src | ||
WORKDIR /go/src/github.com/kubeflow/kubeflow/components/profile-controller | ||
COPY pkg/ pkg/ | ||
COPY cmd/ cmd/ | ||
COPY go.mod . | ||
|
||
ENV GO111MODULE=on | ||
|
||
# Build | ||
RUN go build -gcflags 'all=-N -l' -o manager cmd/manager/main.go | ||
|
||
# Copy the controller-manager into a thin image | ||
FROM ubuntu:latest | ||
WORKDIR / | ||
COPY --from=builder /go/src/github.com/kubeflow/kubeflow/components/profile-controller/manager . | ||
ENTRYPOINT ["/manager"] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,63 @@ | ||
|
||
# Image URL to use all building/pushing image targets | ||
IMG ?= gcr.io/kubeflow-images-public/notebook-controller | ||
TAG ?= $(eval TAG := $(shell date +v%Y%m%d)-$(shell git describe --tags --always --dirty)-$(shell git diff | shasum -a256 | cut -c -6))$(TAG) | ||
GOLANG_VERSION ?= 1.11.5 | ||
|
||
all: test manager | ||
|
||
# Run tests | ||
test: generate fmt vet manifests | ||
echo "Skip test..." | ||
#go test ./pkg/... ./cmd/... -coverprofile cover.out | ||
|
||
# Build manager binary | ||
manager: generate fmt vet | ||
go build -o bin/manager github.com/kubeflow/kubeflow/components/profile-controller/cmd/manager | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would change bin/manager to bin/profiler-controller |
||
|
||
# Run against the configured Kubernetes cluster in ~/.kube/config | ||
run: generate fmt vet | ||
go run ./cmd/manager/main.go | ||
|
||
# Install CRDs into a cluster | ||
install: manifests | ||
kubectl apply -f config/crds | ||
|
||
# Deploy controller in the configured Kubernetes cluster in ~/.kube/config | ||
deploy: manifests | ||
kubectl apply -f config/crds | ||
kustomize build config/default | kubectl apply -f - | ||
|
||
# Generate manifests e.g. CRD, RBAC etc. | ||
manifests: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this won't work since there is no vendor directory. I'll update with the correct command a bit later |
||
go run vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go all | ||
|
||
# Run go fmt against code | ||
fmt: | ||
go fmt ./pkg/... ./cmd/... | ||
|
||
# Run go vet against code | ||
vet: | ||
go vet ./pkg/... ./cmd/... | ||
|
||
# Generate code | ||
generate: | ||
go generate ./pkg/... ./cmd/... | ||
|
||
# Build the docker image | ||
docker-build: test | ||
docker build . -t ${IMG} | ||
@echo "updating kustomize image patch file for manager resource" | ||
sed -i'' -e 's@image: .*@image: '"${IMG}"'@' ./config/default/manager_image_patch.yaml | ||
|
||
# Push the docker image | ||
docker-push: | ||
docker push ${IMG} | ||
|
||
build-gcr: test | ||
docker build -t $(IMG):$(TAG) . | ||
@echo Built $(IMG):$(TAG) | ||
|
||
push-gcr: build-gcr | ||
docker push $(IMG):$(TAG) | ||
@echo Pushed $(IMG):$(TAG) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
version: "1" | ||
domain: kubeflow.org | ||
repo: github.com/kubeflow/kubeflow/components/profile-controller |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
/* | ||
Copyright 2019 The Kubeflow Authors. | ||
|
||
Licensed under the Apache License, Version 2.0 (the "License"); | ||
you may not use this file except in compliance with the License. | ||
You may obtain a copy of the License at | ||
|
||
http://www.apache.org/licenses/LICENSE-2.0 | ||
|
||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
*/ | ||
|
||
package main | ||
|
||
import ( | ||
"flag" | ||
"os" | ||
|
||
"github.com/kubeflow/kubeflow/components/profile-controller/pkg/apis" | ||
"github.com/kubeflow/kubeflow/components/profile-controller/pkg/controller" | ||
"github.com/kubeflow/kubeflow/components/profile-controller/pkg/webhook" | ||
_ "k8s.io/client-go/plugin/pkg/client/auth/gcp" | ||
"sigs.k8s.io/controller-runtime/pkg/client/config" | ||
"sigs.k8s.io/controller-runtime/pkg/manager" | ||
logf "sigs.k8s.io/controller-runtime/pkg/runtime/log" | ||
"sigs.k8s.io/controller-runtime/pkg/runtime/signals" | ||
) | ||
|
||
func main() { | ||
var metricsAddr string | ||
flag.StringVar(&metricsAddr, "metrics-addr", ":8080", "The address the metric endpoint binds to.") | ||
flag.Parse() | ||
logf.SetLogger(logf.ZapLogger(false)) | ||
log := logf.Log.WithName("entrypoint") | ||
|
||
// Get a config to talk to the apiserver | ||
log.Info("setting up client for manager") | ||
cfg, err := config.GetConfig() | ||
if err != nil { | ||
log.Error(err, "unable to set up client config") | ||
os.Exit(1) | ||
} | ||
|
||
// Create a new Cmd to provide shared dependencies and start components | ||
log.Info("setting up manager") | ||
mgr, err := manager.New(cfg, manager.Options{MetricsBindAddress: metricsAddr}) | ||
if err != nil { | ||
log.Error(err, "unable to set up overall controller manager") | ||
os.Exit(1) | ||
} | ||
|
||
log.Info("Registering Components.") | ||
|
||
// Setup Scheme for all resources | ||
log.Info("setting up scheme") | ||
if err := apis.AddToScheme(mgr.GetScheme()); err != nil { | ||
log.Error(err, "unable add APIs to scheme") | ||
os.Exit(1) | ||
} | ||
|
||
// Setup all Controllers | ||
log.Info("Setting up controller") | ||
if err := controller.AddToManager(mgr); err != nil { | ||
log.Error(err, "unable to register controllers to the manager") | ||
os.Exit(1) | ||
} | ||
|
||
log.Info("setting up webhooks") | ||
if err := webhook.AddToManager(mgr); err != nil { | ||
log.Error(err, "unable to register webhooks to the manager") | ||
os.Exit(1) | ||
} | ||
|
||
// Start the Cmd | ||
log.Info("Starting the Cmd.") | ||
if err := mgr.Start(signals.SetupSignalHandler()); err != nil { | ||
log.Error(err, "unable to run the manager") | ||
os.Exit(1) | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
apiVersion: apiextensions.k8s.io/v1beta1 | ||
kind: CustomResourceDefinition | ||
metadata: | ||
creationTimestamp: null | ||
labels: | ||
controller-tools.k8s.io: "1.0" | ||
name: profiles.kubeflow.kubeflow.org | ||
spec: | ||
group: kubeflow.kubeflow.org | ||
names: | ||
kind: Profile | ||
plural: profiles | ||
scope: Namespaced | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Should we make profile name scope global and have namespace name == profile name? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done. I make profile a cluster scope resource, and namespace's names = profile's name There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It looks like its still Namespace scoped. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sorry only updated ksonnet package. |
||
validation: | ||
openAPIV3Schema: | ||
properties: | ||
apiVersion: | ||
description: 'APIVersion defines the versioned schema of this representation | ||
of an object. Servers should convert recognized schemas to the latest | ||
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources' | ||
type: string | ||
kind: | ||
description: 'Kind is a string value representing the REST resource this | ||
object represents. Servers may infer this from the endpoint the client | ||
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds' | ||
type: string | ||
metadata: | ||
type: object | ||
spec: | ||
properties: | ||
owner: | ||
description: The profile owner | ||
type: object | ||
type: object | ||
status: | ||
type: object | ||
version: v1alpha1 | ||
status: | ||
acceptedNames: | ||
kind: "" | ||
plural: "" | ||
conditions: [] | ||
storedVersions: [] |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
# Adds namespace to all resources. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. you may want to add these kustomization files to the new repo for kustomization manifests (kubeflow/manifests) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ack, will do later |
||
namespace: profile-controller-system | ||
|
||
# Value of this field is prepended to the | ||
# names of all resources, e.g. a deployment named | ||
# "wordpress" becomes "alices-wordpress". | ||
# Note that it should also match with the prefix (text before '-') of the namespace | ||
# field above. | ||
namePrefix: profile-controller- | ||
|
||
# Labels to add to all resources and selectors. | ||
#commonLabels: | ||
# someName: someValue | ||
|
||
# Each entry in this list must resolve to an existing | ||
# resource definition in YAML. These are the resource | ||
# files that kustomize reads, modifies and emits as a | ||
# YAML string, with resources separated by document | ||
# markers ("---"). | ||
resources: | ||
- ../rbac/rbac_role.yaml | ||
- ../rbac/rbac_role_binding.yaml | ||
- ../manager/manager.yaml | ||
# Comment the following 3 lines if you want to disable | ||
# the auth proxy (https://github.com/brancz/kube-rbac-proxy) | ||
# which protects your /metrics endpoint. | ||
- ../rbac/auth_proxy_service.yaml | ||
- ../rbac/auth_proxy_role.yaml | ||
- ../rbac/auth_proxy_role_binding.yaml | ||
|
||
patches: | ||
- manager_image_patch.yaml | ||
# Protect the /metrics endpoint by putting it behind auth. | ||
# Only one of manager_auth_proxy_patch.yaml and | ||
# manager_prometheus_metrics_patch.yaml should be enabled. | ||
- manager_auth_proxy_patch.yaml | ||
# If you want your controller-manager to expose the /metrics | ||
# endpoint w/o any authn/z, uncomment the following line and | ||
# comment manager_auth_proxy_patch.yaml. | ||
# Only one of manager_auth_proxy_patch.yaml and | ||
# manager_prometheus_metrics_patch.yaml should be enabled. | ||
#- manager_prometheus_metrics_patch.yaml | ||
|
||
vars: | ||
- name: WEBHOOK_SECRET_NAME | ||
objref: | ||
kind: Secret | ||
name: webhook-server-secret | ||
apiVersion: v1 |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# This patch inject a sidecar container which is a HTTP proxy for the controller manager, | ||
# it performs RBAC authorization against the Kubernetes API using SubjectAccessReviews. | ||
apiVersion: apps/v1 | ||
kind: StatefulSet | ||
metadata: | ||
name: controller-manager | ||
namespace: system | ||
spec: | ||
template: | ||
spec: | ||
containers: | ||
- name: kube-rbac-proxy | ||
image: quay.io/brancz/kube-rbac-proxy:v0.4.0 | ||
args: | ||
- "--secure-listen-address=0.0.0.0:8443" | ||
- "--upstream=http://127.0.0.1:8080/" | ||
- "--logtostderr=true" | ||
- "--v=10" | ||
ports: | ||
- containerPort: 8443 | ||
name: https | ||
- name: manager | ||
args: | ||
- "--metrics-addr=127.0.0.1:8080" |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
apiVersion: apps/v1 | ||
kind: StatefulSet | ||
metadata: | ||
name: controller-manager | ||
namespace: system | ||
spec: | ||
template: | ||
spec: | ||
containers: | ||
# Change the value of image field below to your controller image URL | ||
- image: IMAGE_URL | ||
name: manager |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
# This patch enables Prometheus scraping for the manager pod. | ||
apiVersion: apps/v1 | ||
kind: StatefulSet | ||
metadata: | ||
name: controller-manager | ||
namespace: system | ||
spec: | ||
template: | ||
metadata: | ||
annotations: | ||
prometheus.io/scrape: 'true' | ||
spec: | ||
containers: | ||
# Expose the prometheus metrics on default port | ||
- name: manager | ||
ports: | ||
- containerPort: 8080 | ||
name: metrics | ||
protocol: TCP |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried
make manager
and gotI also tried
make generate
andmake docker-build
with similar errors.The problem is deepcopy assumes a vendor directory exists which isn't the case for go modules.
This is due to the fact that kubebuilder doesn't generate a Makefile that works with go modules.
You may want to add the following targets to the Makefile
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will it work if I just checkin vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go?