Skip to content

Commit

Permalink
Merge branch 'master' into patch/rg_controller_tests
Browse files Browse the repository at this point in the history
  • Loading branch information
frodopwns committed Oct 10, 2019
2 parents 179712f + 48b21da commit 9de1e78
Show file tree
Hide file tree
Showing 59 changed files with 2,704 additions and 349 deletions.
8 changes: 8 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,18 @@ generate-test-certs:
mkdir -p /tmp/k8s-webhook-server/serving-certs
mv tls.* /tmp/k8s-webhook-server/serving-certs/

# Run API unittests
api-test: generate fmt vet manifests
TEST_USE_EXISTING_CLUSTER=false go test -v -coverprofile=coverage.txt -covermode count ./api/... 2>&1 | tee testlogs.txt
go-junit-report < testlogs.txt > report.xml
go tool cover -html=coverage.txt -o cover.html

# Run tests
test: generate fmt vet manifests
TEST_USE_EXISTING_CLUSTER=false TEST_CONTROLLER_WITH_MOCKS=true go test -v -coverprofile=coverage.txt -covermode count ./api/... ./controllers/... ./pkg/resourcemanager/eventhubs/... ./pkg/resourcemanager/resourcegroups/... ./pkg/resourcemanager/storages/... 2>&1 | tee testlogs.txt
go-junit-report < testlogs.txt > report.xml
go tool cover -html=coverage.txt -o cover.html

# Run tests with existing cluster
test-existing: generate fmt vet manifests
TEST_USE_EXISTING_CLUSTER=true TEST_CONTROLLER_WITH_MOCKS=false go test -v -coverprofile=coverage-existing.txt -covermode count ./api/... ./controllers/... ./pkg/resourcemanager/eventhubs/... ./pkg/resourcemanager/resourcegroups/... ./pkg/resourcemanager/storages/... 2>&1 | tee testlogs-existing.txt
Expand Down Expand Up @@ -186,6 +193,7 @@ endif
install-kustomize:
ifeq (,$(shell which kustomize))
@echo "installing kustomize"
mkdir -p /usr/local/kubebuilder/bin
# download kustomize
curl -o /usr/local/kubebuilder/bin/kustomize -sL "https://go.kubebuilder.io/kustomize/$(shell go env GOOS)/$(shell go env GOARCH)"
# set permission
Expand Down
12 changes: 12 additions & 0 deletions PROJECT
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,21 @@ resources:
- group: azure
version: v1
kind: EventhubNamespace
- group: azure
version: v1
kind: SqlServer
- group: azure
version: v1
kind: SqlDatabase
- group: azure
version: v1
kind: SqlFirewallRule
- group: azure
version: v1
kind: KeyVault
- group: azure
version: v1
kind: ConsumerGroup
- group: azure
version: v1
kind: SqlAction
163 changes: 16 additions & 147 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,168 +1,37 @@
# Azure Operator (for Kubernetes)
# Azure Service Operator (for Kubernetes)

[![Build Status](https://dev.azure.com/azure/azure-service-operator/_apis/build/status/Azure.azure-service-operator?branchName=master)](https://dev.azure.com/azure/azure-service-operator/_build/latest?definitionId=36&branchName=master)

> This project is experimental. Expect the API to change. It is not recommended for production environments.
## Introduction

Kubernetes offers the facility of extending it's API through the concept of 'Operators' ([Introducing Operators: Putting Operational Knowledge into Software](https://coreos.com/blog/introducing-operators.html)). This repository contains the resources and code to provision a Resource group and Azure Event Hub using Kubernetes operator.
Kubernetes offers the facility of extending it's API through the concept of 'Operators' ([Introducing Operators: Putting Operational Knowledge into Software](https://coreos.com/blog/introducing-operators.html)).

The Azure Operator comprises of:

- The golang application is a Kubernetes controller that watches Customer Resource Definitions (CRDs) that define a Resource Group and Event Hub

The project was built using

1. [Kubebuilder](https://book.kubebuilder.io/)

## Building and Running from Source

### Prerequisites And Assumptions

1. You have GoLang installed.
2. [Docker](https://docs.docker.com/install/) is installed and running.
3. You have the kubectl command line (kubectl CLI) installed.
4. You have access to a Kubernetes cluster.
- It can be a local hosted Cluster like
[Minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/),
[Kind](https://github.com/kubernetes-sigs/kind) or Docker for desktop installed locally with RBAC enabled.
- If you opt for Azure Kubernetes Service ([AKS](https://azure.microsoft.com/en-au/services/kubernetes-service/)), you can use:
`az aks get-credentials --resource-group $RG_NAME --name $Cluster_NAME`
- Kubectl: Client version 1.14 Server Version 1.12

**Note:** it is recommended to use [Kind](https://github.com/kubernetes-sigs/kind) as it is needed for testing Webhooks.
5. Install [Kubebuilder](https://book.kubebuilder.io/), following the linked installation instructions.
6. [Kustomize](https://github.com/kubernetes-sigs/kustomize) is also needed. This must be installed via `make install-kustomize` (see section below).

Basic commands to check your cluster

```shell
kubectl config get-contexts
kubectl cluster-info
kubectl version
kubectl get pods -n kube-system
```

### Quick Start

If you're using VSCode with [Remote - Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) extensions installed, you can quickly have you're environment set up and ready to go with everything you need to get started.

1. Open this project in VSCode.
2. Inside `.devcontainer`, create a file called `.env` and using the following template, copy your Service Principal's details.

```txt
AZURE_CLIENT_ID=
AZURE_CLIENT_SECRET=
AZURE_SUBSCRIPTION_ID=
AZURE_TENANT_ID=
```

3. Open the Command Pallet (`Command+Shift+P` on MacOS or `CTRL+Shift+P` on Windows), type `Remote-Containers: Open Folder in Container...` and hit enter.
4. VSCode will relaunch and start building our development container. This will install all the necessary dependencies required for you to begin developing.
5. Once the container has finished building, you can now start testing your Azure Service Operator within your own local kubernetes environment.

**Note**: if you do not want to create a kind cluster when starting the devcontainer, comment out `"postCreateCommand": "make set-kindcluster",` in `.devcontainer/devcontainer.json` and reopen the devcontainer.

### Getting started

1. Clone the repository from the following folder `<GOPATH>/src/github.com/Azure`.
An Operator is an application-specific controller that extends the Kubernetes API to create, configure, and manage instances of complex stateful applications on behalf of a Kubernetes user. It builds upon the basic Kubernetes resource and controller concepts but includes domain or application-specific knowledge to automate common tasks.

2. Make sure the environment variable `GO111MODULE=on` is set.
This repository contains the resources and code to provision and deprovision different Azure services using a Kubernetes operator.

3. Update the values in `azure_v1_eventhub.yaml` to reflect the resource group and event hub you want to provision

4. Install [Kind](https://kind.sigs.k8s.io/docs/user/quick-start/)

```shell
GO111MODULE="on" go get sigs.k8s.io/kind@v0.4.0 && kind create cluster
kind create cluster
export KUBECONFIG="$(kind get kubeconfig-path --name="kind")"
kubectl cluster-info
IMG="docker.io/yourimage:tag" make docker-build
kind load docker-image docker.io/yourimage:tag --loglevel "trace"
make deploy
```

5. Create a Service Principal
If you don't have a Service Principal create one from the Azure CLI:

```bash
az ad sp create-for-rbac --role Contributor
```

Then make sure this service principal has rights assigned to provision resources on your Azure account.

6. Set the environment variables `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_SUBSCRIPTION_ID`, `REQUEUE_AFTER`.

If you are running it on Windows the environment variables should not have quotes.

It should be set in this way:
`SET AZURE_TENANT_ID=11xxxx-xxx-xxx-xxx-xxxxx`
and the VSCode should be run from the same session/command window

7. Set up the Cluster

If you are using Kind:

```shell
make set-kindcluster
```

If you are not using Kind, it's a manual process, as follows:

a. Create the namespace

```shell
kubectl create namespace azureoperator-system
```

b. Set the azureoperatorsettings secret

```shell
kubectl --namespace azureoperator-system \
create secret generic azureoperatorsettings \
--from-literal=AZURE_CLIENT_ID="$AZURE_CLIENT_ID" \
--from-literal=AZURE_CLIENT_SECRET="$AZURE_CLIENT_SECRET" \
--from-literal=AZURE_SUBSCRIPTION_ID="$AZURE_SUBSCRIPTION_ID" \
--from-literal=AZURE_TENANT_ID="$AZURE_TENANT_ID"
```

c. [Cert Manager](https://docs.cert-manager.io/en/latest/getting-started/install/kubernetes.html)

```shell
kubectl get secret webhook-server-cert -n azureoperator-system -o yaml > certs.txt
```

you can use `https://inbrowser.tools/` and extract `ca.crt`, `tls.crt` and `tls.key`

8. Install [kustomize](https://github.com/kubernetes-sigs/kustomize) using `make install-kustomize`.
The Azure Operator comprises of:

9. Install the azure_v1_eventhub CRD in the configured Kubernetes cluster folder ~/.kube/config,
- The Custom Resource Definitions (CRDs) for each of the Azure services that the Kubernetes user can provision
- The Kubernetes controller that watches for requests to create Custom Resources for these CRDs and creates them

run `kubectl apply -f config/crd/bases` or `make install`
The project was built using

## How to extend the operator and build your own images
[Kubebuilder](https://book.kubebuilder.io/)

### Updating the Azure operator
## Install the operator

This repository is generated by [Kubebuilder](https://book.kubebuilder.io/).
For information on how to build, test and run the operator, refer to the link below.
[Building, testing and running the operator](/docs/contents.md)

To Extend the operator `github.com/Azure/azure-service-operator`:
## Azure Services supported

1. Run `go mod download` to download dependencies. It doesn't show any progress bar and takes a while to download all of dependencies.
2. Update `api\v1\eventhub_types.go`.
3. Regenerate CRD `make manifests`.
4. Install updated CRD `make install`
5. Generate code `make generate`
6. Update operator `controller\eventhub_controller.go`
7. Update tests and run `make test`
8. Deploy `make deploy`
1. [Resource Group](/docs/resourcegroup/resourcegroup.md)
2. [EventHub](/docs/eventhub/eventhub.md)
3. [Azure SQL](/docs/azuresql/azuresql.md)

If you make changes to the operator and want to update the deployment without recreating the cluster (when testing locally), you can use the `make update` to update your Azure Operator pod. If you need to rebuild the docker image without cache, use `make ARGS="--no-cache" update`.

## Testing

Expand Down
70 changes: 70 additions & 0 deletions api/v1/sqlaction_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/*
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

// SqlActionSpec defines the desired state of SqlAction
type SqlActionSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file
ResourceGroup string `json:"resourcegroup"`
ActionName string `json:"actionname"`
ServerName string `json:"servername"`
}

// SqlActionStatus defines the observed state of SqlAction
type SqlActionStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
Provisioning bool `json:"provisioning,omitempty"`
Provisioned bool `json:"provisioned,omitempty"`
Message string `json:"state,omitempty"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status

// SqlAction is the Schema for the sqlactions API
type SqlAction struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

Spec SqlActionSpec `json:"spec,omitempty"`
Status SqlActionStatus `json:"status,omitempty"`
}

// +kubebuilder:object:root=true

// SqlActionList contains a list of SqlAction
type SqlActionList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []SqlAction `json:"items"`
}

func init() {
SchemeBuilder.Register(&SqlAction{}, &SqlActionList{})
}

func (s *SqlAction) IsSubmitted() bool {
return s.Status.Provisioned || s.Status.Provisioning
}
69 changes: 69 additions & 0 deletions api/v1/sqldatabase_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
/*
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1

import (
sql "github.com/Azure/azure-service-operator/pkg/resourcemanager/sqlclient"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

// SqlDatabaseSpec defines the desired state of SqlDatabase
type SqlDatabaseSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file
Location string `json:"location"`
ResourceGroup string `json:"resourcegroup,omitempty"`
Server string `json:"server"`
Edition sql.DBEdition `json:"edition"`
}

// SqlDatabaseStatus defines the observed state of SqlDatabase
type SqlDatabaseStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
Provisioning bool `json:"provisioning,omitempty"`
Provisioned bool `json:"provisioned,omitempty"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// SqlDatabase is the Schema for the sqldatabases API
type SqlDatabase struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

Spec SqlDatabaseSpec `json:"spec,omitempty"`
Status SqlDatabaseStatus `json:"status,omitempty"`
}

// +kubebuilder:object:root=true

// SqlDatabaseList contains a list of SqlDatabase
type SqlDatabaseList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []SqlDatabase `json:"items"`
}

func init() {
SchemeBuilder.Register(&SqlDatabase{}, &SqlDatabaseList{})
}

func (s *SqlDatabase) IsSubmitted() bool {
return s.Status.Provisioned
}
Loading

0 comments on commit 9de1e78

Please sign in to comment.