Separate documentation from the README (#110)

docs/README.md

@@ -0,0 +1,14 @@
+# Documentation
+
+Please visit the following pages for documentation on using and developing the Solr Operator:
+
+- [Local Tutorial](local_tutorial.md)
+- [Running the Solr Operator](running-the-operator.md)
+- Available Solr Resources
+    - [Solr Clouds](solr-cloud)
+    - [Solr Collections](solr-collection)
+    - [Solr Backups](solr-backup)
+    - [Solr Metrics](solr-prometheus-exporter)
+    - [Solr Collection Aliases](solr-collection-alias)
+- [Development](development.md)
+- [TODO: Architecture Overview](architecture-overview.md)

docs/development.md

@@ -0,0 +1,139 @@
+# Developing the Solr Operator
+
+This page details the steps for developing the Solr Operator, and all necessary steps to follow before creating a PR to the repo.
+
+ - [Setup](#setup)
+    - [Setup Docker for Mac with K8S](#setup-docker-for-mac-with-k8s-with-an-ingress-controller)
+    - [Install the necessary Dependencies](#installing-the-necessary-dependencies)
+ - [Build the Solr CRDs](#build-the-solr-crds)
+ - [Build and Run the Solr Operator](#build-and-run-the-solr-operator)
+    - [Build the Solr Operator](#)
+    - [Running the Solr Operator](#R)
+ - [Steps to take before creating a PR](#before-you-create-a-pr)
+
+## Setup
+
+### Setup Docker for Mac with K8S with an Ingress Controller
+
+Please follow the instructions from the [local tutorial](local_tutorial.md#setup-docker-for-mac-with-k8s).
+
+### Install the necessary dependencies
+
+Install the Zookeeper & Etcd Operators, which this operator depends on by default.
+Each is optional, as described in the [Zookeeper](#zookeeper-reference) section.
+
+```bash
+$ kubectl apply -f example/dependencies
+```
+
+Install necessary dependencies for building and deploying the operator.
+```bash
+$ export PATH="$PATH:$GOPATH/bin" # You likely want to add this line to your ~/.bashrc or ~/.bash_aliases
+$ ./hack/install_dependencies.sh
+```
+
+Beware that you must be running an updated version of `controller-gen`. To update to a compatible version, run:
+
+```bash
+$ go get sigs.k8s.io/controller-tools/cmd/controller-gen@v0.2.2
+```
+
+## Build the Solr CRDs
+
+If you have changed anything in the [APIs directory](/api/v1beta1), you will need to run the following command to regenerate all Solr CRDs.
+
+```bash
+$ make manifests
+```
+
+In order to apply these CRDs to your kube cluster, merely run the following:
+
+```bash
+$ make install
+```
+
+## Build and Run local versions
+
+It is very useful to build and run your local version of the operator to test functionality.
+
+### Building the Solr Operator
+
+#### Building a Go binary
+
+Building the Go binary files is quite straightforward:
+
+```bash
+$ go build
+```
+
+This is useful for testing that your code builds correctly, as well as using the `make run` command detailed below.
+
+#### Building the Docker image
+
+Building and releasing a test operator image with a custom Docker namespace.
+
+```bash
+$ NAMESPACE=your-namespace/ make docker-build docker-push
+```
+
+You can test the vendor docker container by running
+
+```bash
+$ NAMESPACE=your-namespace/ make docker-vendor-build docker-vendor-push
+```
+
+You can control the namespace and version for your solr-operator docker image via the ENV variables:
+- `NAMESPACE`, defaults to `bloomberg/`. **This must end with a forward slash.** This can also include the docker repository information for private repos.
+- `NAME`, defaults to `solr-operator`.
+- `VERSION`, defaults to the git HEAD tag. (e.g. `v0.2.5-1-g06f4e2a`).  
+You can check what version you are using by running `make version`.
+
+The image will be created under the tag `$(NAMESPACE)$(NAME):$(VERSION)` as well as `$(NAMESPACE)$(NAME):latest`.
+
+
+### Running the Solr Operator
+
+There are a few options for running the Solr Operator version you are developing.
+
+- You can deploy the Solr Operator by using our provided [Helm Chart](/helm/solr-operator/README.md).
+You will need to [build a docker image](#building-the-docker-image) for your version of the operator.
+Then update the values for the helm chart to use the version that you have built.
+- There are two useful `make` commands provided to help with running development versions of the operator:
+    - `make run` - This command will start the solr-operator process locally (not within kubernetes).
+    This does not require building a docker image.
+    - `make deploy` - This command will apply the docker image with your local version to your kubernetes cluster.
+    This requires [building a docker image](#building-the-docker-image).
+
+**Warning**: If you are running kubernetes locally and do not want to push your image to docker hub or a private repository, you will need to set the `imagePullPolicy: Never` on your Solr Operator Deployment.
+That way Kubernetes does not try to pull your image from whatever repo it is listed under (or docker hub by default).
+
+## Testing
+
+If you are creating new functionality for the operator, please include that functionality in an existing test or a new test before creating a PR.
+Most tests can be found in the [controller](/controllers) directory, with names that end in `_test.go`.
+
+PRs will automatically run the unit tests, and will block merging if the tests fail.
+
+You can run these tests locally via the following make command:
+
+```bash
+$ make test
+```
+
+## Before you create a PR
+
+The CRD should be updated anytime you update the API.
+
+```bash
+$ make manifests
+```
+
+
+Make sure that you have updated the go.mod file:
+
+```bash
+$ make mod-tidy
+```
+
+Finally, as mentioned on the Repo's README, you will need to sign all commits included in the PR.
+More information can be found on the [organization's contibutions documentation](https://github.com/bloomberg/.github/blob/master/CONTRIBUTING.md#contribution-licensing).

docs/local_tutorial.md

@@ -0,0 +1,233 @@
+# Solr on Kubernetes on local Mac
+
+This tutorial shows how to setup Solr under Kubernetes on your local mac. The plan is as follows:
+
+ 1. [Setup Docker for Mac with K8S](#setup-docker-for-mac-with-k8s)
+ 2. [Install an Ingress Controller to reach the cluster on localhost](#install-an-ingress-controller)
+ 3. [Install Solr Operator](#install-solr-operator)
+ 4. [Start your Solr cluster](#start-your-solr-cluster)
+ 5. [Create a collection and index some documents](#create-a-collection-and-index-some-documents)
+ 6. [Scale from 3 to 5 nodes](#scale-from-3-to-5-nodes)
+ 7. [Upgrade to newer Solr version](#upgrade-to-newer-version)
+ 8. [Install Kubernetes Dashboard (optional)](#install-kubernetes-dashboard-optional)
+ 9. [Delete the solrCloud cluster named 'example'](#delete-the-solrcloud-cluster-named-example)
+
+## Setup Docker for Mac with K8S
+
+```bash
+# Install Homebrew, if you don't have it already
+/bin/bash -c "$(curl -fsSL \
+	https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
+
+# Install Docker Desktop for Mac (use edge version to get latest k8s)
+brew cask install docker-edge
+
+# Enable Kubernetes in Docker Settings, or run the command below:
+sed -i -e 's/"kubernetesEnabled" : false/"kubernetesEnabled" : true/g' \
+    ~/Library/Group\ Containers/group.com.docker/settings.json
+
+# Start Docker for mac from Finder, or run the command below
+open /Applications/Docker.app
+
+# Install Helm, which we'll use to install the operator, and 'watch'
+brew install helm watch
+```
+
+## Install an Ingress Controller
+
+Kubernetes services are by default only accessible from within the k8s cluster. To make them adressable from our laptop, we'll add an ingress controller
+
+```bash
+# Install the nginx ingress controller
+kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud/deploy.yaml
+
+# Inspect that the ingress controller is running by visiting the Kubernetes dashboard 
+# and selecting namespace `ingress-nginx`, or running this command:
+kubectl get all --namespace ingress-nginx
+
+# Edit your /etc/hosts file (`sudo vi /etc/hosts`) and replace the 127.0.0.1 line with:
+127.0.0.1	localhost default-example-solrcloud.ing.local.domain ing.local.domain default-example-solrcloud-0.ing.local.domain default-example-solrcloud-1.ing.local.domain default-example-solrcloud-2.ing.local.domain dinghy-ping.localhost
+```
+
+Once we have installed Solr to our k8s, this will allow us to address the nodes locally.
+
+## Install Solr Operator
+
+Now that we have the prerequisites setup, let us install Solr Operator which will let us easily manage a large Solr cluster:
+
+```bash
+# Download the operator
+OPERATOR_VER=0.2.5
+curl https://codeload.github.com/bloomberg/solr-operator/tar.gz/v$OPERATOR_VER | tar xz
+ln -s -f solr-operator-$OPERATOR_VER solr-operator
+
+# Install the Zookeeper operator (and etcd operator even if we don't use it)
+kubectl apply -f solr-operator/example/dependencies/
+
+# Install the operator (specifying ingressBaseDomain to match our ingressController)
+helm install --set-string ingressBaseDomain=ing.local.domain \
+    solr-operator solr-operator/helm/solr-operator
+```
+
+After installing, you can check to see what lives in the cluster to make sure that the Solr and ZooKeeper operators have started correctly.
+```
+$ kubectl get all
+
+NAME                                       READY   STATUS             RESTARTS   AGE
+pod/solr-operator-8449d4d96f-cmf8p         1/1     Running            0          47h
+pod/zk-operator-674676769c-gd4jr           1/1     Running            0          49d
+
+NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
+deployment.apps/solr-operator              1/1     1            1           49d
+deployment.apps/zk-operator                1/1     1            1           49d
+
+NAME                                       DESIRED   CURRENT   READY   AGE
+replicaset.apps/solr-operator-8449d4d96f   1         1         1       2d1h
+replicaset.apps/zk-operator-674676769c     1         1         1       49d
+```
+
+After inspecting the status of you Kube cluster, you should see a deployment for the Solr Operator as well as the Zookeeper Operator.
+
+## Start an example Solr Cloud cluster
+
+To start a Solr Cloud cluster, we will create a yaml that will tell the Solr Operator what version of Solr Cloud to run, and how many nodes, with how much memory etc.
+
+```bash
+# Create a spec for a 3-node cluster v8.3 with 300m RAM each:
+cat <<EOF > solrCloud-example.yaml
+apiVersion: solr.bloomberg.com/v1beta1
+kind: SolrCloud
+metadata:
+  name: example
+spec:
+  replicas: 3
+  solrImage:
+    tag: "8.3"
+  solrJavaMem: "-Xms300m -Xmx300m"
+EOF
+
+# Install Solr from that spec
+kubectl apply -f solrCloud-example.yaml
+
+# The solr-operator has created a new resource type 'solrclouds' which we can query
+# Check the status live with the 'watch' command. Hit Control-C when done
+watch -dc kubectl get solrclouds
+
+# Open a web browser to see a solr node:
+# Note that this is the service level, so will round-robin between the nodes
+open "http://default-example-solrcloud.ing.local.domain/solr/#/~cloud?view=nodes"
+```
+
+## Create a collection and index some documents
+
+We'll use the Operator's built in collection creation option
+
+```bash
+# Create the spec
+cat <<EOF > collection.yaml
+apiVersion: solr.bloomberg.com/v1beta1
+kind: SolrCollection
+metadata:
+  name: mycoll
+spec:
+  solrCloud: example
+  collection: mycoll
+  autoAddReplicas: true
+  routerName: compositeId
+  numShards: 1
+  replicationFactor: 3
+  maxShardsPerNode: 2
+  collectionConfigName: "_default"
+EOF
+
+# Execute the command and check in Admin UI that it succeeds
+kubectl apply -f collection.yaml
+
+# Check in Admin UI that collection is created
+open "http://default-example-solrcloud.ing.local.domain/solr/#/~cloud?view=graph"
+```
+
+Now index some documents into the empty collection.
+```bash
+curl -XPOST -H "Content-Type: application/json" \
+    -d '[{id: 1}, {id: 2}, {id: 3}, {id: 4}, {id: 5}, {id: 6}, {id: 7}, {id: 8}]' \
+    "http://default-example-solrcloud.ing.local.domain/solr/mycoll/update/"
+```
+
+## Scale from 3 to 5 nodes
+
+So we wish to add more capacity. Scaling the cluster is a breeze.
+
+```
+# Issue the scale command
+kubectl scale --replicas=5 solrcloud/example
+```
+
+After issuing the scale command, start hitting the "Refresh" button in the Admin UI.
+You will see how the new Solr nodes are added.
+You can also watch the status via the `kubectl get solrclouds` command:
+
+```bash
+watch -dc kubectl get solrclouds
+
+# Hit Control-C when done
+```
+
+## Upgrade to newer version
+
+So we wish to upgrade to a newer Solr version:
+
+```
+# Take note of the current version, which is 8.3.1
+curl -s http://default-example-solrcloud.ing.local.domain/solr/admin/info/system | grep solr-i
+
+# Update the solrCloud configuratin with the new version, keeping 5 nodes
+cat <<EOF > solrCloud-example.yaml
+apiVersion: solr.bloomberg.com/v1beta1
+kind: SolrCloud
+metadata:
+  name: example
+spec:
+  replicas: 5
+  solrImage:
+    tag: "8.4"
+  solrJavaMem: "-Xms300m -Xmx300m"
+EOF
+
+# Apply the new config
+# Click the 'Show all details" button in Admin UI and start hitting the "Refresh" button
+# See how the operator upgrades one pod at a time. Solr version is in the 'node' column
+# You can also watch the status with the 'kubectl get solrclouds' command
+kubectl apply -f solrCloud-example.yaml
+watch -dc kubectl get solrclouds
+
+# Hit Control-C when done
+```
+
+## Install Kubernetes Dashboard (optional)
+
+Kubernetes Dashboard is a web interface that gives a better overview of your k8s cluster than only running command-line commands. This step is optional, you don't need it if you're comfortable with the cli.
+
+```
+# Install the Dashboard
+kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta8/aio/deploy/recommended.yaml
+
+# You need to authenticate with the dashboard. Get a token:
+kubectl -n kubernetes-dashboard describe secret \
+    $(kubectl -n kubernetes-dashboard get secret | grep default-token | awk '{print $1}') \
+    | grep "token:" | awk '{print $2}'
+
+# Start a kube-proxy in the background (it will listein on localhost:8001)
+kubectl proxy &
+
+# Open a browser to the dashboard (note, this is one long URL)
+open "http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/overview?namespace=default"
+
+# Select 'Token' in the UI and paste the token from last step (starting with 'ey...')
+```
+
+## Delete the solrCloud cluster named 'example'
+
+```
+kubectl delete solrcloud example
+```