Skip to content
A Kubernetes Operator for deploying and managing clouds of Apache Solr instances.
Go Shell Makefile HTML
Branch: master
Clone or download
HoustonPutman Adding docker pulls and the latest tag version. (#16)
Signed-off-by: Houston Putman <houstonputman@gmail.com>
Latest commit 4798719 Aug 5, 2019

README.md

Solr Operator

Build Status Go Report Card Latest Version Docker Pulls

The Solr Operator manages Apache Solr Clouds within Kubernetes. It is built on top of the Kube Builder framework.

The project is currently in beta (v1beta1), and while we do not anticipate changing the API in backwards-incompatible ways there is no such guarantee yet.

Menu

Getting Started

Running the Solr Operator

Install the Zookeeper & Etcd Operators, which this operator depends on by default.

$ kubectl apply -f example/ext_ops.yaml

Install the SolrCloud CRD & Operator

$ kubectl apply -f config/crds/solr_v1beta1_solrcloud.yaml
$ kubectl apply -f config/operators/solr_operator.yaml

Lifecyle of a Solr Cloud

Creating

Make sure that the solr-operator and a zookeeper-operator are running.

Create an example Solr cloud, with the following configuration.

$ cat example/test_solrcloud.yaml

apiVersion: solr.bloomberg.com/v1beta1
kind: SolrCloud
metadata:
  name: example
spec:
  replicas: 4
  solrImage:
    tag: 8.1.1

Apply it to your Kubernetes cluster.

$ kubectl apply -f example/test_solrcloud.yaml
$ kubectl get solrclouds

NAME      VERSION   DESIREDNODES   NODES   READYNODES   AGE
example   8.1.1     4              2       1            2m

$ kubectl get solrclouds

NAME      VERSION   DESIREDNODES   NODES   READYNODES   AGE
example   8.1.1     4              4       4            8m

Scaling

Increase the number of Solr nodes in your cluster.

$ kubectl scale --replicas=5 solrcloud/example

Deleting

Decrease the number of Solr nodes in your cluster.

$ kubectl delete solrcloud example

Dependent Kubernetes Resources

What actually gets created when the Solr Cloud is spun up?

$ kubectl get all

NAME                                       READY   STATUS             RESTARTS   AGE
pod/example-solrcloud-0                    1/1     Running            7          47h
pod/example-solrcloud-1                    1/1     Running            6          47h
pod/example-solrcloud-2                    1/1     Running            0          47h
pod/example-solrcloud-3                    1/1     Running            6          47h
pod/example-solrcloud-zk-0                 1/1     Running            0          49d
pod/example-solrcloud-zk-1                 1/1     Running            0          49d
pod/example-solrcloud-zk-2                 1/1     Running            0          49d
pod/example-solrcloud-zk-3                 1/1     Running            0          49d
pod/example-solrcloud-zk-4                 1/1     Running            0          49d
pod/solr-operator-8449d4d96f-cmf8p         1/1     Running            0          47h
pod/zk-operator-674676769c-gd4jr           1/1     Running            0          49d

NAME                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)               AGE
service/example-solrcloud-0                ClusterIP   ##.###.###.##    <none>        80/TCP                47h
service/example-solrcloud-1                ClusterIP   ##.###.##.#      <none>        80/TCP                47h
service/example-solrcloud-2                ClusterIP   ##.###.###.##    <none>        80/TCP                47h
service/example-solrcloud-3                ClusterIP   ##.###.##.###    <none>        80/TCP                47h
service/example-solrcloud-common           ClusterIP   ##.###.###.###   <none>        80/TCP                47h
service/example-solrcloud-headless         ClusterIP   None             <none>        80/TCP                47h
service/example-solrcloud-zk-client        ClusterIP   ##.###.###.###   <none>        21210/TCP             49d
service/example-solrcloud-zk-headless      ClusterIP   None             <none>        22210/TCP,23210/TCP   49d

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/solr-operator              1/1     1            1           49d
deployment.apps/zk-operator                1/1     1            1           49d

NAME                                       DESIRED   CURRENT   READY   AGE
replicaset.apps/solr-operator-8449d4d96f   1         1         1       2d1h
replicaset.apps/zk-operator-674676769c     1         1         1       49d

NAME                                       READY   AGE
statefulset.apps/example-solrcloud         4/4     47h
statefulset.apps/example-solrcloud-zk      5/5     49d

NAME                                          HOSTS                                                                                       PORTS   AGE
ingress.extensions/example-solrcloud-common   default-example-solrcloud.test.domain,default-example-solrcloud-0.test.domain + 3 more...   80      2d2h

NAME                                       VERSION   DESIREDNODES   NODES   READYNODES   AGE
solrcloud.solr.bloomberg.com/example       8.1.1     4              4       4            47h

Solr Backups

Solr backups require 3 things:

  • A solr cloud running in kubernetes to backup
  • The list of collections to backup
  • A shared volume reference that can be written to from many clouds
    • This could be a NFS volume, a persistent volume claim (that has ReadWriteMany access), etc.
    • The same volume can be used for many solr clouds in the same namespace, as the data stored within the volume is namespaced.
  • A way to persist the data. The currently supported persistence methods are:
    • A volume reference (this does not have to be ReadWriteMany)
    • An S3 endpoint.

Backups will be tarred before they are persisted.

There is no current way to restore these backups, but that is in the roadmap to implement.

Solr Images

The solr-operator will work with any of the official Solr images currently available.

Zookeeper

Solr Clouds require an Apache Zookeeper to connect to.

The Solr operator gives a few options.

ZK Connection Info

This is an external/internal connection string as well as an optional chRoot to an already running Zookeeeper ensemble. If you provide an external connection string, you do not have to provide an internal one as well.

Provided Instance

If you do not require the Solr cloud to run cross-kube cluster, and do not want to manage your own Zookeeper ensemble, the solr-operator can manage Zookeeper ensemble(s) for you.

Zookeeper

Using the zookeeper-operator, a new Zookeeper ensemble can be spun up for each solrCloud that has this option specified.

The startup parameter zookeeper-operator must be provided on startup of the solr-operator for this parameter to be available.

Zetcd

Using etcd-operator, a new Etcd ensemble can be spun up for each solrCloud that has this option specified. A Zetcd deployment is also created so that Solr can interact with Etcd as if it were a Zookeeper ensemble.

The startup parameter etcd-operator must be provided on startup of the solr-operator for this parameter to be available.

Solr Operator

Solr Operator Input Args

  • -zookeeper-operator Whether or not to use the Zookeeper Operator to create dependent Zookeeepers. Required to use the ProvidedZookeeper.Zookeeper option within the Spec. If true, then a Zookeeper Operator must be running for the cluster. ( true | false , defaults to false)
  • -etcd-operator Whether or not to use the Etcd Operator to create dependent Zetcd clusters. Required to use the ProvidedZookeeper.Zetcd option within the Spec. If true, then an Etcd Operator must be running for the cluster. ( true | false , defaults to false)
  • -ingress-base-url If you desire to make solr externally addressable via ingresses, a base ingress domain is required. Solr Clouds will be created with ingress rules at *.(ingress-base-url). ( optional , e.g. ing.base.domain )

Development

Updating the CRD

The CRD should be updated anytime you update the API.

$ make manifests

Docker

Docker Images

Two Docker images are published to DockerHub, both based off of the same base image.

  • Base Image - Downloads vendor directories, builds operator executable (This is not published, only used to build the following images)
  • Slim Image - Contains only the operator executable
  • Vendor Image - Contains the operator executable as well as all vendored dependencies (at /solr-operator-vendor-sources)

Building

Building and releasing a test operator image with a custom namespace.

$ NAMESPACE=your-namespace make docker-base-build docker-build docker-push

Contributions

We ❤️ contributions.

Have you had a good experience with ? Why not share some love and contribute code, or just let us know about any issues you had with it?

We welcome issue reports here; be sure to choose the proper issue template for your issue, so that we can be sure you're providing the necessary information.

Before sending a Pull Request, please make sure you read our Contribution Guidelines.

License

Please read the LICENSE file here.

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behavior which you have experienced in the project, please contact us at opensource@bloomberg.net.

Security Vulnerability Reporting

If you believe you have identified a security vulnerability in this project, please send email to the project team at opensource@bloomberg.net, detailing the suspected issue and any methods you've found to reproduce it.

Please do NOT open an issue in the GitHub repository, as we'd prefer to keep vulnerability reports private until we've had an opportunity to review and address them.

You can’t perform that action at this time.