THREESCALE-9187 use standalone searchd image

[eguzki]

What

This is follow up work based on #803

We separate Porta and Sphinx into separate images.

This PR intention is to use the standalone Sphinx image.

The image is called searchd, because we will likely need to switch to manticore server at some point due to discontinued support of latest sphinx open source version. And the executable of both is called searchd. Intention is to avoid changing the name going forward.

This is part of https://issues.redhat.com/browse/THREESCALE-8729

Fixes https://issues.redhat.com/browse/THREESCALE-9187

Note for @3scale/qe: The 3scale search feature should be tested on a new deployment as well as after upgrade

Verification steps

• Requires openshift (oc CLI) session

New deployment

• Create new project

oc new-project 3scale-searchd

• Checkout this PR's branch and run the operator

make install
make run

• Deploy 3scale
  • no need to specify S3 credentials
  • Custom Searchd PVC settings
  • wildcardDomain must be something valid (only if it is going to be used later). Usually ${PROJECT_NAME}.apps.${CLUSTER_DOMAIN}

k apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  creationTimestamp: null
  name: aws-auth
stringData:
  AWS_ACCESS_KEY_ID: testID
  AWS_SECRET_ACCESS_KEY: testkey
  AWS_BUCKET: testbucket
  AWS_REGION: us-east-1
type: Opaque
EOF

k apply -f - <<EOF
---
apiVersion: apps.3scale.net/v1alpha1
kind: APIManager
metadata:
  name: apimanager1
spec:
  wildcardDomain: example.net
  resourceRequirementsEnabled: false
  system:
    searchdSpec:
      persistentVolumeClaim:
        resources:
          requests: 2Gi
    fileStorage:
      simpleStorageService:
        configurationSecretRef:
          name: aws-auth
EOF

• wait for deployment to be ready

oc wait --for=condition=available apimanager/apimanager1 --timeout=-1s
apimanager.apps.3scale.net/apimanager1 condition met

The above "wait" ensures the pods are up and running.

• Check the PVC for the system sphinx has indeed 2Gi as requested

❯ k get pvc system-searchd -o wide
NAME             STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE   VOLUMEMODE
system-searchd   Bound    pvc-15dfab4d-3ec7-4e91-8e9b-d79320a849d1   2Gi        RWO            gp2            10m   Filesystem

• Check the system component references to the searchd address

❯ k get dc system-app -o yaml | grep -A 1 THINKING_SPHINX_ADDRESS
          - name: THINKING_SPHINX_ADDRESS
            value: system-searchd
--
        - name: THINKING_SPHINX_ADDRESS
          value: system-searchd
--
        - name: THINKING_SPHINX_ADDRESS
          value: system-searchd
--
        - name: THINKING_SPHINX_ADDRESS
          value: system-searchd

Upgrade from 2.13

• Create new project

oc new-project 3scale-searchd-upgrade

• Checkout current master branch and run the operator

git checkout ede8838d156d1bf06a25391668f7b5bc18e4cc58
make install
make run

• Deploy 3scale from current master
  • no need to specify S3 credentials
  • wildcardDomain must be something valid (only if it is going to be used later). Usually ${PROJECT_NAME}.apps.${CLUSTER_DOMAIN}

k apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  creationTimestamp: null
  name: aws-auth
stringData:
  AWS_ACCESS_KEY_ID: testID
  AWS_SECRET_ACCESS_KEY: testkey
  AWS_BUCKET: testbucket
  AWS_REGION: us-east-1
type: Opaque
EOF

k apply -f - <<EOF
---
apiVersion: apps.3scale.net/v1alpha1
kind: APIManager
metadata:
  name: apimanager1
spec:
  wildcardDomain: example.net
  resourceRequirementsEnabled: false
  system:
    fileStorage:
      simpleStorageService:
        configurationSecretRef:
          name: aws-auth
EOF

• wait for deployment to be ready

oc wait --for=condition=available apimanager/apimanager1 --timeout=-1s
apimanager.apps.3scale.net/apimanager1 condition met

The above "wait" ensures the pods are up and running.

• Let's run upgrade. Checkout this PR's branch and run the operator

git checkout searchd
make install
make run

The upgrade should happen automatically

• wait for deployment to be ready

oc wait --for=condition=available apimanager/apimanager1 --timeout=-1s
apimanager.apps.3scale.net/apimanager1 condition met

The above "wait" ensures the pods are up and running.

• All the deployments should be ready

❯ k get apimanager apimanager1 -o jsonpath='{.status.deployments}' | yq e -P
ready:
  - apicast-production
  - apicast-staging
  - backend-cron
  - backend-listener
  - backend-redis
  - backend-worker
  - system-app
  - system-memcache
  - system-mysql
  - system-redis
  - system-searchd
  - system-sidekiq
  - zync
  - zync-database
  - zync-que

Note that system-sphinx is gone and replaced by system-searchd

• Check that the references to the new system-searchd have been updated as well

❯ k get dc system-app -o yaml | grep -A 1 THINKING_SPHINX_ADDRESS
          - name: THINKING_SPHINX_ADDRESS
            value: system-searchd
--
        - name: THINKING_SPHINX_ADDRESS
          value: system-searchd
--
        - name: THINKING_SPHINX_ADDRESS
          value: system-searchd
--
        - name: THINKING_SPHINX_ADDRESS
          value: system-searchd

❯ k get dc system-sidekiq -o yaml | grep -A 1 THINKING_SPHINX_ADDRESS
        - name: THINKING_SPHINX_ADDRESS
          value: system-searchd

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[akostadinov]

Appears to be working but having some difficulties with the dev ocp cluster. Hopefully tomorrow will be resolved.

[eguzki]

Appears to be working but having some difficulties with the dev ocp cluster. Hopefully tomorrow will be resolved.

Cool. All done from my side. Just waiting for your approval

[akostadinov]

Finally managed to test on a good OCP. Things seem alright to me. Thanks a lot!