libertyEventDrivenSurvey is an example event-driven survey application demonstrating Liberty InstantOn, CloudEvents, KNative, and MicroProfile Reactive Messaging 3.
The way it works is that users scan a QR code presented by the person running the survey and users type in a location (e.g. New York). This is submitted to a microservice that can scale from zero using KNative Serving and quickly using Liberty InstantOn. This web service publishes the location to a Kafka topic. Another microservice that can scale from zero using KNative Eventing and the KNative Kafka Broker and quickly using Liberty InstantOn then receives this event and geocodes the location to a latitude and longitude through a Google Maps API. Finally, this pin works its way back through another Kafka topic and WebSockets back to the map that the person running the survey is showing.
- If using
podman machine:- Set your connection to the
rootconnection:podman system connection default podman-machine-default-root - If the machine has SELinux
virt_sandbox_use_netlinkdisabled (i.e. the following returnsoff):Then, enable it:podman machine ssh "getsebool virt_sandbox_use_netlink"Note that this must be done after every time the podman machine restarts.podman machine ssh "sudo setsebool virt_sandbox_use_netlink 1"
- Set your connection to the
- Build:
mvn clean deploy
- Install Kafka; for example, the Red Hat Streams for Apache Kafka operator
- Create project
amq-streams-kafka - OperatorHub } AMQ Streams } A specific namespace =
amq-streams-kafka - Kafka } Create Instance } my-cluster } Use all default options } Create
- Wait for
Condition: Ready- If you get a warning about "inter.broker.protocol.version", apply the known workaround
- Kafka Topic } Create KafkaTopic }
locationtopic} Create - Kafka Topic } Create KafkaTopic }
geocodetopic} Create
- Create project
- Install KNative; for example, the Red Hat OpenShift Serverless operator
- Install the
kncommand line utility- Alternatively, install the latest version of kn and the kafka plugin
- macOS:
brew install knative/client/kn knative-sandbox/kn-plugins/source-kafka
- macOS:
- Alternatively, install the latest version of kn and the kafka plugin
- Install KNative Serving
- Operators } Installed Operators
- Project =
knative-serving - Red Hat OpenShift Serverless } Knative Serving
- Create KnativeServing
- Click
Create - Wait for the
ReadyCondition inStatus
- Install KNative Eventing
- Operators } Installed Operators
- Project =
knative-eventing - Red Hat OpenShift Serverless } Knative Eventing
- Create KnativeEventing
- Click
Create - Wait for the
ReadyCondition inStatus
- Install the
KNativeKafkabroker- Knative Kafka } Create KnativeKafka
- channel } enabled; bootstrapServers } my-cluster-kafka-bootstrap.amq-streams-kafka.svc:9092
- source } enabled
- broker } enabled; defaultConfig } bootstrapServers } my-cluster-kafka-bootstrap.amq-streams-kafka.svc:9092
- sink } enabled
- Click
Create - Wait for the
ReadyCondition inStatus
- Install the
- Clone repository:
git clone https://github.com/IBM/libertyEventDrivenSurvey - Change into the clone:
cd libertyEventDrivenSurvey - Check the current project is some test project name:
oc project- If not, create and switch to some test project:
oc new-project libertysurvey
- If not, create and switch to some test project:
- Ensure the internal OpenShift registry is available:
oc patch configs.imageregistry.operator.openshift.io/cluster --patch "{\"spec\":{\"defaultRoute\":true}}" --type=merge - Get a Google Maps API key (simple usage should fit within the free tier)
- Create a service account for InstantOn:
oc create serviceaccount instanton-sa - Create an InstantOn SecurityContextConstraints:
oc apply -f lib/instantonscc.yaml - Associate the InstantOn SecurityContextConstraints with the service account:
oc adm policy add-scc-to-user cap-cr-scc -z instanton-sa - To use InstantOn, we need to modify KNative Serving configuration which must be done through the operator:
- Operators } Installed Operators
- Project =
knative-serving - Red Hat OpenShift Serverless } Knative Serving
- Click
knative-serving - Click
YAML - Under
spec, add:config: features: kubernetes.containerspec-addcapabilities: enabled kubernetes.podspec-securitycontext: enabled - Click
Save
- Push
surveyInputServiceto the registry:REGISTRY=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') echo "Registry host: ${REGISTRY}" printf "Does it look good (yes=ENTER, no=Ctrl^C)? " read trash podman login --tls-verify=false -u $(oc whoami | sed 's/://g') -p $(oc whoami -t) ${REGISTRY} podman tag localhost/surveyinputservice $REGISTRY/libertysurvey/surveyinputservice podman push --tls-verify=false $REGISTRY/libertysurvey/surveyinputservice - Create a KNative Service for
surveyInputService(if needed, replacemp.messaging.connector.liberty-kafka.bootstrap.serverswith the AMQ Streams Kafka Cluster bootstrap address):oc apply -f lib/example_surveyinputservice.yaml - Query until
READYisTrue:kn service list surveyinputservice - Open your browser to the URL from the
kn service listoutput above and click onLocation Survey. - Double check logs look good:
oc exec -it $(oc get pod -o name | grep surveyinputservice) -c surveyinputservice -- cat /logs/messages.log - Note that
scale-down-delaydoes not apply to the initial pod creation so the pod will be terminated about 30 seconds after it's initially created. Once a real user request is made to this application, thenscale-down-delaywill apply. Therefore, if you want to tail the logs of the pod, first wait for the initial pod to terminate and then make a request to the application and then you can tail the pod logs.
- Push
surveyAdminServiceto the registry:REGISTRY=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') echo "Registry host: ${REGISTRY}" printf "Does it look good (yes=ENTER, no=Ctrl^C)? " read trash podman login --tls-verify=false -u $(oc whoami | sed 's/://g') -p $(oc whoami -t) ${REGISTRY} podman tag localhost/surveyadminservice $REGISTRY/libertysurvey/surveyadminservice podman push --tls-verify=false $REGISTRY/libertysurvey/surveyadminservice - Copy
lib/example_surveyadminservice.yaml.templateintolib/example_surveyadminservice.yaml, and then:- Replace
INSERT_API_KEYwith your Google Maps API key - Replace
INSERT_URLwith the URL from theserviceInputServiceabove appended withlocation.html - Replace
mp.messaging.connector.liberty-kafka.bootstrap.serverswith the AMQ Streams Kafka Cluster bootstrap address - If needed, replace
SURVEY_LATITUDEandSURVEY_LONGITUDE(defaults to Las Vegas, NV, USA) - Run:
oc apply -f lib/example_surveyadminservice.yaml
- Replace
- Query until
READYisTrue:kn service list surveyadminservice - Open your browser to the URL from the
kn service listoutput above and click onStart New Geolocation Survey. - Double check logs look good:
oc exec -it $(oc get pod -o name | grep surveyadminservice) -c surveyadminservice -- cat /logs/messages.log - Click
Start New Geolocation Survey - Create a KNative Eventing KafkaSource for
surveyAdminService(if needed, replacebootstrapServerswith the AMQ Streams Kafka Cluster bootstrap address):oc apply -f lib/example_surveyadminkafkasource.yaml - Query until
OKis++for all lines:kn source kafka describe geocodetopicsource - Note that
scale-down-delaydoes not apply to the initial pod creation so the pod will be terminated about 30 seconds after it's initially created. Once a real user request is made to this application, thenscale-down-delaywill apply. Therefore, if you want to tail the logs of the pod, first wait for the initial pod to terminate and then make a request to the application and then you can tail the pod logs.
- Push
surveyGeocoderServiceto the registry:REGISTRY=$(oc get route default-route -n openshift-image-registry --template='{{ .spec.host }}') echo "Registry host: ${REGISTRY}" printf "Does it look good (yes=ENTER, no=Ctrl^C)? " read trash podman login --tls-verify=false -u $(oc whoami | sed 's/://g') -p $(oc whoami -t) ${REGISTRY} podman tag localhost/surveygeocoderservice $REGISTRY/libertysurvey/surveygeocoderservice podman push --tls-verify=false $REGISTRY/libertysurvey/surveygeocoderservice - Copy
lib/example_surveygeocoderservice.yaml.templateintolib/example_surveygeocoderservice.yaml, and then:- Replace
INSERT_API_KEYwith your Google Maps API key - Replace
mp.messaging.connector.liberty-kafka.bootstrap.serverswith the AMQ Streams Kafka Cluster bootstrap address - Run:
oc apply -f lib/example_surveygeocoderservice.yaml
- Replace
- Query until
READYisTrue:kn service list surveygeocoderservice - Double check logs look good:
oc exec -it $(oc get pod -o name | grep surveygeocoderservice) -c surveygeocoderservice -- cat /logs/messages.log - Create a KNative Eventing KafkaSource for
surveyGeocoderService(if needed, replacebootstrapServerswith the AMQ Streams Kafka Cluster bootstrap address):oc apply -f lib/example_surveygeocoderkafkasource.yaml - Query until
OKis++for all lines:kn source kafka describe locationtopicsource - Note that
scale-down-delaydoes not apply to the initial pod creation so the pod will be terminated about 30 seconds after it's initially created. Once a real user request is made to this application, thenscale-down-delaywill apply. Therefore, if you want to tail the logs of the pod, first wait for the initial pod to terminate and then make a request to the application and then you can tail the pod logs.
- Submit a location input:
- Using the command line:
- Execute:
curl -k --data "textInput1=New York, NY" "$(kn service list surveyinputservice -o jsonpath="{.items[0].status.url}{'\n'}")/LocationSurvey" - Check for a successful output:
Your submission has been received
- Execute:
- Using the browser:
- Find and open the URL:
kn service list surveyinputservice -o jsonpath="{.items[0].status.url}{'/location.html\n'}" - Click
Location Surveyand submit the form
- Find and open the URL:
- Using the command line:
- Double check logs look good:
oc exec -it $(oc get pod -o name | grep surveygeocoderservice) -c surveygeocoderservice -- tail -f /logs/messages.log
- Tail
surveyinputservicelogs:oc exec -it $(oc get pod -o name | grep surveyinputservice) -c surveyinputservice -- tail -f /logs/messages.log - Tail
surveyadminservicelogs:oc exec -it $(oc get pod -o name | grep surveyadminservice) -c surveyadminservice -- tail -f /logs/messages.log - Tail
surveygeocoderservicelogs:oc exec -it $(oc get pod -o name | grep surveygeocoderservice) -c surveygeocoderservice -- tail -f /logs/messages.log
lib/cleanup_all.sh
- Delete the KafkaSource:
kn source kafka delete geocodetopicsource - Delete the KNative Service:
kn service delete surveyadminservice
- Delete the KafkaSource:
kn source kafka delete locationtopicsource - Delete the KNative Service:
kn service delete surveygeocoderservice
kn service delete surveyinputservice
Only some functions can be tested locally without KNative.
- Run
surveyAdminService:podman run --privileged --rm -e GOOGLE_API_KEY=YOUR_KEY -p 8080:8080 -p 8443:8443 -it localhost/surveyadminservice:latest - Open browser to http://localhost:8080/geolocation.jsp
- Post a
CloudEvent:curl -X POST http://localhost:8080/api/cloudevents/geocodeComplete \ -H "Ce-Source: https://example.com/" \ -H "Ce-Id: $(uuidgen)" \ -H "Ce-Specversion: 1.0" \ -H "Ce-Type: CloudEvent1" \ -H "Content-Type: text/plain" \ -d "40.7127753 -74.0059728 New York, NY" - Switch back to the browser and you should see the point.
- Create Kafka container network if it doesn't exist:
podman network create kafka - Start Kafka if it's not started:
podman run --rm -p 9092:9092 -e "ALLOW_PLAINTEXT_LISTENER=yes" -e "KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka-0:9092" -e "KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093" -e "KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT" -e "KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-0:9093" -e "KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER" -e "KAFKA_CFG_PROCESS_ROLES=controller,broker" -e "KAFKA_CFG_NODE_ID=0" --name kafka-0 --network kafka docker.io/bitnami/kafka - Run
surveyInputService:podman run --privileged --rm --network kafka --rm -p 8080:8080 -p 8443:8443 -it localhost/surveyinputservice:latest - Wait for the message:
[...] CWWKZ0001I: Application surveyInputService started [...] - Access http://localhost:8080/location.html or https://localhost:8443/location.html
- Create Kafka container network if it doesn't exist:
podman network create kafka - Start Kafka if it's not started:
podman run --rm -p 9092:9092 -e "ALLOW_PLAINTEXT_LISTENER=yes" -e "KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka-0:9092" -e "KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093" -e "KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT" -e "KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka-0:9093" -e "KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER" -e "KAFKA_CFG_PROCESS_ROLES=controller,broker" -e "KAFKA_CFG_NODE_ID=0" --name kafka-0 --network kafka docker.io/bitnami/kafka - Run
surveyGeocoderService:podman run --privileged --rm -p 8080:8080 -p 8443:8443 -e "GOOGLE_API_KEY=INSERT_API_KEY" -it localhost/surveygeocoderservice:latest - Post a
CloudEvent:curl -X POST http://localhost:8080/api/cloudevents/locationInput \ -H "Ce-Source: https://example.com/" \ -H "Ce-Id: $(uuidgen)" \ -H "Ce-Specversion: 1.0" \ -H "Ce-Type: CloudEvent1" \ -H "Content-Type: text/plain" \ -d "New York, NY"
- Change directory to the application
GOOGLE_API_KEY=INSERT_API_KEY mvn clean liberty:dev- Open http://localhost:8080/
- https://developer.ibm.com/articles/develop-reactive-microservices-with-microprofile/
- https://openliberty.io/guides/microprofile-reactive-messaging.html
- https://smallrye.io/smallrye-reactive-messaging/latest/concepts/concepts/
- https://openliberty.io/blog/2022/10/17/microprofile-serverless-ibm-code-engine.html
- OpenLiberty/open-liberty#19889
- OpenLiberty/open-liberty#21659


