Microservices application with Istio, gRPC, Redis, BigQuery, Spring Boot, Spring Cloud and Stackdriver
- Smart product finder with autocomplete feature
- Distributed Stackdriver Tracing with Log correlation across gRPC calls
- Istio based Stackdriver Service Monitoring
- Automatic synthetic load generation with Locust
- Complete CI/CD to GKE with Google Cloud Build
- Redis in Master/Slave replication mode (image from Bitnami)
- Autocomplete logic based on ZRANGEBYLEX Redis command
- Tunable cache size
- Dataset from BestBuy (converted to ndjson using
JsonProcessor
) - Developed completely with VSCode
- Inspired by Next '18 Microservices Demo
- Covering slides on Slideshare
Guided one click installation from Google Cloud Shell. Complete stack (incl. all dashboards shown below + Istio fault simulation). No client tooling required.
- Get the Best Buy products dataset and save it to a GCS bucket:
- BQ requires the JSON to be new line delimited. Use the provided utility JSON processor for this:
java -jar src/queryservice/target/queryservice-1.0.0.jar \
--spring.profiles.active=JSON \
--input.json="$PWD"/products.json --exit
- Load it to BQ as dataset
coolretailer.products
from GCS - Create a GCP service account with the following roles:
BigQuery Data Viewer
, BigQuery Job User
, Stackdriver Debugger Agent
, Cloud Trace Agent
, Context Graph Asserter
, Error Reporting Admin
, Logs Writer
, Monitoring Metric Writer
and install the private key as app-gac.json as kubernetes secret app-gac.
- apply Istio manifests in:
setup/istio-manifests
- set
PROJECT_ID
,TAG_NAME
,REDIS_HOST
insetup/kubernetes-manifests/
and apply them
- Spring Cloud Sleuth Stackdriver Trace and Logging enabled.
- Complete CI/CD pipeline with Google Cloud Build.
- Test locally using:
$mvn spring-boot:run \
-DGOOGLE_APPLICATION_CREDENTIALS=path-to-key-file \
-DPROJECT_ID=coolretailer \
-DGOOGLE_CLOUD_PROJECT=coolretailer \
-Dspring-boot.run.arguments=--spring.redis.host=localhost
- Using docker from Cloud Shell:
(If tunneling to the redis service is required)
gcloud container clusters get-credentials coolretailer2 --zone europe-west4-a --project coolretailer \
&& kubectl port-forward $(kubectl get pod --selector="app=redis-master,role=master,tier=backend" --output jsonpath='{.items[0].metadata.name}') 6379:6379
gcloud auth configure-docker -q &&\
docker run \
--network=host \
-v /home/kbhattacharya:/etc/app-gac \
-e GOOGLE_CLOUD_PROJECT=coolretailer \
-e PROJECT_ID=coolretailer \
-e GOOGLE_APPLICATION_CREDENTIALS=/etc/app-gac/app-gac.json \
-e --spring.sleuth.sampler.probability=1.0 \
-e --spring.application.name=ux-local \
-e --spring.cloud.gcp.trace.enabled=true \
-e --spring.cloud.gcp.logging.enabled=true \
-e --spring.cloud.gcp.project-id=coolretailer \
-it gcr.io/coolretailer/queryservice
- Use Cloud Memorystore instead of the Redis deployments (src/setup/kubernetes-manifests/archive/redis):
- Istio TLS setup
- See setup/istio-manifests/secure for simple TLS configuration
- The static content is recommended to be served from a GCS bucket with Cloud CDN. Just drop off the content of
src/ui/static/ui.*
andsrc/ui/static/lib
in a GCS bucket and provideallUsers
READER access.
This is not an official Google product