Skip to content

zutherb/AppStash

Repository files navigation

AppStash - Microservice Phone Shop Application

Overview

This application gives software architects and developers an example how a microservice web application can look like and it simulates a development cluster, which contains continuous integration infrastructure as well as all necessary nodes that are needed to run an online shop. Thus it will furthermore shown, how a distributed online shop can deployed with a multi deployment pipeline and how the distributed system can be monitored. The application is based on the following two online shop applications, which can be found on Github:

Both project was combined to an new online shop that is indeed to sell mobile devices and implements the following use cases. An user is able to:

  • see different kinds of mobile devices catalogs (e.g. mobiles or tablets),
  • create a cart,
  • and order the created cart.

Use Case Online Shop

This use cases are implemented in the following two ways:

  • A Monolitic Webshop, which is represented by a three layered online shop based on Apache Wicket, the Spring Framework and Spring Data that implements all given use cases,
  • and microservice architecture, which is based on a mix of the Monolitic Webshop and a Microservice Catalog Frontend as it is shown in the below deployment diagram. In this mix a so called Microservice Catalog Frontend provides the use case that an user should be able to see the different mobile. Finally the Monolitic Webshop is used by the user to create an order that means Monolitic Webshop represents a microservice on its own for this specific use case. The Microservice Catalog Frontend is based on an AngularJS and Typescript which access different kinds of REST-Services that are implemented in Scala, Spray, Restx and Spring Boot.

Deployment Diagram Online Shop

Used technologies

For simplicity all services are supposed to run on a Java-VM at the moment.

  • Web frontend
    • Based on Angular JS and Typescript
  • Cart service
    • Based on Spring boot and Groovy
    • Redis backend via Spring Data
  • Product backend
    • Based on Scala and Spray
    • MongoDB backend via ReactiveMongo
  • Navigation backend
    • Based on Restx with embedded Jetty
    • MongoDB backend via Jongo

Presentations

Date Event Title
12.01.2015 JUGM Next Generation IT - Qual oder Segen für den Entwickler
23.01.2015 bobkonf Video (german) Microservices und die Jagd nach mehr Konversion - Fluch oder Segen für den Entwickler
26.01.2015 Microservices Meetup Munich Video (german) Microservices und die Jagd nach mehr Konversion
24.02.2015 Microservices Meetup Berlin Microservices & Conversion Hunting - Software architectures for changeableness
09.03.2015 Microservices Meetup Hamburg Microservices und die Jagd nach mehr Konversion
25.03.2015 Javaland Die Jagd nach mehr Konversion - Fluch oder Segen für den Entwickler
15.04.2015 confess Microservices and Conversion Hunting - How to build software architectures for changeableness
21.04.2015 Agile Softwarearchitektur Münster Microservices und die Jagd nach mehr Konversion
23.06.2015 Devoxx Poland Microservices and Conversion Hunting - How to build software architectures for adaptability
20.08.2015 IT Meetup Bali Microservices and Conversion Hunting: Build Architectures for Changeability
14.09.2015 Microservices Meetup Hamburg Microservice-Deployment ganz einfach
15.09.2015 Lightweight Java User Group München Microservice-Deployment ganz einfach
30.09.2015 code.talks Microservices und die Jagd nach mehr Konversion
06.10.2015 Java Forum Nord Microservices und die Jagd nach mehr Konversion
29.10.2015 JavaOne Microservices and Conversion Hunting: Build Architectures for Changeability
03.11.2015 W-Jax Von Null auf Hundert mit Microservices
28.01.2016 DevOps Karlsruhe Meetup Von Null auf Hundert mit Microservices

Articles

Title Language
Microservices und die Jagd nach mehr Konversion – das Heilmittel für erkrankte IT-Architekturen? German
Microservice-Deployment ganz einfach mit Giant Swarm German
Microservice-Deployment ganz einfach mit Kubernetes German
Microservice-Deployment ganz einfach mit Docker Compose German
Microservice-Deployment ganz einfach ohne Docker mit der Linux-Paketverwaltung German
Canary-Releases mit der Very Awesome Microservices Platform (Vamp) German
In 10 Minuten zum Kubernetes Cluster auf AWS German

Directory Layout

The following directory layout shows only the important directories that are necessary to implement the given use cases in the overview.

microservice/           --> all files of the microservice applications are located in this folder
    frontend/           --> all microservice frontend applications are located in this folder
        catalog/        --> an AngularJS frontend application that shows the product catalog and is used to create a cart is located in this directory
        checkout/       --> all files that are needed to glue the checkout form of the monolithic to the microservice catalog frontend
    service/            --> all business services are located in the folder
        cart/           --> a spring boot cart rest service is located in the folder
        navigation/     --> a java based restx navigation rest service is located in the folder
        product/        --> a scala spray product rest service is located in the folder
monolithic/             --> all files of the monolithic application are located in this directory

Prerequisites

You need some dependencies to run the application cluster or to add add own services to the showcase application.

###Running

You need at least 16 GB RAM to run the whole cluster that emulates a whole development environment like you can find it in the must professional software development projects. Furthermore you have to install the following software dependencies on your machine. If you want to run the shop applications with lower memory you has to uses Docker or Kubernetes.

The provisioning should work on other systems as well but is only tested on MacOSX 10.10. That is the reason because the whole instructions that are shown, are related to MacOSX.

Git

  • Git home (download, documentation) is a distributed revision control system.
  • A good place to learn about setting up git is here
  • You should install Git with Homebrew
brew install git

Vagrant

  • Vagrant creates and configures lightweight, reproducible, and portable development environments. You do not have to learn much about Vagrant, but you should be able to install it and execute the following commandline: vagrant up
  • Vagrant (download, documentation)
  • You should install Vagrant with Homebrew
brew tap caskroom/cask
brew install brew-cask
brew cask install virtualbox
brew cask install vagrant
brew cask install vagrant-manager

Ansible

  • Ansible (download, documentation) is a tool for automating infrastructure orchestration. You must not know Ansible, but you have to install Ansible otherwise Vagrant is not able to create the virtual machines.
  • You should install Ansible with Homebrew
brew install ansible

###Boot up the cluster

The only thing you have to do to run the whole microservice cluster is to execute the following commands:

git clone git@github.com:zutherb/AppStash.git appstash
cd appstash/vagrant
vagrant plugin install vagrant-cachier
vagrant plugin install vagrant-hostsupdater
vagrant up

Vagrant will provision each node in the cluster with all the necessary components (e.g. Monitoring, Build, Database, Debian repository and Application Server). The initial setup can take a few minutes.

Deploy on production servers

You have to execute the Production Deployment Builds on the Jenkins CI Server after you have boot up the cluster. Otherwise you can not use the production urls that are given in the next section. Therefore you have to execute the following two builds:

Please check if all builds are green sometimes the catalog ui build fails and must be re run untill it is green.

CI-Node

After you have complete this, the cluster is fully installed and you can start to work with it.

Workings with the application cluster

The Cluster contains of the following nodes:

Vargrant-Name  IP  Hostname  Application Forward
buildserver 10.211.55.200 ci-node Jenkins http://ci.microservice.io:8080/
reposerver 10.211.55.201 ci-repo Artifact Repository (NGINX)
dbserver 10.211.55.202 mongodb-node MongoDB localhost:27017
dbserver 10.211.55.202 redis-node Redis localhost:6379
appserver1 10.211.55.101 app-server-node-1 Legacy Shop http://test.monolith.io:8080/shop/
appserver1 10.211.55.101 app-server-node-1 Probe http://test.monolith.io:8080/probe/ (admin / topsecret)
appserver2 10.211.55.102 app-server-node-2 Legacy Shop http://shop.monolith.io:8080/shop/
appserver2 10.211.55.102 app-server-node-2 Probe http://shop.monolith.io:8080/probe/ (admin / topsecret)
appserver3 10.211.55.103 app-server-node-3 Microservice Shop http://test-shop.microservice.io/
appserver3 10.211.55.104 app-server-node-4 Microservice Shop http://shop.microservice.io/
elasticsearch 10.211.55.100 monitoring-node Kibana http://monitoring.microservice.io/
elasticsearch 10.211.55.100 monitoring-node Nagios http://monitoring.microservice.io/nagios3/ (nagiosadmin / admin123)
elasticsearch 10.211.55.100 monitoring-node Icinga http://monitoring.microservice.io/icinga/ (icingaadmin / admin123)

###CI-Node

A Jenkins build server is running on the CI-Node. Jenkins is an open source continuous integration tool written in Java that provides a continuous integration services for software development which supports diffent SCM tools. Furthermore Jenkins can execute different build scripts like Gradle as well as arbitrary shell scripts and Windows batch commands.

You can reach the jenkins that builds and deploy the monolith and microservice application under the following url http://ci.microservice.io:8080/.

CI-Node

###Monolith Appserver

The monolith online shop is deployed on the Monolith Appserver which is a reference implementation for the given use cases in the Overview. You can reach the online shop under the following url http://shop.monolith.io:8080/shop/ .

Monolith Appserver

Furthermore you can reach the PSI Probe monitoring and log analysis services under the following url http://shop.monolith.io:8080/probe/. The user credentials are admin / topsecret.

PSI Probe is a community-driven fork of Lambda Probe, which is intended to replace the Tomcat Manager and should make it easier to manage and monitor an instance of Apache Tomcat. PSI Probe does not require any changes to an existing app and it provides many features through a web-accessible interface that becomes available simply by deploying it to your server. These features include:

  • Requests: Monitor traffic in real-time, even on a per-application basis.
  • Sessions: Browse/search attributes, view last IP, expire, estimate size.
  • Logs: View contents, download, change levels at runtime.
  • Threads: View execution stack, kill.
  • JVM: Memory usage charts, advise GC.

Probe Probe

Moreover you access the performance monitor JETM under the following url http://shop.monolith.io:8080/shop/performance/ which is a small and free library that is included in the monolith online shop, that helps locating performance problems in existing Java applications. JETM enables developers to track down performance issues on demand, either programmatic or declarative with minimal impact on application performance, even in production.

JETM Overview JETM Request view

JMX is a natural way to have access to technical management, e.g. for tuning, statistics, log levels and so on. Unfortunately, it lacks a lightweight tool to expose mbeans and to browse them securely on any application and environment without heavy infrastructure setup. JMiniX provides such a feature. You can reach JMiniX under the following url http://shop.monolith.io:8080/shop/jmx/.

JMiniX

###Microservice Appserver

The microservice based online shop is deployed on the microservice appserver which is a reference implementation for the given use cases in the Overview. You can reach the online shop under the following url http://shop.microservice.io/ .

Microservice Appserver

The microservice based online shop consists of two frontend parts as you can see in the deployment diagram in the overview section. The first part is an AngularJS Catalog Frontend that makes it possible to see a catalog for mobiles as well as a catalog for tablets. Furthermore a user is able to create a cart. If a user wants to order a created cart there is same clue logic in the monolithic web application that a cart which was created in the AngularJS Catalog Frontend can be order with the checkout of the Wicket online shop on the monolith appserver.

###Monitoring Server

Monitoring a monolithic web application is no major pain as you can see in the monolith appserver section. A distributed web application, like it is shown in the microservice appserver section, is not so easy to monitor. In this small example there is a Ngnix web server that logs all request that comes into it. The Ngnix deliveries a AngularJS Catalog Frontend that represents a A single-page application (SPA). A SPA is a web application that fits on a single web page with the goal of providing a more fluid user experience akin to a desktop application. In addition to the SPA there are the three rest services a cart, product and a navigation service which are need for the different uses cases and are implemented in the programming languages Groovy, Scala and Java. This services must be alive that an user can see the products or create a cart. Furthermore there is a legacy JEE web application which is deployed in a Tomcat Webserver. An user can order its cart with that legacy JEE web application.

Icinga is an open source network and computer system monitoring application. It was originally created as a fork of the Nagios system monitoring application. Icinga is attempting to get past perceived short-comings in Nagios development process, as well as adding new features such as a modern Web 2.0 style user interface, additional database connectors, and a REST API that lets administrators integrate numerous extensions without complicated modification of the Icinga core.

Icinga Status Map Icinga Status Report

Kibana is a browser based analytics and search interface for Elasticsearch that was developed primarily to view Logstash event data. Logstash is a tool that can be used to collect, process and forward events and log messages. Collection is accomplished via number of configurable input plugins including raw socket/packet communication, file tailing and several message bus clients. Once an input plugin has collected data it can be processed by any number of filters which modify and annotate the event data. Finally events are routed to output plugins which can forward the events to a variety of external programs including Elasticsearch, local files and several message bus implementations.

Kibana

Contact

If you have any questions or remarks, please don't hesitate to contact me. For feature requests or general feedback, you can also use the issue tracker

Bernd Zuther

Licensing

This work is open source, and is licensed under the Apache License, Version 2.0.