IMAGE STORAGE SERVICE

Image Storage Service is a microservice based on REST APIs to store and retrieve images. It features REST end-points to add/delete/modify albums and images. It is written using Gin Web Framework and jsoniter to make server high performant and have used Swaggo for API documentation.

Contents

• IMAGE STORAGE SERVICE
  • Contents
  • How it Works
  • Usage
  • API Documentation
  • Performance Metrics
  • Docker
    • Docker Build
    • Docker Run
    • Docker Compose
  • Testing
  • Limitations

How it Works

Image storage service has REST end-points to add/delete/modify albums and images. These images are stored in database(file-system) along with meta-info like which image belongs to which album. On successful operations of storing or retrieving of images or albums this service publishes notification messages to MQTT broker and anyone who wants to listen this notification can subscribe to the topic 'imagestore/notifier'.

Usage

To install Image Storage Service, you need to install Go(version 1.12+ is required) and set your Go workspace.

1. This project uses go modules and provides a make file. You should be able to simply install and start:

$ git clone https://github.com/rishikeshbedre/image-storage-service.git
$ cd image-storage-service
$ make
$ mkdir image-db
$ export HOSTIP=$(hostname -I | awk '{print $1}')
$ ./image-storage-service

2. Prior to starting image storage service, you need to install and start MQTT broker

API Documentation

Image storage service serves the API document using swagger and this interactive document can be accessed by:

$ http://host-address:3333/swagger/index.html

Or else link to the document is shown in the logs. Swagger document can be disabled by setting 'PRODMODE' env variable.

Performance Metrics

Benchmarking for this application is not done.

"As this application uses Gin web framework, the default logs of gin server shows how much time is consumed by each request to send response back. By running sample tests, there was high RAM usage when adding the image to storage and this issue was debugged by running pprof on heap and CPU profile. Later this issue was solved by decreasing the value of memory block used by multipart forms (from 32mb to 8mb). As this microservice will run in container environment and this application is written using golang version above 1.12, 'GODEBUG=madvdontneed=1' flag is set instead of default MADV_FREE and calling FreeOSMemory() in a interval so that memory is freed whenever the application doesn't need it anymore."

Docker

Docker Build

To build the container for this microservice, you need to run this script:

$ git clone https://github.com/rishikeshbedre/image-storage-service.git
$ cd image-storage-service
$ ./extras/build.sh

This script builds the image storage service and then pulls eclipse-mosquitto image from docker-hub and then builds notification subscriber service image which helps in subscribing to notification.

Docker Run

After the docker build stage, you can run the containers using following commands:

$ docker run -it -p 1883:1883 --name mq_broker_service eclipse-mosquitto:1.6.9
$ docker run -it -p 3333:3333 -e HOSTIP=`hostname -I | awk '{print $1}'` -v "$(pwd)"/image-db:/home/app/image-db  --name image-store image-storage-service:0.0.1
$ docker run -it --name notify-sub -e HOSTIP=`hostname -I | awk '{print $1}'`  notification-subscriber-service:0.0.1

Docker Compose

After the docker build stage, you can run the containers using following script:

$ ./extras/run.sh

Testing

To run test just run following command:

$ go mod download
$ make test

Limitations

1. There is no authentication on api routes and the server is running on http mode.
2. There is no field validation or image file validation on api routes.
3. As the microservice uses file-system as database, security has to be maintained at OS.
4. Notification service has to be run in secure mode.