- This is a sample data processing pipeline implementation. Taking a dummy JSON as an input and puts the given payload on a running
Redis
server. Another consumer is running in the application taking the freshly received message and persists it in aMariaDB
instance. Furthermore, A REST endpoint is implemented for retrieving all the messages persisted in JSON format from the database.
Method | Path | Description | Available from UI |
---|---|---|---|
GET | /endpoint | Get all persisted messages from database | ✔ |
POST | /endpoint | Send new message to server | ✔ |
Following the instructions will get you a copy of the project up and running on your local machine for development and testing purposes
- Java
- Spring Boot - The project foundation
- Redis - Message Broker
- MariaDB - Database
- Maven - Build & Dependency Management
- Docker - Containerization
- Clone project to an empty directory
- Run command
docker-compose up -d
Example output:
$ docker-compose up -d
......
Starting demo-redis ...
Starting demo-redis ... done
Starting demo-app ... done
.......
.......
demo-app | [INFO] --- spring-boot-maven-plugin:1.5.10.RELEASE:run (default-cli) @ data-processing-pipeline-demo ---
demo-app |
demo-app | . ____ _ __ _ _
demo-app | /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
demo-app | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
demo-app | \\/ ___)| |_)| | | | | || (_| | ) ) ) )
demo-app | ' |____| .__|_| |_|_| |_\__, | / / / /
demo-app | =========|_|==============|___/=/_/_/_/
demo-app | :: Spring Boot :: (v1.5.10.RELEASE)
- Access to
http://localhost:9000
- Alternativly, test with
Curl
by running:
curl -X POST http://localhost:9000/endpoint -H 'Content-Type: application/json' -d "{"content":"sample message"}"
- Cannot access
http://localhost:9000
Run $ docker container inspect demo-app
to show the docker container IP
$ docker container demo-app inspect
....
....
"Networks": {
"dataprocessingpipeline_default": {
....
....
"IPAddress": "172.21.0.4",
....
....
}
}
Bind for 0.0.0.0:[port] failed: port is already allocated
.
As it says, the specified port is occupied by another process. Change the port in docker-compse.yml
or stop the process using the port
- Test fail due to
java.net.UnknownHostException: [host]
Change the mariadb & redis host on application.properties