Skip to content

KorolevDmitry/Database

Repository files navigation

Database application
Dmitry Korolev, Computer Science Center, 2013

Database designed to use simple key-value (string-string for now) storage.

Solution consists of 3 main parts: Base, Client, Server and Balancer.

=====BASE=====
Module that consists of common parts for others (exceptions, entities and some components).
There are next components: Lexer, Parser, Evaluator, TransactionLogger, TcpListener, TcpSender.
-----Lexer-----
Base implementation for splitting commands on lexemes.
-----Parser-----
Implements logic for parsing client requests.
-----Evaluator-----
Abstract class that implements some common methods for client, server and balancer evaluators.
-----TransactionLogger-----
Storage based on DataStorage that stores queries in key/value manner where key is unique id of query.
-----TcpListener-----
Component to listen tcp channel execute commands by given Evaluator and send EvaluationResult back.
Use additional thread.
-----TcpSender-----
Send requests by tcp channel listen response. Parse it to EvaluationResult and gives back.

=====SERVER=====
Console application that listen port and communicate with client by tcp channel via TcpListener.
There are next components: DataStorage, ServerEvaluator.
For now: will be added one nore component: ServerEngine, that will accumulate all components.
-----DataStorage-----
KeyValue storage with get, add, addOrUpdate, delete methods.
There are 2 implementations:
1) MemoryBasedDataStorage - store data in memory in ConcurrentHashMap;
2) FileBasedDataStorage - store data in files with given split rate (by hash of key).
                          Has index by key with position of value in file. Use ConcurrentHashMap and flush it to file.
                          Store data with ObjectOutputStream.
3) CombinedDataStorage - use both previous storages.
                          When memory limit is overflown it redirect half of data in MemoryBasedDataStorage to
                          FileBasedDataStorage synchronously.
-----ServerEvaluator-----
Analyse requested command and execute it by redirect request to given DataStorage.
Also use TcpSender to handle replication requests between servers.
For now: consider opportunity to reduce switches and move logic of executions to commands.
---===USAGE===---
You can use program with next arguments:
        --help - Print description of acceptable arguments
        --port {port} - Port number to listen requests.
        --mode {mode} - Opportunity to run server in different modes:
                Memory - to store values only in memory.
                File - to store values only in file storage.
                Combined - to store values only in combined storage mode.
        --directory {directory} - Directory for storing files of database. Applied only for Memory and Combined modes.
        --split-rate {split-rate} - Count of files to store database in. Applied only for Memory and Combined modes.
        --memory-max-size {memory-max-size} - Size of database in bytes that will be cached in memory. Applied only for Combined mode.
        --generate-elements-count {generate-elements-count} - Count of sample items to generate (key is integer from 0 till count-1).
        --generate-value_size {generate-value_size} - Size of values in bytes to generate.
--EXAMPLE
--port 1107 --mode memory

=====CLIENT=====
Console application that send requests to servers (choose server by Balancer) via TcpSender.
There are next components: ClientEvaluator.
-----ClientEvaluator-----
Execute client requests.
At first send request to Balancer to get route, than handle request directly to server with data.
Use TcpSender to send request.
Use Parser to parse request.
---===USAGE===---
You can use program with next arguments:
        --list-of-servers {hostname:port[;hostname:port]*} - List of servers to connect to.
There are number of commands:
        get {key} - Get value from storage.
        add {key} {value} - Add value to storage.
        update {key} {value} - Update value from storage.
        add_or_update {key} {value} - Add value to storage if it is not exists and update it in another case.
        delete {key} - Delete value from storage.
        help - Print list of accessible commands.
        quit - Quit.
        add_server "{serverHost}" {serverRole} {master} - Add serverHost to cluster. ServerHost - server:port, ServerRole - master/slave, MASTER - serverHost of master node if it is slave.
        remove_server "{serverHost}" - Remove serverHost from cluster. If it was master - master will be one of slaves.
        get_servers_list - Return cluster configuration
        ping "{serverHost}" - Check if serverHost is alive
        replicate "{serverHostFrom}" "{serverHostTo}" {startIndex} {endIndex} {removeAfterReplicationCompleted} - Replicate data
        update_server "{serverHost}" - Force update server info about cluster

=====Balancer=====
Console application that is responsible for routing commands between client and servers.
There are 2 implementations: StaticBalancer and DynamicBalancer.
StaticBalancer routes commands based on hash code of key.
DynamicBalancer features:
1) pings servers to check its availability (for now once in second);
2) can add/remove master/slave servers and request data replication if needed;
3) use ConsistentHash to reduce replication data (master servers in a ring, each of them can contain number of slaves);
4) can accept and execute commands from command line (for now from client);
5) can watch and modify ring by adding/removing servers and changing server's index directly (in testing).
Do not consider any fail scenarios (the only: node can be switched off for a while and then switched on,
but without any additional replication).
---===USAGE===---
You can use program with next arguments:
        --help - Print description of acceptable arguments
        --port {port} - Port number to listen requests.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Languages