Autoscaling Techniques and Blockchain-based Architectures for Performant and Dependable Complex Distributed Systems

Dependability is an important measure which combines several properties to draw up the quality of a system. It is often combined with performability which quantifies how well the system performs in the presence of faults over a specified period of time. The recent wide spread of Cloud Computing has made such properties paramount, however as modern systems are increasing their complexity, ensuring such properties has become harder.

In this thesis we study solutions to improve performability and dependability of modern complex distributed systems. The thesis is divided in two parts.

In the first part we evaluate automatic scaling techniques to elastically reconfigure the number of resources of a distributed system according to workload changes. We started with the claim that proactivity may lead to a more efficient usage of resources; so we design MYSE, an architecture to automatically scale a distributed system proactively according to workload changes. The promising results obtained from simulations carried us to propose a refinement of MYSE for real-world scenarios, so we designed PASCAL. As a first case study we proposed a solution for a distributed datastore that we tested on a real prototype within the Cassandra datastore. Similarly, we proposed ELYSIUM, an instantiation of PASCAL for stream processing systems which symbiotically combines scaling of operators and resources. We integrated ELYSIUM within the scheduler of a well-known stream processing engine, namely Storm. The obtained results for both case studies backup our initial claim, showing as proactivity may improve performances while saving more resources, since it is possible to reduce under/over-provisioning periods.
(Link to the Autoscaling repo https://github.com/FLombardi-PhD/Autoscaling)

In the second part we assess architectures based on blockchain for tamper-resistant multi-party interactions. We started introducing the SUNFISH project and the design of the Federated Service Ledger infrastructure. Then, we propose SLAVE, a solution which exploits blockchain for log-base disputes resolution among parties. Following, we stated some open research questions to adopt a blockchain as a distributed database, thus we proposed 2LBC, a two-layer architecture that relies on a permissioned blockchain anchored to a permissionless one to ensure high data integrity without sacrificing performances. An evaluation of a real prototype shows the effectiveness of our approach, but highlights how the distributed consensus plays a prominence role for both performance and security of a blockchain-based system. Thus, we evaluated a new family of consensus algorithms designed for blockchain, i.e. Proof-of-Authority, to properly assess which family can offer more guarantees. Through a CAP theorem-based analysis we compared their guarantees with PBFT, figuring out that so far choosing a consensus is strictly dependent from the target scenario.
(Link to the Blockchain repo https://github.com/FLombardi-PhD/Blockchain)

Finally, we concluded the work by discussing a possible integration of the two parts of the thesis to design a decentralised elastic dependable framework.