Library of Alexandria (LoA in short) is a project that aims to collect and archive documents from the internet.
In our modern age new text documents are born in a blink of an eye then (often just as quickly) disappear from the internet. We find it a noble task to save these documents for future generations.
This project aims to support this noble goal in a scalable way. We want to make the archival activity streamlined and easy to do even in a huge (Terabyte / Petabyte) scale. This way we hope that more and more people can start their own collection helping the archiving effort.
We have an extensive documentation for the application suite! It is available in our documentation page!
We have an introduction video on YouTube to show the application in action. The example dataset shown in the video is 33 TB in size with 30 million documents.
We have an official Discord server to help people setting up the application and discussing possible/upcoming updates. Feel free to join.