A Scalable, Decentralized, Fully distributed, Content addressable file storage system using Golang that can handle and stream very large files.
.
├── Makefile
├── README.md
├── crypto.go - This file defines methods to encrypt files using AES encryption.
├── crypto_test.go - This file contains unit tests for the crypto.go file.
├── go.mod
├── go.sum
├── learn.exe - This is a binary executable file that can be run directly without the Golang compiler.
├── main.go - This file contains the configuration for file, crypto, and p2p.
├── p2p
│ ├── encoding.go - This file specifies the encoding to efficiently transfer files.
│ ├── handshake.go - This file contains pre-connection checks. Additional tests can be added before establishing a connection.
│ ├── message.go - This file defines the message type and format, such as whether it is a message or a stream of files.
│ ├── tcp_transport.go - This file defines methods to handle messages between two nodes.
│ ├── tcp_transport_test.go - This file contains unit tests for the tcp_transport.go file.
│ └── transport.go - This file defines structs and interfaces for peer and transport, respectively.
├── server.go - This file defines methods to store and retrieve files from clients and nodes.
├── storage.go - This file defines methods to store, retrieve, and check files on the local disk.
└── storage_test.go - This file contains unit tests for the storage.go file.
2 directories, 18 files
Note
- This repository is ideal for those who want to learn Golang at its peak. It offers a great opportunity to explore and understand how such systems work under the hood.
- While more features could be added, such as automatic peer discovery, automatic file recovery, and load balancers for faster file retrieval, this project is already a perfect starting point for beginners. Keep in mind that the goal is to learn and explore, and there will always be more features to add.
- New features and pull requests are welcome.
- Concurrency: Utilize goroutines to efficiently handle multiple operations.
- Scalability: Designed for horizontal scaling.
- Fault Tolerance: Implements data replication and failover mechanisms.
- Security: Ensures data encryption and secure communications.
- Performance: Optimized for low latency and high throughput.
Feel free to explore and contribute to this project!