refactor: code/docs subfile/chunk_file -> bundle/file_manifest

.github/workflows/containers.yml

@@ -20,7 +20,7 @@ jobs:
 
     strategy:
       matrix:
-        target: [subfile-exchange]
+        target: [file-exchange]
 
     permissions:
       packages: write

.github/workflows/gen-binaries.yml

@@ -23,8 +23,8 @@ jobs:
           GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN }}
         with:
           upload_url: ${{ github.event.release.upload_url }}
-          asset_path: ./target/release/subfile-exchange
-          asset_name: subfile-exchange-${{ github.event.release.tag_name }}-ubuntu
+          asset_path: ./target/release/file-exchange
+          asset_name: file-exchange-${{ github.event.release.tag_name }}-ubuntu
           asset_content_type: binary/octet-stream
   build-macos:
     runs-on: macos-latest
@@ -43,6 +43,6 @@ jobs:
           GITHUB_TOKEN: ${{ secrets.RELEASE_TOKEN }}
         with:
           upload_url: ${{ github.event.release.upload_url }}
-          asset_path: ./target/release/subfile-exchange
-          asset_name: subfile-exchange-${{ github.event.release.tag_name }}-macos
+          asset_path: ./target/release/file-exchange
+          asset_name: file-exchange-${{ github.event.release.tag_name }}-macos
           asset_content_type: binary/octet-stream

Cargo.toml

@@ -1,7 +1,7 @@
 [workspace]
 members = [
-    "subfile-exchange", 
-    "subfile-service", 
+    "file-exchange", 
+    "file-service", 
 ]
 resolver = "2"
 

Dockerfile.subfile-exchange

@@ -5,7 +5,7 @@ COPY . .
 
 RUN ls -a
 
-RUN cargo build --release --bin subfile-exchange
+RUN cargo build --release --bin file-exchange
 
 ########################################################################################
 
@@ -14,6 +14,6 @@ FROM debian:bookworm-slim
 RUN apt-get update && apt-get install -y --no-install-recommends \
     openssl ca-certificates \
     && rm -rf /var/lib/apt/lists/*
-COPY --from=build /root/target/release/subfile-exchange /usr/local/bin/subfile-exchange
+COPY --from=build /root/target/release/file-exchange /usr/local/bin/file-exchange
 
-ENTRYPOINT [ "/usr/local/bin/subfile-exchange" ]
+ENTRYPOINT [ "/usr/local/bin/file-exchange" ]

README.md

@@ -1,17 +1,17 @@
-# Subfile-exchange
+# File Exchange
 
 ## Introduction 
 Enable file sharing as a exchange, aim for a decentralized, efficient, and verifiable market, with scalable, performant, and secure software.
 
-Subfile-exchange is a decentralized, peer-to-peer data sharing platform designed for efficient and verifiable file sharing. It leverages a combination of technologies including Hash commitments on IPFS for file discovery and verification, chunk data transfer and micropayments reducing trust requirements between clients and servers, and HTTPS over HTTP2 for secure and efficient data transfer. The system is built with scalability, performance, integrity, and security in mind, aiming to create a robust market for file sharing.
+File Exchange is a decentralized, peer-to-peer data sharing platform designed for efficient and verifiable file sharing. It leverages a combination of technologies including Hash commitments on IPFS for file discovery and verification, chunk data transfer and micropayments reducing trust requirements between clients and servers, and HTTPS over HTTP2 for secure and efficient data transfer. The system is built with scalability, performance, integrity, and security in mind, aiming to create a robust market for file sharing.
 
 
-Subfile-exchange leverages IPFS for file discovery and verification, ensuring that each piece of data shared is authentic and unaltered. The use of SHA2-256 for hashing provides a balance of speed and security, making the system both fast and impenetrable to known cryptographic attacks. Furthermore, the adoption of HTTPS over HTTP2 with range requests ensures that all data transfers are not only swift but also secure, safeguarding against common internet vulnerabilities and minimizing risks per transaction.
+File Exchange leverages IPFS for file discovery and verification, ensuring that each piece of data shared is authentic and unaltered. The use of SHA2-256 for hashing provides a balance of speed and security, making the system both fast and impenetrable to known cryptographic attacks. Furthermore, the adoption of HTTPS over HTTP2 with range requests ensures that all data transfers are not only swift but also secure, safeguarding against common internet vulnerabilities and minimizing risks per transaction.
 
 
 ## Target Audience
 
-This documentation is tailored for individuals who have a basic understanding of decentralized technologies, peer-to-peer networks, and cryptographic principles. Whether you are an indexer running various blockchain nodes looking for sharing and verifying your data, an indexer looking to launch service for a new chain, or simply a user interested in the world of decentralized file sharing, this guide aims to provide you with a clear and comprehensive understanding of how Subfile-service operates.
+This documentation is tailored for individuals who have a basic understanding of decentralized technologies, peer-to-peer networks, and cryptographic principles. Whether you are an indexer running various blockchain nodes looking for sharing and verifying your data, an indexer looking to launch service for a new chain, or simply a user interested in the world of decentralized file sharing, this guide aims to provide you with a clear and comprehensive understanding of how File Service operates.
 
 ## Features
 
@@ -49,7 +49,7 @@ You may learn background information on various components of the exchange
 
 #### [Design Principle](docs/architecture.md)
 
-#### [Packaging](docs/subfile_manifest.md)
+#### [Entity Definition](docs/manifest.md)
 
 #### [Contracts](docs/contracts.md)
 

contributing.md

@@ -1,7 +1,7 @@
 
-# Contributing to Subfile-exchange
+# Contributing to File Hosting Service
 
-Welcome to the Subfile Exchange! Thanks a ton for your interest in contributing.
+Welcome to the File Hosting Service! Thanks a ton for your interest in contributing.
 
 If you run into any problems feel free to create an issue. PRs are much appreciated for simple things. If it's something more complex we'd appreciate having a quick chat in GitHub Issues or the Graph Discord server.
 

docs/architecture.md

@@ -2,24 +2,24 @@
 
 ### Decentralized Architecture
 
-Unlike traditional centralized systems where data is stored and managed through a single entity, Subfile-service distributes data across a network of nodes. This decentralization ensures resilience against failures and attacks as there is no single point of failure; it also enhances privacy and control as users are not reliant on a central authority for data management.
+Unlike traditional centralized systems where data is stored and managed through a single entity, File Service distributes data across a network of nodes. This decentralization ensures resilience against failures and attacks as there is no single point of failure; it also enhances privacy and control as users are not reliant on a central authority for data management.
 
 The architecture is underpinned by a peer-to-peer (P2P) network framework, where each node in the network can act as a client and/or a server. This setup facilitates direct file sharing among users without the need for intermediaries, and as more users join and contribute to the network, the system becomes more robust and capable of handling larger volumes of data.
 
 
 ### Generic Diagram
 
-![Diagram](./subfile-exchange.png)
+![Diagram](./file-exchange.png)
 
 
 ### Key Components
 
 1. **File Hasher** ensures data integrity. It uses the SHA2-256 hashing algorithm to process files. The hasher chunks files into manageable sizes (currently 1MB), hashes each chunk, and then organizes these hashes into a Merkle tree structure (Currently we are using an ordered list, but should be relatively simple to update to use the tree structure as it optimizes the verification process ($O(n)$ versus $O(log(n)$ for a single chunk verification where $n$ is the number of chunks), but require 2x memory usage for the hash data structure).
 
-2. **Subfile Publisher** is responsible for preparing and publishing files onto the network. It takes files, processes them through the File Hasher to generate a chunk_file.yaml containing chunk hashes, and then publishes this data to IPFS. The Subfile Builder/Publisher also constructs a subfile manifest, which contains metadata and other relevant information about the files.
+2. **Manifest Publisher** is responsible for preparing and publishing files onto the network. It takes files, processes them through the File Hasher to generate a file_manifest.yaml containing chunk hashes, and then publishes this data to IPFS. The Manifest Builder/Publisher also constructs a file/bundle manifest, which contains metadata and other relevant information about the files.
 
 3. **IPFS Client** connects to the IPFS network as it is used for posting files to the network and retrieving them. IPFS plays a crucial role in discovering and verifying files, as it allows for content-addressable storage.
 
-4. **Subfile Server** requires an operator mnemonic for initialization and handles various tasks such as retrieving files from IPFS, managing subfiles services, verifying file integrity against chunk hashes, and managing API endpoints. The server also implements routes for various functionalities like health checks, version information, and file availability.
+4. **File Server** requires an operator mnemonic for initialization and handles various tasks such as retrieving files from IPFS, managing file services, verifying file integrity against chunk hashes, and managing API endpoints. The server also implements routes for various functionalities like health checks, version information, and file availability.
 
-5. **Subfile Client** is used to request and receive files. It handles the construction of requests, including the addition of authentication tokens and, in future iterations, will manage budgeting for file downloads. The client is responsible for ensuring that the received files are complete and authentic by verifying each chunk against the hashes provided by the File Hasher.
+5. **File Downloader** is used to request and receive files. It handles the construction of requests, including the addition of authentication tokens and, in future iterations, will manage budgeting for file downloads. The client is responsible for ensuring that the received files are complete and authentic by verifying each chunk against the hashes provided by the File Hasher.

docs/client_guide.md

@@ -19,13 +19,13 @@ To minimize trust requirements, the client employs a chunk-based payment system.
 
 ## CLI Usage
 
-The client operates through a command-line interface (CLI) for simplicity and ease of use. Client would need to determine the subfile that contains the dataset they desire. This may mean looking at subfile manifests or in the future a tool that matches subfiles by the provided critieria. 
+The client operates through a command-line interface (CLI) for simplicity and ease of use. Client would need to determine the Bundle that contains the dataset they desire. This may mean looking at Bundle manifests or in the future a tool that matches manifests by the provided critieria. 
 
-After determining the subfile CID, client should supply a local path for writing the subfile corresponding files, a wallet for payments or a free query auth token, and a list of indexer endpoints (this should be handled by gateway or a scraping client).
+After determining the Bundle CID, client should supply a local path for writing the Bundle corresponding files, a wallet for payments or a free query auth token, and a list of indexer endpoints (this should be handled by gateway or a scraping client).
 
 ### CLI example
 ```
-➜  subfile-exchange git:(main) ✗ cargo run -p subfile-exchange downloader \
+➜  file-exchange git:(main) ✗ cargo run -p file-exchange downloader \
    --ipfs-hash QmHash \
    --indexer-endpoints http://localhost:5678,http://localhost:5677 \
    --free-query-auth-token 'Bearer auth_token' \
@@ -49,7 +49,7 @@ To use the client effectively, you will need:
 ### Getting Started
 
 1. Download and install the source code.
-2. Gather configurations: Identify the CID of the desired subfile, registered indexer endpoints, a local path for storing the downloaded files, private key (or mnemonics) of a wallet valid for Escrow payments, (optional) Obtain a free query auth token for limited access.
+2. Gather configurations: Identify the CID of the desired Bundle, registered indexer endpoints, a local path for storing the downloaded files, private key (or mnemonics) of a wallet valid for Escrow payments, (optional) Obtain a free query auth token for limited access.
 3. Use the CLI commands to download files.
 
 Enjoy seamless access to a vast world of data!
@@ -58,9 +58,9 @@ Enjoy seamless access to a vast world of data!
 
 The client prioritizes user safety and security. It employs secure communication protocols and wallet management practices. However, users should always be mindful of potential risks:
 
-- Choosing subfiles: Verify correctness of the subfile before initiating file requests.
+- Choosing manifests: Verify correctness of the Bundle before initiating file requests.
 - Securing wallet: Implement strong key protection and other security measures for the wallet.
-- Staying informed: Updated on the latest security threats, invalid subfiles, and updates for the client software.
+- Staying informed: Updated on the latest security threats, invalid manifests, and updates for the client software.
 
 ### Join the Community