Unable to Index Documents with Custom IPFSBlockStorage (pure browser environment) #1180

orwithout · 2024-05-07T02:27:42Z

Description
I am using @orbitdb/core with helia for managing IPFS nodes entirely within a browser environment. I encounter issues when attempting to index documents using custom IPFSBlockStorage.

Expected Behavior
Documents should be indexed and retrievable when using custom IPFSBlockStorage options.

Actual Behavior
Documents fail to be indexed or retrieved when IPFSBlockStorage configurations are applied. However, removing these configurations results in normal indexing and retrieval of documents.

Steps to Reproduce

Configure @orbitdb/core with IPFSBlockStorage for entryStorage, headsStorage, and indexStorage.
Add a document using the add method.
Attempt to retrieve the document using the searchId method.

Additional Information

Browser version: Google Chrome 123.0.6312, Microsoft Edge 123.2420
Operating System: Windows 2022
@orbitdb/core Version: 2.2.0
Helia Version: 4.2.1

Relevant Code

js:

// documents_db.js
import { createHelia } from 'helia';
import { IDBBlockstore } from 'blockstore-idb';
import { IDBDatastore } from 'datastore-idb';
import { createOrbitDB , Documents, IPFSBlockStorage, IPFSAccessController } from '@orbitdb/core';
import { gossipsub } from "@chainsafe/libp2p-gossipsub";
import { identify } from "@libp2p/identify";

class DocumentStore {
  constructor() { this.init(); };

  async init() {
    this.datastore = new IDBDatastore('/datastore');
    this.blockstore = new IDBBlockstore('/blockstore');
    await this.datastore.open();
    await this.blockstore.open();
    this.helia = await createHelia({
      libp2p: {
        services: {
          pubsub: gossipsub({allowPublishToZeroTopicPeers: true}),
          identify: identify()
        }
      },
      keychain: { pass: 'very-strong-password' },
      datastore: this.datastore,
      blockstore: this.blockstore,
      start: true
    });

    console.log('Helia ID:', this.helia.libp2p.peerId.toString());
    setInterval(async () => {
        const peers = await this.helia.libp2p.peerStore.all();
        console.log(`Total number of store peers: ${peers.length}`);
        console.log('Connected peers count:', this.helia.libp2p.getPeers().length);
    }, 5000);

    this.orbitdb = await createOrbitDB({ ipfs: this.helia, directory: '/my_odb'});
    this.db = await  this.orbitdb.open(`${'my_documents_db'}`, {
        create: true,
        type: 'documents',
        Database: Documents({ indexBy: 'id'} ),
        AccessController: IPFSAccessController({ write: ['*'] }),
        entryStorage: await IPFSBlockStorage({ ipfs: this.helia, pin: true }),
        headsStorage: await IPFSBlockStorage({ ipfs: this.helia, pin: true }),
        indexStorage: await IPFSBlockStorage({ ipfs: this.helia, pin: true })
    });
    console.log('odb.address:', this.db.address);
  }

  async add(doc) { return await this.db.put(doc); }
  async searchId(ids) { return await this.db.query(doc => ids.includes(doc.id)); }
  async delete(_id) { return await this.db.del(_id); }
}

const documentStore = new DocumentStore();
export default documentStore;

html:

<!-- documents_odb.html -->
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"><title>FileIDB Module Test</title>
</head>
<body>
    <h1>Test FileIDB Module</h1>
    <lable>①</lable> <button id="addData">Add Sample Data</button><br><br>
    <lable>②</lable> <input type="text" id="searchQuery" placeholder="Search ID, example: 1"> <button id="searchData">Search Data</button>

    <script type="module">
        import documentStore from './documents_odb.js';

        document.getElementById('addData').addEventListener('click', async () => {
            const sampleData = {
                id: "123",
                type: "text/markdown",
                name: "Sample File",
                content: "This is a test file content.",
            };
            try {
            await documentStore.add(sampleData);
                console.log('Data added successfully:', sampleData);
            } catch (error) {
                console.error('Error adding data:', error);
            }
        });

        document.getElementById('searchData').addEventListener('click', async () => {
            const query = document.getElementById("searchQuery").value;
            try {
                const results = await documentStore.searchId(query);
                console.log('found: ',results);
            } catch (error) {
                console.log('found error: ',error.message);
            }
        });
    </script>
</body>
</html>

Console outputs

After removing these three lines:

// In the documents_db.js file: 
        entryStorage: await IPFSBlockStorage({ ipfs: this.helia, pin: true }),
        headsStorage: await IPFSBlockStorage({ ipfs: this.helia, pin: true }),
        indexStorage: await IPFSBlockStorage({ ipfs: this.helia, pin: true })

The text was updated successfully, but these errors were encountered:

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to Index Documents with Custom IPFSBlockStorage (pure browser environment) #1180

Unable to Index Documents with Custom IPFSBlockStorage (pure browser environment) #1180

orwithout commented May 7, 2024 •

edited

Unable to Index Documents with Custom IPFSBlockStorage (pure browser environment) #1180

Unable to Index Documents with Custom IPFSBlockStorage (pure browser environment) #1180

Comments

orwithout commented May 7, 2024 • edited

orwithout commented May 7, 2024 •

edited