FAQ: My data isn't appearing! An explanation of db.load() vs db.events.on("replicated", callback)

[aphelionz]

There's been a number of questions lately about data not appearing in the store when you expect, so I thought it would be a good time to take a second to explain the process of opening a database and getting data out of it. The database can be either local or remote.

Important Note: While content discovery can happen via the IPFS distributed hash table, due to the limitations of browser-based IPFS nodes, it's still best if the necessary peers are directly connected via a call to ipfs.swarm.connect

The basic flow of data can be summarized as follows:

1. Calling orbitdb.open with default options (or its various shorthand methods orbitdb.keyvalue, orbitdb.docs etc) will create a local database and database manifest in the IPFS instance that OrbitDB is interacting with. In traditional database parlance, you can view this as creating an empty schema.
2. You'll want to then call db.load for both local and remote databases which loads data from the local IPFS into memory for use. Basically it's saying, "give me the data this node knows about"
3. For remote databases, you'll need to define a listenerFunction and register it as an event listener on db.events.on('replicated', function listenerFunction() {}). This will be fired when data is loaded from a remote source, and again every time those remote data change.

In code, this might look something like:

var db = await orbitdb.keyvalue(dbAddress)
await db.load() // Database contents are now in memory
db.events.on("replicated", () => {
  // any remote database content is now replicated to your IPFS node
})

Feel free to respond to this issue with further questions! We want to make sure we're explaining this properly.

[aphelionz]

Leaving it here but flagging as potential to transfer to the Field Manual. Maybe we can replace this pinned issue with just a link to the Field Manual and a quick blurb.

[koo5]

it's not clear what "creating a database" means here. Does this reset some pointer somewhere? What if i "create" a database that already exists locally? What if i create a database that already exists on another node and then sync?

[tabcat]

https://github.com/orbitdb/orbit-db/blob/main/API.md#orbitdbcreatename-type-options

What if i "create" a database that already exists locally?

this is will error without the overwrite option present. use orbitdb.open to open existing databases.

What if i create a database that already exists on another node and then sync?

If the database addresses are the same then the database is the same.

[mar1n3r0]

1. I am having an issue where the data in a local database is not persisted. For example if I put and get in the same run it's there. If I put, close, open and try to get it again it's not there.

2. Using the Control center locally I can't open a previously created local database. In case I create one in the UI it's not accessible outside of it. They also have different IPFS multihash formats. Example:

The one created in the UI:
/orbitdb/zdpuAyPXXpCuZTmnbZ2zPm5hau4mUADAjUh8Qrj2m1pKMagJ4/demands_supplies

The one created in the app code:
/orbitdb/bafyreibe2lnmluj2y4byq6dnb4jbyxinvfbiz5lj7myasprln5pqtmcarm/demand_supply

[haydenyoung]

Thanks @aphelionz for the comprehensive guide. Unfortunately, I feel the replication method is still not functioning correctly. Here is my workflow:

1. I create a db locally using db.keyvalue() and db.load(),
2. I save some data using db.set(),
3. I open the database created in 1. using db.open() followed by db.load(). The database is open but no records are replicated (and replicated event is NOT fired),
4. I save data again using db.set(). Replication is successful.

My assumption is (and please tell me if it is wrong) that records added at step 2 should be available and loaded into the remote db during step 3?

Also, the fact that step 4 completes successfully would suggest both dbs can see each other and replication IS working.

I think the problem myself and @mar1n3r0 are experiencing are similar, the only difference I can see is that the orbitdb hash I use is the same on both local and remote because I use Metamask to handle the identity.

[haydenyoung]

Closing. db.load is no longer available in v1.0 of OrbitDB. This discussion probably does not apply to the latest release.