Web5.connect() with react-native

[pax-k]

You can now use Web5.connect() with react-native.

Need to add ReadableStream and Blob polyfills. Haven't found anything out of the box, might need to write one myself.

[amika-sq]

I believe that Web5.connect() won't be called from the identity agents like the wallet. This concept is going through a pretty major refactor currently by @frankhinek on the web5-js side.

Would like to hold off on thoroughly reviewing this in until we have a solid understanding of what this re-write entails for clients. @frankhinek if you have any insights here, would greatly appreciate it!

[shamilovtim]

We should be able to achieve the goals of this PR as soon as refactors upstream are complete. I'm uncertain of whether we can use asyncstorage at all given that it only stores strings and DWN stores binary.

[pax-k]

I was able to come a little bit more closer to running Web5.connect() in react-native, but unfortunately we cannot spend any extra time.

Findings:

• Blob and ReadableStream have to be polyfilled in index.js:

import "@tbd54566975/web5-react-native-polyfills";
import { polyfillGlobal } from "react-native/Libraries/Utilities/PolyfillFunctions";

import { Blob } from "blob-polyfill";

export const polyfill = () => {
  const { ReadableStream } = require("web-streams-polyfill/ponyfill/es6");
  polyfillGlobal("ReadableStream", () => ReadableStream);
  polyfillGlobal("Blob", () => Blob);
};

polyfill();

• eventually i used expo-level as storage, but if react-native-leveldb encounters a store location containing slashes, it will fail, so I had to fix it like this:

export function Level(location, options) {
  const cleanLocation = location
    .split("/")
    .filter((x) => x)
    .join("-");
  console.warn("Level", cleanLocation, options);
  return new ExpoLevel(cleanLocation, {
    ...options,
    createIfMissing: true,
    errorIfExists: false,
  });

• which means that I had to patch all hardcoded store locations from the web5 lib, from data/did-res-cache to data-did-res-cache for example.

• I removed syncing from web5 lib, and I was able to get my did:

      const { web5, did: myDid } = await Web5.connect({
        techPreview: { dwnEndpoints: ["https://dwn.hyphen-cloud.xyz"] },
      });
      console.log("myDid", myDid);

• but web5.dwn.records.create() failed with error "Content was invalid" from node_modules/ipfs-unixfs-importer/src/dag-builder/index.ts, because content didn't match any if clause.

• i implemented streamToAsyncIterable:

function streamToAsyncIterable(stream) {
    const readable = stream;

    return {
        [Symbol.asyncIterator]() {
            return {
                next() {
                    return new Promise((resolve, reject) => {
                        readable.once('data', data => {
                            resolve({ value: data, done: false });
                        });
                        readable.once('end', () => {
                            resolve({ done: true });
                        });
                        readable.once('error', reject);
                    });
                }
            };
        }
    };
}

....

function contentAsAsyncIterable (content: Uint8Array | AsyncIterable<Uint8Array> | Iterable<Uint8Array>): AsyncIterable<Uint8Array> {
  try {
    if (content instanceof Uint8Array) {
      return (async function * () {
        yield content
      }())
    } else if (isIterable(content)) {
      return (async function * () {
        yield * content
      }())
    } else if (isAsyncIterable(content)) {
      return content
    } else {
      return streamToAsyncIterable(content) 
    }
  } catch {
    throw errCode(new Error('Content was invalid'), 'ERR_INVALID_CONTENT')
  }

• but I get TypeError: Object is not async iterable which I tried to fix in the first place. content is actually a Node Stream which comes from node_modules/@tbd54566975/web5-user-agent/src/utils.ts. If I try to console.log the stream contents or the blob contents, I indeed get the string I'm trying to write to the DWN, so I'm sure the secret is somewhere around here.

I will keep an eye on your react-native progress, but we need to progress with our demo and I'm forced to mock the following methods, bypass dwn entirely and store data in async-storage directly as k:v JSONs:

web5.dwn.protocols.query
web5.dwn.protocols.configure
web5.dwn.records.delete
web5.dwn.records.create

We relied on web5 to build our sdk around it, works great on web, but our product is mobile, and we're now burned haha

Thanks for your effort!

cc @amika-sq @shamilovtim

[pax-k]

Actually WebView can be used as an execution environment for web5, which could solve the problem of running Web5.connect() on a mobile device, with full sync, local dwn, no swappable storage, etc - no need for polyfills, workarounds, etc, works out of the box. Will push a demo tomorrow

• Create the necessary handlers
• use postMessage as a bridge
• hide WebView

Will let you know how it goes