Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable PubSub in the Browser #518

Closed
daviddias opened this issue Jan 31, 2017 · 21 comments · Fixed by #1059

Comments

@daviddias
Copy link
Member

commented Jan 31, 2017

Following: #493 (comment)

PubSub will be unavailable as a browser API until https://github.com/ipfs/http-api-spec/issues/116 happens.

@daviddias

This comment has been minimized.

Copy link
Member Author

commented May 2, 2017

Update on this thread:

We currently have PubSub disabled on the browser as a way to prevent misunderstanding how the cancellation of the API works. Unfortunately, due to what is noted in -- #493 (comment) -- we simply can't cancel requests as go-ipfs expects in the browser. However, it seems to me that removing support for it entirely is not the optimal solution, as for a lot of use cases, only a few amount of PubSub channels will be open from a client, keeping the maintenance of those at bare minimum.

Proposal: enable PubSub and be explicit that cancels aren't actually being canceled, just silenced.

@mitar

This comment has been minimized.

Copy link
Contributor

commented May 2, 2017

I think we should enable PubSub on client and document cancelation issue.

@Morantron

This comment has been minimized.

Copy link

commented May 20, 2017

I've found and ultra-hacky workaround to cancel fetch requests without XHR here ( https://github.com/Morantron/poor-mans-cancelable-fetch ).

The whole idea is to make requests in a separate WebWorker, and sending a signal to the worker to terminate itself, which also aborts the request :trollface:

Eventually fetch requests will be able to be aborted ( whatwg/fetch#447 ), but the final API hasn't been decided yet.

I know it's super hackish, but if it works ™️ , it can serve as workaround until fetch requests are abortable.

@mitar

This comment has been minimized.

Copy link
Contributor

commented May 21, 2017

Great hack! I like it!

@daviddias

This comment has been minimized.

Copy link
Member Author

commented Jun 25, 2017

Oh wow! @Morantron wanna submit a PR with that feature? Can you make sure to add a check to see if the browser has WebWorker support?

@Morantron

This comment has been minimized.

Copy link

commented Jun 27, 2017

@diasdavid Sure! I'm gonna see if I can turn the hack into some kind of FetchController polyfill, and then make the needed changes in js-ipfs-api.

Looks like IE11 does not support fetch, so I guess PubSub won't work given that XHR does not play well with http streams.

@jedd-ahyoung

This comment has been minimized.

Copy link

commented Jul 21, 2017

Hey, all, hate to ask a potentially obvious (and stupid) question.....but why not just use standard XHR which is cancellable? Believe me, I support the fetch standard and I love what it provides, but if we aren't using a service worker to intercept requests or anything like that, the main difference comes down to error handling.

XHR requests are cancellable and would solve the issue until the fetch standard includes true cancellable requests. The other alternative to this to cancel requests on the client side would be to run a service worker, and have the service worker respond to the request with a 400 Cancelled or something like that when a cancellation is requested. (Basically, you're manually sending a response to an initiated HTTP request to terminate it early.)

@daviddias

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2017

@jedd-ahyoung xhr is not streamable, pubsub/subscribe is a stream. We need to use something like fetch or make this endpoint available Websockets.

@mitar

This comment has been minimized.

Copy link
Contributor

commented Jul 21, 2017

Yes, why not websockets? Or dataframes from WebRTC?

@daviddias

This comment has been minimized.

Copy link
Member Author

commented Jul 21, 2017

This is not a decision at the level of this client library, it is rather at the http-api spec level. Moving to a complete RPC API where WebSockets is used as a transport has been proposed: https://github.com/ipfs/http-api-spec/issues/116

@daviddias daviddias changed the title PubSub does not work in the browser Enable PubSub in the Browser Nov 23, 2017

@daviddias

This comment has been minimized.

Copy link
Member Author

commented Nov 23, 2017

Update

The decision on this one is to either do:

a) Enable PubSub and fake subscriptions - #518 (comment)
b) Use WebWorkers to cancel subscriptions - #518 (comment)

Either is fine as a first iteration.

@nunofmn

This comment has been minimized.

Copy link
Member

commented Nov 24, 2017

It's important to notice that Edge 16 and Firefox 57 now can cancel fetch requests by using AbortController and AbortSignal. But Chrome still doesn't support this API...

@fazo96 fazo96 referenced this issue Dec 19, 2017
@victorb

This comment has been minimized.

Copy link
Member

commented Feb 25, 2018

Seems Chrome has done some work on it (AbortController), last patch from February 5th this year https://bugs.chromium.org/p/chromium/issues/detail?id=750599

WebKit, no status yet: https://bugs.webkit.org/show_bug.cgi?id=174980

@victorb

This comment has been minimized.

Copy link
Member

commented Aug 6, 2018

According to https://developer.mozilla.org/en-US/docs/Web/API/AbortController#Browser_compatibility most browsers now work with AbortController (except IE, shocker)

image

@shunkino

This comment has been minimized.

Copy link
Contributor

commented Aug 28, 2018

Hi, I'm new to the ipfs community and impressed by the all works done so far.
I find the PubSub functionality very interesting and wish it be working in browsers too.

Here are my findings. Maybe I'm totally off the track, please let me know if anything is wrong.
In my understanding, the problem here is that we can't cancel requests in browser by the code bellow.
https://github.com/ipfs/js-ipfs-api/blob/master/src/pubsub.js#L94
But, as @nunofmn suggest if we can use AbortController, we can cancel the fetch.

Since js-ipfs-api uses stream-http as http, aborting function should be implemented using it's API. When looking at the code of the stream-http library, abort() to the client request is implemented at v2.7.2. Thus, required version of stream-http by package.json, v2.8.3, should already have this API. Looks like just calling abort() to the request object is enough(=same as node).

So, isn't this mean we can use same codebase as node to get this "cancellable fetch" working in the browser? I'd love to test my idea but I'm not sure how to do so yet, so just posting comment. Hope this helps.

@alanshaw

This comment has been minimized.

Copy link
Member

commented Aug 28, 2018

This sounds reasonable to me @shunkino - I'd ❤️ to see a proposal PR 😄

@alanshaw alanshaw referenced this issue Sep 6, 2018
@shunkino

This comment has been minimized.

Copy link
Contributor

commented Sep 7, 2018

--- UPDATED ---
Find out that my code works on Firefox (Quantum 61.0.2 on MacOS)...
Research and status on the problem mentioned below were about Google Chrome (version 69.0.3497.81).

Still not sure why the test on Chrome failed...
---

I've made a change to the /src/pubsub.js according to my findings above. But something is still wrong, and I was debugging that these few days. Unfortunately, I'm stuck now and need help...

I modified the pubsub.js as below. It's basically adding condition to detect whether the browser have the AbortController functionality.
https://gist.github.com/shunkino/baf26def13aca1ae6949d3ff1e0f939b

Modified version seems to work properly inside browser, but failing the test in interface-ipfs-core.

    .pubsub.publish
      ✓ should error on string messags
      ✓ should publish message from buffer (42ms)
      ✓ should publish 10 times within time limit (251ms)
    .pubsub.subscribe
      single node
        1) should subscribe to one topic
        2) should subscribe to one topic (promised)
        3) should subscribe to one topic with options
        4) should subscribe to one topic with options (promised)
        5) should subscribe to topic multiple times with different handlers
        6) should allow discover option to be passed
      multiple connected nodes
        ✓ should receive messages from a different node (77ms)
        7) "after each" hook for "should receive messages from a different node"
    .pubsub.unsubscribe
      8) should subscribe and unsubscribe 10 times
    .pubsub.peers
      ✓ should not error when not subscribed to a topic (51ms)
      9) should not return extra peers
      10) should return peers for a topic - one peer
      11) should return peers for a topic - multiple peers
    .pubsub.ls
      ✓ should return an empty list when no topics are subscribed (14ms)
      12) should return a list with 1 subscribed topic
      13) should return a list with 3 subscribed topics

To identify why the test is failing, I created the small app based on the example.
(I deployed the test app to /ipfs/Qme1GWMpHuErWGcGXbF7NsDBNKB5yMsT8sSeRP3s8wopQ1/ 😜 You can debug it inside browser. Click testFunction button to test.)

Looks like callback here is not called.

Furthermore, Inside the unsubscribe(), callback given to the eos() was not called. I don't know anything about end-of-stream and once library it depends. Also it was difficult to dig deeper so I'm stack here now.

Suspicious factors:

  • Event emitting from stream-http, readable-stream
  • Function wrapper in eos(), once library

Same function worked perfectly in node.js. Any comment and advice are welcome, thank you.

@klueq

This comment has been minimized.

Copy link

commented Oct 1, 2018

Is there a way to enable pubsub locally?

@whyrusleeping

This comment has been minimized.

Copy link
Member

commented Dec 10, 2018

any updates here?

@mccoysc

This comment was marked as off-topic.

Copy link

commented Dec 13, 2018

@whyrusleeping i like your cat........

@daviddias

This comment has been minimized.

Copy link
Member Author

commented May 23, 2019

@shunkino did you happen to make a PR with your progress?

@alanshaw alanshaw referenced this issue Jul 25, 2019
1 of 1 task complete
alanshaw added a commit that referenced this issue Aug 28, 2019
feat: browser pubsub (#1059)
This PR enabled pubsub in the browser and paves the way for a switch to using `fetch` by default and allowing for cancelable requests via the use of `AbortController`.

It's mostly the work done in ipfs-shipyard/js-ipfs-http-client-lite#1 but adapted a bit for use here.

If approved, we can start work moving the other commands to use `fetch`. The work in https://github.com/ipfs-shipyard/js-ipfs-http-client-lite has proven the hard parts (uploading files) are all possible using the `fetch` API.

Since `fetch` is promise based, when moving the other commands it makes sense to just switch to async/await as per ipfs/js-ipfs#1670 (and callbackify instead of promisify).

Depends on:

* [x] ipfs/interface-js-ipfs-core#505

resolves #518
refs ipfs/js-ipfs#2093
resolves #932

License: MIT
Signed-off-by: Alan Shaw <alan.shaw@protocol.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.