New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove foreign fetch #1188

Closed
jakearchibald opened this Issue Aug 17, 2017 · 23 comments

Comments

Projects
None yet
10 participants
@jakearchibald
Collaborator

jakearchibald commented Aug 17, 2017

Discussed in #1173.

Due to problems with double-keying, unclear trial results, and unclear use-cases, we're going to remove foreign fetch from the spec (and fetch spec).

We can reexamine use-cases later and look to reintroduce it in another form once we have better data.

@jakearchibald jakearchibald self-assigned this Aug 17, 2017

@Jxck

This comment has been minimized.

Show comment
Hide comment
@Jxck

Jxck Aug 18, 2017

what is double-keying and why that disables foreign-fetch ?
(I'm +1 for foreign fetch, Separation of Concern in service worker)

Jxck commented Aug 18, 2017

what is double-keying and why that disables foreign-fetch ?
(I'm +1 for foreign fetch, Separation of Concern in service worker)

@mattto

This comment has been minimized.

Show comment
Hide comment
@mattto

mattto Aug 22, 2017

Member

Double-keying refers to the system some browsers do for separating data set by a cross-origin iframe from the top frame, something like this: https://bugzilla.mozilla.org/show_bug.cgi?id=565965

Member

mattto commented Aug 22, 2017

Double-keying refers to the system some browsers do for separating data set by a cross-origin iframe from the top frame, something like this: https://bugzilla.mozilla.org/show_bug.cgi?id=565965

@Jxck

This comment has been minimized.

Show comment
Hide comment
@Jxck

Jxck Aug 22, 2017

thanks @mattto, but how this suffers foreign-fetch ?

Jxck commented Aug 22, 2017

thanks @mattto, but how this suffers foreign-fetch ?

@jakearchibald

This comment has been minimized.

Show comment
Hide comment
@jakearchibald

jakearchibald Aug 22, 2017

Collaborator

One of the use-cases we had was font caching, where fonts.google.com would have its own service worker that handled its own caching strategies.

With double keying, when example.com uses the fonts.google.com foreign worker, it has storage and execution keyed to example.com+fonts.google.com. When jakearchibald.com uses the same foreign worker, the storage and execution is keyed to jakearchibald.com+fonts.google.com.

This results in the same fonts being stored multiple times, for each combination.

Collaborator

jakearchibald commented Aug 22, 2017

One of the use-cases we had was font caching, where fonts.google.com would have its own service worker that handled its own caching strategies.

With double keying, when example.com uses the fonts.google.com foreign worker, it has storage and execution keyed to example.com+fonts.google.com. When jakearchibald.com uses the same foreign worker, the storage and execution is keyed to jakearchibald.com+fonts.google.com.

This results in the same fonts being stored multiple times, for each combination.

@Jxck

This comment has been minimized.

Show comment
Hide comment
@Jxck

Jxck Aug 22, 2017

@jakearchibald make sense, thanks!

Jxck commented Aug 22, 2017

@jakearchibald make sense, thanks!

@jakearchibald

This comment has been minimized.

Show comment
Hide comment
@jakearchibald

jakearchibald Aug 25, 2017

Collaborator

A foreign-fetch use-case: Kinda polyfilling something like cache digests https://twitter.com/mjackson/status/901090486739378177.

Double-keying would somewhat get in the way here, although you'd be able to send details of the double-keyed storage back to the CDN.

Collaborator

jakearchibald commented Aug 25, 2017

A foreign-fetch use-case: Kinda polyfilling something like cache digests https://twitter.com/mjackson/status/901090486739378177.

Double-keying would somewhat get in the way here, although you'd be able to send details of the double-keyed storage back to the CDN.

annevk added a commit to whatwg/fetch that referenced this issue Sep 4, 2017

annevk added a commit to whatwg/fetch that referenced this issue Sep 5, 2017

@Jxck

This comment has been minimized.

Show comment
Hide comment
@Jxck

Jxck Sep 8, 2017

is there potensial alternative for foreign-fetch?

Third party like Ad probider, analytics, CDN seems has a use cases, and I'm waitkng for foreign-fetch for avoid/separate handling fetch of these 3rdP request.

Jxck commented Sep 8, 2017

is there potensial alternative for foreign-fetch?

Third party like Ad probider, analytics, CDN seems has a use cases, and I'm waitkng for foreign-fetch for avoid/separate handling fetch of these 3rdP request.

@jakearchibald

This comment has been minimized.

Show comment
Hide comment
@jakearchibald

jakearchibald Sep 11, 2017

Collaborator

@Jxck I haven't heard a proposal that meets the use cases while preserving privacy.

Collaborator

jakearchibald commented Sep 11, 2017

@Jxck I haven't heard a proposal that meets the use cases while preserving privacy.

@mkruisselbrink

This comment has been minimized.

Show comment
Hide comment
@mkruisselbrink

mkruisselbrink Sep 12, 2017

Collaborator

+1 to removing foreign fetch from the spec. Maybe less clear what to do about Link: headers and elements for installing service workers? Either remove completely as well, or limit to processing these only on top-level (documents/workers) loads?

Collaborator

mkruisselbrink commented Sep 12, 2017

+1 to removing foreign fetch from the spec. Maybe less clear what to do about Link: headers and elements for installing service workers? Either remove completely as well, or limit to processing these only on top-level (documents/workers) loads?

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Sep 14, 2017

Member

Please also ensure tests are updated and browser bugs get filed. See whatwg/fetch#596 (comment) for details.

Member

annevk commented Sep 14, 2017

Please also ensure tests are updated and browser bugs get filed. See whatwg/fetch#596 (comment) for details.

@rektide

This comment has been minimized.

Show comment
Hide comment
@rektide

rektide Sep 14, 2017

One example use case I was working towards- I wanted to build a audioscrobbling service. It would let any media player register that the user was viewing something. Without foreign fetch, this is impossible to do in an offline case.

I was also working on a library 0hub to fulfill some of my early hopes for navigator-connect, which is enabling discovery. Rather than have to know about my audioscrobbling service, I was hoping to make a service where other services could register. My scrobbler could register itself, as could other scrobblers, and then anyone who wanted to post scrobbles could query for any scrobbling services and push to all of them.

Later down the road I intended to implement a feed reader around this premise.

This is truly one of the saddest things I have ever heard for the web, my lightyears & lightyears. Offline will be savagely ruined, not web at all, if we can only work offline with ourselves. The web has to have some kind of functional, interconnected offline capability. It has to.

I totally would not expect radical tech like this to have fast uptake. It needs half a decade of people playing around with it and learning about it and mainstreaming and libraries. We barely have service workers. Please, let new trials begin. Soon. This is incredibly deeply saddening to hear of.

rektide commented Sep 14, 2017

One example use case I was working towards- I wanted to build a audioscrobbling service. It would let any media player register that the user was viewing something. Without foreign fetch, this is impossible to do in an offline case.

I was also working on a library 0hub to fulfill some of my early hopes for navigator-connect, which is enabling discovery. Rather than have to know about my audioscrobbling service, I was hoping to make a service where other services could register. My scrobbler could register itself, as could other scrobblers, and then anyone who wanted to post scrobbles could query for any scrobbling services and push to all of them.

Later down the road I intended to implement a feed reader around this premise.

This is truly one of the saddest things I have ever heard for the web, my lightyears & lightyears. Offline will be savagely ruined, not web at all, if we can only work offline with ourselves. The web has to have some kind of functional, interconnected offline capability. It has to.

I totally would not expect radical tech like this to have fast uptake. It needs half a decade of people playing around with it and learning about it and mainstreaming and libraries. We barely have service workers. Please, let new trials begin. Soon. This is incredibly deeply saddening to hear of.

@mjackson

This comment has been minimized.

Show comment
Hide comment
@mjackson

mjackson Sep 15, 2017

Thanks for posting my tweet here, @jakearchibald 😅

Just wanted to chime in and say that I think the possibility of using foreign fetch to improve caching behavior for a CDN like unpkg.com is really appealing, especially with the advent of web modules. FF would make it possible to build better support for a module-level cache.

mjackson commented Sep 15, 2017

Thanks for posting my tweet here, @jakearchibald 😅

Just wanted to chime in and say that I think the possibility of using foreign fetch to improve caching behavior for a CDN like unpkg.com is really appealing, especially with the advent of web modules. FF would make it possible to build better support for a module-level cache.

@phamann

This comment has been minimized.

Show comment
Hide comment
@phamann

phamann Sep 15, 2017

Maybe less clear what to do about Link: headers and elements for installing service workers?

I would like to also echo this, whilst I'm not too concerned about loosing foreign fetch for the time being. The ability to install a Service Worker via a Link header opened a lot of very interesting potentials for delivering dynamic client-side caching logic via CDNs that act as a proxy for the first party domain, without having to compromise security or mutate html document responses. The above mentioned cache-digests polyfills is one such example.

phamann commented Sep 15, 2017

Maybe less clear what to do about Link: headers and elements for installing service workers?

I would like to also echo this, whilst I'm not too concerned about loosing foreign fetch for the time being. The ability to install a Service Worker via a Link header opened a lot of very interesting potentials for delivering dynamic client-side caching logic via CDNs that act as a proxy for the first party domain, without having to compromise security or mutate html document responses. The above mentioned cache-digests polyfills is one such example.

MXEBot pushed a commit to mirror/chromium that referenced this issue Sep 17, 2017

Chris Palmer Commit Bot
Remove the mention of Foreign Fetch from the SW Security FAQ.
Foreign Fetch is being removed from the Service Worker spec.
(w3c/ServiceWorker#1188 (comment))

TBR=rsesek

Bug: None
Change-Id: If84db57f7d62d065e389f97bbc100ae5d5e6f84b
Reviewed-on: https://chromium-review.googlesource.com/669740
Reviewed-by: Chris Palmer <palmer@chromium.org>
Commit-Queue: Chris Palmer <palmer@chromium.org>
Cr-Commit-Position: refs/heads/master@{#502510}
@rektide

This comment has been minimized.

Show comment
Hide comment
@rektide

rektide Mar 28, 2018

This can be closed, as per #1207, I believe. But I would very much like a clearer path on knowing what challenges are to re-open it, and to hear thoughts on what can be done to help advance this fantastically hugely important capability that greatly facilitates & is necessary for a useful offline web.

rektide commented Mar 28, 2018

This can be closed, as per #1207, I believe. But I would very much like a clearer path on knowing what challenges are to re-open it, and to hear thoughts on what can be done to help advance this fantastically hugely important capability that greatly facilitates & is necessary for a useful offline web.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Mar 28, 2018

Member

You'd need to come up with a way of adding them without making tracking worse.

Member

annevk commented Mar 28, 2018

You'd need to come up with a way of adding them without making tracking worse.

@annevk annevk closed this Mar 28, 2018

@jozanza

This comment has been minimized.

Show comment
Hide comment
@jozanza

jozanza May 31, 2018

Does anyone know the status/direction of foreign fetch? It still seems incredibly useful, and I'd love a chance to build something with it. Are there plans for future Origin Trials in Chrome? Or is this feature entirely deprecated with no plans to be implemented any longer?

jozanza commented May 31, 2018

Does anyone know the status/direction of foreign fetch? It still seems incredibly useful, and I'd love a chance to build something with it. Are there plans for future Origin Trials in Chrome? Or is this feature entirely deprecated with no plans to be implemented any longer?

@jeffposnick jeffposnick referenced this issue May 31, 2018

Merged

Foreign fetch was removed from the SW spec. #6207

3 of 4 tasks complete
@mattto

This comment has been minimized.

Show comment
Hide comment
@mattto

mattto Jun 1, 2018

Member

Chrome has no plans currently to reimplement foreign fetch.

Member

mattto commented Jun 1, 2018

Chrome has no plans currently to reimplement foreign fetch.

@jakearchibald

This comment has been minimized.

Show comment
Hide comment
@jakearchibald

jakearchibald Jun 1, 2018

Collaborator

@jozanza what are you wanting to do with it?

Collaborator

jakearchibald commented Jun 1, 2018

@jozanza what are you wanting to do with it?

@jozanza

This comment has been minimized.

Show comment
Hide comment
@jozanza

jozanza Jun 1, 2018

@jakearchibald unless I’m misunderstanding how it works, foreignfetch seems like a huge boon to webrtc. I’d want to use it to cache offers/answers for RTCPeerConnection signaling. And once connected, it could also be used to scalably relay media streams without a CDN.

jozanza commented Jun 1, 2018

@jakearchibald unless I’m misunderstanding how it works, foreignfetch seems like a huge boon to webrtc. I’d want to use it to cache offers/answers for RTCPeerConnection signaling. And once connected, it could also be used to scalably relay media streams without a CDN.

@jakearchibald

This comment has been minimized.

Show comment
Hide comment
@jakearchibald

jakearchibald Jun 1, 2018

Collaborator

@jozanza Are you speaking as the person who'd own the RTCPeerConnection signaling server, or the person who'd run the site using the RTCPeerConnection signaling server?

Collaborator

jakearchibald commented Jun 1, 2018

@jozanza Are you speaking as the person who'd own the RTCPeerConnection signaling server, or the person who'd run the site using the RTCPeerConnection signaling server?

@jozanza

This comment has been minimized.

Show comment
Hide comment
@jozanza

jozanza Jun 1, 2018

@jakearchibald I started writing and was having a hard time describing what I thinking clearly. So I wrote some pseudo code here. It shows more or less what I was hoping would be possible:

async function sendMediaToPeer({ config, from, to }) {
  // Create a peer connection
  const pc = new RTCPeerConnection(config);
  // Get user's video/audio stream
  const stream = await navigator.mediaDevices.getUserMedia({
    video: true,
    audio: true
  });
  // Do all the signaling with a foreign fetch service worker 🤞
  pc.onnegotiationneeded = async () => {
    // Create offer
    const offer = await pc.createOffer();
    // Add media stream
    pc.addStream(stream);
    await pc.setLocalDescription(offer);
    // Gather all ice candidates for simplified signaling
    while (true) {
      if (pc.iceGatheringState === "complete") break;
      await new Promise(f => setTimeout(f, 100));
    }
    // Send offer, gets intercepted by service worker
    // The worker can store the offer with the Cache API
    const res = await fetch(`${API_ROOT}/${from}/offers`, {
      method: "POST",
      headers: { "content-type": "application/json" },
      body: JSON.stringify({ to, offer })
    });
    if (!res.ok) throw new Error("Could not create offer");
    // The offer is now cached in the service worker so
    // We can just poll for answer from the intended peer
    // (They would use the service worker to post their answer)
    while (true) {
      const res = await fetch(`${API_ROOT}/${to}/answers`);
      const answer = await res.json();
      if (answer) {
        // Aaaand we're connected! :)
        pc.setRemoteDescription(answer);
        break;
      }
    }
  };
}

tl;dr It'd be pretty amazing, if all of the signaling between peers could be done in a serverless manner by relying on a common foreign fetch service worker storing offers/answer with "a single, authoritative cache instance".

And the signaling code could obviously be even cleaner if the foreign fetch service worker also supported onmessage() and Client.postMessage(). It could almost be a replacement for WebSocket server at that point. But again, I'm not sure if I totally misunderstood what foreign fetch is intended for and capable of. Admittedly, this all just seems too good to be possible 😅.

jozanza commented Jun 1, 2018

@jakearchibald I started writing and was having a hard time describing what I thinking clearly. So I wrote some pseudo code here. It shows more or less what I was hoping would be possible:

async function sendMediaToPeer({ config, from, to }) {
  // Create a peer connection
  const pc = new RTCPeerConnection(config);
  // Get user's video/audio stream
  const stream = await navigator.mediaDevices.getUserMedia({
    video: true,
    audio: true
  });
  // Do all the signaling with a foreign fetch service worker 🤞
  pc.onnegotiationneeded = async () => {
    // Create offer
    const offer = await pc.createOffer();
    // Add media stream
    pc.addStream(stream);
    await pc.setLocalDescription(offer);
    // Gather all ice candidates for simplified signaling
    while (true) {
      if (pc.iceGatheringState === "complete") break;
      await new Promise(f => setTimeout(f, 100));
    }
    // Send offer, gets intercepted by service worker
    // The worker can store the offer with the Cache API
    const res = await fetch(`${API_ROOT}/${from}/offers`, {
      method: "POST",
      headers: { "content-type": "application/json" },
      body: JSON.stringify({ to, offer })
    });
    if (!res.ok) throw new Error("Could not create offer");
    // The offer is now cached in the service worker so
    // We can just poll for answer from the intended peer
    // (They would use the service worker to post their answer)
    while (true) {
      const res = await fetch(`${API_ROOT}/${to}/answers`);
      const answer = await res.json();
      if (answer) {
        // Aaaand we're connected! :)
        pc.setRemoteDescription(answer);
        break;
      }
    }
  };
}

tl;dr It'd be pretty amazing, if all of the signaling between peers could be done in a serverless manner by relying on a common foreign fetch service worker storing offers/answer with "a single, authoritative cache instance".

And the signaling code could obviously be even cleaner if the foreign fetch service worker also supported onmessage() and Client.postMessage(). It could almost be a replacement for WebSocket server at that point. But again, I'm not sure if I totally misunderstood what foreign fetch is intended for and capable of. Admittedly, this all just seems too good to be possible 😅.

@joymon

This comment has been minimized.

Show comment
Hide comment
@joymon

joymon Sep 13, 2018

Does this mean if my site is in example.com and consuming api.example.com, the API requests cannot be intercepted by fetch due to cross domain limits?

joymon commented Sep 13, 2018

Does this mean if my site is in example.com and consuming api.example.com, the API requests cannot be intercepted by fetch due to cross domain limits?

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Sep 14, 2018

Member

They can be intercepted by example.com, not by api.example.com.

Member

annevk commented Sep 14, 2018

They can be intercepted by example.com, not by api.example.com.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment