Define agents and agent clusters #2521

Merged
merged 2 commits into from Apr 26, 2017

Conversation

@annevk
Member

annevk commented Apr 11, 2017

This puts the infrastructure in place to define SharedArrayBuffer in
detail. See also #2260.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 11, 2017

Member

This requires #2520 to land first. Ideally we also sort out tc39/ecma262#882 first, though with implementations close to shipping somehow this is better than nothing.

The other open issue is worklets (see https://drafts.css-houdini.org/worklets/), but that is potentially easy. @bfgeek can a worklet ever be owned by something other than a document? Do we want to allow shared memory with worklets, ever?

Member

annevk commented Apr 11, 2017

This requires #2520 to land first. Ideally we also sort out tc39/ecma262#882 first, though with implementations close to shipping somehow this is better than nothing.

The other open issue is worklets (see https://drafts.css-houdini.org/worklets/), but that is potentially easy. @bfgeek can a worklet ever be owned by something other than a document? Do we want to allow shared memory with worklets, ever?

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 11, 2017

Member

cc @bakulf @lars-t-hansen

@bzbarsky I suspect you may also find this interesting and worthy of nitpicking as it effectively attempts to formalize the boundaries of processes and shared memory within the browser.

Member

annevk commented Apr 11, 2017

cc @bakulf @lars-t-hansen

@bzbarsky I suspect you may also find this interesting and worthy of nitpicking as it effectively attempts to formalize the boundaries of processes and shared memory within the browser.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 11, 2017

Member

I'd like to not merge this until we get tc39/ecma262#882 sorted out. We can have the discussion over there, but this is a pretty serious mismatch with my understanding of the purpose of the agent formalism. If we did this then there'd be no point in agents and we could just use realms.

Member

domenic commented Apr 11, 2017

I'd like to not merge this until we get tc39/ecma262#882 sorted out. We can have the discussion over there, but this is a pretty serious mismatch with my understanding of the purpose of the agent formalism. If we did this then there'd be no point in agents and we could just use realms.

@bfgeek

This comment has been minimized.

Show comment
Hide comment
@bfgeek

bfgeek Apr 11, 2017

Member

@annevk

Worklets can only be owned by a document at the moment, I don't see I need yet to break that at all yet.

Shared memory - no, at the moment. We'd also disallow the futex apis. There may be a need later on, but we can cross that bridge when we come to it.

Member

bfgeek commented Apr 11, 2017

@annevk

Worklets can only be owned by a document at the moment, I don't see I need yet to break that at all yet.

Shared memory - no, at the moment. We'd also disallow the futex apis. There may be a need later on, but we can cross that bridge when we come to it.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 11, 2017

Member

@bfgeek okay, so we'd basically put each worklet in its own agent cluster. Then it can create SharedArrayBuffer (as each JavaScript environment can) but not share it with anyone.

Member

annevk commented Apr 11, 2017

@bfgeek okay, so we'd basically put each worklet in its own agent cluster. Then it can create SharedArrayBuffer (as each JavaScript environment can) but not share it with anyone.

@annevk annevk referenced this pull request in w3c/css-houdini-drafts Apr 11, 2017

Open

[worklets] relationship to JavaScript agent clusters #380

@bfgeek

This comment has been minimized.

Show comment
Hide comment
@bfgeek

bfgeek Apr 11, 2017

Member

@annevk Great, that makes sense at the moment.

Member

bfgeek commented Apr 11, 2017

@annevk Great, that makes sense at the moment.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 12, 2017

Member

Learned something new from @flagxor in WebAudio/web-audio-api#1194 (comment): https://tc39.github.io/ecma262/#sec-agentcansuspend. We'll need to actually define the value of the [[CanBlock]] field of agents (true for workers, false elsewhere it sounds like, including worklets).

Not sure if we need to define any other values, though if tc39/ecma262#882 (comment) ends up being applied, there might be a lot. (I'm not sure we want to block on defining all those though and waiting for another ECMAScript revision. SharedArrayBuffer seems close to shipping, so we should focus on the overall outline and then work through the specifics. In particular, not having it part of the serialize and deserialize algorithm for another month seems strictly worse.)

Member

annevk commented Apr 12, 2017

Learned something new from @flagxor in WebAudio/web-audio-api#1194 (comment): https://tc39.github.io/ecma262/#sec-agentcansuspend. We'll need to actually define the value of the [[CanBlock]] field of agents (true for workers, false elsewhere it sounds like, including worklets).

Not sure if we need to define any other values, though if tc39/ecma262#882 (comment) ends up being applied, there might be a lot. (I'm not sure we want to block on defining all those though and waiting for another ECMAScript revision. SharedArrayBuffer seems close to shipping, so we should focus on the overall outline and then work through the specifics. In particular, not having it part of the serialize and deserialize algorithm for another month seems strictly worse.)

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 12, 2017

Member

I landed a new approach based on the updates in tc39/ecma262#882. I'll need decisions on w3c/css-houdini-drafts#385 and w3c/ServiceWorker#1115 though and @domenic will need to agree with using global objects as our way of defining agents until such a time arrives where can properly allocate agent clusters and agents at specific points in the various standards.

Member

annevk commented Apr 12, 2017

I landed a new approach based on the updates in tc39/ecma262#882. I'll need decisions on w3c/css-houdini-drafts#385 and w3c/ServiceWorker#1115 though and @domenic will need to agree with using global objects as our way of defining agents until such a time arrives where can properly allocate agent clusters and agents at specific points in the various standards.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 12, 2017

Member

Still going through my morning email but I wanted to add a reminder to us that we should be able to write web platform tests for the [[CanBlock]] stuff in each different context. (Maybe not worklets yet since nobody's shipped them?)

Member

domenic commented Apr 12, 2017

Still going through my morning email but I wanted to add a reminder to us that we should be able to write web platform tests for the [[CanBlock]] stuff in each different context. (Maybe not worklets yet since nobody's shipped them?)

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 12, 2017

Member

Oh, I actually read your message now. I'm not really OK with saying global objects === agents. I think we should not merge anything until we have event loop === agents. It's just way too misleading for implementers otherwise.

Member

domenic commented Apr 12, 2017

Oh, I actually read your message now. I'm not really OK with saying global objects === agents. I think we should not merge anything until we have event loop === agents. It's just way too misleading for implementers otherwise.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 12, 2017

Member

I'm not saying global objects === agents. I suggest reading the PR first. I think event loop === agents is the wrong way to go as they're obviously very different.

Member

annevk commented Apr 12, 2017

I'm not saying global objects === agents. I suggest reading the PR first. I think event loop === agents is the wrong way to go as they're obviously very different.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 12, 2017

Member

OK, I'll read the PR. I still am pretty sure event loops are === agents in implementations and that we want to reflect that in the spec, since SAB makes so much of the event loop machinery observable (i.e. ordering between independent event loops). But we'll see. Sorry for jumping the gun.

Member

domenic commented Apr 12, 2017

OK, I'll read the PR. I still am pretty sure event loops are === agents in implementations and that we want to reflect that in the spec, since SAB makes so much of the event loop machinery observable (i.e. ordering between independent event loops). But we'll see. Sorry for jumping the gun.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 12, 2017

Member

I could be convinced that we want to align event loops (in particular those for browsing contexts) with the scope of agents and have agents effectively hold an event loop, but I don't think that would actually match implementations. E.g., if you have a cross-origin <iframe> you cannot document.domain with, it's likely in the same event loop (out-of-process iframes are still somewhat experimental), but definitely a different agent cluster. And it also wouldn't be observable as you cannot share memory with that <iframe>. (Nevertheless, we could still reduce the scope of event loops as implementations could on-observably merge them again anyway.)

Member

annevk commented Apr 12, 2017

I could be convinced that we want to align event loops (in particular those for browsing contexts) with the scope of agents and have agents effectively hold an event loop, but I don't think that would actually match implementations. E.g., if you have a cross-origin <iframe> you cannot document.domain with, it's likely in the same event loop (out-of-process iframes are still somewhat experimental), but definitely a different agent cluster. And it also wouldn't be observable as you cannot share memory with that <iframe>. (Nevertheless, we could still reduce the scope of event loops as implementations could on-observably merge them again anyway.)

@domenic

OK, so I think this approach works, but it's very confusing right now because it makes it look like we define a distinct agent for each Window.

Maybe a way to phrase this is to flip it and define five classes of agents (one for each type of global), and then say something about how e.g. each URSOBC creates a single Window-agent. (As does each WorkerGlobalScope, WorkletGlobalScope, etc.) You can then add a note about how with these definitions, every Realm belongs to an agent.


Upon re-reading the spec I see I had previously misread and thought it said that each URSOBC got its own event loop. I think that may still be necessary with the stuff SAB makes observable, but I am not sure, and we don't need to block on redefining event loop, since this approach appears good. I will file a separate issue to check on that.

source
+
+ <p>JavaScript defines the concept of an <span>agent</span>. Until such a time that this standard
+ has a better handle on lifetimes, we define an <span>agent</span> from the perspective of a
+ <span>global object</span>:

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

I'm a little unclear on what lifetimes have to do with this, and what would be wrong with a definition based on global objects that we plan on fixing in the future

@domenic

domenic Apr 12, 2017

Member

I'm a little unclear on what lifetimes have to do with this, and what would be wrong with a definition based on global objects that we plan on fixing in the future

This comment has been minimized.

@annevk

annevk Apr 13, 2017

Member

In the future I'd just want CreateAgentCluster followed by CreateAgent at the appropriate places. But we don't clearly define when those things happen today.

@annevk

annevk Apr 13, 2017

Member

In the future I'd just want CreateAgentCluster followed by CreateAgent at the appropriate places. But we don't clearly define when those things happen today.

This comment has been minimized.

@domenic

domenic Apr 13, 2017

Member

Huh. I'm not sure exactly what the advantages would be of specifying the exact creation time of an agent, but I guess it seems like a reasonable path. Maybe make that more explicit, e.g.

NOTE: in the future, when this specification has a better handle on lifetimes, we hope to define exactly when agents and agent clusters are created.

@domenic

domenic Apr 13, 2017

Member

Huh. I'm not sure exactly what the advantages would be of specifying the exact creation time of an agent, but I guess it seems like a reasonable path. Maybe make that more explicit, e.g.

NOTE: in the future, when this specification has a better handle on lifetimes, we hope to define exactly when agents and agent clusters are created.

This comment has been minimized.

@annevk

annevk Apr 13, 2017

Member

The advantage is that it's much more clear to implementations when they can allocate a new process and where the boundaries are. That is, the lifetimes are more clear.

@annevk

annevk Apr 13, 2017

Member

The advantage is that it's much more clear to implementations when they can allocate a new process and where the boundaries are. That is, the lifetimes are more clear.

source
+ <span>global object</span>:
+
+ <dl class="switch">
+ <dt><code>Window</code></dt>

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

This is confusing. It makes it look like each Window has an agent. But e.g. a same-origin iframe's Window should not have its own agent, since that Window's realm is part of the parent Window's agent. Remember per AWB's comment: realms cannot be shared between two agents.

@domenic

domenic Apr 12, 2017

Member

This is confusing. It makes it look like each Window has an agent. But e.g. a same-origin iframe's Window should not have its own agent, since that Window's realm is part of the parent Window's agent. Remember per AWB's comment: realms cannot be shared between two agents.

This comment has been minimized.

@annevk

annevk Apr 13, 2017

Member

Isn't it clear from the definition what the agent is?

@annevk

annevk Apr 13, 2017

Member

Isn't it clear from the definition what the agent is?

This comment has been minimized.

@domenic

domenic Apr 13, 2017

Member

Not to me. The <dl> structure makes it look like you are defining a distinct agent for each global object.

@domenic

domenic Apr 13, 2017

Member

Not to me. The <dl> structure makes it look like you are defining a distinct agent for each global object.

This comment has been minimized.

@annevk

annevk Apr 13, 2017

Member

That's not what the paragraph before says.

@annevk

annevk Apr 13, 2017

Member

That's not what the paragraph before says.

This comment has been minimized.

@domenic

domenic Apr 13, 2017

Member

It says an agent is defined "from the perspective of" a global object. I have no idea what that means precisely, but my best guess is that you look at each global object in turn, and from its perspective you generate an agent, using the following definition.

@domenic

domenic Apr 13, 2017

Member

It says an agent is defined "from the perspective of" a global object. I have no idea what that means precisely, but my best guess is that you look at each global object in turn, and from its perspective you generate an agent, using the following definition.

This comment has been minimized.

@annevk

annevk Apr 13, 2017

Member

Okay, I don't want to generate agents, since you can't do that after the fact. I just want to explain their scope and features in relation to global objects, which are a fairly well understood concept.

@annevk

annevk Apr 13, 2017

Member

Okay, I don't want to generate agents, since you can't do that after the fact. I just want to explain their scope and features in relation to global objects, which are a fairly well understood concept.

This comment has been minimized.

@domenic

domenic Apr 13, 2017

Member

Did you see my comment above suggesting a way around this?

Maybe a way to phrase this is to flip it and define five classes of agents (one for each type of global), and then say something about how e.g. each URSOBC creates a single Window-agent. (As does each WorkerGlobalScope, WorkletGlobalScope, etc.) You can then add a note about how with these definitions, every Realm belongs to an agent.

I don't think this is "generating agents"; it's just defining what agents exist, given a known set of globals.

@domenic

domenic Apr 13, 2017

Member

Did you see my comment above suggesting a way around this?

Maybe a way to phrase this is to flip it and define five classes of agents (one for each type of global), and then say something about how e.g. each URSOBC creates a single Window-agent. (As does each WorkerGlobalScope, WorkletGlobalScope, etc.) You can then add a note about how with these definitions, every Realm belongs to an agent.

I don't think this is "generating agents"; it's just defining what agents exist, given a known set of globals.

This comment has been minimized.

@annevk

annevk Apr 13, 2017

Member

I missed that, thanks. I can give that a try. It would actually be useful to have names for these agents in discussion as well.

@annevk

annevk Apr 13, 2017

Member

I missed that, thanks. I can give that a try. It would actually be useful to have names for these agents in discussion as well.

source
+ <dd><p>The <span>agent</span>'s [[CanBlock]] is false, and its set of <span data-x="JavaScript
+ realm">JavaScript realms</span> consists of this <code>Window</code> object's <span
+ data-x="concept-global-object-realm">Realm</span>, and any other <span
+ data-x="concept-global-object-realm">Realm</span> of <code>Window</code> objects whose

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

Nit: Realms

@domenic

domenic Apr 12, 2017

Member

Nit: Realms

source
+ <dt><code>Window</code></dt>
+ <dd><p>The set containing <var>globalObject</var>'s <span>agent</span>'s set of realms's
+ <span data-x="concept-realm-global">global objects</span> and any
+ <code>DedicatedWorkerGlobalScope</code> object whose <span>the worker's owners</span> contains

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

Nits: "DedicatedWorkerGlobalScope objects"; "the worker's owners" -> "owners"

@domenic

domenic Apr 12, 2017

Member

Nits: "DedicatedWorkerGlobalScope objects"; "the worker's owners" -> "owners"

source
+ <li><var>globalObject</var>,</li>
+ <li>the <span data-x="concept-relevant-global">relevant global object</span> of
+ <var>globalObject</var>'s single <span data-x="the worker's owners">the worker's
+ owner</span>,</li>

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

Nit: just "owner"

@domenic

domenic Apr 12, 2017

Member

Nit: just "owner"

source
+ <var>globalObject</var>'s single <span data-x="the worker's owners">the worker's
+ owner</span>,</li>
+ <li>any <span data-x="concept-relevant-global">relevant global object</span> of
+ <var>globalObject</var>'s <span>the worker's workers</span> that is not a

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

Nit: linking text should be just "workers"

@domenic

domenic Apr 12, 2017

Member

Nit: linking text should be just "workers"

source
+ <li>the <span data-x="concept-relevant-global">relevant global object</span> of
+ <var>globalObject</var>'s single <span data-x="the worker's owners">the worker's
+ owner</span>,</li>
+ <li>any <span data-x="concept-relevant-global">relevant global object</span> of

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

Do workers have relevant global objects? I think the relevant global object of a worker is the global it was created in, not the WorkerGlobalScope that was created. Not sure what the spec uses for this; maybe "associated WorkerGlobalScope" or something.

@domenic

domenic Apr 12, 2017

Member

Do workers have relevant global objects? I think the relevant global object of a worker is the global it was created in, not the WorkerGlobalScope that was created. Not sure what the spec uses for this; maybe "associated WorkerGlobalScope" or something.

This comment has been minimized.

@annevk

annevk Apr 13, 2017

Member

"the worker's workers" are global objects.

@annevk

annevk Apr 13, 2017

Member

"the worker's workers" are global objects.

This comment has been minimized.

@annevk

annevk Apr 13, 2017

Member

Or documents, and the "or documents" is important for why I use relevant global object here.

@annevk

annevk Apr 13, 2017

Member

Or documents, and the "or documents" is important for why I use relevant global object here.

This comment has been minimized.

@domenic

domenic Apr 13, 2017

Member

I see, I seem to have been just confused here.

@domenic

domenic Apr 13, 2017

Member

I see, I seem to have been just confused here.

source
+ <code>ServiceWorkerGlobalScope</code>)</dt>
+ <dd><p>The set containing <var>globalObject</var> and any <span
+ data-x="concept-relevant-global">relevant global object</span> of <var>globalObject</var>'s
+ <span>the worker's workers</span> that is not a <code>SharedWorkerGlobalScope</code> objects.</p>

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

Nits: just "workers" for the link text; no plural on "objects"

@domenic

domenic Apr 12, 2017

Member

Nits: just "workers" for the link text; no plural on "objects"

source
+
+ <div class="example">
+ <p>The following pairs of global objects are each within the same <span>can-share-with global
+ objects</span>, and thus can use <code>SharedArrayBuffer</code> instances to share memory with

This comment has been minimized.

@domenic

domenic Apr 12, 2017

Member

Nit: phrasing is awkward here, which is part of why I used "unit of" before. "Set of" might also work, either as part of the dfn or not.

@domenic

domenic Apr 12, 2017

Member

Nit: phrasing is awkward here, which is part of why I used "unit of" before. "Set of" might also work, either as part of the dfn or not.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 13, 2017

Member

Thoughts:

  1. Getting review is awesome. All standards should do it.
  2. Defining things on top of browsing contexts is a little messy, but unless figure out how to reword all that and test it, I don't see it changing soon. I filed #2528 as a follow-up.
  3. It might actually be good if the Worklet specification defined Agent allocation itself and we'd refer to it when defining how it sits in an agent subcluster. That makes it much easier to put multiple global objects in the same agent as it allocates them all closely together in the same spot. And instead of a worklet holding a collection of global objects it would end up holding a worklet agent. (Technically giving each global object its own agent is not observable, but it feels wrong.)
  4. I've tentatively decided that service workers cannot block based on initial feedback in w3c/ServiceWorker#1115 and my own opinion. I don't have verification of what implementations actually do though.
  5. If the current text looks okay to you apart from nits (and issues outlined above) I think we should also have it vetted by some other people, in particular senior engineers and those involved in the creation of agents. Before we do that I can generate an HTML copy that makes it easier to review.
Member

annevk commented Apr 13, 2017

Thoughts:

  1. Getting review is awesome. All standards should do it.
  2. Defining things on top of browsing contexts is a little messy, but unless figure out how to reword all that and test it, I don't see it changing soon. I filed #2528 as a follow-up.
  3. It might actually be good if the Worklet specification defined Agent allocation itself and we'd refer to it when defining how it sits in an agent subcluster. That makes it much easier to put multiple global objects in the same agent as it allocates them all closely together in the same spot. And instead of a worklet holding a collection of global objects it would end up holding a worklet agent. (Technically giving each global object its own agent is not observable, but it feels wrong.)
  4. I've tentatively decided that service workers cannot block based on initial feedback in w3c/ServiceWorker#1115 and my own opinion. I don't have verification of what implementations actually do though.
  5. If the current text looks okay to you apart from nits (and issues outlined above) I think we should also have it vetted by some other people, in particular senior engineers and those involved in the creation of agents. Before we do that I can generate an HTML copy that makes it easier to review.
@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 13, 2017

Member

Reportedly Firefox only sets [[CanBlock]] to true for dedicated worker agents. I'm not sure yet why not also for shared worker agents.

Member

annevk commented Apr 13, 2017

Reportedly Firefox only sets [[CanBlock]] to true for dedicated worker agents. I'm not sure yet why not also for shared worker agents.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 13, 2017

Member

Pushed some editorial things. Really like this approach; thanks for taking it on.

I think the tests we should land for this are basically the [[CanBlock]] tests. I can again work on those (but again feel free if I haven't taken over). It sounds like worklets should not get [[CanBlock]], but we're not sure yet about shared workers? @flagxor @binji thoughts from the Blink side? @lars-t-hansen @bakulf thoughts from Gecko? (Since those are the only two browsers implementing shared workers right now.)

We'll have other tests for the actual agent cluster boundaries as part of #2518.

I don't think we should necessarily block this on getting clarity on worklets' agent clusters since worklets are so young and fresh. But it would be good to keep the pressure on @bfgeek and others :).

Member

domenic commented Apr 13, 2017

Pushed some editorial things. Really like this approach; thanks for taking it on.

I think the tests we should land for this are basically the [[CanBlock]] tests. I can again work on those (but again feel free if I haven't taken over). It sounds like worklets should not get [[CanBlock]], but we're not sure yet about shared workers? @flagxor @binji thoughts from the Blink side? @lars-t-hansen @bakulf thoughts from Gecko? (Since those are the only two browsers implementing shared workers right now.)

We'll have other tests for the actual agent cluster boundaries as part of #2518.

I don't think we should necessarily block this on getting clarity on worklets' agent clusters since worklets are so young and fresh. But it would be good to keep the pressure on @bfgeek and others :).

@domenic domenic added the needs tests label Apr 13, 2017

@annevk annevk referenced this pull request in w3c/css-houdini-drafts Apr 14, 2017

Open

Why is there a responsible browsing context, but no document? #224

domenic added a commit to web-platform-tests/wpt that referenced this pull request Apr 14, 2017

Test [[CanBlock]] in various agents
Follows whatwg/html#2521. This downgrades the resources submodule due to #5568 which prevents these tests (and many, many other tests in WPT) from working.

Worklets are not tested at this time due to spec uncertainty.

@domenic domenic referenced this pull request in web-platform-tests/wpt Apr 14, 2017

Merged

Test [[CanBlock]] in various agents #5569

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 14, 2017

Member

This LGTM. We have w3c/css-houdini-drafts#380 tracking worklets and tests for [[CanBlock]] at web-platform-tests/wpt#5569. I've created a rendered version of the spec for wider review at https://dl.dropboxusercontent.com/u/20140634/agents/multipage/webappapis.html#integration-with-the-javascript-agent-formalism although I'm not sure who we should be pinging.

Member

domenic commented Apr 14, 2017

This LGTM. We have w3c/css-houdini-drafts#380 tracking worklets and tests for [[CanBlock]] at web-platform-tests/wpt#5569. I've created a rendered version of the spec for wider review at https://dl.dropboxusercontent.com/u/20140634/agents/multipage/webappapis.html#integration-with-the-javascript-agent-formalism although I'm not sure who we should be pinging.

domenic added a commit to web-platform-tests/wpt that referenced this pull request Apr 14, 2017

Test [[CanBlock]] in various agents
Follows whatwg/html#2521. This downgrades the resources submodule due to #5568 which prevents these tests (and many, many other tests in WPT) from working.

Worklets are not tested at this time due to spec uncertainty.
@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 14, 2017

Member

@binji pointed out that the atomics APIs are useless in shared workers in Chrome (1/2 shared worker implementers), since without nested workers there will be no other realms in the agent cluster for you to share with. This might argue for setting [[CanBlock]] to false there.

But he also says (and I agree) that it's better to be more lenient than more restrictive. So we're OK keeping [[CanBlock]] to true, as in this PR.

Member

domenic commented Apr 14, 2017

@binji pointed out that the atomics APIs are useless in shared workers in Chrome (1/2 shared worker implementers), since without nested workers there will be no other realms in the agent cluster for you to share with. This might argue for setting [[CanBlock]] to false there.

But he also says (and I agree) that it's better to be more lenient than more restrictive. So we're OK keeping [[CanBlock]] to true, as in this PR.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 14, 2017

Member

@pizlonator your review on the latest plan would be especially appreciated; it's pretty far from Apple's implementation, as you described it in #2361 (comment), but you said you didn't feel strongly, so I hope you'd be OK updating.

The draft spec for "agent clusters" is at https://dl.dropboxusercontent.com/u/20140634/agents/multipage/webappapis.html#integration-with-the-javascript-agent-formalism. The idea is that within an agent cluster, postMessaging will share memory. Outside the agent cluster, postMessaging will fail, causing the target to receive a messageerror event (see #2530) instead of a message event.

What do you think?

Member

domenic commented Apr 14, 2017

@pizlonator your review on the latest plan would be especially appreciated; it's pretty far from Apple's implementation, as you described it in #2361 (comment), but you said you didn't feel strongly, so I hope you'd be OK updating.

The draft spec for "agent clusters" is at https://dl.dropboxusercontent.com/u/20140634/agents/multipage/webappapis.html#integration-with-the-javascript-agent-formalism. The idea is that within an agent cluster, postMessaging will share memory. Outside the agent cluster, postMessaging will fail, causing the target to receive a messageerror event (see #2530) instead of a message event.

What do you think?

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 15, 2017

Member

Thanks for creating https://dl.dropboxusercontent.com/u/20140634/agents/multipage/webappapis.html#integration-with-the-javascript-agent-formalism. I'd like @bzbarsky @foolip @lars-t-hansen @smaug---- @othermaciej @travisleithead to take a look as agents and agent clusters define rather fundamental architecture of a browser. To be clear, I don't think we should block on all of them having reviewed this before landing, but waiting a couple working days seems good.

Member

annevk commented Apr 15, 2017

Thanks for creating https://dl.dropboxusercontent.com/u/20140634/agents/multipage/webappapis.html#integration-with-the-javascript-agent-formalism. I'd like @bzbarsky @foolip @lars-t-hansen @smaug---- @othermaciej @travisleithead to take a look as agents and agent clusters define rather fundamental architecture of a browser. To be clear, I don't think we should block on all of them having reviewed this before landing, but waiting a couple working days seems good.

source
+ subclusters</span>, if <var>subcluster</var> contains an <span>agent</span> that is also
+ contained by one of the <span data-x="agent subcluster">agent subclusters</span> in
+ <var>cluster</var>, then <span data-x="set append">append</span> <var>subcluster</var> to
+ <var>cluster</var>.</p></li>

This comment has been minimized.

@domenic

domenic Apr 19, 2017

Member

Did you need to repeat this step until it stabilizes, like @foolip suggested?

@domenic

domenic Apr 19, 2017

Member

Did you need to repeat this step until it stabilizes, like @foolip suggested?

This comment has been minimized.

@annevk

annevk Apr 19, 2017

Member

Sigh, take 3n:

Let allSubclusters be all agent subclusters.

While allSubclusters contains an agent subcluster subcluster which contains an agent that is also contained by an agent subcluster contained by cluster, remove subcluster from allSubclusters and append subcluster to cluster.

@annevk

annevk Apr 19, 2017

Member

Sigh, take 3n:

Let allSubclusters be all agent subclusters.

While allSubclusters contains an agent subcluster subcluster which contains an agent that is also contained by an agent subcluster contained by cluster, remove subcluster from allSubclusters and append subcluster to cluster.

This comment has been minimized.

@domenic

domenic Apr 19, 2017

Member

That "while" clause is getting pretty long... let's see. How about this?

  1. Let subclustersInCluster be « cluster ».
  2. Let allSubclusters be a set containing all agent subclusters.
  3. While true:
    1. Let agentsInClusterSoFar be the set containing all agents in all the agent subclusters in subclustersInCluster.
    2. If allSubclusters contains any agent subcluster subcluster that contains an agent also in agnentsInClusterSoFar, then
      1. Append subcluster to subclustersInCluster.
      2. Remove subcluster from allSubclusters.
    3. Otherwise, break.
  4. Return the set containing all agents in all the agent subclusters in subclustersInCluster.
@domenic

domenic Apr 19, 2017

Member

That "while" clause is getting pretty long... let's see. How about this?

  1. Let subclustersInCluster be « cluster ».
  2. Let allSubclusters be a set containing all agent subclusters.
  3. While true:
    1. Let agentsInClusterSoFar be the set containing all agents in all the agent subclusters in subclustersInCluster.
    2. If allSubclusters contains any agent subcluster subcluster that contains an agent also in agnentsInClusterSoFar, then
      1. Append subcluster to subclustersInCluster.
      2. Remove subcluster from allSubclusters.
    3. Otherwise, break.
  4. Return the set containing all agents in all the agent subclusters in subclustersInCluster.

This comment has been minimized.

@annevk

annevk Apr 20, 2017

Member

@foolip any final thoughts on this?

@annevk

annevk Apr 20, 2017

Member

@foolip any final thoughts on this?

This comment has been minimized.

@foolip

foolip Apr 20, 2017

Member

I can't find any bug in what @domenic suggests, although I'm reading it knowing what I expect it to do. Since we don't expect this to be mapped 1:1 to some bit of implementation code, a note explaining what this is intended to achieve might be good.

@foolip

foolip Apr 20, 2017

Member

I can't find any bug in what @domenic suggests, although I'm reading it knowing what I expect it to do. Since we don't expect this to be mapped 1:1 to some bit of implementation code, a note explaining what this is intended to achieve might be good.

This comment has been minimized.

@foolip

foolip Apr 20, 2017

Member

This would be a bit of a rewrite, but what might make this all a bit easier to follow is if instead of defining agent subclusters, we define a "can share memory with" relation between two agents A and B, which is true iff any of the (tweaked) conditions currently used to define subclusters hold, or if (recursively) there exists another agent C so that the relation holds for AC and CB.

It would be important that the relation is symmetric, and https://en.wikipedia.org/wiki/Connected_component_(graph_theory) would be a way to express the same thing.

Take this suggestion only if you also think it'd be a simplification and not just bikeshedding.

@foolip

foolip Apr 20, 2017

Member

This would be a bit of a rewrite, but what might make this all a bit easier to follow is if instead of defining agent subclusters, we define a "can share memory with" relation between two agents A and B, which is true iff any of the (tweaked) conditions currently used to define subclusters hold, or if (recursively) there exists another agent C so that the relation holds for AC and CB.

It would be important that the relation is symmetric, and https://en.wikipedia.org/wiki/Connected_component_(graph_theory) would be a way to express the same thing.

Take this suggestion only if you also think it'd be a simplification and not just bikeshedding.

This comment has been minimized.

@annevk

annevk Apr 20, 2017

Member

That does seem clearer. I can give that a go. (Eventually it'll be very clear since we'll just define whenever an agent cluster is created and then when you append an agent to it, etc.)

@annevk

annevk Apr 20, 2017

Member

That does seem clearer. I can give that a go. (Eventually it'll be very clear since we'll just define whenever an agent cluster is created and then when you append an agent to it, etc.)

domenic added a commit to web-platform-tests/wpt that referenced this pull request Apr 19, 2017

Test [[CanBlock]] in various agents
Follows whatwg/html#2521.

Worklets are not tested at this time as their API is still in flux.
@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 21, 2017

Member

@foolip that was a great suggestion, unless I got it wrong again.

Member

annevk commented Apr 21, 2017

@foolip that was a great suggestion, unless I got it wrong again.

source
- </ul>
+ agents, we define which <span data-x="agent">agents</span> <dfn>can share memory with</dfn>
+ another. Then all the <span data-x="agent">agents</span> that can <span>can share memory
+ with</span> eachother form an <span>agent cluster</span>.</p>

This comment has been minimized.

@othermaciej

othermaciej Apr 21, 2017

"each other" should be two words.

@othermaciej

othermaciej Apr 21, 2017

"each other" should be two words.

@othermaciej

This comment has been minimized.

Show comment
Hide comment
@othermaciej

othermaciej Apr 21, 2017

That seems a lot easier to follow. Is there a way to see a version of the spec with the latest version of the pull request applied?

That seems a lot easier to follow. Is there a way to see a version of the spec with the latest version of the pull request applied?

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 21, 2017

Member

It seems to me the relation as defined is not symmetric. It seems a similar-origin window agent can share memory with a dedicated worker agent that it owns, but there's nothing saying that the decidicated worker agent can share memory with the similar-origin window agent that created it.

Member

domenic commented Apr 21, 2017

It seems to me the relation as defined is not symmetric. It seems a similar-origin window agent can share memory with a dedicated worker agent that it owns, but there's nothing saying that the decidicated worker agent can share memory with the similar-origin window agent that created it.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 21, 2017

Member

OK, after spending some time trying to adapt this, I think we should go back to the transitive closure definition and spec text, with some small tweaks. Let me explain.

"All the agents that can share memory with each other" is not precise enough to understand. You need to reference some mathematical concept.

@foolip seems to have been going for using the "connected component" concept. There are two approaches here:

  • Define an edge relationship ("can directly share memory with") between agents, and then say that you view this as a graph with edges and then the agent clusters are the connected components. I started on this, but see below.
  • Define a reachability equivalence relation ("can share memory with") between agents, and then say that the equivalence classes of that equivalence relation form the agent clusters. This is not fun because getting transitivity requires lots of extra definition text.

So we need to define an edge relationship (symmetric). Doing this ends up looking very similar to the previous version's <dl>.

But now we have a binary relationship. Why don't we just use the transitive closure concept, instead of bringing graph theory into the picture?

I will upload a revision that goes in this direction, and people can see what they think. I'll also upload a separate preview so you can compare them side by side.

Member

domenic commented Apr 21, 2017

OK, after spending some time trying to adapt this, I think we should go back to the transitive closure definition and spec text, with some small tweaks. Let me explain.

"All the agents that can share memory with each other" is not precise enough to understand. You need to reference some mathematical concept.

@foolip seems to have been going for using the "connected component" concept. There are two approaches here:

  • Define an edge relationship ("can directly share memory with") between agents, and then say that you view this as a graph with edges and then the agent clusters are the connected components. I started on this, but see below.
  • Define a reachability equivalence relation ("can share memory with") between agents, and then say that the equivalence classes of that equivalence relation form the agent clusters. This is not fun because getting transitivity requires lots of extra definition text.

So we need to define an edge relationship (symmetric). Doing this ends up looking very similar to the previous version's <dl>.

But now we have a binary relationship. Why don't we just use the transitive closure concept, instead of bringing graph theory into the picture?

I will upload a revision that goes in this direction, and people can see what they think. I'll also upload a separate preview so you can compare them side by side.

@othermaciej

This comment has been minimized.

Show comment
Hide comment
@othermaciej

othermaciej Apr 21, 2017

This clause is still pretty hard to follow:

"any dedicated worker agent whose single realm's global object's owner set contains an item whose relevant Realm belongs to agent."

It sounds like the effect is that you can directly share memory with any dedicated worker that you create. But this ends up stated with five levels of indirection through nontrivial definitions (plus you have to work out that the owner set of a dedicated worker will only ever contain the realm that created it, something that's not obvious from the definition of owner set).

Is there a way to more directly express what is intended while maintaining precision?

This clause is still pretty hard to follow:

"any dedicated worker agent whose single realm's global object's owner set contains an item whose relevant Realm belongs to agent."

It sounds like the effect is that you can directly share memory with any dedicated worker that you create. But this ends up stated with five levels of indirection through nontrivial definitions (plus you have to work out that the owner set of a dedicated worker will only ever contain the realm that created it, something that's not obvious from the definition of owner set).

Is there a way to more directly express what is intended while maintaining precision?

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 21, 2017

Member

I agree that's not super-easy to parse. I can't offhand think of any way to make it easier while still using only defined terms (although I can keep thinking about it). But we could certainly add

NOTE: this effectively means any dedicated workers created by windows in agent

Member

domenic commented Apr 21, 2017

I agree that's not super-easy to parse. I can't offhand think of any way to make it easier while still using only defined terms (although I can keep thinking about it). But we could certainly add

NOTE: this effectively means any dedicated workers created by windows in agent

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 21, 2017

Member

If we switch worker's owner set to only contain global objects (i.e. Windows or WorkerGlobalScopes, not Documents or WorkerGlobalScopes) as has been discussed previously, then we could:

  • Define "the agent's global objects"
  • Define "a worker global object is owned by another global object"

and then the three bullets for "can directly share memory with" would reduce to

  • Any dedicated worker agent whose global object is owned by one of agent's global objects
  • Any agent for which any of its global objects owns agent's global object
  • Any dedicated worker agent whose global object is owned by agent's global object

My understanding though is that this switch needs a bit more research first and we'd prefer to not hold up this PR for it.

We could approximate this today by defining "the agent's Documents" (for similar-origin window agents only) and then define "a worker global object is owned by a Document or WorkerGlobalScope", which would yield

  • Any dedicated worker agent whose global object is owned by one of agent's Documents
  • Any agent for which any of its global objects or Documents owns agent's global object
  • Any dedicated worker agent whose global object is owned by agent's global object

@othermaciej, I'm curious what you think of this, as it does introduce more indirection through definitions, although IMO they are reasonably intuitive ones. Also would love @annevk's thoughts.

Member

domenic commented Apr 21, 2017

If we switch worker's owner set to only contain global objects (i.e. Windows or WorkerGlobalScopes, not Documents or WorkerGlobalScopes) as has been discussed previously, then we could:

  • Define "the agent's global objects"
  • Define "a worker global object is owned by another global object"

and then the three bullets for "can directly share memory with" would reduce to

  • Any dedicated worker agent whose global object is owned by one of agent's global objects
  • Any agent for which any of its global objects owns agent's global object
  • Any dedicated worker agent whose global object is owned by agent's global object

My understanding though is that this switch needs a bit more research first and we'd prefer to not hold up this PR for it.

We could approximate this today by defining "the agent's Documents" (for similar-origin window agents only) and then define "a worker global object is owned by a Document or WorkerGlobalScope", which would yield

  • Any dedicated worker agent whose global object is owned by one of agent's Documents
  • Any agent for which any of its global objects or Documents owns agent's global object
  • Any dedicated worker agent whose global object is owned by agent's global object

@othermaciej, I'm curious what you think of this, as it does introduce more indirection through definitions, although IMO they are reasonably intuitive ones. Also would love @annevk's thoughts.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 22, 2017

Member

I don't really understand why you switched back so I don't understand if the new text is okay. E.g., now you have that a dedicated worker can share with its parent. But do you have that a dedicated worker can share with a child, if it's a dedicated worker?

Also, one of the realizations I had, which I applied in my patch, is that similar-origin window agent, dedicated, shared, and service worker agent, all have the same relationship with a child descendant worker agent.

If you think we should define both directions, I think it would have been clearer to just add that a dedicated worker agent can share with its parent, as that's always true.

Member

annevk commented Apr 22, 2017

I don't really understand why you switched back so I don't understand if the new text is okay. E.g., now you have that a dedicated worker can share with its parent. But do you have that a dedicated worker can share with a child, if it's a dedicated worker?

Also, one of the realizations I had, which I applied in my patch, is that similar-origin window agent, dedicated, shared, and service worker agent, all have the same relationship with a child descendant worker agent.

If you think we should define both directions, I think it would have been clearer to just add that a dedicated worker agent can share with its parent, as that's always true.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 22, 2017

Member

(I can probably investigate if we can change owner set next week. The main thing is what happens with initial about:blank documents. Should be fairly trivial...)

Member

annevk commented Apr 22, 2017

(I can probably investigate if we can change owner set next week. The main thing is what happens with initial about:blank documents. Should be fairly trivial...)

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 22, 2017

Member

I don't really understand why you switched back so I don't understand if the new text is okay.

The main thing is that it needs to be symmetric and reflexive. Putting that in paragraph form got fairly unwieldy. Even though, as you say, you can use one sentence to encapsulate the parent-child relationship, it's a pretty obscure sentence, and so the <dl> form seems clearer to me, with explicit distinction between the document-owner case and the worker-owner case.

The other issues were that "can share memory with" was a misleading name (since e.g. a window can share with a worker nested inside another worker, but couldn't according to your definition). And that "all the agents that can share memory with each other" was not helpful or precise compared to using transitive closure over binary relations more clearly.

That said, I'm not wedded to <dl> over paragraph form for defining "can share memory directly with", as long as we ensure the paragraph ends up reflexive and symmetric. The main addition I made that I feel is a big improvement is my paragraph starting "Given the above definition, can directly share memory with is a reflexive and symmetric...".

But do you have that a dedicated worker can share with a child, if it's a dedicated worker?

Somehow that got dropped, dang -_-.

If you think we should define both directions, I think it would have been clearer to just add that a dedicated worker agent can share with its parent, as that's always true.

Yeah, I started doing that, but it ended up being a very long paragraph that was confusing to read compared to the <dl>.

Member

domenic commented Apr 22, 2017

I don't really understand why you switched back so I don't understand if the new text is okay.

The main thing is that it needs to be symmetric and reflexive. Putting that in paragraph form got fairly unwieldy. Even though, as you say, you can use one sentence to encapsulate the parent-child relationship, it's a pretty obscure sentence, and so the <dl> form seems clearer to me, with explicit distinction between the document-owner case and the worker-owner case.

The other issues were that "can share memory with" was a misleading name (since e.g. a window can share with a worker nested inside another worker, but couldn't according to your definition). And that "all the agents that can share memory with each other" was not helpful or precise compared to using transitive closure over binary relations more clearly.

That said, I'm not wedded to <dl> over paragraph form for defining "can share memory directly with", as long as we ensure the paragraph ends up reflexive and symmetric. The main addition I made that I feel is a big improvement is my paragraph starting "Given the above definition, can directly share memory with is a reflexive and symmetric...".

But do you have that a dedicated worker can share with a child, if it's a dedicated worker?

Somehow that got dropped, dang -_-.

If you think we should define both directions, I think it would have been clearer to just add that a dedicated worker agent can share with its parent, as that's always true.

Yeah, I started doing that, but it ended up being a very long paragraph that was confusing to read compared to the <dl>.

@othermaciej

This comment has been minimized.

Show comment
Hide comment
@othermaciej

othermaciej Apr 22, 2017

@domenic I'm not entirely clear on what text your proposed three bullets would be replacing. Also not sure if the three Document-mentioning bullets would be instead or in addition.

This is now deep bike shedding, but I wonder if this would be simpler if the spec didn't explicitly rely on the notion of transitive closure or two levels of relations, but rather stated the equivalence relation laws as well as the sole needed unidirectional relationship. Equivalence relations are always their own transitive closure.

I'd imagine this being something like:

In addition, any agent A can share memory with

  • [.. a rule that says any agent owned by A, or any agent that owns A, whichever is simpler to define ..]
  • itself
  • any agent B such that B can share memory with A
  • any agent B such that there exists an agent C, where A can share memory with C and C can share memory with B

Then there has to be only one rule specific to agent clusters (you share with dedicated workers you own). You explicitly state the equivalence relation laws without needed to state rules that happen to make them true. And you have all you need to define equivalence classes.

It's way more obvious that this is symmetric than the original bullets for example, where you would have to read a lot of other definitions and algorithms to confirm that. I think it also ends up being a smaller number of rules, if you count the invocation of transitive closure as a rule.

@domenic I'm not entirely clear on what text your proposed three bullets would be replacing. Also not sure if the three Document-mentioning bullets would be instead or in addition.

This is now deep bike shedding, but I wonder if this would be simpler if the spec didn't explicitly rely on the notion of transitive closure or two levels of relations, but rather stated the equivalence relation laws as well as the sole needed unidirectional relationship. Equivalence relations are always their own transitive closure.

I'd imagine this being something like:

In addition, any agent A can share memory with

  • [.. a rule that says any agent owned by A, or any agent that owns A, whichever is simpler to define ..]
  • itself
  • any agent B such that B can share memory with A
  • any agent B such that there exists an agent C, where A can share memory with C and C can share memory with B

Then there has to be only one rule specific to agent clusters (you share with dedicated workers you own). You explicitly state the equivalence relation laws without needed to state rules that happen to make them true. And you have all you need to define equivalence classes.

It's way more obvious that this is symmetric than the original bullets for example, where you would have to read a lot of other definitions and algorithms to confirm that. I think it also ends up being a smaller number of rules, if you count the invocation of transitive closure as a rule.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 23, 2017

Member

I just wanted to make a point about the <dl> without going into the alternative discussed above, which seems reasonable. The reason I realized it doesn't work is that if we ignore worklet agents, all agents have a child relationship with dedicated worker agents, including dedicated worker agents. And only dedicated worker agents have a parent relationship. So if you grouped related <dt>s together you'd end up with dedicated worker agents twice or you'd end up with duplicated text inside the dedicated worker agent bit both of which would make it harder to analyze what is going on.

Member

annevk commented Apr 23, 2017

I just wanted to make a point about the <dl> without going into the alternative discussed above, which seems reasonable. The reason I realized it doesn't work is that if we ignore worklet agents, all agents have a child relationship with dedicated worker agents, including dedicated worker agents. And only dedicated worker agents have a parent relationship. So if you grouped related <dt>s together you'd end up with dedicated worker agents twice or you'd end up with duplicated text inside the dedicated worker agent bit both of which would make it harder to analyze what is going on.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 24, 2017

Member

@othermaciej with that definition for "can share memory with", what would you then say about "agent cluster"? And do you think the first bullet point could be defined as I defined it?

Member

annevk commented Apr 24, 2017

@othermaciej with that definition for "can share memory with", what would you then say about "agent cluster"? And do you think the first bullet point could be defined as I defined it?

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 24, 2017

Member

New sketch:


A similar-origin window agent, dedicated worker agent, shared worker agent, or service worker agent, ownerAgent, can share memory with a dedicated worker agent whose single realm's global object's owner set includes an item whose relevant realm is contained by ownerAgent.

In addition, any agent A can share memory with

  • A
  • any agent B such that B can share memory with A
  • any agent B such that there exists an agent C, where A can share memory with C and C can share memory with B.

An agent cluster consists of all agents that can share memory with each other.


This is pretty much the same as what I had, with the addition of @othermaciej's suggestion of being more explicit about the relationships in prose.

Member

annevk commented Apr 24, 2017

New sketch:


A similar-origin window agent, dedicated worker agent, shared worker agent, or service worker agent, ownerAgent, can share memory with a dedicated worker agent whose single realm's global object's owner set includes an item whose relevant realm is contained by ownerAgent.

In addition, any agent A can share memory with

  • A
  • any agent B such that B can share memory with A
  • any agent B such that there exists an agent C, where A can share memory with C and C can share memory with B.

An agent cluster consists of all agents that can share memory with each other.


This is pretty much the same as what I had, with the addition of @othermaciej's suggestion of being more explicit about the relationships in prose.

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 24, 2017

Member

@othermaciej

I'm not entirely clear on what text your proposed three bullets would be replacing.

This would replace the current three bullets in my draft that read:

  • Any dedicated worker agent whose global object's owner set contains a Document object whose relevant Realm belongs to agent.
  • Any agent with a realm that is the relevant Realm of an item in agent's global object's owner set.
  • Any dedicated worker agent whose global object is in agent's global object's worker set.

Also not sure if the three Document-mentioning bullets would be instead or in addition.

These would be instead of; they are something we could accomplish today without reworking worker ownership to be global-based instead of Document-based.


I wonder if this would be simpler if the spec didn't explicitly rely on the notion of transitive closure or two levels of relations, but rather stated the equivalence relation laws as well as the sole needed unidirectional relationship.

This makes me uncomfortable but not in a way that I can back up. I guess I don't remember seeing an equivalence relation defined this way before, but I admit it has been many years since my math major. If you and Anne are OK with it then I guess I can be as well.

I'd also like to point out that you have two levels of relations too, the unidirectional "owns" and the equivalence relation "can share memory with". But your two-level version is indeed simpler.

I guess maybe my discomfort is wondering whether the definition is too circular, as opposed to a multi-stage definition of: A = owns, B = reflexive closure of A, C = symmetric closure of B, D = transitive closure of C. (And then you might have to repeat until it stabilizes?) Probably it is fine for the HTML spec though, which isn't a math textbook...

@annevk:

with that definition for "can share memory with", what would you then say about "agent cluster"?

New sketch:

This seems OK along the lines of @othermaciej's sketch, modulo my above discomfort. However:

  • IMO "whose single realm's global object's owner set includes an item whose relevant realm is contained by ownerAgent" is confusing compared to breaking down into document-owner cases and worker-owner cases. Maybe we can live with it though, and hope it gets cleaned up when we consolidate the ownership to be only about globals.
  • "All agents that can share memory with each other" is not very precise. IMO it would be improved by something like:

Given the above definition, can share memory with is an equivalence relation: it is reflexive, symmetric, and transitive. We can then define agent clusters to be the equivalence classes of can share memory with.

(or even just the last sentence by itself.)

Member

domenic commented Apr 24, 2017

@othermaciej

I'm not entirely clear on what text your proposed three bullets would be replacing.

This would replace the current three bullets in my draft that read:

  • Any dedicated worker agent whose global object's owner set contains a Document object whose relevant Realm belongs to agent.
  • Any agent with a realm that is the relevant Realm of an item in agent's global object's owner set.
  • Any dedicated worker agent whose global object is in agent's global object's worker set.

Also not sure if the three Document-mentioning bullets would be instead or in addition.

These would be instead of; they are something we could accomplish today without reworking worker ownership to be global-based instead of Document-based.


I wonder if this would be simpler if the spec didn't explicitly rely on the notion of transitive closure or two levels of relations, but rather stated the equivalence relation laws as well as the sole needed unidirectional relationship.

This makes me uncomfortable but not in a way that I can back up. I guess I don't remember seeing an equivalence relation defined this way before, but I admit it has been many years since my math major. If you and Anne are OK with it then I guess I can be as well.

I'd also like to point out that you have two levels of relations too, the unidirectional "owns" and the equivalence relation "can share memory with". But your two-level version is indeed simpler.

I guess maybe my discomfort is wondering whether the definition is too circular, as opposed to a multi-stage definition of: A = owns, B = reflexive closure of A, C = symmetric closure of B, D = transitive closure of C. (And then you might have to repeat until it stabilizes?) Probably it is fine for the HTML spec though, which isn't a math textbook...

@annevk:

with that definition for "can share memory with", what would you then say about "agent cluster"?

New sketch:

This seems OK along the lines of @othermaciej's sketch, modulo my above discomfort. However:

  • IMO "whose single realm's global object's owner set includes an item whose relevant realm is contained by ownerAgent" is confusing compared to breaking down into document-owner cases and worker-owner cases. Maybe we can live with it though, and hope it gets cleaned up when we consolidate the ownership to be only about globals.
  • "All agents that can share memory with each other" is not very precise. IMO it would be improved by something like:

Given the above definition, can share memory with is an equivalence relation: it is reflexive, symmetric, and transitive. We can then define agent clusters to be the equivalence classes of can share memory with.

(or even just the last sentence by itself.)

@othermaciej

This comment has been minimized.

Show comment
Hide comment
@othermaciej

othermaciej Apr 24, 2017

@annevk

An agent cluster consists of all agents that can share memory with each other.

If you want to be fussy about math terms, what you'd say instead is:
Can share memory with defines an equivalence relation. An agent cluster consists of all agents in the same equivalence class with respect to the can share memory with equivalence relation.

I think the way you said it is clear enough though.

@domenic To be an equivalence relation, it's only necessary that the three equivalence properties be true. Often for interesting mathematical structures, these properties are implicit, and we prove them to be true.

But sometimes one gives a definition of a relation that includes these properties explicitly. For example, in the Peano axioms for arithmetic, the the reflexive, symmetric and transitive properties of equality are axioms, and essentially form the definition of the equality relation on natural numbers. Based on this, it's completely kosher from a math perspective to give a definition of an equivalence relation where reflexive, symmetric and transitive properties are explicitly part of the definition.

I also don't think it's a circular definition, any more than the usual definition of the Fibonacci numbers is circular. It's a definition with two base cases (dedicated worker ownership and reflexivity) and two recurrence relations that let you extend the base case. Taking the four basic rules as axioms, you can prove doubly indirect transitivity (a = b /\ b = c /\ c = d => a=d) as a theorem, just as you could in Peano arithmetic.

I'm not sure this level of mathematical rigor is even necessary, but it is actually there.

@annevk

An agent cluster consists of all agents that can share memory with each other.

If you want to be fussy about math terms, what you'd say instead is:
Can share memory with defines an equivalence relation. An agent cluster consists of all agents in the same equivalence class with respect to the can share memory with equivalence relation.

I think the way you said it is clear enough though.

@domenic To be an equivalence relation, it's only necessary that the three equivalence properties be true. Often for interesting mathematical structures, these properties are implicit, and we prove them to be true.

But sometimes one gives a definition of a relation that includes these properties explicitly. For example, in the Peano axioms for arithmetic, the the reflexive, symmetric and transitive properties of equality are axioms, and essentially form the definition of the equality relation on natural numbers. Based on this, it's completely kosher from a math perspective to give a definition of an equivalence relation where reflexive, symmetric and transitive properties are explicitly part of the definition.

I also don't think it's a circular definition, any more than the usual definition of the Fibonacci numbers is circular. It's a definition with two base cases (dedicated worker ownership and reflexivity) and two recurrence relations that let you extend the base case. Taking the four basic rules as axioms, you can prove doubly indirect transitivity (a = b /\ b = c /\ c = d => a=d) as a theorem, just as you could in Peano arithmetic.

I'm not sure this level of mathematical rigor is even necessary, but it is actually there.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 25, 2017

Member

I think I found a variant of @bzbarsky's scenario that doesn't have the guarantees @lars-t-hansen (and JavaScript) is looking for (unless polling and getting nothing back is sufficient).

A embeds B through an element FRAME and A and B do not cooperate with each other in any way. B opens an auxiliary browsing context B' and one child worker B''. Then A removes FRAME. At this point B' is still "alive" and B and B'' are no longer running (event loop stopped dead in its track, no final tasks processed or anything). So B and B'' have no way of telling B' and B' can only determine they stopped running through some kind of self-implemented timeout model.

This follows from https://html.spec.whatwg.org/multipage/browsers.html#a-browsing-context-is-discarded which is a little more brutal than navigating (which gives you unload).

I will continue with fixing the wording in this PR, but that might be something that is interesting to look into further.

I came upon this while investigating the lifetime model of workers and whether they could move to be attached to global objects rather than documents as discussed in the latter part of #411. Implementations are very much aligned on documents at the moment so we should not change that here, even if it leads to slightly less elegant wording.

Member

annevk commented Apr 25, 2017

I think I found a variant of @bzbarsky's scenario that doesn't have the guarantees @lars-t-hansen (and JavaScript) is looking for (unless polling and getting nothing back is sufficient).

A embeds B through an element FRAME and A and B do not cooperate with each other in any way. B opens an auxiliary browsing context B' and one child worker B''. Then A removes FRAME. At this point B' is still "alive" and B and B'' are no longer running (event loop stopped dead in its track, no final tasks processed or anything). So B and B'' have no way of telling B' and B' can only determine they stopped running through some kind of self-implemented timeout model.

This follows from https://html.spec.whatwg.org/multipage/browsers.html#a-browsing-context-is-discarded which is a little more brutal than navigating (which gives you unload).

I will continue with fixing the wording in this PR, but that might be something that is interesting to look into further.

I came upon this while investigating the lifetime model of workers and whether they could move to be attached to global objects rather than documents as discussed in the latter part of #411. Implementations are very much aligned on documents at the moment so we should not change that here, even if it leads to slightly less elegant wording.

annevk added some commits Apr 11, 2017

Define agents and agent clusters
Define the infrastructure for SharedArrayBuffer. This also clarifies
along which boundaries a browser implementation can use processes and
threads.

Tests: web-platform-tests/wpt#5569.

Follow-up to define similar-origin window agents upon a less shaky
foundation is #2528. Because of that, similar-origin window agents
are the best place to store state that would formerly go on unit of
related similar-origin browsing contexts.

tc39/ecma262#882 is follow-up to define
agents in more detail; in particular make their implicit realms slot
explicit. w3c/css-houdini-drafts#224 is
follow-up to define worklet ownership better which is needed to
define how they relate to agent (sub)clusters.

Fixes part of #2260. Fixes #851. Fixes
w3c/ServiceWorker#1115. Fixes most of
w3c/css-houdini-drafts#380 (no tests and no
nice grouping of multiple realms in a single agent as that is not
needed).
@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 25, 2017

Member

These two new sections can also be reviewed here: https://html5.org/temp/agents.html#integration-with-the-javascript-agent-formalism. Not all the links will work though.

Member

annevk commented Apr 25, 2017

These two new sections can also be reviewed here: https://html5.org/temp/agents.html#integration-with-the-javascript-agent-formalism. Not all the links will work though.

@lars-t-hansen

This comment has been minimized.

Show comment
Hide comment
@lars-t-hansen

lars-t-hansen Apr 25, 2017

@annevk

unless polling and getting nothing back is sufficient

It probably is not, though that may be a matter of how you spin it...

A heartbeat mechanism that sends a packet every k seconds may be sufficient in principle but is awful for battery, etc.

@annevk

unless polling and getting nothing back is sufficient

It probably is not, though that may be a matter of how you spin it...

A heartbeat mechanism that sends a packet every k seconds may be sufficient in principle but is awful for battery, etc.

@domenic

LGTM, let's merge this (and probably the tests too).

annevk added a commit to web-platform-tests/wpt that referenced this pull request Apr 26, 2017

Test [[CanBlock]] in various agents
Follows whatwg/html#2521.

Worklets are not tested at this time as their API is still in flux.
@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 26, 2017

Member

I found a minor issue with a filename of one of the tests that I pushed a fix for.

Safari: doesn't support shared or service workers and handles dedicated and windows per this PR. So no bug needed I think.

Chrome and Edge: SharedArrayBuffer is not defined.

Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1359745.

Member

annevk commented Apr 26, 2017

I found a minor issue with a filename of one of the tests that I pushed a fix for.

Safari: doesn't support shared or service workers and handles dedicated and windows per this PR. So no bug needed I think.

Chrome and Edge: SharedArrayBuffer is not defined.

Firefox: https://bugzilla.mozilla.org/show_bug.cgi?id=1359745.

@annevk annevk merged commit 4db8654 into master Apr 26, 2017

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@annevk annevk deleted the annevk/agents branch Apr 26, 2017

@domenic

This comment has been minimized.

Show comment
Hide comment

domenic added a commit to web-platform-tests/wpt that referenced this pull request Apr 26, 2017

Test [[CanBlock]] in various agents
Follows whatwg/html#2521.

Worklets are not tested at this time as their API is still in flux.
@othermaciej

This comment has been minimized.

Show comment
Hide comment
@othermaciej

othermaciej Apr 26, 2017

Do the tests also check same-/similar-origin frames and windows? I would expect Safari may not handle that right given Phil's comments above (and we'd be happy to fix it).

Do the tests also check same-/similar-origin frames and windows? I would expect Safari may not handle that right given Phil's comments above (and we'd be happy to fix it).

@domenic

This comment has been minimized.

Show comment
Hide comment
@domenic

domenic Apr 26, 2017

Member

They do; see web-platform-tests/wpt#5003 for details. In particular the case

postMessage between window and non-same-origin-domain iframe that is still in the same URSOBC

We're still finishing up the tests for SharedArrayBuffer, which is where most of the agent cluster stuff comes into play; the only tests merged so far (and thus the only bugs filed so far) are about the [[CanBlock]] aspect of agent clusters.

Member

domenic commented Apr 26, 2017

They do; see web-platform-tests/wpt#5003 for details. In particular the case

postMessage between window and non-same-origin-domain iframe that is still in the same URSOBC

We're still finishing up the tests for SharedArrayBuffer, which is where most of the agent cluster stuff comes into play; the only tests merged so far (and thus the only bugs filed so far) are about the [[CanBlock]] aspect of agent clusters.

@annevk

This comment has been minimized.

Show comment
Hide comment
@annevk

annevk Apr 26, 2017

Member

The last PR in the series of shared memory changes to the HTML Standard is #2518. That actually defines how a SharedArrayObject is serialized and deserialized. The tests @domenic mentions will land together with that PR ideally.

Member

annevk commented Apr 26, 2017

The last PR in the series of shared memory changes to the HTML Standard is #2518. That actually defines how a SharedArrayObject is serialized and deserialized. The tests @domenic mentions will land together with that PR ideally.

@binji binji referenced this pull request in w3ctag/design-reviews May 9, 2017

Closed

SharedArrayBuffer + Atomics #165

3 of 5 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment