New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What is an Agent? #882

Open
annevk opened this Issue Apr 11, 2017 · 16 comments

Comments

Projects
None yet
8 participants
@annevk
Contributor

annevk commented Apr 11, 2017

I'm trying to figure out how to embed the Agent concept into the HTML Standard. It's clear that a worker, shared worker, service worker, or worklet all represent a single agent.

It's not clear for Window/document. @lars-t-hansen suggests it's meant to match the concept of document, however that would mean we'd end up with two Agents and one global for initial about:blank navigation.

Previously it was suggested it's meant to match the event loop concept, but that's also not true as that much more closely matches what you call "execution thread" in scope (not necessarily in function, although I think it's close).

So it would be good to hear about some invariants or ways to test what is actually implemented and maybe get the specification clarified what it actually means since it's rather vague.

SharedArrayBuffer integration issue for the HTML Standard: whatwg/html#2260 (also whatwg/html#2361 and whatwg/html#2518).

@lars-t-hansen

This comment has been minimized.

Contributor

lars-t-hansen commented Apr 11, 2017

Well, "document". You can't have an agent (in any meaningful sense) without a global object / Realm since every execution record requires a realm and an agent only works in terms of processing execution records. So when I say document I probably really mean the instantiated ES environment you get when you have a document. I know about:blank is a thing but it seems like a special case that could be adapted to fit agents somehow.

@annevk

This comment has been minimized.

Contributor

annevk commented Apr 11, 2017

The instantiated ES environment would match up with the Window object I think (which ends up being reused for initial about:blank navigation whereas the document gets replaced). That would mean Agents are 1:1 with what HTML calls settings objects, or settings object's global objects.

See also https://html.spec.whatwg.org/#realms-settings-objects-global-objects (if the Realms API becomes a thing that'll need to change obviously and someone should work through the implications of that changing).

@domenic

This comment has been minimized.

Member

domenic commented Apr 11, 2017

Hmm. I was pretty sure that agents were supposed to be event loops. For example there are many realms within a single agent.

@domenic

This comment has been minimized.

Member

domenic commented Apr 11, 2017

For example I thought that two same-origin iframes were in the same agent. E.g. they share a list of promise jobs (which are agent-scoped). In general I thought agent was supposed to represent any code which can synchronously access other code's memory from JavaScript today. And agent cluster was supposed to represent any code which can access other code's memory using SAB, which has more capabilities than JavaScript today.

@lars-t-hansen is that not the case?

@domenic

This comment has been minimized.

Member

domenic commented Apr 11, 2017

If agents are 1:1 with realms we should delete the agent concept from ES262 and just use realms everywhere.

@erights

This comment has been minimized.

erights commented Apr 11, 2017

Maybe I am missing some context, but Agent and Realm are quite clearly totally distinct concepts. Realm is about a set of global objects. Agent is about concurrency. Job queues are per-Agent rather than per-Realm. In JavaScript, any given Realm must be within one Agent, but an Agent can contain any number of Realms. I don't see how these can be confused.

In the browser, Origin cross-cuts Agent. Any Realm must also be within one Origin and an Origin can contain any number of Realms. But neither Origins nor Agents contain each other. An Agent can contain Realms from different Origins and an Origin can contain Realms running in different Agents.

At the last TC39 meeting, where we discussed reifying Realms with a Realm API, the question came up about reifying Agents and Job queues into an Agent API. This should be done eventually, but will be almost completely distinct from the effort to create a Realm API.

@annevk

This comment has been minimized.

Contributor

annevk commented Apr 11, 2017

@erights that also matches my understanding, with the further clarification that Agent encompasses all Realms that can possibly execute each other's functions within the same event loop (all "similar-origin" Window objects), but not more. So a.yes.no and b.yes.no are same Agent and a.yes.no a.maybe.no are not. (That distinction matters if we accurately want to define who can have shared memory and who cannot.)

However, when I talked with @lars-t-hansen about it on IRC today he said that Agent roughly corresponded to "document".

@allenwb

This comment has been minimized.

Member

allenwb commented Apr 11, 2017

Agents can't be 1:1 with realms because realms to do have any of the resource listed in the first paragraph of the Agent Definition privately associated with them.

It appears to be that while the definition of Agent in 8.7 is probably adequate to described the semantics of inter-Agent sharing shared buffers, it is missing a few things to complete the ECMAScript language semantics.

The following needed to be added as constituent elements of an ECMAScript Agent:

  • A set of Realms. The realms in the set are private to the Agent and may not be accessible to any other agent.
  • A heap of SharedArrayBuffer objects that whose access may be shared with other Agents
  • A set of well-known symbol values
  • A GlobalSymbolRegistry

An event loop is not included because it is not (yet?) an ECAMScript language concept. But in an implementation it manifests via the per agent job queue and job schedule.

@allenwb allenwb closed this Apr 11, 2017

@allenwb allenwb reopened this Apr 11, 2017

@domenic

This comment has been minimized.

Member

domenic commented Apr 11, 2017

To be a bit clearer, this is an issue cross-cutting ES and HTML, so that is why we are talking about things like origins and event loops that are outside the scope of ES.

It sounded like from @lars-t-hansen's description that in implementations agents and realms are 1:1 (and would be forever). In that case I was suggesting we merge the two spec concepts. We could also keep them separate (just like we keep global object and Realm separate, instead of e.g. using lots of slots on the global object) but that would just be a spec formalism, and IMO a dubious one.

But it sounds like that is probably a moot discussion since at least @erights, @allenwb, and myself believe that @lars-t-hansen's description of agents was mistaken. In particular @allenwb's

The realms in the set are private to the Agent and may not be accessible to any other agent.

definitely makes it impossible for each agent to contain only one realm (in a browser) since many realms can access each other synchronously (in a browser).

What remains is the confusion that the creator of agents (@lars-t-hansen) has a different conception of them than us three, and believes they are 1:1 with realms. It'd be ideal to sort that out.

@jfbastien

This comment has been minimized.

jfbastien commented Apr 11, 2017

If WebAssembly gets its own threads (WorkerLite™), would these be Agents? Is that out of scope for TC39? I think we intend to re-use the wording and want it to inter-operate, e.g. SharedArrayBuffer can be shared between WebAssembly and JavaScript.

@syg

This comment has been minimized.

Member

syg commented Apr 11, 2017

@domenic @annevk It doesn't make sense for Agents to be 1:1 with Realms, neither in spec nor in implementation. If they were 1:1 in spec, it would mean currently cross-Realm execution with SAB would be undefined. If they were 1:1 in implementation, that'd mean each global had its own event loop. So all that is just a long-winded version of "I agree with what @erights said". I'd interpret Lars's "document" and ES environment comment as saying that an ES environment must be present for the concept of Agent to be sensible, not anything about its bijection with document.

@jfbastien A by-the-letter reading of 8.7 means wasm threads aren't Agents because they don't have any ES-isms like Realms, ES execution contexts, etc. For the sake of interop, I think we'd need to factor out a notion of Agent that doesn't include ES things but does include stuff in that table, the various boolean flags and the candidate execution for the memory model.

@binji

This comment has been minimized.

binji commented Apr 11, 2017

@syg I started to work on that for the WebAssembly atomics proposal here WebAssembly/design#1019, but it's pretty informal.

@annevk

This comment has been minimized.

Contributor

annevk commented Apr 12, 2017

Yeah, I later recalled the whole debate about naming agents continents so @lars-t-hansen must have meant some other kind of document.

I like @allenwb's suggestions. I think we could go even further and have CreateAgentCluster and CreateAgent abstract operations that detail the arguments a host is expected to provide (such as [[CanBlock]] for agents), though it might be quite some time until we have refactored the HTML Standard enough to be able to use these properly (our lifetime story is not great).

(WebAssembly integration can be discussed at #887.)

@lars-t-hansen

This comment has been minimized.

Contributor

lars-t-hansen commented Apr 12, 2017

What remains is the confusion that the creator of agents (@lars-t-hansen) has a different conception of them than us three, and believes they are 1:1 with realms. It'd be ideal to sort that out.

I think when Anne says "document" he means something much more specific than I mean, and there is every reason to be skeptical when I casually use something that sounds like they may be HTML terms.

The final intent of the interaction of SAB and HTML was recorded here: http://tc39.github.io/ecmascript_sharedmem/shmem.html#WebBrowserEmbedding. Per discussions with Domenic we settled on a mapping s.t. in an HTML embedding, an Agent is realized by having the event loop provide the the executing thread.[*] This was to allow for situations where code in one realm (the tab) reaches into another realm (same-origin iframe, opened window). Accounting for the HTML embedding and browser semantics influenced the SAB spec a little bit, in that (a) the executing thread can be shared (if the tab reaches into an iframe they cannot be running concurrently) and (b) shared threads cannot block (or programs would deadlock quickly). Other than that, I tried to keep the Agent spec as open as possible.

[*] I dislike saying that an Agent "is" an event loop because an Agent is already defined in the ES spec and it is its own thing. The embedding has to provide the machinery the spec requires to drive evaluation however. Those "holes in the spec" are not well spelled out, see also @allenwb's list. (It does not help the matter that the link quoted above states that an agent "is" an event loop, nor does my informal use of HTML terms.)

@annevk

This comment has been minimized.

Contributor

annevk commented Apr 12, 2017

Glad it was just confusion. (By document I do indeed mean an object that is hold by the Window object (the global object in use for non-worker scenarios) and typically has a 1:1 relationship with that object, but not always.)

I think as a minimum we should give Agent a [[Realms]] slot and then we can say which realms belong to a single agent and what that agent's [[CanBlock]] should be. That also avoids comparisons with the event loop and changing the definition of the event loop (as it is currently defined in HTML it can be used across agent clusters and that is also how user agents implement it, definitely for cross-origin iframes, though this may change in the future).

@domenic

This comment has been minimized.

Member

domenic commented Apr 13, 2017

So shall we leave this open to track @allenwb's #882 (comment), or should we maybe close an open a new issue to track that specific work item?

annevk added a commit to whatwg/html that referenced this issue Apr 16, 2017

Define agents and agent clusters
Define the infrastructure for SharedArrayBuffer. This also clarifies
along which boundaries a browser implementation can use processes and
threads.

shaky foundation. Because of that, similar-origin window agents are
the best place to store state that would formerly go on unit of
related similar-origin browsing contexts.

tc39/ecma262#882 is follow-up to define
agents in more detail; in particular make their implicit realms slot
explicit. w3c/css-houdini-drafts#224 is
follow-up to define worklet ownership better which is needed to
define how they relate to agent (sub)clusters.

Fixes part of #2260. Fixes
w3c/ServiceWorker#1115. Fixes most of
w3c/css-houdini-drafts#380 (no tests and no
nice grouping of multiple realms in a single agent as that is not
needed).

annevk added a commit to whatwg/html that referenced this issue Apr 19, 2017

Define agents and agent clusters
Define the infrastructure for SharedArrayBuffer. This also clarifies
along which boundaries a browser implementation can use processes and
threads.

shaky foundation. Because of that, similar-origin window agents are
the best place to store state that would formerly go on unit of
related similar-origin browsing contexts.

tc39/ecma262#882 is follow-up to define
agents in more detail; in particular make their implicit realms slot
explicit. w3c/css-houdini-drafts#224 is
follow-up to define worklet ownership better which is needed to
define how they relate to agent (sub)clusters.

Fixes part of #2260. Fixes
w3c/ServiceWorker#1115. Fixes most of
w3c/css-houdini-drafts#380 (no tests and no
nice grouping of multiple realms in a single agent as that is not
needed).

annevk added a commit to whatwg/html that referenced this issue Apr 25, 2017

Define agents and agent clusters
Define the infrastructure for SharedArrayBuffer. This also clarifies
along which boundaries a browser implementation can use processes and
threads.

Tests: web-platform-tests/wpt#5569.

Follow-up to define similar-origin window agents upon a less shaky
foundation is #2528. Because of that, similar-origin window agents
are the best place to store state that would formerly go on unit of
related similar-origin browsing contexts.

tc39/ecma262#882 is follow-up to define
agents in more detail; in particular make their implicit realms slot
explicit. w3c/css-houdini-drafts#224 is
follow-up to define worklet ownership better which is needed to
define how they relate to agent (sub)clusters.

Fixes part of #2260. Fixes #851. Fixes
w3c/ServiceWorker#1115. Fixes most of
w3c/css-houdini-drafts#380 (no tests and no
nice grouping of multiple realms in a single agent as that is not
needed).

annevk added a commit to whatwg/html that referenced this issue Apr 26, 2017

Define agents and agent clusters
Define the infrastructure for SharedArrayBuffer. This also clarifies
along which boundaries a browser implementation can use processes and
threads.

Tests: web-platform-tests/wpt#5569.

Follow-up to define similar-origin window agents upon a less shaky
foundation is #2528. Because of that, similar-origin window agents
are the best place to store state that would formerly go on unit of
related similar-origin browsing contexts.

Follow-up for better agent shutdown notifications: #2581.

tc39/ecma262#882 is follow-up to define
agents in more detail; in particular make their implicit realms slot
explicit. w3c/css-houdini-drafts#224 is
follow-up to define worklet ownership better which is needed to
define how they relate to agent (sub)clusters.

Fixes part of #2260. Fixes #851. Fixes
w3c/ServiceWorker#1115. Fixes most of
w3c/css-houdini-drafts#380 (no tests and no
nice grouping of multiple realms in a single agent as that is not
needed).

inikulin added a commit to HTMLParseErrorWG/html that referenced this issue May 9, 2017

Define agents and agent clusters
Define the infrastructure for SharedArrayBuffer. This also clarifies
along which boundaries a browser implementation can use processes and
threads.

Tests: web-platform-tests/wpt#5569.

Follow-up to define similar-origin window agents upon a less shaky
foundation is whatwg#2528. Because of that, similar-origin window agents
are the best place to store state that would formerly go on unit of
related similar-origin browsing contexts.

Follow-up for better agent shutdown notifications: whatwg#2581.

tc39/ecma262#882 is follow-up to define
agents in more detail; in particular make their implicit realms slot
explicit. w3c/css-houdini-drafts#224 is
follow-up to define worklet ownership better which is needed to
define how they relate to agent (sub)clusters.

Fixes part of whatwg#2260. Fixes whatwg#851. Fixes
w3c/ServiceWorker#1115. Fixes most of
w3c/css-houdini-drafts#380 (no tests and no
nice grouping of multiple realms in a single agent as that is not
needed).

inikulin added a commit to HTMLParseErrorWG/html that referenced this issue May 9, 2017

Define agents and agent clusters
Define the infrastructure for SharedArrayBuffer. This also clarifies
along which boundaries a browser implementation can use processes and
threads.

Tests: web-platform-tests/wpt#5569.

Follow-up to define similar-origin window agents upon a less shaky
foundation is whatwg#2528. Because of that, similar-origin window agents
are the best place to store state that would formerly go on unit of
related similar-origin browsing contexts.

Follow-up for better agent shutdown notifications: whatwg#2581.

tc39/ecma262#882 is follow-up to define
agents in more detail; in particular make their implicit realms slot
explicit. w3c/css-houdini-drafts#224 is
follow-up to define worklet ownership better which is needed to
define how they relate to agent (sub)clusters.

Fixes part of whatwg#2260. Fixes whatwg#851. Fixes
w3c/ServiceWorker#1115. Fixes most of
w3c/css-houdini-drafts#380 (no tests and no
nice grouping of multiple realms in a single agent as that is not
needed).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment