Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Private members break proxies #106

Closed
jods4 opened this issue Jun 14, 2018 · 407 comments
Labels

Comments

@jods4
Copy link

@jods4 jods4 commented Jun 14, 2018

I commented on an old issue in tc39/proposal-private-fields#102 but after reading the answer I feel like I should re-open an issue.

It's been discussed before: current PrivateName spec doesn't tunnel through proxies.
In my opinion the consequences have been under-evaluated.

Proxies are already quirky: they don't work with internal slots (bye Number, Date, Promise and friends), they change the identity of this (no equality, no WeakMap).
But at least, they work on classes.

So they are useful. Libraries and frameworks can provide many features that motivated building proxies: automatic logging, lazy objects, dependency detection and change tracking.

My point here is that PrivateName and Proxy don't work together. You have to choose one of the two features and give up on the other. Partitionning JS features in this way is terrible.

Here is a basic example.
Let's say a library provides logReads, a function that writes on the console every member that is read.

function logReads(target) { 
  return new Proxy(target, {
    get (obj, prop, receiver) {
      console.log(prop);
      return target[prop];
    },
  });
}

Now let's say I'm writing an application and I use private fields for encapsulation, because they're nice.

class TodoList {
  #items = [];
  threshold = 50;

  countCheap() {
    return this.#items.reduce((n, item) => item.price < this.threshold ? n : n + 1, 0);
  }
}

I would like to use that nice logging library to better understand what happens when I run my code.
Seems legit from a naive user's perspective:

let list = logReads(new TodoList);
list.countCheap(); // BOOOM

Ahaha gotcha! And if you don't know the source code, why it crashes when inside a proxy might be a unpleasant mystery.

Please reconsider. All it takes to solve this is make PrivateName tunnel through proxy (no interception, encapsulation is fine).

Don't think that returning bound functions from the proxy will solve this. It might seem better but creates many new issues of its own.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Jun 14, 2018

The purpose of Proxy is not to be a transparent wrapper alone - it also requires a membrane (iow, you need to control access to every builtin). Specifically yes, your proxy has to return a proxy around every object it provides, including functions.

It would be nice if Proxy was indeed a transparent replacement for any object, but that’s not it’s purpose and as such, it does not - and can not - work that way.

@jods4

This comment has been minimized.

Copy link
Author

@jods4 jods4 commented Jun 14, 2018

@ljharb I am not sure I fully understand your answer, could you explain a little more?

Is logReads above foolish? It's supposed to log (non-private) members access on TodoList (not deep, just that instance) and it actually works today (as long as you don't rely on WeakMap).

How should I write logReads so that it works then?

Looking at MDN page on proxies:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy
I have the impression several examples won't work anymore if you apply them to a class with private members.

Let's take the first one, "Basic example". It provides default values: anything undefined on proxified object is 37.

var handler = {
    get: function(obj, prop) {
        return prop in obj ?
            obj[prop] :
            37;
    }
};

var p = new Proxy({}, handler);
p.a = 1;
p.b = undefined;

console.log(p.a, p.b); // 1, undefined
console.log('c' in p, p.c); // false, 37

Wrap TodoList instead:

var p = new Proxy(new TodoList, handler);
p.c; // 37, yay.
p.activeCount(); // BOOOM
@bakkot

This comment has been minimized.

Copy link
Contributor

@bakkot bakkot commented Jun 14, 2018

You need to proxy methods if you want to make your proxy transparent. Of the top of my head, something like

const objectProxies = new WeakMap;
const proxiedFunctions = new WeakMap;
function logReads(target) { 
  const proxy = new Proxy(target, {
    get (obj, prop, receiver) {
      console.log(prop);
      const value = target[prop];
      if (typeof value === 'function') {
        if (proxiedFunctions.has(value)) {
          return proxiedFunctions.get(value);
        }
        const newFun = new Proxy(value, {
          apply (original, thisArg, args) {
            return original.apply(objectProxies.has(thisArg) ? objectProxies.get(thisArg) : thisArg, args);
          },
        });
        proxiedFunctions.set(value, newFun);
        return newFun;
      }
      return value;
    },
  });
  objectProxies.set(proxy, target);
  return proxy;
}

// Some random business class, using private fields because they're nice
class TodoList {
  #items = [];

  get getter() {
    return this.#items.reduce((n, item) => item.done ? n : n + 1, 0);
  }

  method() {
    return this.#items.reduce((n, item) => item.done ? n : n + 1, 0);
  }
}

// Try to use the class while logging reads
let list = logReads(new TodoList);
list.getter; // works!
list.method(); // works!
list.method.call(logReads(new TodoList)); // works!
list.method === logReads(new TodoList).method; // true!

Yes, the basic examples in the docs will break if you try to use them on an object with private fields, just as they'll break if you try to use them on an object which has been put into a WeakMap, or on an object where someone is testing equality, or an object with internal slots other than those for an array or function. But it's important to keep in mind here that private fields aren't actually giving the designer of the class any power they didn't have previously, just making it more ergonomic (and optimizable and explicit and so on). Anyone intending to provide transparent proxies has always needed to do the above work.

I don't think this is a bug in the design of WeakMaps or proxies (or private fields). This is how they're intended to work together. Proxies are not intended to allow you to violate invariants in other people's code.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Jun 14, 2018

I’d do get(obj, prop, receiver) { return Reflect.has(obj, prop) ? Reflect.get(obj, prop, receiver) : 37; } - but if the property is a function, the only option you have is to return a proxy for that function that does the same thing on the apply trap.

@jods4

This comment has been minimized.

Copy link
Author

@jods4 jods4 commented Jun 14, 2018

Please consider:

  1. Proxies from the MDN docs or like my example exist and work (within existing limitations) today. When coders decide to replace their typescript private fieldwith real #field, they will be quite surprised that their application breaks. This can be very tricky in large projects where responsibility of conflicting code is split across teams. Adopting private fields might break legitimate code.

  2. @bakkot your example "works" as in "doesn't TypeError" but doesn't achieve what my function does. Specifically it never logs items, which is the very purpose of the proxy: detect what fields are accessed when running opaque code.
    The problem is this bit of code:
    original.apply(objectProxies.has(thisArg) ? objectProxies.get(thisArg)
    It calls the function transparently with the original this instead of proxy. That makes PrivateName work, but that bypasses the proxy for anything accessed inside the function.
    That previously working code is impossible to replicate once private fields are introduced.

@ljharb

but if the property is a function, the only option you have is to return a proxy for that function that does the same thing on the apply trap.

See 2. above. It doesn't achieve what I intend to do.

  1. You keep coming back at WeakMap... Yes, it's a trap. I would be glad if new features could be more compatible with older ones. But please observe that the comparison is not as good as you make it to be. Sometimes WeakMap do work, private fields never do. For instance, if you wrap your ViewModel in a proxy for dependency detection from the very beginning, right when it's constructed, then only the proxified this is seen in code and it works using WeakMap and this comparison. You only need to ensure you don't share both the target and its proxy. Private fields never work when invoked on a proxy.

  2. Damn that's large chunk of code for seemingly basic needs. Funny how no introduction to proxies looks like that, makes me wonder how many people use those "fragile" proxies in real code?

I have made several points showing that not tunneling private fields is bad.
I have proposed a reasonable and simple solution: tunnel private fields through proxies. Everything above just works.

You seem opposed to that idea. What is your motivation? What makes not tunneling private fields through proxies better for the language or the community? What benefits outweight the drawbacks I have mentionned?

@bakkot

This comment has been minimized.

Copy link
Contributor

@bakkot bakkot commented Jun 15, 2018

@jods4,

Adopting private fields might break legitimate code.

That's true anyway.

Specifically it never logs items, which is the very purpose of the proxy: detect what fields are accessed when running opaque code.

Assuming you meant some other, public field (it certainly shouldn't log #items!) - yes, that's true. But introspecting on what code you don't own is doing has never been a goal of proxies. You still get all the traps for your code, to which you can provide the proxy rather than its target.

For instance, if you wrap your ViewModel in a proxy for dependency detection from the very beginning, right when it's constructed

If the constructor is your code, and you want it instances of it to have both proxy behavior and a private field, you should just put the private fields on the proxy itself: instead of

class ViewModel extends Object /* or whatever */ {
  #priv = 0;
  method(){ return this.#priv; }
}
foo = new Proxy(new ViewModel, { /* handler */ });
foo.method(); // throws

do

function adapt(base, handler) {
  return class extends base {
    constructor(...args){
      return new Proxy(super(...args), handler);
    }
  };
}

class ViewModel extends adapt(Object /* or whatever */, { /* handler */ }) {
  #priv = 0;
  method(){ return this.#priv; }
}
foo = new ViewModel;
foo.method(); // works

Then you get private fields and also the proxy traps.

Funny how no introduction to proxies looks like that, makes me wonder how many people use those "fragile" proxies in real code?

Yes, I think it's kind of a shame that the documentation often does not make it clear that making proxies actually transparent is not trivial. But this is already the case. I don't think that justifies changing the semantics of private fields in the language.

You seem opposed to that idea. What is your motivation? What makes not tunneling private fields through proxies better for the language or the community? What benefits outweight the drawbacks I have mentionned?

The major benefits in my mind are:

  1. Other people don't get to violate invariants of my code. If I have
class Factory {
  static #nextId = 0;
  #id = Factory.#nextid++;
  constructor() {
    Object.defineProperty(this, 'prop', { value: 'always present' });
  }
}

then I can trust that there is one and only one object with a given #id. I can further trust that objects with #id have any other invariants I care to enforce: in this example, that they have a prop property whose value is "always present". Since the whole point of private fields is to allow class authors to maintain and reason about invariants of their code without exposing their internals to the world, this is not something to lightly forgo.

  1. The mental model for private fields is consistent: fields are attached to objects by the constructor; objects which were not created by the constructor do not have the private field, and as such attempting to access them will throw.
@zenparsing

This comment has been minimized.

Copy link
Member

@zenparsing zenparsing commented Jun 15, 2018

There are good reasons mentioned in this thread for leaving the current behavior as-is. Remember that one of the use cases for "private" fields is being able to more easily self-host fully encapsulated built-ins and platform APIs.

I also think that @jods4 makes some important observations: that if private field usage were to proliferate among user code it will break certain usage patterns based on proxies. Furthermore, this consequence is not particularly obvious to users given the current ("just replace _ with #") syntax.

(To me, this is another argument for having different syntax for different semantics.)

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Jun 15, 2018

It's a brand check - that's the point. It's supposed to change behavior.

@jods4

This comment has been minimized.

Copy link
Author

@jods4 jods4 commented Jun 15, 2018

@bakkot

[introducing private can break...] That's true anyway.

Do you have examples? If the field you're encapsulating was truly private I don't see how.
If it was used by outside code, sure. You can't encapsulate a non-encapsulated field. But then it's good that it breaks as it shows problems in your design.

The example I give worked and I don't see why it shouldn't work with private fields.

Assuming you meant some other, public field

Yes, sorry. Of course I meant public threshold field not private items.

But introspecting on what code you don't own is doing has never been a goal of proxies.

I'm not in the know of ES editors motivations, but...

  • I don't see this information anywhere, so it's rather hard to get.
  • MDN describes proxies as a way to intercept operations, quote: "analogous to the concept of traps in operating systems". That seems in line with what I'm doing.
  • I clearly remember proxies being touted as a way to replace Object.observe when it was dropped from the spec. That's more or less the example I'm giving you here.
  • If I should only intercept my own code, then really you shouldn't have bothered with something as complex as proxies. I can instrument my own code alright without them.
  • Maybe it wasn't an explicit goal but it works fine. Breaking it is a step backwards.

If the constructor is your code [...]

That kind of would work. It's quite constrained, though, notably when it comes to using base classes. Correct me if I'm wrong but for example I don't think it's gonna work well if the base class uses private fields itself?

And that's one problem right here: encapsulation means you shouldn't know nor care whether base class has private fields or not. Yet with proxies it's a crucially observable difference.

Consider that I might not control my base classes. It might come from another team or a library.
A new version of the library that encapsulate its private fields can break consumer code. -> Brittle base class problem, not good.

I don't think that justifies changing the semantics of private fields in the language.

Private fields are not in the language now, so we don't change their semantic per se. We change the proposal.
I have demonstrated that changing the proposal can reduce breakage during private fields adoption, that seems worth considering to me.

Moving on to the [two] major benefits [of sticking with how it is]:

  1. I can trust that there is one and only one object with a given #id.
    The whole point of private fields is to allow class authors to maintain and reason about invariants of their code without exposing their internals to the world.

I don't get how tunneling private access through proxy to the actual target would break that.
Note that I don't want to intercept private access. I want it to tunnel to the real target and work.
Can you provide an actual example where tunneling would permit creating 2 Factory instances with the same #id?

  1. The mental model for private fields is consistent: fields are attached to objects by the constructor; objects which were not created by the constructor do not have the private field, and as such attempting to access them will throw.

I see what you mean and I personnally wouldn't agree.

Public fields are also attached to objects by the constructor, objects which were not created by the constructor do not have the public field, and yet they work on a proxy.

I could give more reasons why I disagree here but anyway I think this point is much weaker than problems above. In itself it can't justify not tunneling private fields.

@jods4

This comment has been minimized.

Copy link
Author

@jods4 jods4 commented Jun 15, 2018

I don't know if it matters to you but it seems using Proxies pretty much as I'm describing here is on VueJS roadmap:

Reactivity system will be rewritten with Proxies with various improvements
https://github.com/vuejs/roadmap

I can't speak for VueJS authors but I'm pretty sure classes using private fields internally won't work in VueJS vnext.
I know Aurelia is considering the same move.

@bakkot

This comment has been minimized.

Copy link
Contributor

@bakkot bakkot commented Jun 15, 2018

Do you have examples?

Sure:

class Foo {
  metadata = makeMeta(this); // may return null
  getName() {
    return (this.metadata || {}).name;
  }
}
Foo.prototype.method.call({});

metadata really is effectively private in that code outside the class never attempts to access it, but making it private will still break this code.

I agree that's not ideal, but I don't think it's worth giving up the privacy model. Other usages of private fields will want to rely on the this.#id throwing for non-instances.

If I should only intercept my own code, then really you shouldn't have bothered with something as complex as proxies. I can instrument my own code alright without them.

Sorry, let me be more precise: you can use proxies to observe (some of) the behavior of code which consumes objects you provide it. So if you're sitting between Alice and Bob, when Alice gives you an object you can wrap it in a proxy before giving it to Bob. But you can't wrap it in a proxy before giving it to Alice.

I should have said "introspecting on what code you don't own is doing with objects it creates".

Correct me if I'm wrong but for example I don't think it's gonna work well if the base class uses private fields itself?

Yes, in that case you'd need to proxy the base class as well.

Can you provide an actual example where tunneling would permit creating 2 Factory instances with the same #id?

A proxy for an object is not the object itself. let x = new Factory and new Proxy(x, {}) are two different objects with the same #id. If code in my class treats them as the same, that will break. So will any other code relying on object identity of things which have the private field, such as this linked list class which avoids cycles (unless private fields tunnel through proxies):

class Node {
  #child = null;
  link(child) {
    let cursor = child;
    while (cursor !== null) {
      if (cursor === this) {
        throw new TypeError('Cycle!');
      }
      cursor = cursor.#child;
    }
    this.#child = child;
  }
  getTail() {
    let cursor = this;
    while (cursor.#child !== null) {
      cursor = cursor.#child;
    }
    return cursor;
  }
}
let x = new Node;
x.link(new Proxy(x, {}); // currently throws, but with tunneling it works
x.getTail(); // currently always terminates, but with tunneling it now spins in this example

Also, though, I want to focus on the other point I made in that paragraph, since it is the core conflict. The point of private fields is to let class authors maintain invariants without exposing their internals to the world. Invariants are not just the values in private fields. Allowing code outside the class to create a proxy for the class's instances which still have their private fields allows code outside the class to break those invariants.

@bakkot

This comment has been minimized.

Copy link
Contributor

@bakkot bakkot commented Jun 15, 2018

encapsulation means you shouldn't know nor care whether base class has private fields or not. Yet with proxies it's a crucially observable difference.

I do want to say that this is a fairly strong concern. I just don't think it outweighs the cost of allowing code outside the class to violate invariants it is enforcing.

@erights, you've thought more about proxies than I have, I would love to get your commentary here.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Jun 15, 2018

@jods4 I think you missed something important....

If I'm right, you were thinking that your handler.get would receive "#items" as the value for prop. That's the problem you think would cause an issue since this["#items"] would return undefined. According to the spec, however, prop would not be "#items", but rather the PrivateFieldName associated with "#items" in the associated class's [[PrivateFieldNames]]. Just think of that as a Symbol(). It's not significantly different. As such, prop would be set to that PrivateFieldName value.

Now given your code, it might still fail since this wouldn't have a member by that name, and this proposal's spec says that [] access to [[PrivateFieldValues]] is a no go. However, if you replaced return target[prop]; with return Reflect.get(target, prop, receiver); it should work since it would continue through the ES engine's internal get handler. Proxies aren't broken by this proposal, but they do have to be handled with even more care than normal.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Jun 15, 2018

@bakkot Unless I got something wrong in my other post, isn't this an issue for hard private?

Since the PrivateFieldName for every used private field would necessarily pass through a get or set handler of a Proxy wrapped around an instance with private fields, it's possible (albeit really convoluted) to not only get the PrivateFieldNames, but also access and alter them on an instance using a combination of Proxy and Reflect. The only way to stop this given the current proposal would be to do as @jods4 is expecting and break Proxy with respect to private fields.

A modification of the proposal (as given here: #104) can solve this without breaking anything else. Using that suggestion, the handler.get method would never be called for private fields.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Jun 15, 2018

@rdking yes - which is also why the get handler does not fire for private field access currently.

@bakkot

This comment has been minimized.

Copy link
Contributor

@bakkot bakkot commented Jun 15, 2018

@rdking, the get trap on proxies is not called for private field access as currently spec'd, and as far as I can tell no one believes it does or is proposing that it should.

@jods4

This comment has been minimized.

Copy link
Author

@jods4 jods4 commented Jun 16, 2018

@rdking

you were thinking that your handler.get would receive "#items" as the value for prop

No. There was one slightly confusing typo in my first comment where I wrote items instead of threshold. I don't want to see private fields in get. I think it would be better if they pierced through the proxy.

@bakkot
About code breaking when introducing private in encapsulated code: nice one, JS is a language with weird features!
I would say that passing arbitrary this to a function, with .call() or .apply(), was not encapsulated code in the first place. If you use an arbitrary receiver for a function, you take responsibility that your receiver shares a compatible implementation as the intended receiver. Assuming you know about the implementation is not encapsulation.

That was a good find but I still don't see an example where encapsulated code will break when introducing real private fields.

About breaking invariants:
It's a tricky discussion because Proxy is very special and intended to behave like the object it wraps. When you say "there are two different objects with the same #id" it's not completely true. There's still only one Factory with that #id, and some proxies of it.

Let's look at your linked-list example. I think it supports my point more than yours.

  1. Consider how you would have done that pre-private fields. A WeakMap maybe? Cool thing is that with a WeakMap it works, even with proxies. You don't have cycles but you can have both the proxy and its target in the chain -> that should be expected since proxies have their own identity... this is outside of this discussion.

  2. Quick aside: @ljharb said private fields were design to work like WeakMap with nicer syntax. Note how they do not: we went from "actually works" to "TypeError".

  3. Your implementation doesn't support proxies because of this === cursor, which is a well-known limitation of proxies. So basically you're using private fields to detect the proxy and throw.
    TC39 has previously discussed whether the platform should expose isProxy() and maybe getProxyTarget(). They explicitely decided not to. Private fields are a great way to work around that decision as demonstrated here -> bad!

  4. Let's say I know I am not gonna insert cycles and I wish to use your linked list... and proxies. No can't do. This is what I meant when I said you're partitioning the web platform in two: proxies or private fields, pick one, don't mix.

To sum up: I still think everything would be better if private fields tunneled.
The real problem with your example is this vs proxies, which is something unfortunate but out of scope here.
You could document this limitation, or you could work-around it with something a little more clever than this === cursor. In fact, private fields can help you do that.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Jun 16, 2018

@jods4 you’d have to wrap accesses in a try/catch to avoid the type error from the private field brand check, if that’s what you wanted.

You can already write an isProxy for every builtin object type except arrays and plain objects. This proposal means that a proxy for an object with private fields can also be distinguished; but that would be the same if your current implementation did such a brand check.

@jods4

This comment has been minimized.

Copy link
Author

@jods4 jods4 commented Jun 16, 2018

@ljharb After looking at all code examples in this thread, my personal summary would be:

Cons of current spec:

  • Creates a brittle base class problem.
  • Introduction of private inside well encapsulated classes can break existing code.
  • Divides JS features: either a piece of code is implemented using privates, or it can be called from a proxy, not both.
  • Makes writing proxies even more complicated and less performant than they are today.
  • Functionnalities possible today and desired by major frameworks (VueJS, Aurelia) become impossible if target uses private fields internally.
  • Although they are an encapsulation primitive, private fields are very much noticeable from outside code.

Pros of current spec:

  • More similar to WeakMap, although not the same. Replacing WeakMap by private can lead to TypeError.
  • (edited) Simpler to implement for browsers.
  • (edited) Can easily enforce brand check: nothing else than instances of your class can be used as receiver of your implementation, including proxies. (I put that under Pros but it is precisely the point we're debating -- I think proxies are special objects and being able to substitue an object for a proxy adds a lot of value to them).

IMHO you failed to provide strong examples of why tunneling would be bad or why not tunneling would be better.
Feel free to expand my list but right now my opinion is that there is no doubt private fields should tunnel through proxies.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Jun 16, 2018

  • I'm not sure how private state makes base classes more brittle - I'd argue it's quite the opposite.
  • Certainly introducing a brand check can break existing code, as will making public properties private, but that's fine - changing existing code can always break it.
  • Making Proxies easy isn't something that I find compelling; they're an advanced feature, not intended to be used casually.
  • can you elaborate? React, for example, would benefit highly by being able to maximize the privacy of its internal structure.
  • in what way are private fields "noticeable" from outside code?

Tunneling would add a very high amount of complexity to Proxies, for the negligible value imo of making Proxies, an advanced feature, easier to use.

@bakkot

This comment has been minimized.

Copy link
Contributor

@bakkot bakkot commented Jun 16, 2018

If you use an arbitrary receiver for a function, you take responsibility that your receiver shares a compatible implementation as the intended receiver. Assuming you know about the implementation is not encapsulation.

I'd say the same for passing a proxy instead of a real object, personally.

It's a tricky discussion because Proxy is very special and intended to behave like the object it wraps.

Not in general: only with respect to code which doesn't have access to the original object.

Consider how you would have done that pre-private fields. A WeakMap maybe? Cool thing is that with a WeakMap it works, even with proxies.

It actually doesn't, not with the obvious implementation:

const childMap = new WeakMap;
class Node {
  constructor() {
    childMap.set(this, null);
  }
  link(child) {
    let cursor = child;
    while (cursor !== null) {
      if (cursor === this) {
        throw new TypeError('Cycle!');
      }
      cursor = childMap.get(cursor);
    }
    childMap.set(this, child);
  }
}
let x = new Node;
x.link(new Proxy(x, {}); // spins - that's even worse than throwing!

This is because I haven't checked that child is either an instance of Node or null, which I got for free with the obvious private field implementation. (This is what I mean about "private fields are good for enforcing invariants".)

And if I do introduce such a check - for example, if I put if (child !== null && !weakMap.has(child)) throw new TypeError('not a Node'); as the first line of link - then I can't link to a proxy (unless I first link that proxy to something else, in which case Node's consumers are now depending on implementation details of the particulars of how it's enforcing that the chain only contains Nodes, which is bad).

In any case, while you might be able to come up with an implementation which does allow a Node and a proxy for the node as distinct elements in the chain with distinct children, it's not obvious to me that that's desirable. In fact I'd expect that to be a huge source of bugs.

TC39 has previously discussed whether the platform should expose isProxy() and maybe getProxyTarget(). They explicitely decided not to. Private fields are a great way to work around that decision as demonstrated here -> bad!

As I understand it that decision was very specifically because it would allow people to detect a proxy without having access to its target, which wasn't a power we wanted to grant. The case of a class is detecting if you give it a proxy for an instance it created is very much not the concern. (And classes can already can already do that, of course, just by sticking their instances in a WeakSet when they construct them.)

Again, the point is that proxies are useful when you're sitting between Alice and Bob, wrapping objects that Alice gives you before handing them off to Bob. They are not so useful for wrapping objects Alice gives you before giving them back to Alice.

Let's say I know I am not gonna insert cycles and I wish to use your linked list... and proxies. No can't do.

Yeah. We've worked hard in the design of private fields to ensure that classes can enforce their own invariants. That does mean that people outside of the class who want to do something unusual and be trusted to enforce the invariants themselves don't get to.

Might have more comments this evening; gotta run now.

@jods4

This comment has been minimized.

Copy link
Author

@jods4 jods4 commented Jun 16, 2018

@bakkot WeakMap implementation does work in a rather obvious way, you just have to use cursor == null instead of cursor === null. Bonus point: you now have lazy initialization. No need to init the child field for leaves.

Now take that working implementation, that could be in the wild today, replace WeakMap by a private field and boom.

I'd say the same for passing a proxy instead of a real object, personally.

Yeah, we already agreed that proxies are not fully transparent. Turns out that in practice it works in a lot of useful situations. Few people use WeakMap for private fields today, but with the new syntax it's gonna change.

They are not so useful for wrapping objects Alice gives you before giving them back to Alice.

Can you point out some official source for that claim? When they were in development, one motivation was that it would help replace Object.observe and that's very much the opposite of what you're saying. Most examples one can found (including on MDN) are not the kind you describe.

Even if it is true and intended usage, what does it change? Where does it fit in my list of Pros and Cons? There are things that people are doing today, that are working and that you are going to break and make impossible. Expected usage doesn't matter as much as practice.

it would allow people to detect a proxy without having access to its target

Strictly speaking your linked list code doesn't have access to the target, but to its class. I guess that's what you mean.

enforce their own invariants.

OK I'm gonna rephrase that with the only way you demonstrated it and edit my Pros list with it: private fields forbid calling your implementation with a class other than your own, including proxies.

@ljharb

I'm not sure how private state makes base classes more brittle - I'd argue it's quite the opposite.

Yes, private state helps better encapsulation -- it's great.
But when it comes to tunneling or not, if you don't tunnel you create a brittle base class situation for anyone whoe is using proxies in his code.

// library v1
export class Lib {
  _age = 18;
  canDrinkBeer() { return this._age > 16 }
}

// consumer of lib
class Buddy extends Lib { 
  // Do my own stuff with it, no private in sight here
}

var proxy = new Proxy(new Buddy, { }); // for whatever reason
proxy.canDrinkBeen();  // public api, all fine

// library v2
export class Lib {
  #age = 18;
  canDrinkBeer() { return #age > 16 }
}

// After consumer upgrades, same perfectly fine code breaks.

Certainly introducing a brand check can break existing code, as will making public properties private, but that's fine - changing existing code can always break it.

Yes, changing code always has potential of breaking stuff. It's bad and we should reduce breaking changes as much as we can.

In preceding example, I was not willingly introducing a brand check. I was just making truly private, state that has always been (conceptually) private.

Consumer was only using public API. In the end, I believe that it is unexpected that it broke.

Making Proxies easy isn't something that I find compelling; they're an advanced feature, not intended to be used casually.

I find that shocking. Good design should be as easy to use as possible, no matter the intended target.

Code has to be written, tested, delivered to browser and run. Making stuff hard to use on purpose is offending the whole industry, who has to pay the ensuing cost.

If there is a problem with using a feature "casually" then maybe it was poorly designed.

can you elaborate?

Assuming you speak of frameworks: VueJS and Aurelia are considering using proxies for automatic ViewModel observation: i.e. detect when properties change so that they can update the UI accordingly, i.e. replace Object.observe. Look at VueJS roadmap, I linked it above.

The point here is that it won't work if there are private fields in ViewModel classes, or their hierarchy. Private fields are a very tempting feature so there will be conflict there for sure.

React, for example, would benefit highly by being able to maximize the privacy of its internal structure.

Yes, private fields are great. VueJS and Aurelia will benefit greatly as well. It's just the interaction with proxies that is going to create friction.

in what way are private fields "noticeable" from outside code?

See my example above. Consumer code crashes when library was updated to use private fields.

Tunneling would add a very high amount of complexity to Proxies, for the negligible value imo of making Proxies, an advanced feature, easier to use.

Maybe, but proxies have made their way into JS. They are advanced but some people, sometimes in frameworks or libraries, use them. You need to support them.

(edit): and very importantly, this is not about making proxies easier to use. It's enabling scenarios that are possible today but will be impossible once the target class uses private fields.

You don't need to do anything complicated to get a TypeError in your face.

class ViewModel {
  #count = 0;
  increment() { #count++; }
}
var p = new Proxy(new ViewModel, {}); // totally empty proxy
p.increment(); // Boom!
@jods4

This comment has been minimized.

Copy link
Author

@jods4 jods4 commented Jun 16, 2018

@bakkot I was thinking about the "intended" usage of proxies and it just makes no sense to me.
In today JS, there is absolutely no difference between passing a proxy to a function on Alice or passing the proxy as a receiver this for a function on Bob. In the end it's just functions and parameters.

In fact, the same things are broken on both sides: keeping references and comparing identity; attaching data with a WeakMap. In fact, Alice is more likely to extend the object with a WeakMap than Bob. Usually coders put their private directly on instances and don't bother with weak maps.

And your complex membrane code for that use case is not even 100%. If Alice wants to pass parameters dynamically, she might do stuff like Bob.prototype.someFunc.apply(proxy, [a,b,c]). Proxy ended up in Bob even in your intended use case.

I feel like your making a circular argument. The only difference between a function on Alice and one on Bob are upcoming private fields. So you're saying that we shouldn't use a proxy that way because it's how you want private fields to work.

Did I miss something?

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Jun 16, 2018

You don't need to do anything complicated

The design of Proxy requires that you do have to do complicated things to be able to successfully use them, including wrapping effectively every property value in a Proxy before returning it in a [[Get]]. This might be poor design, it might not be, but it's none the less the design.

@isiahmeadows

This comment has been minimized.

Copy link

@isiahmeadows isiahmeadows commented Jun 17, 2018

@ljharb

Here's an interesting tidbit that might inform the discussion: the spec, as far as I can tell, doesn't even state whether internal slots should or shouldn't be read through (it's undefined behavior), but that's what I'm observing. If you want a test, this should run without error if an implementation reads internal slots through proxies, and throw a TypeError on the second line if it doesn't:

var object = new Proxy([], {})
object.push(1)

My personal opinion on this is whatever engines do with internal slots, private slots should work identically. Anything else is surprising, as it breaks the encapsulation and transparency of proxies, as well as the existence of private fields within an object (whose existence should generally be private itself). Also, doing otherwise would make builtins impossible to implement at the language level, and IIRC userland implementations of builtins is one of the driving reasons for this proposal. (Correct me if I'm wrong here.)

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Jun 17, 2018

@isiahmeadows Array.prototype.push does not check internal slots; the only array method that does is Array.isArray. Thus, it shouldn't throw a TypeError in any case. Arrays are the special case that do tunnel through proxies, by explicitly specifying it.

@erights

This comment has been minimized.

Copy link

@erights erights commented Jun 17, 2018

Proxies and WeakMaps were designed, and initially motivated, to support the creation of membranes. Proxies used standalone cannot be transparent, and cannot reasonably approximate transparency. Membranes come reasonably close to transparently emulating a realm boundary. For classes with private members, the emulation is essentially perfect. Let's take the class code

class Foo {
  #priv;
  constructor() {
    this.#priv = {};
  }
  both(other) {
    return [this.#priv, other.#priv];
  }
}

new Foo().both(new Foo()); // works

// foreignFoo is a Foo instance from another realm
// or foreignFoo is a membrane proxy for a Foo instance
// across a membrane boundary

new Foo().both(foreignFoo); // throws
foreignFoo.both(new Foo()); // throws

foreignFoo.both(foreignFoo); // works

The Foo code evaluated in another realm creates a behaviorally identical but distinct Foo class. Instances of each foo class can see into other instances of the same foo class but not instances of the other foo class. The expression foreignFoo.both, in the inter-realm case, evaluates to the both method from the foreign Foo class, i.e., the one that foreignFoo is an instance of. In the membrane case, foreignFoo.both evaluates to a proxy to the both method of the Foo class on the other side of the membrane.

The non-transparent case is that, for the builtins, like Date, with internal slots, the internal slots are visible across realms, which is broken but entrenched, and so is much too late to fix. Across a realm boundary, the internal slots act as-if Date is defined by a class and the internal slots are defined as Date's private members, and so don't work across the boundary as shown in the Foo example above.

There is much confusion about Array.isArray. Those thinking about proxies used standalone, which can never be reasonably transparent anyway, think that the so-called exception made for this in the proxy rules is a mistake. Instead, since Array.isArray does work across realms, it is consistent for it to work across a membrane boundary. Likewise typeof f === "function" works across realms and likewise works across membrane boundaries.

This is because the four-way distinction between normal objects, arrays, functions, and primitive values is fundamental to JavaScript. For example, the JSON.stringify algorithm distinguishes between objects, arrays, functions, and primitive values. The JSON.stringify algorithm thus works transparently across a membrane boundary because this four-way distinction is preserved.

By contrast, JSON.stringify has no specific knowledge of Date and RegExp, and no one would expect it to. Concepts like Date and RegExp are more like concepts that could have been provided by user code as classes. Across a membrane boundary, Date and RegExp in fact act as if they were provided by classes, which is less surprising than how they actually act across realm boundaries.

attn @ajvincent @tvcutsem

@isiahmeadows

This comment has been minimized.

Copy link

@isiahmeadows isiahmeadows commented Jun 18, 2018

@ljharb Good point - I didn't catch that. new Proxy(new Date(), {}).valueOf() fails correctly, and I forgot that Array.prototype.push is generic like 99% of the other Array methods. 😄

And now that I think about it closer, proxies don't have all the methods of dates, etc., so that behavior is in fact defined.

Edit: Example from Node's REPL:

> new Proxy(new Date(), {}).valueOf()
TypeError: this is not a Date object.
    at Proxy.valueOf (<anonymous>)
@isiahmeadows

This comment has been minimized.

Copy link

@isiahmeadows isiahmeadows commented Jun 19, 2018

@erights If proxies get turned into membranes, it might be worth it to start also translating the various builtins to use private fields as well, to keep it consistent. They would need to be a different type of private field, since they often are used across multiple classes. (Still think my proposal could help alleviate that issue by simply letting all builtins be defined within its own scope, scoped above the realm with a few builtins.)

@littledan

This comment has been minimized.

Copy link
Member

@littledan littledan commented Jun 24, 2018

Thanks for your explanations, @bakkot, @erights and others. The high-level point here seems to be, the semantics of private fields with Proxy follows the original design of Proxy; they do not transparently form part of a membrane because Proxy does not transparently form a membrane tunneling requires some particular logic in the Proxy and is not built-in. Any attempt at revisiting this property might be better done with something like a standard library for the membrane pattern.

@zenparsing

To me, this is another argument for having different syntax for different semantics.

I'm not sure if using -> rather than .# would imply for programmers that the semantics with respect to Proxies are so different. Maybe broader feedback from JS programmers could give us more information here.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Apr 17, 2019

@mbrowne i assume it would be some mechanism to end up with a way that i could, as a class author, opt in to a Proxy tunneling me - and then that I, as a Proxy creator, could tap into that tunneling. I'd expect it to work for private fields and internal slots, and ideally, WeakMap identity, such that Function.prototype.toString.call(x) does not throw for an x that is a Proxy for a function, for example. I haven't thought through whether class author opt-in would be always required or not; nor have I thought through the mechanism for doing so - but I think it's important that internal slots and private fields behave the same way through a Proxy.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 18, 2019

@ljharb To come up with the kind of proposal you're hoping for, it would be helpful to understand why Proxy doesn't forward internal slots. It would be completely wasteful if someone offered a proposal that wouldn't even be given a chance because it violated some intention unknown to the proposer.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Apr 18, 2019

Totally fair. I don't know the reason there - I do know that it's generally considered unfortunate that Array.isArray tunnels while nothing else does.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 18, 2019

Is it also fair to assume that no proposal adjusting the semantics of class-fields would be considered acceptable?

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Apr 18, 2019

Any follow-on proposal would need to assume that all observable semantics of class fields couldn't be changed, if that's what you mean.

@mbrowne

This comment has been minimized.

Copy link

@mbrowne mbrowne commented Apr 18, 2019

Any follow-on proposal would need to assume that all observable semantics of class fields couldn't be changed, if that's what you mean.

So among other things, that means that the following would still need to produce a type error by default, is that correct?

class K { #x; get x() { return this.#x; } }
let k = new Proxy(new K, { });
k.x  // TypeError

...unless the user explicitly opted into tunneling somehow.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Apr 18, 2019

@mbrowne fair point; generally things that throw are permitted to stop throwing - but i think the safer default would be to not allow tunneling without an explicit opt-in, so yes.

@mhofman

This comment has been minimized.

Copy link

@mhofman mhofman commented Apr 18, 2019

In my opinion this is not a problem specific to private fields, but as @ljharb explains above (#106 (comment)), to any hidden data access based on this, including internal slots or WeakMap.

The only solution I can imagine is to have a symbol on objects which have such hidden data, to be notified when a proxy is created for the object. I.e. the class author opt-in.
A new proxy constructor would then check for that symbol. (proxy creator opt-in)

Symbol.newProxy = Symbol();

Proxy.inform = function(target, handler) {
    const proxy = new Proxy(target, handler);
    if (typeof target[Symbol.newProxy] == "function")
        target[Symbol.newProxy](proxy);
    return proxy;
};

const privates = new WeakMap();

class Person {
    constructor(name) {
        privates.set(this, {});
        this.name = name;
    }

    [Symbol.newProxy](proxy) {
        privates.set(proxy, privates.get(this));
    }

    get name() {
        return privates.get(this).name;
    }

    set name(value) {
        privates.get(this).name = value;
    }

    sayHi(to) {
        return `Hello${to ? " " + privates.get(to).name : ""}, I am ${
            privates.get(this).name
        }.`;
    }
}

const john = new Person("John");
const jane = new Person("Jane");

const proxyJohn = Proxy.inform(john, {});
const proxyJane = Proxy.inform(jane, {});
const proxyProxyJohn = Proxy.inform(proxyJohn, {});

console.log(proxyJohn.name); // John
console.log(proxyJane.sayHi(proxyProxyJohn)); // Hello John, I am Jane.

The symbol could also be used as a marker to let the Proxy creator that it uses hidden data, in case the creator would prefer to simply not intercept internal behavior:

function createProxy(target) {
    const handler =
        typeof target[Symbol.newProxy] == "function"
            ? {
                  get(target, p, receiver) {
                      if (receiver === proxy) {
                          const ret = Reflect.get(target, p);
                          if (typeof ret == "function") {
                              return new Proxy(ret, handler);
                          }
                          return ret;
                      }
                      return Reflect.get(target, p, receiver);
                  },

                  set(target, p, value, receiver) {
                      if (receiver === proxy) {
                          return Reflect.set(target, p, value);
                      }
                      return Reflect.set(target, p, value, receiver);
                  },

                  apply(fn, thisArg, args) {
                      if (thisArg === proxy) {
                          return Reflect.apply(fn, target, args);
                      }
                      return Reflect.apply(fn, thisArg, args);
                  },
              }
            : {};

    const proxy = new Proxy(target, handler);

    return proxy;
}

const mildProxyJane = createProxy(jane);

console.log(mildProxyJane.name); // Jane
console.log(mildProxyJane.sayHi()); // Hello, I am Jane.
@bakkot

This comment has been minimized.

Copy link
Contributor

@bakkot bakkot commented Apr 19, 2019

The only solution I can imagine

If the class wants to cooperate with proxies, there's already a straightforward if moderately inconvenient solution which doesn't require notifying the class of anything:

let hidden = new WeakMap;

class Foo {
  _ = this;
  #priv = 1;

 constructor() {
   hidden.set(this, 2);
 }

  method() {
    return this._.#priv + hidden.get(this._);
  }
}

(new Proxy(new Foo, {})).method(); // 3
@mhofman

This comment has been minimized.

Copy link

@mhofman mhofman commented Apr 19, 2019

If the class wants to cooperate with proxies, there's already a straightforward if moderately inconvenient solution which doesn't require notifying the class of anything

Except if the proxy is intrusive and automatically wraps anything accessed by get into a Proxy object.

The point of my suggestion is 2 fold:

  • Provide a standardized hook so that cooperating objects with hidden data can handle being proxied. Such objects can decide how to behave. E.g. for private class fields, we could decide to copy the [[PrivateFieldValues]] internal slot to the proxy object.
  • Inform proxy creators that the object will likely not work if being proxied plainly.
@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 19, 2019

@ljharb

Any follow-on proposal would need to assume that all observable semantics of class fields couldn't be changed, if that's what you mean.

That's exactly what I meant. My point was partially made by @mbrowne. The rest is below.

@mhofman

In my opinion this is not a problem specific to private fields, but as @ljharb explains above (#106 (comment)), to any hidden data access based on this, including internal slots or WeakMap.

That's almost right. The Proxy problem with WeakMap has nothing to do with hidden data. It's more of a pointer comparison problem if I were to relate it to something like C++. The beauty here is that the WeakMap issue can be remedied directly in ES without engine modification. So I see that as more of an annoyance than a serious technical issue.

The real problem is that internal slots are not forwarded by Proxy objects. All private fields did was clumsily inherit the existing problem. That's been the argument TC39 has been offering as to why they have thus far had no real intention of remedying the problem in this proposal.

The point of my question was because of the possible directions any remedying proposal can take:
  • Let Proxy objects forward internal slot.
    • Though we don't seem to understand why it wasn't done in the first place, I can see how it might become troublesome as even the exotic Proxy object has the internal slots for Object plus at least two others to hold the [[Target]] and [[Handler]]. Accessing the Proxy objects own internal slots could become very complicated and hard to optimize if it forwarded slots, even with some intelligence. So I don't think that's going to be a solution
  • Provide exemptions for the known intrinsic objects using internal slots as has already been done for Array
    • Too messy. Too hard to maintain. Too fragile. Not at all future resistant. So no.
  • Use a flagging mechanism to signal that instances of a class are willing to tunnel.
    • Awkward. It would only be necessary if the class has private fields. It also adds an extra encumbrance to an already slow Proxy object.
    • Counter-intuitive. As it presently stands, an object that only has a single, non-function field can be Proxy wrapped without difficulty or extra flags. So the natural intuition of most developers will be that since private fields are undetectable, it shouldn't matter whether or not the afore mentioned field is public or private. The jury is out on this one.
  • Add a new handler to Proxy and let it barely have enough parameter info to decide whether to access via the [[Target]] or the Proxy.
    • Potential to slow Proxy accesses even further, but that's mostly up to the developer, so it's a wash.
  • Rework private-fields so that the observable semantics stay 100% compatible with the existing proposal, but the internal details completely avoids the use of internal slots.
    • Has the advantage of not incurring additional speed penalties, but delays the proposal.
    • TC39 is of the impression that they've already thought through all possible viable scenarios and that what is in this proposal is the best they can get a consensus on, and as such, are unlikely to give any such proposal a chance.
  • In the end, the only real paths are either tamper with how Proxy works, or tamper with this proposal. There's nothing in between. If we are so adamant about not modifying class-fields, then we're forced to modify something developers already understand to accommodate something new that didn't necessarily need to require the accommodation.

Long story short:

We can either modify something already part of ES, or modify this proposal. I don't see where it makes more sense to modify something already part of ES to add something new when none of the features of the new thing had to be implemented in a way that forced the modification of the existing thing.

I'm just asking that room be left open for considering a different approach that avoids the problem while preserving the semantics.

@mhofman

This comment has been minimized.

Copy link

@mhofman mhofman commented Apr 19, 2019

That's almost right. The Proxy problem with WeakMap has nothing to do with hidden data. It's more of a pointer comparison problem if I were to relate it to something like C++. The beauty here is that the WeakMap issue can be remedied directly in ES without engine modification. So I see that as more of an annoyance than a serious technical issue.

I'm not sure I agree here. WeakMap is basically the userland way of implementing internal slots or any protected data related to the this instance. We should strive to solve the issue not just for the private fields slot, but for all objects with private data, whether it's part of the language, implemented by the host, or in ES by the end developer.

  • Let Proxy objects forward internal slot.
    • Though we don't seem to understand why it wasn't done in the first place, I can see how it might become troublesome as even the exotic Proxy object has the internal slots for Object plus at least two others to hold the [[Target]] and [[Handler]]. Accessing the Proxy objects own internal slots could become very complicated and hard to optimize if it forwarded slots, even with some intelligence. So I don't think that's going to be a solution

Performance-wise I don't know if having the proxy and target share the same [[PrivateFieldValues]] list would really be a problem.

  • Use a flagging mechanism to signal that instances of a class are willing to tunnel.
    • Awkward. It would only be necessary if the class has private fields. It also adds an extra encumbrance to an already slow Proxy object.

Yes awkward, but it doesn't change the existing behavior of Proxy. It's an opt-in from target objects with hidden data (not just private fields), and would have the equivalent opt-in from proxy creators.
Again I'm not sure where the performance impact is. The flag would only need to be checked when creating the proxy object, and when using the opt-in creation method.

    • Counter-intuitive. As it presently stands, an object that only has a single, non-function field can be Proxy wrapped without difficulty or extra flags. So the natural intuition of most developers will be that since private fields are undetectable, it shouldn't matter whether or not the afore mentioned field is public or private. The jury is out on this one.

I don't see it. As a developer I don't know if doing obj.foo = "bar" is simply writing a property's value, or invoking a setter (unless I lookup the property definition). If there is a getter/setter, I have no guarantee that the actual backing store might not be in some hidden data structure that breaks when used through a proxy.
Is the problem more prevalent with the current private fields proposal, yes. Is it non-existent today, definitely not.

Long story short:

We can either modify something already part of ES, or modify this proposal. I don't see where it makes more sense to modify something already part of ES to add something new when none of the features of the new thing had to be implemented in a way that forced the modification of the existing thing.

I don't think it's an either/or, I think doing both could be a solution: Add to Proxy so it can be used explicitly on objects with hidden data (with opt-in), and make classes with private fields opt-in such mechanism by default.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 19, 2019

@mhofman

I'm not sure I agree here.

I'm not saying it's not a problem worth solving. I think it's a core design anomaly in Proxy that it is not identity transparent. It's not bad, just unfortunate. What I am saying is that this issue is structurally different than the issue for internal slots (an identity problem vs a forwarding problem).

Performance-wise I don't know if having the proxy and target share the same [[PrivateFieldValues]] list would really be a problem.

The problem is that this would be short-sighted. Forwarding [[PrivateFieldValues]] would do nothing to help native objects and their subclasses become directly Proxy-ready. But if you solve the problem generally for all internal slots, then the private-fields problem is already solved.

I don't see it. As a developer I don't know if doing obj.foo = "bar" is simply writing a property's value, or invoking a setter (unless I lookup the property definition).

Nor should you. You shouldn't need to care about that at all. For a developer trying to wrap 3rd-party objects in a non-membrane proxy to observe, the private API should be invisible, impenetrable, and insubstantial. From the Proxy's perspective, it should be as if an equivalent object only having the public API was the thing being proxied.

I don't think it's an either/or, I think doing both could be a solution

I think it should be both. What I don't agree with is the idea of forcing a change on an existing, well-known, viable implementation to accommodate a new feature with an awkward, unfortunate, and unnecessary encumbrance in its implementation. The problem is that the board isn't likely to accept a proposal to modify private-fields unless it is left with no other recourse. As it already stands, they'd rather not address the issue at all in the current proposal just to push something out the door.

@mhofman

This comment has been minimized.

Copy link

@mhofman mhofman commented Apr 19, 2019

The problem is that this would be short-sighted. Forwarding [[PrivateFieldValues]] would do nothing to help native objects and their subclasses become directly Proxy-ready. But if you solve the problem generally for all internal slots, then the private-fields problem is already solved.

I think it should be both. What I don't agree with is the idea of forcing a change on an existing, well-known, viable implementation to accommodate a new feature with an awkward, unfortunate, and unnecessary encumbrance in its implementation. The problem is that the board isn't likely to accept a proposal to modify private-fields unless it is left with no other recourse. As it already stands, they'd rather not address the issue at all in the current proposal just to push something out the door.

Forwarding all internal slots, not just the [[PrivateFieldValues]] slot through a proxy is bigger than the private fields in this proposal. It is a change to the existing and actively in use Proxy object.
I am not sure what you're advocating for anymore? Make the private fields special with proxy, which could be done within the scope of this proposal only, or improving Proxy so that any object with hidden data can work with proxies, but requires a separate proposal?

I personally think that we should improve Proxy for all use-cases.
I'm not sure how host objects implement hidden data internally, but I doubt they leverage internal slots. It's more likely they check the identity of this.

Similarly, babel transpiles private fields using WeakMap, the same way some of us have been associating hidden data to our instances for years.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 19, 2019

I am not sure what you're advocating for anymore?

From where I sit, there are only 2 viable options:

  1. Rewrite private fields so they don't need to use internal slots but keep 100% compatibility with the existing semantics.
  2. Re-design Proxy so that it forwards any reference to an internal slot that is not its own.

I think both need to be done. The current design of private-fields is a hinderance to multiple design patterns due only to the semantic approach. I've already proven that it's possible to do without internal slots. The adjustment to Proxy needs to be done so Native objects will still fully work when proxied.

It's more likely they check the identity of this.

From what I've seen in every implementation whose source I could view, that's nearly it. It ends up being an RTTI problem. The pointer/reference to the native object Proxy can't be used to retrieve the native object pointer/reference, so the internal code doesn't get an object with the expected function table. Instant failure.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Apr 20, 2019

Note that one of the intended designs for Proxy is to have more than one potential target - so transparently forwarding internal slots or private fields to the actual target of the proxy would not necessarily work well. There’d need to be a way for the Proxy handler to dynamically determine the target for the private lookup - without having any other say as to how that lookup is handled.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 21, 2019

Note that one of the intended designs for Proxy is to have more than one potential target...

That's comes as a surprise given the single-target design. But now I understand more of the problem a little better. A Proxy is always free to ignore [[Target]] while processing handlers. I imagine that how the intent was designed in.

Are you saying that if internal slots were forwarded, they'd have to allow themselves to be re-targetable via Proxy? That's counter-intuitive. I would think that much like a Proxy for an empty object, internal slots not owned by the Proxy would automatically forward to [[Target]]. There's no recourse for them to be re-targeted as there's no invariant to catch and allow such a re-targeting. Adding such invariants would remove their candidacy for use as private field containers.

@ljharb

This comment has been minimized.

Copy link
Member

@ljharb ljharb commented Apr 21, 2019

I agree with those as difficulties; i'm more pointing out that there are intended usage of Proxy that would break by passing any information implicitly to the [[Target]].

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 22, 2019

Accepting that for what it is, then I will replace item 2 as follows:

  1. Modify Proxy so as to provide the Proxy developer with the ability to control directly whether or not a particular access will be handled by the by the Proxy.

This is essence of the idea behind the idea from @isiahmeadows that I modified.

@mhofman

This comment has been minimized.

Copy link

@mhofman mhofman commented Apr 22, 2019

Note that one of the intended designs for Proxy is to have more than one potential target - so transparently forwarding internal slots or private fields to the actual target of the proxy would not necessarily work well.

Thanks for clarifying that @ljharb. I am still a little confused as of the terminology here. When you mention multiple potential target, would the Proxy object have it's internal [[Target]] slot change, or is it just the handler that accesses a different target object based on some internal logic?

It seems that giving proxy objects an internal [[Target]] slot is explicitly creating a unique relation between the proxy object and its target. As a developer, I'm not sure I would find abnormal to have internal / hidden data of a target not work if that target is not the one explicitly provided when creating the proxy.
Also, do you see a lot of use cases where a proxy handler would have the logic to pick a different target, yet wouldn't know enough of the internal implementation of such target to not know they would break if accessed through an unrelated proxy object?

However, the ability to change the explicit target of a Proxy could be an interesting one. A new Proxy creation API (e.g. Proxy.swappable) could help solve some use cases of re-targeting, akin to the revocable use case.
With such an API, the proxy object would still be related to a single target at any given point, and the internal / hidden data can be explicitly re-wired when a target swap occurs.

@rdking :

Accepting that for what it is, then I will replace item 2 as follows:

  1. Modify Proxy so as to provide the Proxy developer with the ability to control directly whether or not a particular access will be handled by the by the Proxy.

I'm still not convinced modifying the Proxy object in itself is necessary.

First, changing the behavior of proxy objects created by the existing APIs would be a problem as some existing code surely relies on the existing behavior (no equivalence or forwarding of private / hidden data).

As I proposed above, I think you only need to extend the Proxy creation to make explicit by the proxy creator that they're ok with the target being informed that a new Proxy instance is targeting them. At that point the target implementation (host object, private fields proposal, end user JS code, etc.) can wire their internal / hidden data however is appropriate (e.g. set the [[PrivateFieldValues]] slot of the proxy object to the one of the target, or register the proxy object in an equivalence map).

This approach could be made compatible with target swaps of proxy objects, you just need to inform the new target of the proxy (and possibly inform the old target they are no longer targeted by that proxy instance).

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 22, 2019

@mhofman

I'm still not convinced modifying the Proxy object in itself is necessary.

To solve the problem presented in this thread? No, it's not necessary at all hence item 1 in #106 (comment). However, unless I'm mistaken, @ljharb and several other TC39 members would rather keep the current proposal as is and fix the issue by solving the underlying problem: the fact that any internal slot-related code is subject to break when wrapped by a proxy. This apparently comes with the hope that identity-related issues for Proxy and WeakMap will also be solved at the same time.

...some existing code surely relies on the existing behavior...

The [[Handler]].resolve() approach eliminates risk there. It becomes up to the using developer to ensure that an appropriate handler is written so as not to break any code being used.

This approach could be made compatible with target swaps of proxy objects...

The problem here is that there is no way to catch access attempts against an internal slot. Since they are not properties of the object, they do not pass through the Invariant functions. So it really isn't at all possible to change the target of an internal slot access. Just flagging the Proxy instance with a symbol would be insufficient for altering that target.

That's why I offer the following:

  1. To solve the immediate issue with private-fields vs Proxy: https://github.com/rdking/proposal-proxy-safe-private
  2. To solve the long-term issue with internal slots vs Proxy: #106 (comment)
@mhofman

This comment has been minimized.

Copy link

@mhofman mhofman commented Apr 22, 2019

The [[Handler]].resolve() approach eliminates risk there. It becomes up to the using developer to ensure that an appropriate handler is written so as not to break any code being used.

The proxy creator can already write a handler today that makes sure the target doesn't break by invoking methods with the target as thisArg instead of the proxy. The caveat is that it can't intercept further internal calls to public methods of the target.

I'm not sure how your resolve() idea solves that though. What is the isPrivate parameter exactly ? Who decides when it's true? I don't think a solution specific to private fields is the way to go, and I don't see how you can generalize it to other implementations of hidden data.

The problem here is that there is no way to catch access attempts against an internal slot. Since they are not properties of the object, they do not pass through the Invariant functions. So it really isn't at all possible to change the target of an internal slot access. Just flagging the Proxy instance with a symbol would be insufficient for altering that target.

Which is why I'm not advocating to catch access attempts, but simply to let the target know ahead of time that it may be accessed by a proxy instance, so that when it is, it's capable of resolving the this proxy to find the related target hidden data.
That resolution would be done by the target's implementation, not by the proxy handler.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 22, 2019

I'm not sure how your resolve() idea solves that though.

It's meant to be an intrinsic access redirector. Under normal circumstances, when you do this:

foo.bar = 2

the engine calls

foo.[[Set]]("bar", 2, foo);

If foo is a Proxy object, the above [[Set]] function calls the set proxy handler if it exist or else redirects the call to:

foo.[[Target]].[[Set]]("bar", 2, foo);

The resolve() function will work by altering this path. Before attempting the above, if the proxy handler contains a "resolve" function, the following will be called:

foo.[[Handler]].resolve([[Target]], foo, isPrivate(field), "set")

the result of which becomes the new receiver for the action. If [[Target]] is returned, then the Proxy becomes blind to the resulting actions. If the Proxy is returned, then the appropriate proxy handler is triggered.

What is the isPrivate parameter exactly ? Who decides when it's true?

This parameter is set true when the access attempt is against something the Proxy is incapable of monitoring. That includes but is not necessarily limited to private fields and internal slots.

I don't think a solution specific to private fields is the way to go, and I don't see how you can generalize it to other implementations of hidden data.

Most other implementations of hidden data rely on identity and/or closures Proxy can't see closures even enough to crash because of them. No issue there. For identity based methods, WeakMap's inability to unwrap a Proxy can be remedied by having WeakMap trigger this resolve function, using the result as the key instead of the original object. Hidden data approaches outside of those 2 types won't be affected one way or the other.

@mhofman

This comment has been minimized.

Copy link

@mhofman mhofman commented Apr 23, 2019

Most other implementations of hidden data rely on identity and/or closures Proxy can't see closures even enough to crash because of them. No issue there. For identity based methods, WeakMap's inability to unwrap a Proxy can be remedied by having WeakMap trigger this resolve function, using the result as the key instead of the original object. Hidden data approaches outside of those 2 types won't be affected one way or the other.

This is where I wholly disagree.

Who decides when the resolve handler should be used to test identity? Why WeakMap and not Map, Set or Array.indexOf? Where should it stop?
Also, it probably shouldn't apply to all WeakMap instances as external users (proxy creator or end-user code) may want to associate data with the proxy object but not the target. Do we need to add new constructors for all those objects?

Either you do identity test for proxy objects everywhere, or you do it nowhere. Doing it everywhere is obviously too costly.
What you need is a way to make the hidden data implementation aware of the equivalence between proxy and target objects, so that it doesn't have to rely on hooks into the identity test. My proposal is one approach to accomplish this.

@isiahmeadows

This comment has been minimized.

Copy link

@isiahmeadows isiahmeadows commented Apr 23, 2019

Just unsubscribed...it's gotten a bit too noisy and personally, this bug doesn't really affect me much in practice.

@mbrowne

This comment has been minimized.

Copy link

@mbrowne mbrowne commented Apr 23, 2019

It's totally understandable for people to unsubscribe if it's getting too noisy for them personally. Of course, following the discussion and caring about the issue are two different things. IMO anyone who cares about the language should care about this issue being solved, for reasons I explained previously (and the issue doesn't affect me personally at all). And that requires detailed technical discussion. At some point (I'm not sure when) this discussion should be moved somewhere else, but I'm not sure where. I think we're working toward a proposal but are still figuring out what direction to take...I know @rdking has a repo already for a proposal that would address at least one half of the issue, but that proposal is basically a whole reimplementation of class-fields (how they work behind the scenes, that is) and I don't think that's the best meeting place since it's probably a non-starter with the committee. And it doesn't have anything like consensus among the rest of us either (myself included).

When we get to the point of creating an actual proposal, we should think about what committee members to invite to the discussion. The committee members who were most actively involved in class-fields might not necessarily be the same members who are most interested in proxies. It would be great to identify more committee members who are actively interested in helping to solve this issue.

@rdking

This comment has been minimized.

Copy link

@rdking rdking commented Apr 23, 2019

@mbrowne That's precisely what I'm after. @ljharb asked for a proposal and also provided some limiting information. I could easily contrive a proposal based on what I currently have but it wouldn't be sufficient to meet even half of the use cases out there. So I'm in information gathering mode. I would be grateful to all who add their opinion of what's needed.

Though admittedly, I'm beginning to think we need to move this conversation to a different repo since it's starting to get a bit off topic for this repo. To that end, I've created https://github.com/rdking/proposal-proxy-transparency

@mhofman My reply to you will appear there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.