Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow overloading of a.b field access syntax #1974

Closed
StefanKarpinski opened this issue Jan 10, 2013 · 249 comments
Closed

allow overloading of a.b field access syntax #1974

StefanKarpinski opened this issue Jan 10, 2013 · 249 comments
Assignees
Labels

Comments

@StefanKarpinski
Copy link
Member

@StefanKarpinski StefanKarpinski commented Jan 10, 2013

Brought up here: #1263.

@stevengj
Copy link
Member

@stevengj stevengj commented Jan 10, 2013

The ability to use dots as syntactic sugar for mutator/accessor methods would be nice for lots of things. I've always appreciated this in languages that provide it, so that you can turn structure fields into more complicated abstractions without breaking the API.

@toivoh
Copy link
Contributor

@toivoh toivoh commented Jan 10, 2013

+1

@JeffBezanson
Copy link
Member

@JeffBezanson JeffBezanson commented May 6, 2013

I have an absolutely awesome way to implement this.

@johnmyleswhite
Copy link
Member

@johnmyleswhite johnmyleswhite commented May 6, 2013

Interested in talking about it? I know that Tom Short is really interested in having this for DataFrames, although I've come to be increasingly skeptical about the wisdom of using this feature.

@stevengj
Copy link
Member

@stevengj stevengj commented Oct 22, 2013

This would make calling Python code (via PyCall) significantly nicer, since currently I'm forced to do a[:b] instead of a.b.

@stevengj
Copy link
Member

@stevengj stevengj commented Jan 13, 2014

@JeffBezanson, any chance of having this for 0.3? Would be great for inter-language interop, both for PyCall and for JavaCall (cc @aviks).

@ihnorton
Copy link
Member

@ihnorton ihnorton commented Jan 13, 2014

@JeffBezanson if not, is there any chance you could give some direction on how you want this implemented? (I have an absolutely awesome way to implement this.)

@StefanKarpinski
Copy link
Member Author

@StefanKarpinski StefanKarpinski commented Jan 13, 2014

In my experience, there is no faster nor surer way to get Jeff to implement something than to implement a version of it that he doesn't like ;-)

@JeffBezanson
Copy link
Member

@JeffBezanson JeffBezanson commented Jan 14, 2014

The basic idea is that you implement

getfield(x::MyType, ::Field{:name}) = ...

so that you can overload it per-field. That allows access to "real" fields to keep working transparently. With suitable fallbacks getfield(::MyType, ::Symbol) also works.

The biggest issue is that modules have special behavior with respect to .. In theory, this would just be another method of getfield, but the problem is that we need to resolve module references earlier since they basically behave like global variables. I think we will have to keep this a special case in the behavior of .. There is also a bit of a compiler efficiency concern, due to analyzing (# types) * (# fields) extra function definitions. But for that we will just see what happens.

@bfredl
Copy link
Contributor

@bfredl bfredl commented Jan 17, 2014

@JeffBezanson Do you also refer to const behavior in modules? It would be useful to have a user type emulating a module and be able to tell the compiler when the result of a dynamic field lookup is infact constant. (another approach would be to start with an actual module and be able to "trap" a failed jl_get_global and inject new bindings on demand)

I would find that to be very useful in combination with #5395. Then one be able to intercept a call to a undefined function or method MyMod.newfunction(new signature) and generate bindings to a (possibly large) API on demand. This would then be cached as usual const bindings I guess.

@cdsousa
Copy link
Contributor

@cdsousa cdsousa commented Jan 30, 2014

Let me, a simple Julia newbie, present a little concern: I think the possibility to overload the dot operator might imply that field access "purity" is somehow lost.

The user would generally lose the knowledge if doing a.b is just an access to a reference/value or if there can be a huge function machinery being called behind. I'm not sure how that could be bad though, it is just a feeling...

On the other hand, I see that indeed this is a big wish for syntax sugar for many cases (PyCall, Dataframes...), which is perfectly understandable.
Maybe it is time for .. #2614?

@johnmyleswhite
Copy link
Member

@johnmyleswhite johnmyleswhite commented Jan 30, 2014

I support doing this.

But the purity does have something to say for it, even if one can use names(Foo) to figure out what the real components of Foo are.

The purity argument is closely related to the main practical concern I have, which is how one handles name conflicts when the fields of the type interfere with names you might hope to use. In DataFrames, I think we'd resolve this by banning the use of columns and colindex as column names, but wanted to know what people's plan was for this.

@cdsousa
Copy link
Contributor

@cdsousa cdsousa commented Jan 30, 2014

I guess getfield(x::MyType, ::Field{:foo}) = ... would have to be forbidden when MyType has a field foo, otherwise the access to the real field would be lost (or a way to force access to the field would have to be available).
But then getfield could only be defined for concrete types, since abstract ones know nothing about fields.

(Meanwhile, I stumbled upon this about C++.)

@JeffBezanson
Copy link
Member

@JeffBezanson JeffBezanson commented Jan 30, 2014

It's not a major problem. We can provide something like Core.getfield(x, :f) to force access to the real fields.

@cdsousa
Copy link
Contributor

@cdsousa cdsousa commented Jan 30, 2014

Ok, maybe I'm sold. But then defining a shortcut to Core.getfield(x, :f) (e.g., x..f) will be nice, otherwise internal code of types overloading the . for all symbols (dataframes, probably dictionaries) have to be crowded with Core.getfields.

@toivoh
Copy link
Contributor

@toivoh toivoh commented Jan 30, 2014

I'm not worried about the purity aspect - until we have this, the only code
that should be using field access at all is code that belongs to the
implementation of a given type. When field access is part of an api, you
have to document it, as with any api. I agree that it might be handy with
some shortcut syntax for core.getfield though, when writing those
implementations.

@cdsousa
Copy link
Contributor

@cdsousa cdsousa commented Jan 30, 2014

It had already been pointed out in #4935, but let's pull it to here: dot overloading can overlap a little with classical Julian multiple dispatch if not properly used, since we can start doing

getfield(x::MyType, ::Field{:size}) = .........
for i=1:y.size .....

instead of

size(x::MyType) = ..........
for i=1:size(y) ....

While the dot would be great to access items in collections (Dataframes, Dicts, PyObjects), it can somehow change the way object properties (not fields) are accessed.

@nalimilan
Copy link
Contributor

@nalimilan nalimilan commented Jan 30, 2014

I think one thing to consider is that if you can overload accessing field, you should also be able to overload setting a field. Else this will be inconsistent and frustrating. Are you OK to go that far?

@stevengj
Copy link
Member

@stevengj stevengj commented Jan 30, 2014

@nalimilan, one absolutely needs a setfield! in addition to getfield. (Similar to setindex! vs. getindex for []). I don't think this is controversial.

@johnmyleswhite
Copy link
Member

@johnmyleswhite johnmyleswhite commented Jan 30, 2014

Agree with @stevengj: DataFrames will definitely be implementing setfield! for columns.

@lindahua
Copy link
Contributor

@lindahua lindahua commented Jan 30, 2014

I support this.

Experience with other languages (e.g. C# and Python) does show that the dot syntax does have a lot of practical value. The way that it is implemented through specialized methods largely addresses the concern of performance regression.

It is, however, important to ensure that the inlineability of a method won't be seriously affected by this change. For example, something like f(x) = g(x.a) + h(x.b) won't become suddenly un-inlineable after this lands.

If we decide to make this happen, it is useful to also provide macros to make the definition of property easier, which might look like:

# let A be a type, and foo a property name
@property (a::A).foo = begin
    # compute the return the property value
end

# for simpler cases, this can be simplified to
@property (a::A).foo2 = (2 * a.foo)

# set property 
@setproperty (a::A).foo v::V begin
    # codes for setting value v to a property a.foo
end

Behind the scene, all these can be translated to the method definitions.

vtjnash added a commit that referenced this issue Dec 7, 2017
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974
@vtjnash vtjnash mentioned this issue Dec 7, 2017
2 of 2 tasks complete
vtjnash added a commit that referenced this issue Dec 7, 2017
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974
vtjnash added a commit that referenced this issue Dec 11, 2017
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974
vtjnash added a commit that referenced this issue Dec 15, 2017
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974
vtjnash added a commit that referenced this issue Dec 17, 2017
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974
vtjnash added a commit that referenced this issue Dec 18, 2017
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.p` and `x.p = v`,
respectively.

This forces inference constant propagation through get/setproperty,
since it is very likely this method will yield better information after specializing on the field name
(even if `convert` is too big to make us want to inline the generic version and trigger the heuristic normally).

closes #16195 (and thus also closes #16226)
fix #1974
@Liso77
Copy link

@Liso77 Liso77 commented Dec 25, 2017

Pls change milestone to 1.0! :)

@nalimilan
Copy link
Contributor

@nalimilan nalimilan commented Dec 25, 2017

@Liso77 What do you mean? This is already implemented on git master, as noted in the status when closing.

@Liso77
Copy link

@Liso77 Liso77 commented Dec 25, 2017

@nalimilan sorry if I understand it wrong! But I thought that as 1.x are labeled postponed things which are going to be solved after 1.0. And this is solved now...

@ChrisRackauckas
Copy link
Contributor

@ChrisRackauckas ChrisRackauckas commented Dec 26, 2017

Open source is a decentralized community. The milestone sets what should be complete by 1.0, but contributors and work on whatever they want. In this case someone wanted this in 1.0, so they contributed the code to get it there.

@kdheepak
Copy link
Contributor

@kdheepak kdheepak commented Dec 26, 2017

@Liso77 As I understand it, this will not be in v1.0. The Julia v1.0 feature freeze date was set to Dec 15th, but this issue was closed on Dec 17th so I think we can expect it in a 1.x release. Core devs can correct me if my interpretation is incorrect.

@tomaklutfu
Copy link
Contributor

@tomaklutfu tomaklutfu commented Dec 26, 2017

No, it is merged into master and will be in the next release.

@Liso77
Copy link

@Liso77 Liso77 commented Dec 26, 2017

:) Well! I just thought that it is good to label 1.0 what is going out in 1.0. If it is not wanted then sorry for disturbing! :)

@KristofferC
Copy link
Contributor

@KristofferC KristofferC commented Dec 26, 2017

I think the NEWS file is a better way to see what is going into 1.0.

@nalimilan nalimilan removed this from the 1.x milestone Dec 26, 2017
@nalimilan
Copy link
Contributor

@nalimilan nalimilan commented Dec 26, 2017

The milestone was added to mean "can be implemented without breaking compatibility in the 1.x release series", but then since the code was ready it's been merged anyway before the 1.0 feature freeze. I've removed the milestone for clarity, but at this point everything which is merged in master will be in 1.0.

@kdheepak
Copy link
Contributor

@kdheepak kdheepak commented Dec 26, 2017

Thanks for the clarification! This is exciting! The NEWS file was also particularly enlightening.

@Liso77
Copy link

@Liso77 Liso77 commented Dec 26, 2017

Thanks for remove! I am also very glad it will come in 1.0! :)

@oschulz
Copy link

@oschulz oschulz commented Dec 28, 2017

I wonder if there's a way to support these new "dynamically-defined fields" in auto-complete, e.g. by allowing to overload fieldnames?. This could be very powerful, for interactive use, e.g. when dealing with DataFrames (assuming they'll will support df.column_name in the future) with many columns and/or long column names.

I guess at the moment the REPL (tab-expansion), IJulia, etc. look at the type definition, not the instance? But maybe that could be changed, for interactive use. It's probably impossible to support in an IDE like Juno, though, as instances are not available during coding.

@yurivish
Copy link
Contributor

@yurivish yurivish commented Dec 28, 2017

@oschulz, you can already overload fieldnames:

julia> struct Foo; foo; end

julia> fieldnames(Foo)
1-element Array{Symbol,1}:
 :foo

julia> Base.fieldnames(::Type{Foo}) = [:bar, :baz]

julia> fieldnames(Foo)
2-element Array{Symbol,1}:
 :bar
 :baz

And it looks like fieldnames is what the REPL looks at:

julia> x = Foo(3)
Foo(3)

julia> x.ba<tab>
bar baz
@oschulz
Copy link

@oschulz oschulz commented Dec 28, 2017

@yurivish right - but is it "safe" to do so, currently? I'm not sure what else relies on fieldnames.

@KristofferC
Copy link
Contributor

@KristofferC KristofferC commented Dec 28, 2017

If not safe, it should be possible to define a complete_fieldnames(x) = fieldnames(x), use complete_fieldnames for the REPL completions, and overload that for custom completions.

@oschulz
Copy link

@oschulz oschulz commented Dec 28, 2017

Yes, this is why I was bringing this up - in case something needs to be changed/added in Base, so that REPL and friends can take advantage of it later. In view of the 0.7 feature freeze ...

@JeffBezanson
Copy link
Member

@JeffBezanson JeffBezanson commented Dec 28, 2017

This is why I'm glad we called the function getproperty; it helps clarify that it doesn't refer to the same thing as fieldnames. Interfering with fieldnames could definitely cause serious problems.

@StefanKarpinski
Copy link
Member Author

@StefanKarpinski StefanKarpinski commented Dec 28, 2017

We might need a corresponding propertynames to give the valid property values. Of course, that may not be a well-defined concept, so perhaps we want something more specific to completion.

@oschulz
Copy link

@oschulz oschulz commented Dec 28, 2017

I had a feeling that this may require a bit of deeper analysis and input from the experts (thanks!) - should we open a new issue for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.