New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow overloading of a.b field access syntax #1974

Closed
StefanKarpinski opened this Issue Jan 10, 2013 · 249 comments

Comments

@StefanKarpinski
Member

StefanKarpinski commented Jan 10, 2013

Brought up here: #1263.

@stevengj

This comment has been minimized.

Member

stevengj commented Jan 10, 2013

The ability to use dots as syntactic sugar for mutator/accessor methods would be nice for lots of things. I've always appreciated this in languages that provide it, so that you can turn structure fields into more complicated abstractions without breaking the API.

@toivoh

This comment has been minimized.

Member

toivoh commented Jan 10, 2013

+1

@JeffBezanson

This comment has been minimized.

Member

JeffBezanson commented May 6, 2013

I have an absolutely awesome way to implement this.

@johnmyleswhite

This comment has been minimized.

Member

johnmyleswhite commented May 6, 2013

Interested in talking about it? I know that Tom Short is really interested in having this for DataFrames, although I've come to be increasingly skeptical about the wisdom of using this feature.

@stevengj

This comment has been minimized.

Member

stevengj commented Oct 22, 2013

This would make calling Python code (via PyCall) significantly nicer, since currently I'm forced to do a[:b] instead of a.b.

@stevengj

This comment has been minimized.

Member

stevengj commented Jan 13, 2014

@JeffBezanson, any chance of having this for 0.3? Would be great for inter-language interop, both for PyCall and for JavaCall (cc @aviks).

@ihnorton

This comment has been minimized.

Member

ihnorton commented Jan 13, 2014

@JeffBezanson if not, is there any chance you could give some direction on how you want this implemented? (I have an absolutely awesome way to implement this.)

@StefanKarpinski

This comment has been minimized.

Member

StefanKarpinski commented Jan 13, 2014

In my experience, there is no faster nor surer way to get Jeff to implement something than to implement a version of it that he doesn't like ;-)

@JeffBezanson

This comment has been minimized.

Member

JeffBezanson commented Jan 14, 2014

The basic idea is that you implement

getfield(x::MyType, ::Field{:name}) = ...

so that you can overload it per-field. That allows access to "real" fields to keep working transparently. With suitable fallbacks getfield(::MyType, ::Symbol) also works.

The biggest issue is that modules have special behavior with respect to .. In theory, this would just be another method of getfield, but the problem is that we need to resolve module references earlier since they basically behave like global variables. I think we will have to keep this a special case in the behavior of .. There is also a bit of a compiler efficiency concern, due to analyzing (# types) * (# fields) extra function definitions. But for that we will just see what happens.

@bfredl

This comment has been minimized.

Contributor

bfredl commented Jan 17, 2014

@JeffBezanson Do you also refer to const behavior in modules? It would be useful to have a user type emulating a module and be able to tell the compiler when the result of a dynamic field lookup is infact constant. (another approach would be to start with an actual module and be able to "trap" a failed jl_get_global and inject new bindings on demand)

I would find that to be very useful in combination with #5395. Then one be able to intercept a call to a undefined function or method MyMod.newfunction(new signature) and generate bindings to a (possibly large) API on demand. This would then be cached as usual const bindings I guess.

@cdsousa

This comment has been minimized.

Contributor

cdsousa commented Jan 30, 2014

Let me, a simple Julia newbie, present a little concern: I think the possibility to overload the dot operator might imply that field access "purity" is somehow lost.

The user would generally lose the knowledge if doing a.b is just an access to a reference/value or if there can be a huge function machinery being called behind. I'm not sure how that could be bad though, it is just a feeling...

On the other hand, I see that indeed this is a big wish for syntax sugar for many cases (PyCall, Dataframes...), which is perfectly understandable.
Maybe it is time for .. #2614?

@johnmyleswhite

This comment has been minimized.

Member

johnmyleswhite commented Jan 30, 2014

I support doing this.

But the purity does have something to say for it, even if one can use names(Foo) to figure out what the real components of Foo are.

The purity argument is closely related to the main practical concern I have, which is how one handles name conflicts when the fields of the type interfere with names you might hope to use. In DataFrames, I think we'd resolve this by banning the use of columns and colindex as column names, but wanted to know what people's plan was for this.

@cdsousa

This comment has been minimized.

Contributor

cdsousa commented Jan 30, 2014

I guess getfield(x::MyType, ::Field{:foo}) = ... would have to be forbidden when MyType has a field foo, otherwise the access to the real field would be lost (or a way to force access to the field would have to be available).
But then getfield could only be defined for concrete types, since abstract ones know nothing about fields.

(Meanwhile, I stumbled upon this about C++.)

@JeffBezanson

This comment has been minimized.

Member

JeffBezanson commented Jan 30, 2014

It's not a major problem. We can provide something like Core.getfield(x, :f) to force access to the real fields.

@cdsousa

This comment has been minimized.

Contributor

cdsousa commented Jan 30, 2014

Ok, maybe I'm sold. But then defining a shortcut to Core.getfield(x, :f) (e.g., x..f) will be nice, otherwise internal code of types overloading the . for all symbols (dataframes, probably dictionaries) have to be crowded with Core.getfields.

@toivoh

This comment has been minimized.

Member

toivoh commented Jan 30, 2014

I'm not worried about the purity aspect - until we have this, the only code
that should be using field access at all is code that belongs to the
implementation of a given type. When field access is part of an api, you
have to document it, as with any api. I agree that it might be handy with
some shortcut syntax for core.getfield though, when writing those
implementations.

@cdsousa

This comment has been minimized.

Contributor

cdsousa commented Jan 30, 2014

It had already been pointed out in #4935, but let's pull it to here: dot overloading can overlap a little with classical Julian multiple dispatch if not properly used, since we can start doing

getfield(x::MyType, ::Field{:size}) = .........
for i=1:y.size .....

instead of

size(x::MyType) = ..........
for i=1:size(y) ....

While the dot would be great to access items in collections (Dataframes, Dicts, PyObjects), it can somehow change the way object properties (not fields) are accessed.

@nalimilan

This comment has been minimized.

Contributor

nalimilan commented Jan 30, 2014

I think one thing to consider is that if you can overload accessing field, you should also be able to overload setting a field. Else this will be inconsistent and frustrating. Are you OK to go that far?

@stevengj

This comment has been minimized.

Member

stevengj commented Jan 30, 2014

@nalimilan, one absolutely needs a setfield! in addition to getfield. (Similar to setindex! vs. getindex for []). I don't think this is controversial.

@johnmyleswhite

This comment has been minimized.

Member

johnmyleswhite commented Jan 30, 2014

Agree with @stevengj: DataFrames will definitely be implementing setfield! for columns.

@lindahua

This comment has been minimized.

Member

lindahua commented Jan 30, 2014

I support this.

Experience with other languages (e.g. C# and Python) does show that the dot syntax does have a lot of practical value. The way that it is implemented through specialized methods largely addresses the concern of performance regression.

It is, however, important to ensure that the inlineability of a method won't be seriously affected by this change. For example, something like f(x) = g(x.a) + h(x.b) won't become suddenly un-inlineable after this lands.

If we decide to make this happen, it is useful to also provide macros to make the definition of property easier, which might look like:

# let A be a type, and foo a property name
@property (a::A).foo = begin
    # compute the return the property value
end

# for simpler cases, this can be simplified to
@property (a::A).foo2 = (2 * a.foo)

# set property 
@setproperty (a::A).foo v::V begin
    # codes for setting value v to a property a.foo
end

Behind the scene, all these can be translated to the method definitions.

vtjnash added a commit that referenced this issue Dec 7, 2017

implement getfield overloading
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974

@vtjnash vtjnash referenced this issue Dec 7, 2017

Merged

implement getfield overloading #24960

2 of 2 tasks complete

vtjnash added a commit that referenced this issue Dec 7, 2017

implement getfield overloading
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974

vtjnash added a commit that referenced this issue Dec 11, 2017

implement getfield overloading
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974

vtjnash added a commit that referenced this issue Dec 15, 2017

implement getfield overloading
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974

vtjnash added a commit that referenced this issue Dec 17, 2017

implement getfield overloading
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.f` and `x.f = v`,
respectively.

fix #16226 (close #16195)
fix #1974

vtjnash added a commit that referenced this issue Dec 18, 2017

implement getfield overloading (#24960)
New functions `Base.getproperty` and `Base.setproperty!`
can be overloaded to change the behavior of `x.p` and `x.p = v`,
respectively.

This forces inference constant propagation through get/setproperty,
since it is very likely this method will yield better information after specializing on the field name
(even if `convert` is too big to make us want to inline the generic version and trigger the heuristic normally).

closes #16195 (and thus also closes #16226)
fix #1974
@Liso77

This comment has been minimized.

Liso77 commented Dec 25, 2017

Pls change milestone to 1.0! :)

@nalimilan

This comment has been minimized.

Contributor

nalimilan commented Dec 25, 2017

@Liso77 What do you mean? This is already implemented on git master, as noted in the status when closing.

@Liso77

This comment has been minimized.

Liso77 commented Dec 25, 2017

@nalimilan sorry if I understand it wrong! But I thought that as 1.x are labeled postponed things which are going to be solved after 1.0. And this is solved now...

@ChrisRackauckas

This comment has been minimized.

Contributor

ChrisRackauckas commented Dec 26, 2017

Open source is a decentralized community. The milestone sets what should be complete by 1.0, but contributors and work on whatever they want. In this case someone wanted this in 1.0, so they contributed the code to get it there.

@kdheepak

This comment has been minimized.

Contributor

kdheepak commented Dec 26, 2017

@Liso77 As I understand it, this will not be in v1.0. The Julia v1.0 feature freeze date was set to Dec 15th, but this issue was closed on Dec 17th so I think we can expect it in a 1.x release. Core devs can correct me if my interpretation is incorrect.

@tomaklutfu

This comment has been minimized.

tomaklutfu commented Dec 26, 2017

No, it is merged into master and will be in the next release.

@Liso77

This comment has been minimized.

Liso77 commented Dec 26, 2017

:) Well! I just thought that it is good to label 1.0 what is going out in 1.0. If it is not wanted then sorry for disturbing! :)

@KristofferC

This comment has been minimized.

Contributor

KristofferC commented Dec 26, 2017

I think the NEWS file is a better way to see what is going into 1.0.

@nalimilan nalimilan removed this from the 1.x milestone Dec 26, 2017

@nalimilan

This comment has been minimized.

Contributor

nalimilan commented Dec 26, 2017

The milestone was added to mean "can be implemented without breaking compatibility in the 1.x release series", but then since the code was ready it's been merged anyway before the 1.0 feature freeze. I've removed the milestone for clarity, but at this point everything which is merged in master will be in 1.0.

@kdheepak

This comment has been minimized.

Contributor

kdheepak commented Dec 26, 2017

Thanks for the clarification! This is exciting! The NEWS file was also particularly enlightening.

@Liso77

This comment has been minimized.

Liso77 commented Dec 26, 2017

Thanks for remove! I am also very glad it will come in 1.0! :)

@oschulz

This comment has been minimized.

oschulz commented Dec 28, 2017

I wonder if there's a way to support these new "dynamically-defined fields" in auto-complete, e.g. by allowing to overload fieldnames?. This could be very powerful, for interactive use, e.g. when dealing with DataFrames (assuming they'll will support df.column_name in the future) with many columns and/or long column names.

I guess at the moment the REPL (tab-expansion), IJulia, etc. look at the type definition, not the instance? But maybe that could be changed, for interactive use. It's probably impossible to support in an IDE like Juno, though, as instances are not available during coding.

@yurivish

This comment has been minimized.

Contributor

yurivish commented Dec 28, 2017

@oschulz, you can already overload fieldnames:

julia> struct Foo; foo; end

julia> fieldnames(Foo)
1-element Array{Symbol,1}:
 :foo

julia> Base.fieldnames(::Type{Foo}) = [:bar, :baz]

julia> fieldnames(Foo)
2-element Array{Symbol,1}:
 :bar
 :baz

And it looks like fieldnames is what the REPL looks at:

julia> x = Foo(3)
Foo(3)

julia> x.ba<tab>
bar baz
@oschulz

This comment has been minimized.

oschulz commented Dec 28, 2017

@yurivish right - but is it "safe" to do so, currently? I'm not sure what else relies on fieldnames.

@KristofferC

This comment has been minimized.

Contributor

KristofferC commented Dec 28, 2017

If not safe, it should be possible to define a complete_fieldnames(x) = fieldnames(x), use complete_fieldnames for the REPL completions, and overload that for custom completions.

@oschulz

This comment has been minimized.

oschulz commented Dec 28, 2017

Yes, this is why I was bringing this up - in case something needs to be changed/added in Base, so that REPL and friends can take advantage of it later. In view of the 0.7 feature freeze ...

@JeffBezanson

This comment has been minimized.

Member

JeffBezanson commented Dec 28, 2017

This is why I'm glad we called the function getproperty; it helps clarify that it doesn't refer to the same thing as fieldnames. Interfering with fieldnames could definitely cause serious problems.

@StefanKarpinski

This comment has been minimized.

Member

StefanKarpinski commented Dec 28, 2017

We might need a corresponding propertynames to give the valid property values. Of course, that may not be a well-defined concept, so perhaps we want something more specific to completion.

@oschulz

This comment has been minimized.

oschulz commented Dec 28, 2017

I had a feeling that this may require a bit of deeper analysis and input from the experts (thanks!) - should we open a new issue for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment