Skip to content
This repository

keyword arguments #485

Closed
multinormal opened this Issue February 28, 2012 · 60 comments
Chris Rose

Unless I've missed this in the manual, Julia does not currently have named parameters. I have found this language feature to dramatically improve the readability, correctness, and maintainability of complex mathematical software.

I would like the following features (mostly common to R, and hacked onto Matlab using their string-value pairing convention):

  • The ability, when defining a function, to specify mandatory and optional parameters.
  • The ability, when defining a function, to specify default values for optional parameters (including the ability to specify that a certain function be called to obtain the default value when the function is called).
  • The ability, when calling a function, to use either parameter names (verbose, but clear) or parameter position (terse, but unclear).
  • The ability, when calling a function, to place named parameters in any order.

If I had to call it, I'd say Objective-C's requirement that parameter names be used is better than also allowing parameter position, but I appreciate this looks weird to users of some languages (particularly Matlab) and is perhaps too verbose in many cases (and may discourage use of anonymous functions).

I appreciate that Julia encourages defining multiple functions of the same name but of different types to solve the optional-parameters-with-default-values problem, but I suggest that providing syntactical sugar that allows a function to be defined in one place, with default values for parameters, but which is equivalent to defining multiple functions of different type, may be a good solution. Novice programmers may be confused by the ability to define multiple versions of the same function, and poor programmers are going to spread multiple definitions of a given function all over the place.

Stefan Karpinski

This is actually something we've discussed a lot and wanted from the very beginning. It just hasn't been implemented yet. The basic outline of how keyword parameters are likely to work:

  • Each paramater is either positional or given by a keyword, not both.
  • No dispatch is done on keyword arguments.
  • Keyword arguments always have default values.
  • Positional parameters don't have default values, but you can use multiple dispatch to simulate defaults.

I know that kind of disagrees with a lot of your feature requests. The main consideration here is making keyword arguments not completely impossibly complicated to understand in the presence of multiple dispatch. For what it's worth, I initially envisioned this very much like your description — brimming with features and possibilities. However, I think that the above bare bones feature set will ensure clarity and simplicity as much as possible. It's much easier to disentangle scattered method definitions than it is to figure out the interactions between two features like keyword arguments and multiple dispatch, unless their potential for complex interactions is kept at an absolute minimum.

Regarding default values for positional parameters, I was initially for that, but I found that using multiple dispatch to express these things is both clearer and more powerful. For example, Regex constructors are defined like this:

type Regex
    pattern::ByteString
    options::Int32
    regex::Array{Uint8}
    extra::Ptr{Void}

    function Regex(pat::String, opts::Integer, study::Bool)
        pat = cstring(pat); opts = int32(opts)
        if (opts & ~PCRE_OPTIONS_MASK) != 0
            error("invalid regex option(s)")
        end
        re = pcre_compile(pat, opts & PCRE_COMPILE_MASK)
        ex = study ? pcre_study(re) : C_NULL
        new(pat, opts, re, ex)
    end
end
Regex(p::String, s::Bool)    = Regex(p, 0, s)
Regex(p::String, o::Integer) = Regex(p, o, false)
Regex(p::String)             = Regex(p, 0, false)

If you only supply one additional argument, it can either be a boolean, indicating whether to study the regex pattern or not, or it can be an integer, providing regex options. I don't think this could possibly be any clearer with parameter defaults, and it would introduce a lot of new syntax, which we'd like to avoid. In this particular case, use of default values would also obliterate the clean separation between inner and outer constructor methods.

Chris Rose

I'm really pleased to hear that keyword arguments are on the agenda. I'm not suitably qualified to comment on your design, but from the point of view of someone who has used enough languages, written a lot of numerical code in various languages, and had to help PhD students and postdocs fix their code, I can say that keywords can be really useful.

Looking again at Julia's type definition and implicit constructor definition, I worry that seemingly innocuous changes to the type definition (again, by inexperienced programmers) will change the meaning of code elsewhere; I think there's a case to be made for using keywords in constructors. Consider the following:

type Foo
  bar
  baz
end

function f(foo)
  foo.bar / foo.baz
end

f(Foo(1, 2))

Now imagine the code is edited by a new PhD student, who has done little programming before, but needs to write some code to solve a research question in time to meet a tight conference deadline. For whatever reason, they change the definition of Foo to be:

type Foo
  baz
  bar
end

Now, the function f computes baz/bar rather than bar/baz, because they didn't understand the language.

However, if you mandate the use of keyword arguments when calling the implicitly-defined constructors, the code could look like this:

type Foo
  bar
  baz
end

function f(foo)
  foo.bar / foo.baz
end

f(Foo(bar=1, baz=2))

… and the code will be protected from the new PhD student's attempt to “clean up” the definition of Foo.

I acknowledge that in designing a language you are also picking an audience. I'm really excited that you're building a language that looks very much like what I've wanted for over ten years now.

(As an aside, I'd be interested to know the syntax you have in mind for associating keywords with their values. I quite like Mathematica's ->; the PhD students often confuse = and ==, so I'd be against overloading the equals sign further.)

Harlan Harris

Stefan, one concern I have with using multiple dispatch, as you suggest, is that many of the statistical functions I'm familiar with in R, as well as a fair number of functions in Matlab, can have very long lists of options. Like, a dozen or more optional arguments. Requiring the author to write all possible subsets is prohibitive. This said, I understand (sorta) that/how Julia's design makes an R-like implementation of optional named arguments very difficult. What if, instead, there was a standardized convention and data structure to pass things that are options (rather than parameters) in a single generally-last argument? Something like:

function model(x, y, z, o:Options)
  o = Options("foo"=7, "bar"="hi mom", o) # later arguments replace earlier ones
  blahblahblah
end

It'd be better if the options could be bare words instead of strings. Ah, maybe a HashTable? Options({:foo => 7, :bar => "hi mom"})?

Jeff Bezanson
Owner

Yes, using multiple dispatch only works for optional positional arguments. You only need n definitions for n optional arguments. But, syntactic sugar for this is entirely possible, such as:

f(x, y=0, z=1) = x+2y+3z

becomes

f(x,y,z) = x+2y+3z
f(x,y) = f(x,y,1)
f(x) = f(x,0,1)

Keyword arguments are a totally different thing. Those will effectively be passed in a separate stack-allocated data structure. I don't recall the details of R's named arguments, but I suspect they don't matter. My thinking is that it will probably be confusing and/or expensive to dispatch based on keyword arguments, so they will be extra "out of band" info that gets passed along. We will eventually be able to do the full deal, including capturing and delegating keywords to other functions, etc.

Harlan Harris

I see. Yeah, the other trick is that if you only want to pass in an optional 14th argument, you either have to specify the first 13, or you have to pass in a bunch of commas, like Matlab: f(a, b, , , , , , , , , , 12). And who can tell what that means?

I really like the idea of "out of band" nature of keyword arguments that don't interact with the dispatch mechanism! Sounds like an elegant solution. Would an Options object implementation work, perhaps with some syntactic sugar, or would you want to make it completely separate from the argument list?

Jeff Bezanson
Owner

Of course, you can pass an options object of your own design at any point. We can provide an interface for "splicing" options like we do for positional arguments now: in f(x...), x can be anything iterable and its contents will be used as the arguments. In theory then we could allow, say, a HashTable to supply keyword arguments. The only benefit would be that the system could "destructure" the options into variables for you.

Stefan Karpinski

There's one other benefit: checking that you only passed valid named parameters. In Ruby, where keyword arguments just become a hash passed into a method, it's a massive pain to check that someone didn't use an invalid keyword.

If you have a function call with 14 non-varargs positional arguments, you're doing it wrong — you should be using keyword arguments. I think the bigest concern I have about keyword arguments is delegation: you have a bunch of methods for the same function and they all end up calling the same core method but all of them can take the same keyword arguments. You don't want to have to list and pass all the keywords every time. So we need a construct that allows delegation of keyword arguments to another method. But it can be more complicated than that: a method that calls a more core method allows an additional keyword argument that it then handles itself. One particularly nasty case would be that a single method takes all the keywords of two different methods that it calls and wants to delegate some to one and some to the other. I think that may be a case where explicit delegation is required. The explicit delegation could happen all in once place, at least and then additional methods that call that core method would just delegate the pooled keywords en masse.

Samuel John

I'd appreciate keyword args very much. One of the things really done right in Python :-)
Looking forward to this in Julia.

Tom Short
Collaborator
tshort commented July 12, 2012

This topic has been quiet for a while. Here is a proposal for a fairly easy way to add keyword arguments.

Proposal: pass keyword arguments as a tuple with the first element being a symbol representing the left-hand side, and the second element being the evaluated right-hand side. The following:

myfun(a = [1,2], b = "qwerty")

becomes:

myfun((:a, [1,2]), (:b, "qwerty"))

The most common way this would be used is with varargs:

function myfun(args...)
    # Do stuff with args. 
    ...
end

This does leave it up to the function author to manage and convert args to something useful. I think this could be handled much like Tim Holy's options.jl currently does.

function simplefun(x, opts...)
    @defaults opts a=3 b=2 c=2*b
    println(a)
    println(b)
    println(c)
    anotherfun(opts...)
end

Tim's @defaults macro could be adapted to also handle a tuple of tuples. Delegation would be handled by passing "opts..." as shown above.

The function author could handle things differently. For R-style DataFrame creation, this might be more appropriate:

function DataFrame(args::Tuple...)
    d = DataFrame()   # blank DataFrame
    for (k, v) in args
        d[string(k)] = v
    end
    d
end

Then, a DataFrame could be created as:

DataFrame(a = shuffle([1:5]), 
          b = randn(5))

The advantages of this approach are:

  • I don't think it's a big change to Julia's parser.

  • There's a pretty clear separation of positional and keyword arguments.

  • It's easy to delegate keyword arguments.

Jeff Bezanson
Owner

That is a clever approach. We could even modify the parser to insert the equivalent of the @defaults expression for you when the function signature contains keyword syntax.

But, there are a few disadvantages. It becomes tricky to have functions with both keyword args and ordinary varargs. This is also a leaky abstraction; one cannot distinguish keyword args from ordinary arguments that happen to be tuples with symbols. There is also less opportunity to optimize calls since the compiler doesn't know what's really going on. It would be better to avoid allocating tuples; we could alternate symbols and values, or put all keyword args in one tuple. Maybe having a special KeywordArg type would help, since then at least you could separate keyword args from other varargs.

Tom Short
Collaborator
tshort commented July 14, 2012

Good points. Let's consider the following function:

myfun(x1, x2, args...) = ...
# now call it
myfun(x1, x2, col = "red", y1, (a, b))

The positional arguments are straightforward. It would be more difficult to separate out keyword argument from the non-keyword arguments after it, especially the last one that is a tuple. I think your idea of a KeywordArg type would help separate these and may help with allocation issues. The varargs could come through as:

(KeywordArg(:col, "red"), y1, (a, b))

I like that better than my tuple idea and better than alternating symbols and values (that could be leaky, too).

Samuel John

@tshort

myfun(x1, x2, kw1="red", x3)

...is not even allowed in Python.

SyntaxError: non-keyword arg after keyword arg

Tom Short
Collaborator

Interesting. That is allowed in R, but I'm fine with not allowing that.

Samuel John

@tshort though what is possible and nice in Python (but I am not saying we need this for julia):

def myfun(a,b):
    print a, b

myfun("hello", "world")
myfun(b="world", a="hello")

"helloworld"
"helloworld"

So by naming the "ordinary" args during a call, we have an extra safty. But at the expense of a dictionary (hash map) lookup, which is not what julia aims for.

Stefan Karpinski

@samueljohn, that is definitely a non-starter in the presence of multiple dispatch where different methods can have different parameter names.

Tom Short
Collaborator

@StefanKarpinski, something like

myfun(x1, x2, kw1="red", x3)

could work with multiple dispatch if myfun is defined using varargs, and keyword and plain arguments are passed to those:

function myfun(x1, x2, ops...) 
    # do something with ops to capture keyword and plain arguments
    ...
end
Steven G. Johnson
Collaborator

I needed keywords for my PyCall package, in order to call Python functions with keyword arguments. Thanks to Julia's macros, this was fairly painless: you just do func(arg1, arg2, ..., @pykw kw1=val1 kw2=val2 ...), and @pykw converts the keyword arguments into a dictionary that gets passed to Python.

When it comes to syntax, however, this brings up an important use case: the set of keywords may not be known in advance in dynamic settings. If Julia ever supports keyword arguments directly, it would be good to keep this possibility in mind.

Let me propose a possible syntax. A function is declared as

function foo(arg1::Type1, arg2, args...; kw1=default1, kw2::Type2=default2, kws...)
      ....
end

and called as

foo(argval1, argval2, argval3, argval4; kw1=val1, kw3=val3)

Multiple dispatch is done on positional arguments only, but typechecking is performed on keyword arguments if their type was specified. The identifiers arg1, arg2, kw1, and kw2 are defined as local variables inside foo, with the keyword arguments taking on their default values if they were not specified by the caller. If an args... positional argument is declared, then args contains an array of any additional values (here, [argval3, argval4]) that were passed before the ;, as usual for varargs functions. If a kws... argument is declared, then kws is a Dict{Symbol,Any} that contains any unrecognized keywords passed by the caller and their values (here, {:kw3 => val3}). If a kws... argument is not declared, then it is an error for the caller to pass unrecognized keywords (i.e., keywords not specified in the function declaration).

Jeff Bezanson

This formulation of keyword arguments sounds good. I think it is basically identical to what we've been thinking.

Stefan Karpinski

I'm still not entirely sure about the semicolons, although my experience with the sorting ABI has certainly led me to have a desire for default values to make it easier to express lots of methods more succinctly. That api would also hugely benefit from keywords. E.g. sort(words, by=lowercase) reads rather nicely. Another interesting data point in the design of keywords from the sorting API (and from DataFrames, if I recall correctly), is the fact that dispatch on types starts to look an awful lot like keyword arguments. I'm thinking of sort(words, Sort.By(lowercase)). Keyword arguments could be implemented by having each function-keyword pair have a type and having the appropriate methods to dispatch on that. I.e. f(a, b, k=val) would mean something like f(a, b, Kw_f_k(val)) where Kw_f_k is a type that just wraps a single value and allows f to dispatch appropriately. This is probably not a good actual implementation strategy, but it's worth thinking about how these relate.

Viral B. Shah
Owner

We should definitely get keyword arguments as a language feature sooner in the early releases, since they will lead to a lot of API churn in base, as well as in packages. As everyone knows, the later we do this, the more difficult and painful it will be.

Steven G. Johnson
Collaborator

@StefanKarpinski, you need semicolons or something similar to separate keyword arguments from positional arguments, because you have to disambiguate whether x=y is a keyword x argument or simply an expression assigning y to x and hence passing the value of y as a positional argument. Think especially of a function combining varargs with keywords, or keywords that are only known dynamically (as in my case).

Patrick O'Leary
Collaborator

or simply an expression assigning y to x

#590 made this a syntax error to reserve it for use in keyword arguments.

Mike Nolta
Collaborator

I've put together a very basic implementation of keyword args on the mn/kwargs branch to toy around with. Probably riddled w/ bugs, but the simple stuff appears to work. See f4a2427 for more details.

Steven G. Johnson
Collaborator

@pao, ah, I didn't realize that. Then there is no syntactic reason why keyword arguments can't just be arbitrarily mixed with positional arguments, with the latter interpreted as if the keyword arguments were not there. (Though would probably be more readable to have the keywords all at the end, maybe the language shouldn't enforce this stylistic preference.)

John Myles White
Collaborator

I kind of like the idea of enforcing the stylistic preference, but R does allow keyword arguments to occur anywhere and uses position only when keywords are absent.

Stefan Karpinski

I think mixing keyword and positional arguments is kind of horribly mess to read, so I'm inclined not to allow it. It's always easier to disallow and then later allow than to go in the other direction.

Tom Short
Collaborator

Has anyone else tried Mike's kwargs branch? Feature wise, it looks good to me.

https://github.com/JuliaLang/julia/tree/mn/kwargs

John Myles White
Collaborator

Definitely a really exciting start. Perhaps it's beyond what multiple dispatch can achieve, but any chance that we could name the keyword arguments in the list of methods instead of using ::Keywords? I think that would help the self-documentation process that Jeff recently set in motion.

toivoh
Collaborator

Keyword arguments would be really nice to have. But, being a huge fan of multiple dispatch in Julia, I wouldn't want to dismiss dispatch based on keyword arguments definitively just yet. I think that we will be in a much better position to decide upon that once we have accumulated some experience with using keyword arguments in Julia. Perhaps we will end up missing it.

So, I would suggest to implement keyword arguments that don't affect dispatch, but keeping an eye to the possibility of adding dispatch later. One way to keep it open: If we had dispatch on keyword args, it would be nullified if all methods would accept all keyword args (and then throw an error if they couldn't handle the given args). So if we were to add dispatch on keyword args later, it could be by adding a way to define methods that don't accept all keyword args.

Some other things that I think deserve to be considered:

  • When collecting catch-all keyword arguments, e.g. kws..., what should they be collected in? I'm not sure that a Dict is the best answer. It seems pretty inconsistent with varargs, which are collected in a tuple. Perhaps something like the "structurally-typed records" that Jeff talks about in #49 would be another possibility?
  • We should be careful with the syntax for splicing keyword arguments into a function call so that it doesn't collide with positional arguments syntax. I think that it would be very unfortunate if the kind of arguments spliced by f(kws...) would be dependent on the type of kws. Perhaps f(kws....) for slicing keyword args? (There's probably better syntaxes.)
Jeff Bezanson

The comment thread for mike's first commit on his branch (f4a2427) has become a hotbed of keyword args design and implementation. There are many good things about mike's basic approach of doing everything with front-end rewrites. I have my own version on jb/kwargs. So far I have managed to cram all the ugliness into one case (passing keywords to a function that doesn't accept any), which becomes very ugly. But if we can solve that we might be all set.

Chris Laws

Hey guys. I'm new to both julia and github.

Anyway, I started looking into julia, but I thought it needed keyword arguments before I started to use it.
And I couldn't really find anyone working on it at the time.
So I started tinkering a few days ago, mostly interested in how one might use macros to add keyword arguments.

For what it's worth, I've posted my module Keyword at https://github.com/claws1983/Keyword.jl
It is modeled after Rs S4 methods.

Be warned that this is the first (and only) julia code I have written.
But

  • it is pretty feature rich
  • under many cases should only have a runtime cost of object_id and == against a constant value (not sure how expensive an object_id call is).
  • I have several examples.

Take what you want from it (or say that it's not a good idea).
Either way, it was great seeing how one can use macros to really extend a language.
And I will surely be using julia in the future.

Glad to you you guys are working on this.
I'm busy (or should be busy) studying for a test, so I'm not going to make much more progress for a few months.

Chris Laws claws1983 referenced this issue from a commit February 28, 2013
Mike Nolta fast and slow keywords
Slow keywords are separated by '=>'. The function call

    f(a, b, c, k => "s", j => 5)

gets converted to

    f(Keywords(k=>s,j=>5), a, b, c)

That's it. Slow keywords are purely dynamic; they cannot be part of
method definitions.

Fast keywords are separated by '='. The method definition

    f(a, x=1, y=2) = ...

creates the following four definitions

    f(a) = f__x__y(a, 1, 2)
    f__x(a, x) = f__x__y(a, x, 2)
    f__y(a, y) = f__x__y(a, 1, y)
    f__x__y(a, x, y) = ...

Then the function call `f(y=9, a)` get dispatched to `f__y(a, 9)`.
ee32a01
Harlan Harris

@claws1983 , you should check out an earlier implementation like this: https://github.com/JuliaLang/Options.jl

Chris Laws

Thanks, I initially wanted to use that as the back-end for the dots\allow-other-keys portion. And simply provide for tighter integration. (An idea I had seen someone else suggest.) But I was having trouble figuring out how to use macros in macros. I think I know what I was doing wrong now and might look at that again at some point.

Options.jl doesn't really solve the problem of dispatch-able arguments, especially if we are talking about ones with default values.

For the latter, suppose my function calls draw which
1) dispatches based on where it should "draw" and
2) has a default for where it should "draw".
In my function, I want to use the default but allow the user to override this. Keyword.jl let's me do this without knowing the details of draw's implementation. That is, I don't need to know what to draw to by default.

@def_generic draw(what_to_draw, where_to_draw_it=>new_plot_area())
@def_generic foo(x,_)
@def_method function foo(x,_) 
    ##do something to x
    @KC draw(what_to_draw=>x,_)
end

Now the user can do something like ...

@KC foo(x=>1) 

canvas1 = canvas()
@KC foo(x=>1, where_to_draw_it=>canvas1)

And foo never needed to know what the default for where_to_draw_it was. Which is good, since that information is part of draw. Now, if someone wants to change the default for draw, they are free to do so without breaking foo. And draw is free to dispatch on where_to_draw_it.

But it looks like julia will soon have a built-in keyword implementation.

P.S., I probably need support of positional and keyword invocation.

Jeff Bezanson
Owner

A big design issue here is whether we want to have keyword argument behavior officially defined in terms of existing calling behavior. For example, saying that f(x, y=1) is equivalent to f(Keywords(:y,1), x) or similar. Another way to look at this is defining the ABI for keywords; such a definition tells you how to make calls with keywords using the existing julia C API.

The advantages of this formulation are that it's easy to implement, and f(args...) = other(args...) still works for universal delegation. The disadvantages are that a function with no keyword arguments can get a Keywords object, leading to strange behavior instead of a simple "function does not take keywords" error. There is also confusion between passing keyword arguments, and intentionally passing around a Keywords object in order to manipulate it.

The alternative is to change the ABI some other way. We could expand the signature (jl_value_t *F, jl_value_t **args, uint32_t nargs) to (jl_value_t *F, jl_value_t **args, uint32_t nargs, uint32_t nkw), adding a keyword argument count. Then this would only be accessible within julia via special keyword syntax. This would need extensive changes to the system and I'd rather avoid that.

Still another option is to give each function a shadow function for handling keywords, so f(x, y=1) becomes something like f.kwfunc(x, :y, 1). This version is somewhere in between. It's less invasive, but you would still need to explicitly delegate keywords with f(a...; kw...) = other(a...; kw...).

Stefan Karpinski

As a design point to consider, I would point out how much trouble Ruby has ended up having with their cute approach to making keyword arguments just a hash object passed at the end. Of course, we can avoid some of that by having a special type, but the ambiguity between passing an object and really using keyword syntax worries me. If nothing else, it can be a bit of a security nightmare; of course, I think the solution there is to be much more careful with what kinds of magic one does with data from external sources, but it does concern me.

Steven G. Johnson
Collaborator

I tend to prefer the separation of positional vs. keyword arguments with a semicolon, i.e. f(x,y; z=3), requiring explicit delegation f(a...; kw...) = other(a...; kw...). This eliminates the ambiguity (and allows assignments to be used as positional arguments, and syntactic sugar for default arguments #1817, etcetera), at a very slight cost in verbosity.

Jeff Bezanson
Owner

Not requiring the semicolon in calls allows some nice syntax like dict(a=1,b=2).

toivoh
Collaborator

I would also really prefer not to let e.g. f(args...) accept keyword arguments. To me, the Julia-visible behavior of keyword arguments seems the most important to get right at this point, since it will be the hardest to change later. How to implement it under the covers, I will have to defer to others.

About syntax: I think that we would open up for way to much confusion with a syntax that makes one assignment in the call f(x=1; y=2) be an assignment, and the other specify a keyword argument. If we want to be able to do assignment in function calls, we should use a different syntax for passing in keyword arguments, like f(1, y=>2) (which would also allow to use assignments in calls, e.g. f(x=1, y=>(z=2))).

Stefan Karpinski

I really don't think assignment in function calls is common enough to block the traditional and nice keyword argument syntax and I wouldn't really be happy about having to write Dict(; a=1, b=2). Especially for collection constructors, there are a number of circumstances where I can see wanting to call the constructor with zero positional arguments and some keyword arguments (e.g. Dict(size=100)). On the method declaration side, I'm much more ok with using the semicolon to separate positional from keyword arguments, thereby opening up the possibility of default positional values and/or required keyword arguments (or dispatch on keyword presence/absence) in addition to keyword arguments with default values.

Steven G. Johnson
Collaborator

@StefanKarpinski, requiring the semicolon in method definitions but omitting it in function calls (and disallowing non-keyword assignments in function-call arguments) seems like a good compromise.

Presumably function delegation with keywords would then work as e.g. f(args...; kw...) = other(args..., kw....).

John Myles White
Collaborator

I also like the idea of semicolons going into methods definitions, but not function calls. I also like the idea of disallowing non-keyword assignment occurring inside function call arguments.

Jacob Quinn

So I'm trying to catch up on the discussion here and I've tried rewriting some of the ODBC package with keywords in mind (which I think would be helpful).

function connect(; dsn::String="",username::String="",password::String="")
    #function body  
end
#method definition for optional arguments
connect(datasource::String) = connect(dsn=datasource,username="",password="")
#call
co = connect(dsn="mydsn")
#would keywords be optional?
co = connect("mydsn")

function query(conn,querystring; output::Union(String,Array{String,1})="DataFrame",delim::Union(Char,Array{Char,1})=',')
    #function body
end
query(querystring::String) = query(conn,querystring,output="DataFrame",delim=',')

So I realize this is a little all over the place, but here are a few questions that don't seem to be answered yet:
1. How would you use a keyword argument without a default value? In my case, I'd like 'dsn' to be keyword argument, but not necessarily supply a default, since there is none really (and "" would probably end up returning an ambiguous error)
2. How would you designate a keyword as optional? Do we still have to rely on additional methods (as in my connect() method definition above)? Not that I'm against using additional methods, I'm actually a big fan and have felt my desire for keywords drastically reduced because they're such a handy way to provide defaults.
3. Would arguments be able to be positional AND keyword? e.g. in my connect() function, could I declare the main function with keywords, and additional methods without? relying on positions?

If someone has some example code somewhere that shows various uses of the proposed design, that'd probably help us all get a better/clearer idea of how the actual implementation would end up.

Sorry if I'm ignorant, but it seems there have been a few discussions on the subject in various locations and I've had a hard time tracking down all the ideas/details.

Stefan Karpinski

An optional keyword argument would have a default supplied; a non-optional one would have no default supplied. I'm against allowing the same argument to be supplied positionally or by keyword. In my experience it means there are more ways to call a function and two different call sites could be doing the exact same thing but look completely different and the only way I can know that is by knowing the functions signature by heart, which is rather annoying.

Jeff Bezanson
Owner

I'm glad people are starting to like the way I'm planning to implement it anyway ;)
Positional and keyword arguments will be disjoint; a given argument cannot be both. And all keyword arguments will be optional. If we really want it will be possible to add required keyword args later by allowing

function f(x; req_kw)
    ...

in definitions (a keyword arg after ; with no =).

I'm working on this now and it's rather complex when you use every feature:

function f(a, b=0, xs...; k=1, ks...)

Just think what we have to go through to call that function...

Jeff Bezanson
Owner

Ok, keyword arguments (and optional positional arguments) are now basically feature-complete on my branch. You can even pass keywords from an associative container, the same way any iterable can be used with ....

Stefan Karpinski

I say merge and let the glorious chaos ensue!

Tom Short
Collaborator
Alessandro Andrioni

Jeff, how keyword arguments will work with the block syntax? I'd love to use them in MPFR to allow things like:

with_mpfr do rounding=RoundToNearest, precision=256, trap_overflow=true
    do_stuff()
end
Jeff Bezanson
Owner

This would work fine:

with_mpfr(rounding=RoundToNearest, precision=256, trap_overflow=true) do
    do_stuff()
end

since do is just a call to with_mpfr, plus passing a function (the block) as the first argument.

Alessandro Andrioni

Oh, I see! This is really amazing, and I'd definitely love to have it in time for the 0.2 freeze!

John Myles White
Collaborator

Having this merged would be great.

Jeff Bezanson
Owner

Merged --- in the usual timeframe of slightly before it's totally ready.

Caveats:

  • The ability to sort keywords at compile time is not there yet, and will hugely improve performance in some cases.
  • The ASTs for these things are not obvious, and that might need to change (i.e. they are not just calls with assignments inside)
  • Debug info and stack traces are not quite perfect
  • It might be possible to dispatch based on whether any named arguments were passed, but this is not fully supported yet
  • Anonymous functions do not yet support optional or named arguments
  • Reflective things like which need to be fixed up
Jeff Bezanson JeffBezanson closed this April 03, 2013
John Myles White
Collaborator

Such great news. This was the one remaining weakness of the language for me.

Viral B. Shah
Owner

This is one of those epic moments!

Miles Lubin

Awesome

Stefan Karpinski

Yes, this is going to be a huge.

Tim Holy
Collaborator

Fantastic!

Steven Pav

Is there a TL;DR summary of how this now works?

thomasmcoffee

Is there a TL;DR summary of how this now works?

By trial and error, I have determined some rules for how this works beyond what is described in the docs. The following examples illustrate (as of Julia 0.2.0-2314.r91b975ce):

function f(a, b, c = 2, d = 3, e = 5, xs...; v = 7, w = 8, x = 10, ks...)
    ks = Dict(zip(ks...)...)
    y = ks[:y]
    z = ks[:z]
    tuple(a, b, c, d, e, xs..., v, w, x, y, z)
end

function test1()
    # returns (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
    f(1, a = 0, b = 0, z = 12, 2, (3, 4, 5, 6, 7)..., w = 9; {:y => 11, :v => 8}...)
end

function test2()
    # returns (1, 2, 3, 4, 5, 8, 9, 10, 11, 12)
    f((1, 2)..., (3, 4)..., z = 12; c = 0, e = 0, {:y => 11, :w => 9, :v => 0}..., w = 0, {:b => 0, :v => 8}..., y = 0)
end

Some rules for function defintion:

  • non-default single arguments cannot follow default arguments
  • non-default arguments cannot follow positional collections (xs...)
  • positional collections (xs...) cannot follow ;

Some rules for function calls:

  • positional arguments cannot follow ;
  • keyword-argument collections (ks...) must follow ;
  • positional argument assignments, including default values in function definition, take precedence over keyword assignments
  • later occurrences of keyword assignments take precedence over earlier ones, but all assignments in keyword-argument collections (ks...) are considered later than all single keyword-argument assignments

To me, this suggests a couple of questions:

  1. Should the implementation make keyword assignments available inside the function as an Associative type? Right now, the above example uses ks = Dict(zip(ks...)...), which loses the ordering, but allows random access --- I could not find any built-ins to provide random access to the Array that appears by default.

  2. Do the above rules about ordering and precedence in argument syntax make sense? They currently allow very confusing code, like the examples above, but also provide some useful features, for instance, overriding a subset of one collection of options by another.

It was also not apparent to me if there's a good reason why tuple(a, xs...) works, but (a, xs...) does not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.