-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create Base.Fix
as general Fix1
/Fix2
for partially-applied functions
#54653
Conversation
Questions:
|
Base.FixN
for general partially-applied functionsBase.Fix
for general partially-applied functions
Base.Fix
for general partially-applied functionsBase.Fix
as general Fix1
/Fix2
for partially-applied functions
@gooroodev can you review this? |
This comment was marked as spam.
This comment was marked as spam.
+1 for |
The implementation turns out to be pretty simple and everything is type stable and compiled. I do like it, it feels like this is what a general julia> sum_1 = Base.Fix(sum; dims=(1,))
(::Fix{typeof(sum), 0, Tuple{}, @NamedTuple{dims::Tuple{Int64}}}) (generic function with 1 method)
julia> sum_1(ones(3, 2))
1×2 Matrix{Float64}:
3.0 3.0
|
More general fixargs would indeed be nice to have! Are And just a minor comment regarding usecase examples: the closure stability example is easily resolved in current Julia as well, without any additions. Simply replace
Also see a short & performant implementation in https://github.com/JuliaAPlavin/AccessorsExtra.jl/blob/master/src/fixargs.jl. It takes a somewhat different approach compared to this PR: basically, stores a tuple of fixed That approach is more flexible in allowing to fix any arguments, eg 1, 2, and 4 – not just consecutive args, as this PR does. Basically the conceptual difference is between "fix this arg(s) and propagate others" (this PR) vs "propagate this arg and fix others" (AccessorsExtra.jl). Though the latter can be generalized to allow |
No, Re: closure example, yes I agree there are many ways to solve it. I just wanted to give it as an example because
Thanks! To summarize, that linked package uses: function (fa::FixArgs)(arg)
args = map(x -> x isa Placeholder ? arg : x, fa.args)
fa.f(args...; fa.kwargs...)
end The implementation on this PR roughly does, for function (f::Fix{F,N,T,K})(args::Vararg{Any,M}; kws...) where {F,N,T,K,M}
return f.f(args[begin:begin+(N-2)]..., f.x..., args[begin+(N-1):end]...; f.kws..., kws...)
end I should note I don't feel particularly attached to either. For the same scenario, which one is easier on the compiler? I would expect they compile to the exact same thing in either case, so does it even matter? And if the Placeholder approach, how would it look like if you had multiple With the I guess you could potentially do some syntax like |
I see... It just seems to be the most minimal straightforward extension of the current state.
I had in mind more like |
I see, thanks. I guess what you had in mind is semantically similar to #24990. Let's just wait for triage to give an opinion on all of this as I want to hear initial thoughts from them. |
I would opt for one of the proposals (of which there are several) that goes all the way and form a general closure type here. Further, it's not much more work to allow inputs to be re-ordered into arbitrary positions. We should use the name In terms of specifying the "open" positions, something like this (the name needs work): struct Closure{P, F, T} # require T<:Tuple?
f::F # function to be called
fixed::T # fixed arguments, to be inserted in positions not indicated in P
end
# pass positional arguments to positions 3 and 1
# fill the rest from the closed values
Closure{(3,1)}(foo, (12, 14))(13, 11) == foo(11, 12, 13, 14)
# still need a syntax to indicate where Vararg
# inputs would go, if we want that
# would they be passed as a tuple or splatted?
# backwards compatibility
Base.Fix1(f, arg1) = Closure{(2,)}(f, (arg1,))
Base.Fix2(f, arg2) = Closure{(1,)}(f, (arg2,))
# provide a macro syntax
Closure{(3,1)}(foo, (2, )) == @closure (x, y) -> foo(y, 2, x)
# the difference between a @closure anonymous function and
# a true anonymous function is that the @closure version would
# build an explicit struct
# this would block constant propagation
# but might avoid recompilation, just like Fix1/Fix2 |
I will state up front that I really really want to avoid making a macro for general anonymous functions, and spurring yet another 500-comment thread on currying syntax like #24990. I would like to keep this PR relegated to implementing a Above anything else, all I want is for Base.Fix(f, n, arg) to work. This is something I see as a much-needed feature. With that syntax alone you could already chain together a sequence of All of the other features I put into this PR are not necessary. Even the fixed vararg and kwargs are optional from my point of view (but still nice if people are on board). If triage only wants For more general closure types I think it's worth having a longer discussion elsewhere. |
Also, the indicated syntax cannot be type stable without I don't actually care whether there's a macro or not. That said, I don't think I think that the macro I proposed runs up against many of the ambiguities that tanked the curry proposals because the macro I proposed does not curry. It simply uses anonymous function syntax with the goal to create a |
Oh I see, sorry, I misinterpreted. If it's just the name I'd certainly be happy to rename it.
I see, what about if if Edit: Oops, I realized you said
I wondered this too but it turns out it's not an issue because the constructor is so simple! If It's the same reason writing out |
Hi @admsev, please do not use the JuliaLang/julia repo as a testing ground for developing your AI tool. |
Actually I slightly preferred the function as the first argument. While there isn't much reason to use this particular constructor with It's purely my aesthetic preference, but I still like the typevar syntax (a la |
Wait, I don't understand why you say that For example: x = 0.5
f = Fix(/, 1, x)
g = Fix1(/, x)
@test f(2.0) == g(2.0) |
Triage is happy with merging Some folks want more, some want to stop at The justifications based on "makes compilation faster" are sound for now, but it's not good to work around compiler limitations. Some folks (including Jeff and I) really prefer the minimal design because there are fewer arbitrary choices. |
Clarification:
|
All suggestions implemented. (Edit: ignore the other comments I posted, I temporarily forgot how |
Thanks for accommodating all the design changes and shifting the implementation as needed! I'm excited to have this in 1.12 :) |
Awesome, thanks! |
|
…tions (JuliaLang#54653) This PR generalises `Base.Fix1` and `Base.Fix2` to `Base.Fix{N}`, to allow fixing a single positional argument of a function. With this change, the implementation of these is simply ```julia const Fix1{F,T} = Fix{1,F,T} const Fix2{F,T} = Fix{2,F,T} ``` Along with the PR I also add a larger suite of unittests for all three of these functions to complement the existing tests for `Fix1`/`Fix2`. ### Context There are multiple motivations for this generalization. **By creating a more general `Fix{N}` type, there is no preferential treatment of certain types of functions:** - (i) No limitation that you can only fix positions 1-2. You can now fix any position `n`. - (ii) No asymmetry between 2-argument and n-argument functions. You can now fix an argument for functions with any number of arguments. Think of this like if `Base` only had `Vector{T}` and `Matrix{T}`, and you wished to generalise it to `Array{T,N}`. It is an analogous situation here: `Fix1` and `Fix2` are now *aliases* of `Fix{N}`. - **Convenience**: - `Base.Fix1` and `Base.Fix2` are useful shorthands for creating simple anonymous functions without compiling new functions. - They are common throughout the Julia ecosystem as a shorthand for filling arguments: - `Fix1` https://github.com/search?q=Base.Fix1+language%3Ajulia&type=code - `Fix2` https://github.com/search?q=Base.Fix2+language%3Ajulia&type=code - **Less Compilation**: - Using `Fix*` reduces the need for compilation of repeatedly-used anonymous functions (which can often trigger compilation of new functions). - **Type Stability**: - `Fix`, like `Fix1` and `Fix2`, captures variables in a struct, encouraging users to use a functional paradigm for closures, preventing any potential type instabilities from boxed variables within an anonymous function. - **Easier Functional Programming**: - Allows for a stronger functional programming paradigm by supporting partial functions with _any number of arguments_. Note that this refactors `Fix1` and `Fix2` to be equal to `Fix{1}` and `Fix{2}` respectively, rather than separate structs. This is backwards compatible. Also note that this does not constrain future generalisations of `Fix{n}` for multiple arguments. `Fix{1,F,T}` is the clear generalisation of `Fix1{F,T}`, so this isn't major new syntax choices. But in a future PR you could have, e.g., `Fix{(n1,n2)}` for multiple arguments, and it would still be backwards-compatible with this. --------- Co-authored-by: Dilum Aluthge <dilum@aluthge.com> Co-authored-by: Lilith Orion Hafner <lilithhafner@gmail.com> Co-authored-by: Alexander Plavin <alexander@plav.in> Co-authored-by: Neven Sajko <s@purelymail.com>
This PR generalises
Base.Fix1
andBase.Fix2
toBase.Fix{n}
, to allow fixing a single positional argument of a function.With this change, the implementation of these is simply
Along with the PR I also add a larger suite of unittests for all three of these functions to complement the existing tests for
Fix1
/Fix2
.Context
There are multiple motivations for this generalization.
By creating a more general
Fix{N}
type, there is no preferential treatment of certain types of functions:n
.No asymmetry between positional arguments and keyword arguments. You can now fix a keyword argument.Think of this like if
Base
only hadVector{T}
andMatrix{T}
, and you wished to generalise it toArray{T,N}
.It is an analogous situation here:
Fix1
andFix2
are now aliases ofFix{N}
.Base.Fix1
andBase.Fix2
are useful shorthands for creating simple anonymous functions without compiling new functions.Fix1
https://github.com/search?q=Base.Fix1+language%3Ajulia&type=codeFix2
https://github.com/search?q=Base.Fix2+language%3Ajulia&type=codeFix*
reduces the need for compilation of repeatedly-used anonymous functions (which can often trigger compilation of new functions).Fix
, likeFix1
andFix2
, captures variables in a struct, encouraging users to use a functional paradigm for closures, preventing any potential type instabilities from boxed variables within an anonymous function.Note that this refactors
Fix1
andFix2
to be equal toFix{1}
andFix{2}
respectively, rather than separate structs. This is backwards compatible.Also note that this does not constrain future generalisations of
Fix{n}
for multiple arguments.Fix{1,F,T}
is the clear generalisation ofFix1{F,T}
, so this isn't major new syntax choices. But in a future PR you could have, e.g.,Fix{(n1,n2)}
for multiple arguments, and it would still be backwards-compatible with this.Details
As the names suggest,
Fix1
andFix2
, they can only inject arguments at the first and second index. Furthermore, they are also constrained to work on 2-argument functions exclusively. It seems at various points there had been interest in extending this (see links below) but nobody had gotten around to it so far.This implementation of
Base.Fix
generalises the form as follows:With this more general type, I also rewrite
Fix1
andFix2
in this PR.With
Fix{n}
, the code which is executed is roughly as follows:This means that the captured variable
x
is inserted at theN
-th position. Keywords are also captured and inserted at the end.This adds several unittests, a docstring, as well as type stability checks, for which it seems to succeed.
I also run the new
Fix1
andFix2
test suites added by this PR on theFix
version of each struct.Examples
Simple examples:
To fix the argument
f
with an anonymous function:whereas now it becomes:
I have some usecases like this in SymbolicRegression.jl. I want to fix a single option in a function, and then repeatedly call that function throughout some loop with different input.
Fix1
andFix2
are not general enough for this as they only allow 2-argument functions.A more common use-case I have is to set
MIME"text/plain"
inprint
for tests, which can now be done asFix
is no longer limited to 2 arguments:without needing to re-compile at each instance.
In a reduction:
Fix1 and Fix2 are useful for short lambda functions like
to reduce compilation and often improve type stability.
With this new change, you aren't limited to only 2-arg functions, so you can use things like fma in this context:
where this will evaluate as x -> affine(x, 0.5, 2.0).
Another example is a mapreduce, where you would typically want to fix the map and reduction in applying:
In data processing pipelines:
Fix can be used to set any number of arguments, and of course also be chained together repeatedly as there is no restriction on 2-arg functions. It makes functional programming easier than with only Fix1 and Fix2.
For example, in a processing pipeline:
For dispatching on keyword-fixed functions:
Say that I would like to dispatch on
sum
for some type, ifdims
is set to an integer. You can do this as follows:which would result in any use of
Fix(sum; dims=1)
operating onVector{MyType}
to call this special function.Real-world examples
I would like to use this in my code. Here are some examples:
For example,
which could now be done with
map(Fix{2}(michaelis_menten, p), eachcol(X), t)
, reducing compilation costs, and avoiding any potential issues with capturing variables in the closure.Another one would be this:
with this you could write
Fix(round; digits=3)
and not need to re-compile an anonymous function for each new outer method.Features in other languages
Here are some of the most related features in other languages (all that I could find; there's probably more)
Groovy's
.ncurry
(Expand)
In Apache Groovy there is the
<function>.ncurry(index, args...)
to insert arguments at a given index. This syntax is semantically identical toBase.Fix
.From the documentation
Python's
functools.partial
(Expand)
In Python, there is no differentiating between args and kwargs – every function can be passed kwargs. Therefore,
functools.partial
is semantically similar toFix
:which would be equivalent to
Fix{2}(f, 2.0)
.C++'s
std::bind
(Expand)
In modern C++, one can use
std::bind
to insert placeholders. This is semantically closer to an anonymous function in Julia, though it binds the arguments in a way similar toFix1
andFix2
do:The C++ approach was also briefly mentioned on discourse back in 2018.
Closes:
Base.Fix1
andBase.Fix2
#36181Related issues:
Other approaches:
Base.Fix1
andBase.Fix2
#36181Base
. I have written theBase.Fix
in this PR from scratch based on the same patterns asFix1
andFix2
but with varargs.FixArgs
implementation in this package is much more closely related to this PR. It works in a similar way although though stores the full signature with aPlaceholder()
set to the replaced arg.Semi-related issues:
MadeFix
work forVararg
so that you can insert multiple arguments at the specified index.stoFix
.Fix(f, Val(1), arg)
syntax toFix{1}(f, arg)
.s, and rewroteFix1
andFix2
in terms ofFix
.Int64
n
Fix{n}
is exclusively for a single positional keyword argument.