-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Factor out bnv derivs #137
Conversation
Have/will you re-run the benchmark script to make sure the changes haven't slowed down the objective function evaluations? |
the_mean::Vector{T1}, the_cov::Matrix{T2}, weight::T3; | ||
calculate_siginv_deriv::Bool=true) | ||
calculate_siginv_deriv::Bool=true) = begin |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's use function
syntax everywhere for functions rather than begin/end blocks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed. Sorry about this, I just missed this in the merge.
On Tue, Mar 8, 2016 at 10:39 AM, Kyle Barbary notifications@github.com
wrote:
In src/bivariate_normals.jl
#137 (comment):the_mean::Vector{T1}, the_cov::Matrix{T2}, weight::T3;
calculate_siginv_deriv::Bool=true)
calculate_siginv_deriv::Bool=true) = begin
Let's use function syntax everywhere for functions rather than begin/end
blocks.—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/pull/137/files#r55405316.
The file |
""" -> | ||
type SensitiveFloat{ParamType <: CelesteTypes.ParamSet, NumType <: Number} | ||
# Actually a single value, but an Array to avoid memory allocation | ||
v::Vector{NumType} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While we're looking at SensitiveFloat, I wanted to ask why v
is an array instead of just v::Numtype
here. The effect of using an array is opposite of what the comment implies.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I found when looking at memory allocation for the full Hessian that memory
was allocated each time the value of a SensitiveFloat was changed and (in
some cases at least) even accessed. I don't know enough about the julia
compiler to account for this, but when I switched to using an array for the
value the memory allocation disappeared. So when I say "to avoid memory
allocation" I mean "to avoid memory allocation whenever the value is used
or changed" rather than to avoid memory allocation up front.
On Tue, Mar 8, 2016 at 10:51 AM, Kyle Barbary notifications@github.com
wrote:
In src/SensitiveFloat.jl
#137 (comment):+A function value and its derivative with respect to its arguments.
+
+Attributes:
- v: The value
- d: The derivative with respect to each variable in
P-dimensional VariationalParams for each of S celestial objects
in a local_P x local_S matrix.
- h: The second derivative with respect to each variational parameter,
in the same format as d. This is used for the full Hessian
with respect to all the sources.
- hs: An array of per-source Hessians. This will generally be reserved
+""" ->for the Hessian of brightness values that depend only on one source.
+type SensitiveFloat{ParamType <: CelesteTypes.ParamSet, NumType <: Number}Actually a single value, but an Array to avoid memory allocation
- v::Vector{NumType}
While we're looking at SensitiveFloat, I wanted to ask why v is an array
instead of just v::Numtype here. The effect of using an array is opposite
of what the comment implies.—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/pull/137/files#r55407321.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any luck duplicating the phenomenon---memory allocation on access---in a minimal working example?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't try. Do you think it's important?
On Tue, Mar 8, 2016 at 12:04 PM, Jeffrey Regier notifications@github.com
wrote:
In src/SensitiveFloat.jl
#137 (comment):+A function value and its derivative with respect to its arguments.
+
+Attributes:
- v: The value
- d: The derivative with respect to each variable in
P-dimensional VariationalParams for each of S celestial objects
in a local_P x local_S matrix.
- h: The second derivative with respect to each variational parameter,
in the same format as d. This is used for the full Hessian
with respect to all the sources.
- hs: An array of per-source Hessians. This will generally be reserved
+""" ->for the Hessian of brightness values that depend only on one source.
+type SensitiveFloat{ParamType <: CelesteTypes.ParamSet, NumType <: Number}Actually a single value, but an Array to avoid memory allocation
- v::Vector{NumType}
Any luck duplicating the phenomenon---memory allocation on access---in a
minimal working example?—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/pull/137/files#r55418802.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might turn out the be nothing, but it's kind of unsettling. Could it be related to #125 ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typically, if there's no type instability, modifying a bitstype field like Float64 should not allocate. Example:
julia> type A
val::Float64
end
julia> function update!(a)
for i=1:10_000_000
a.val = rand()
end
end
julia> a = A(0.0);
julia> @time update!(a)
0.048819 seconds (19.14 k allocations: 826.487 KB)
julia> @time update!(a)
0.039082 seconds (4 allocations: 160 bytes)
I don't know whether this is absolutely always true though.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The problem occurred in accumulate_source_brightness!. I also recall that
before I refactored ElboDeriv, there was a construction where we defined
fsm = (fsm[1], fsm[2])
and then referred to
fsm[i].v
Is it possible that the construction of this tuple was causing some type
instability that led to this memory allocation? I do think that not every
reference to a SF's value caused allocation. This is interesting if
someone wants to try switching back and looking at memory allcoation, but I
do think there are bigger fish to fry before the SC deadline.
On Tue, Mar 8, 2016 at 12:46 PM, Kyle Barbary notifications@github.com
wrote:
In src/SensitiveFloat.jl
#137 (comment):+A function value and its derivative with respect to its arguments.
+
+Attributes:
- v: The value
- d: The derivative with respect to each variable in
P-dimensional VariationalParams for each of S celestial objects
in a local_P x local_S matrix.
- h: The second derivative with respect to each variational parameter,
in the same format as d. This is used for the full Hessian
with respect to all the sources.
- hs: An array of per-source Hessians. This will generally be reserved
+""" ->for the Hessian of brightness values that depend only on one source.
+type SensitiveFloat{ParamType <: CelesteTypes.ParamSet, NumType <: Number}Actually a single value, but an Array to avoid memory allocation
- v::Vector{NumType}
Typically, if there's no type instability, modifying a bitstype field like
Float64 should not allocate. Example:julia> type A
val::Float64
endjulia> function update!(a)
for i=1:10_000_000
a.val = rand()
end
endjulia> a = A(0.0);
julia> @time update!(a)
0.048819 seconds (19.14 k allocations: 826.487 KB)julia> @time update!(a)
0.039082 seconds (4 allocations: 160 bytes)I don't know whether this is absolutely always true though.
—
Reply to this email directly or view it on GitHub
https://github.com/jeff-regier/Celeste.jl/pull/137/files#r55424483.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, if fsm
is a Vector before the assignment
fsm = (fsm[1], fsm[2])
then you'll have a type instability.
@rgiordan Just checking: are you going to remove SensitiveFloat.jl and make the other minor changes as part of the next PR finishing the work? BTW |
@kbarbary , yes, I'll fix them. Thanks for catching it. (I think I'll leave v as an array for the moment if you don't mind, though.) Thank you for the pointer about rebasing. I am learning a lot on this project these days! |
Yeah I agree it's not the time to fiddle with |
…rivs Factor out bnv derivs Former-commit-id: c9efc9a
This is a first step towards fitting the PSF in Celeste. I'd like to submit the PR now to avoid nasty merges while I finish the rest. Tests pass on my machine, but currently waiting on Travis.