Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Small updates to Complete Polynomials #26

Merged
merged 11 commits into from
Feb 7, 2017
Merged

Small updates to Complete Polynomials #26

merged 11 commits into from
Feb 7, 2017

Conversation

cc7768
Copy link
Member

@cc7768 cc7768 commented Dec 23, 2016

This adds some vector methods for generating basis functions of complete polynomials. Specializing on vectors gives a (small) speedup and is more convenient on some occasions.

Define the following

using BasisMatrices
using BenchmarkTools

z = [1.0, 2.0, 3.0, 4.0, 5.0];
zbig = repeat(z', outer=[10000, 1]);

Timings on this branch are

julia> @benchmark complete_polynomial(z, 4)
BenchmarkTools.Trial: 
  memory estimate:  9.81 kb
  allocs estimate:  141
  --------------
  minimum time:     2.921 μs (0.00% GC)
  median time:      3.054 μs (0.00% GC)
  mean time:        3.934 μs (18.97% GC)
  maximum time:     230.539 μs (96.51% GC)
  --------------
  samples:          10000
  evals/sample:     8
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @benchmark complete_polynomial(z', 4)
BenchmarkTools.Trial: 
  memory estimate:  10.02 kb
  allocs estimate:  145
  --------------
  minimum time:     3.303 μs (0.00% GC)
  median time:      3.430 μs (0.00% GC)
  mean time:        4.385 μs (18.22% GC)
  maximum time:     251.962 μs (96.67% GC)
  --------------
  samples:          10000
  evals/sample:     8
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @benchmark complete_polynomial(zbig, 4)
BenchmarkTools.Trial: 
  memory estimate:  9.62 mb
  allocs estimate:  142
  --------------
  minimum time:     1.165 ms (0.00% GC)
  median time:      1.740 ms (0.00% GC)
  mean time:        1.557 ms (10.81% GC)
  maximum time:     2.354 ms (20.19% GC)
  --------------
  samples:          3205
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

Time on master is

julia> @benchmark complete_polynomial(z', 4)
BenchmarkTools.Trial: 
  memory estimate:  10.02 kb
  allocs estimate:  145
  --------------
  minimum time:     3.225 μs (0.00% GC)
  median time:      3.484 μs (0.00% GC)
  mean time:        4.345 μs (16.75% GC)
  maximum time:     237.701 μs (96.36% GC)
  --------------
  samples:          10000
  evals/sample:     8
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @benchmark complete_polynomial(zbig, 4)
BenchmarkTools.Trial: 
  memory estimate:  9.62 mb
  allocs estimate:  142
  --------------
  minimum time:     1.172 ms (0.00% GC)
  median time:      1.665 ms (0.00% GC)
  mean time:        1.589 ms (8.99% GC)
  maximum time:     2.443 ms (14.88% GC)
  --------------
  samples:          3139
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

@coveralls
Copy link

Coverage Status

Coverage decreased (-1.1%) to 83.113% when pulling ef3ea7b on complete_update into 499b3a9 on master.

@codecov-io
Copy link

codecov-io commented Dec 23, 2016

Codecov Report

Merging #26 into master will not impact coverage by -2.95%.

@@            Coverage Diff             @@
##           master      #26      +/-   ##
==========================================
- Coverage   84.25%   81.31%   -2.95%     
==========================================
  Files           9        9              
  Lines         972     1038      +66     
==========================================
+ Hits          819      844      +25     
- Misses        153      194      +41
Impacted Files Coverage Δ
src/complete.jl 70.37% <63.41%> (-27.25%)
src/basis.jl 91.07% <ø> (-8.93%)
src/basis_structure.jl 92% <ø> (-5%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 499b3a9...fd8d3b0. Read the comment docs.

@cc7768
Copy link
Member Author

cc7768 commented Dec 23, 2016

Bad news... There's going to be more in this PR.

My most recent commit adds functionality for computing derivative of basis functions for vectors of variables. Use is shown below:

z = [1.0, 2.0, 3.0]

der_wrt_x = complete_polynomial_der(z, 3, 1)
der_wrt_y = complete_polynomial_der(z, 3, 2)
der_wrt_z = complete_polynomial_der(z, 3, 3)

produces


julia> [der_wrt_x der_wrt_y der_wrt_z]
20×3 Array{Float64,2}:
 0.0   0.0   0.0
 1.0   0.0   0.0
 2.0   0.0   0.0
 3.0   0.0   0.0
 4.0   1.0   0.0
 6.0   0.0   1.0
 2.0   1.0   0.0
 4.0   4.0   0.0
 6.0   3.0   2.0
 3.0   0.0   1.0
 9.0   0.0   6.0
 0.0   1.0   0.0
 0.0   4.0   0.0
 0.0  12.0   0.0
 0.0  12.0   4.0
 0.0   3.0   2.0
 0.0   9.0  12.0
 0.0   0.0   1.0
 0.0   0.0   6.0
 0.0   0.0  27.0

Notice computing the derivatives of basis functions is a little slower than computing the basis functions themselves. There may be a way to close this gap.

julia> @benchmark complete_polynomial_der(z, 3, 1)
BenchmarkTools.Trial: 
  memory estimate:  1.48 kb
  allocs estimate:  21
  --------------
  minimum time:     967.750 ns (0.00% GC)
  median time:      1.003 μs (0.00% GC)
  mean time:        1.128 μs (8.41% GC)
  maximum time:     102.294 μs (96.80% GC)
  --------------
  samples:          10000
  evals/sample:     16
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @benchmark complete_polynomial(z, 3)
BenchmarkTools.Trial: 
  memory estimate:  1.48 kb
  allocs estimate:  21
  --------------
  minimum time:     670.787 ns (0.00% GC)
  median time:      702.546 ns (0.00% GC)
  mean time:        816.580 ns (12.34% GC)
  maximum time:     10.819 μs (91.23% GC)
  --------------
  samples:          10000
  evals/sample:     164
  time tolerance:   5.00%
  memory tolerance: 1.00%

@coveralls
Copy link

Coverage Status

Coverage decreased (-3.3%) to 80.929% when pulling 699d5df on complete_update into 499b3a9 on master.

@cc7768
Copy link
Member Author

cc7768 commented Dec 23, 2016

Now there is a derivative function for matrices as well.

julia> zbig = repeat(z', outer=[10000, 1]);

julia> @benchmark complete_polynomial(zbig, 3)
BenchmarkTools.Trial: 
  memory estimate:  1.53 mb
  allocs estimate:  22
  --------------
  minimum time:     157.047 μs (0.00% GC)
  median time:      182.758 μs (0.00% GC)
  mean time:        208.480 μs (11.96% GC)
  maximum time:     811.686 μs (65.62% GC)
  --------------
  samples:          10000
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

julia> @benchmark complete_polynomial_der(zbig, 3, 1)
BenchmarkTools.Trial: 
  memory estimate:  1.53 mb
  allocs estimate:  22
  --------------
  minimum time:     1.265 ms (0.00% GC)
  median time:      1.277 ms (0.00% GC)
  mean time:        1.312 ms (1.90% GC)
  maximum time:     1.996 ms (18.79% GC)
  --------------
  samples:          3808
  evals/sample:     1
  time tolerance:   5.00%
  memory tolerance: 1.00%

This one seems to pay a higher performance penalty than the vector version. Would love for someone who knows more voodoo than me to have a look to see whether it can be sped up.

@coveralls
Copy link

Coverage Status

Coverage decreased (-5.3%) to 78.978% when pulling 808836c on complete_update into 499b3a9 on master.

@coveralls
Copy link

Coverage Status

Coverage decreased (-2.9%) to 81.31% when pulling a048740 on complete_update into 499b3a9 on master.

Copy link
Member

@sglyon sglyon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks really good.

The main thing I want to address is if there is any way to get the nice z::Vector functionality without so much code duplication. Let's do some more benchmarks and tests on that to see what we should do.

@@ -70,7 +70,8 @@ export BasisFamily, Cheb, Lin, Spline, Basis,
# functions
export nodes, get_coefs, funfitxy, funfitf, funeval, evalbase,
derivative_op, row_kron, evaluate, fit!, update_coefs!,
complete_polynomial, complete_polynomial!, n_complete
complete_polynomial, complete_polynomial!, n_complete,
complete_polynomial_der, complete_polynomial_der!
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking that given that you've introduced a Derivative{D} type, that we could get away with only having the complete_polynomial and complete_polynomial! functions be exported. We could then infer if the user wanted a derivative based on whether or not a Derivative type was passed and/or a derivative keyword argument?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great point.

I don't think we can use a keyword argument for derivatives (it wouldn't dispatch on it) -- I think the following protocol seems sensible

# computes 3rd order complete polynomial for z
complete_polynomial(z, 3)

# computes derivatives with respect to variable 1 of 3rd order complete polynomial
complete_polynomial(z, 3, 1)

Thoughts?

)
out
end
end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I 100% agree that having methods that operate on a single point (passed as a Vector) is beneficial. We need to support this.

However, this seems like a lot of code duplication...

I wonder if the small difference in timing between the new method here for z::Vector and calling on z' has to do with copying the memory for doing the transpose? If so, maybe we could try defining the method on vector so it just calls the Matrix one with reshape(z, 1, length(z)) so there isn't a copy?

Copy link
Member Author

@cc7768 cc7768 Jan 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that it is kind of a lot of code duplication -- Some of it was a playground for me to try and learn some stuff. A bit surprisingly, complete_polynomial(reshape(z, 1, length(z)), 4) actually does worse than complete_polynomial(z', 4) does. It's also slower even if I just pass in the already reshaped z argument.

FWIW, I use a slightly different way of constructing the points for vectors than for matrices (slight might even be an overstatement of the difference), but it does seem to continue being faster even as the number of dimensions and degree of polynomials are ramped up (we're talking about microseconds here...).

Thoughts? If specializing turns out to keep it faster then my vote would be to keep it separate for now, but I strongly suspect there are ways to close the gaps (I just haven't figured out how).

src/complete.jl Outdated
function complete_polynomial{T}(z::Vector{T}, d::Int)
nvar = length(z)
out = Array(T, n_complete(nvar, d))
complete_polynomial!(z, Degree{d}(), out)::Vector{T}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized that we can avoid the type assert by just returning out (implicitly of course 😉 ) after calling complete_polynomial! instead of relying on the return variable of complete_polynomial!.

Copy link
Member Author

@cc7768 cc7768 Jan 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

Great. I'll fix this everywhere (with explicit returns).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm. We still get intermediate type instability because it doesn't know what type gets returned by complete_polynomial!(z, Degree{d}(), out)... Is that a problem? It really seems like it should be able to infer that type.

Copy link
Member

@sglyon sglyon Jan 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that the inability to infer the return type of complete_polynomial! should matter.

What I had in mind was this:

function complete_polynomial{T}(z::Vector{T}, d::Int)
    nvar = length(z)
    out = Array(T, n_complete(nvar, d))
    complete_polynomial!(z, Degree{d}(), out)
    out
end

We don't really care what the output of complete_polynomial! is because we never use it.

@coveralls
Copy link

Coverage Status

Coverage decreased (-3.009%) to 81.25% when pulling fd8d3b0 on complete_update into 499b3a9 on master.

@sglyon sglyon merged commit ea33dbe into master Feb 7, 2017
@sglyon sglyon deleted the complete_update branch April 25, 2017 15:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants