--code-coverage=user --inline=no does not work witih @inline and @noinline

[jlapeyre]

The *.jl.cov file shows '-' for all lines preceded by @inline or @noinline. However, a commented out line will be credited with coverage:

246 #  zchop{T<:Real}(x::T, eps=zeps) = abs(x) > convert(T,eps) ? x : zero(T)
- @inline zchop{T<:Real}(x::T, eps=zeps) = abs(x) > convert(T,eps) ? x : zero(T)

[timholy]

Are you essentially saying this is an off-by-one bug in our line numbering when there's a macro in front of the function call?

[jlapeyre]

No, I only pasted a bit of output. I am trying to reproduce it now, but it I am not sure exactly what triggers it. Often, no *.cov file is generated. If I exit and start Julia a few times, eventually it will be generated.

Every line of code that can be tested is tested. I always see '-' in front of lines that have @inline or @noinline. Once, I commented out 12 or so lines of code, most with @inline and copied the code below it and removed all the @inlines. This time the coverage numbers were there, but in front of the commented out code only. Right now I am back to all -. I did not comment exactly the same portion.

[jlapeyre]

Found it! Notice that one line has @noinline rather than @inline

        - module ZChop
        - using Compat
        - export zchop, zchop!
        - 
        - const zeps = 1e-14
        - 
        -  zchop{T<:Real}(x::T, eps=zeps) = abs(x) > convert(T,eps) ? x : zero(T)
        -  zchop{T<:Integer}(x::T, eps=zeps) = abs(x) > eps ? x : zero(T)
        -  zchop{T<:Complex}(x::T, eps=zeps) = complex(zchop(real(x),eps),zchop(imag(x),eps))
        -  zchop(a::AbstractArray, eps=zeps) =
        -         (b = similar(a); for i in 1:length(a) b[i] = zchop(a[i],eps) end ; b)
        -  zchop!(a::AbstractArray, eps=zeps) = (for i in 1:length(a) a[i] = zchop(a[i],eps) end ; a)
        -  zchop(x::Union(String,Char),eps=zeps) = x
        -  zchop(x::MathConst,eps=zeps) = zchop(float(x),eps)
        - zchop(x::Expr,eps=zeps) = Expr(x.head,zchop(x.args)...)
        - zchop(x,eps=zeps) =  applicable(start,x) ? map((x)->zchop(x,eps),x) : x
        - zchop(x) = applicable(start,x) ? map(zchop,x) : x
        - 
      246 # @noinline zchop{T<:Real}(x::T, eps=zeps) = abs(x) > convert(T,eps) ? x : zero(T)
       15 # @inline zchop{T<:Integer}(x::T, eps=zeps) = abs(x) > eps ? x : zero(T)
       14 # @inline zchop{T<:Complex}(x::T, eps=zeps) = complex(zchop(real(x),eps),zchop(imag(x),eps))
       26 # @inline zchop(a::AbstractArray, eps=zeps) =
      133 #         (b = similar(a); for i in 1:length(a) b[i] = zchop(a[i],eps) end ; b)
      102 # @inline zchop!(a::AbstractArray, eps=zeps) = (for i in 1:length(a) a[i] = zchop(a[i],eps) end ; a)
        4 # @inline zchop(x::Union(String,Char),eps=zeps) = x
        3 # @inline zchop(x::MathConst,eps=zeps) = zchop(float(x),eps)
        7 # zchop(x::Expr,eps=zeps) = Expr(x.head,zchop(x.args)...)
        3 # zchop(x,eps=zeps) =  applicable(start,x) ? map((x)->zchop(x,eps),x) : x
        6 # zchop(x) = applicable(start,x) ? map(zchop,x) : x

[timholy]

What's versioninfo()?

[timholy]

Did you insert all those comments? I have never seen it attribute runs to comments.

Could it be #1334? I'm not so worried about the @inline and @noinline, but if you have other macros elsewhere, I wonder if the line numbers are just borked.

[jlapeyre]

Warning: could not attach metadata for @simd loop.
Julia Version 0.4.0-dev+2588
Commit 24dbedc (2015-01-09 16:13 UTC)
Platform Info:
System: Linux (x86_64-linux-gnu)
CPU: Intel(R) Core(TM) i7-2760QM CPU @ 2.40GHz
WORD_SIZE: 64
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
LAPACK: libopenblas
LIBM: libopenlibm
LLVM: libLLVM-3.3

[jlapeyre]

I had no comments in the original code. No coverage was reported. So I copied and pasted all the code. Then commented out the original to save it. Then removed all the inline macros from the new copy. That's why it looks this way. If I remove all comments and all inline and noinline macros, then everything works as expected.

[jlapeyre]

Depending on how much I comment out, some of the coverage numbers appear before lines that don't make sense, even if they were not commented out. I'll try to find that again. Maybe it contains a clue.

[timholy]

I'm still not sure I understand what you're showing, and what you did. But when you run --code_coverage, also set --inline=no. Inlining can completely bork your results.

It still sounds like you have a genuine issue that's probably a bug (or some other oddity), but I confess I'm not following. Very explicit, step-by-step instructions may be necessary.

[jlapeyre]

Here is another one. Notice the placement of the last three coverage numbers:

        - module ZChop
        - using Compat
        - export zchop, zchop!
        - 
        - const zeps = 1e-14
        - 
      246  zchop{T<:Real}(x::T, eps=zeps) = abs(x) > convert(T,eps) ? x : zero(T)
       15  zchop{T<:Integer}(x::T, eps=zeps) = abs(x) > eps ? x : zero(T)
       14  zchop{T<:Complex}(x::T, eps=zeps) = complex(zchop(real(x),eps),zchop(imag(x),eps))
       26  zchop(a::AbstractArray, eps=zeps) =
      133         (b = similar(a); for i in 1:length(a) b[i] = zchop(a[i],eps) end ; b)
      102  zchop!(a::AbstractArray, eps=zeps) = (for i in 1:length(a) a[i] = zchop(a[i],eps) end ; a)
        4  zchop(x::Union(String,Char),eps=zeps) = x
        3  zchop(x::MathConst,eps=zeps) = zchop(float(x),eps)
        7 
        3 # @noinline zchop{T<:Real}(x::T, eps=zeps) = abs(x) > convert(T,eps) ? x : zero(T)
        6 # @inline zchop{T<:Integer}(x::T, eps=zeps) = abs(x) > eps ? x : zero(T)
        - # @inline zchop{T<:Complex}(x::T, eps=zeps) = complex(zchop(real(x),eps),zchop(imag(x),eps))
        - # @inline zchop(a::AbstractArray, eps=zeps) =
        - #         (b = similar(a); for i in 1:length(a) b[i] = zchop(a[i],eps) end ; b)
        - # @inline zchop!(a::AbstractArray, eps=zeps) = (for i in 1:length(a) a[i] = zchop(a[i],eps) end ; a)
        - # @inline zchop(x::Union(String,Char),eps=zeps) = x
        - # @inline zchop(x::MathConst,eps=zeps) = zchop(float(x),eps)
        - zchop(x::Expr,eps=zeps) = Expr(x.head,zchop(x.args)...)
        - zchop(x,eps=zeps) =  applicable(start,x) ? map((x)->zchop(x,eps),x) : x
        - zchop(x) = applicable(start,x) ? map(zchop,x) : x
        - 
        - end # module ZChop
        - 

[timholy]

If you can provide a small, completely stand-alone test case that I can run on my own machine, it would help a lot.

[jlapeyre]

Ok!

[jlapeyre]

Here is a test case.

https://github.com/jlapeyre/CovInline.jl

The behavior is complicated enough that it seems non-deterministic to me. But, I don't know if it really is; ie in the sense that it depends on the rest of the system. While I was editing the code, the coverage reports suddenly failed to generate. I exited and restarted julia about 15 times trying different things to generate a report. I always (or almost ?) have to delete the old coverage file in order for a new one to be generated. So there was no .cov file, and still it would not appear. Finally, I changed one of the tests so that it should fail. It did fail and a coverage report was generated. I then edited the test so that it would pass as before. Then the coverage tests were generated as as before.

I included the .jl.cov file in the repo. I should note that I also got different results sometimes. The functions that are not single lines, ie. not like func(x) = x sometimes do show coverage for the line that does some computation. However, in the .cov file that I uploaded, this is not the case.

[timholy]

Thanks!

I'm not seeing the kind of "random" behavior you describe. However, I can verify that any function tagged with @inline gives me no coverage, even when I run julia like this:

julia --inline=no --code-coverage=user runtests.jl

[timholy]

One thought on your non-determinism: you're remembering to quit your julia session after running the tests, right? It only writes the .cov files when you quit.

[jlapeyre]

Ah, that must be it! I either did not read it, or forgot. Thanks.

[timholy]

OK, sorry this took a bit, but it should be fixed now.

[jlapeyre]

Thanks!