Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print what is being testing during tests and timing #308

Merged
merged 1 commit into from
Nov 18, 2020
Merged

Conversation

oxinabox
Copy link
Member

@oxinabox oxinabox commented Nov 17, 2020

The test on TravisCI have been timing out lately; due to lack of output.
I think it is because Travis is being super-slow.
but as we get more tests the possibility for it to go over 10 minutes without output increases.
We previously added print(" ") between testsets to stop this.
This PR goes further and just adds a print per include.
This is also nice for testing locally, as it gives a sense of progress.

Beyond this this PR also prints timing info so we can see which tests are taking longest.
And can catch them before any get to 10 minutes.

This is what the output looks like

Testing ChainRules.jl
Testing rulesets/Base/base.jl:   65.677611 seconds (149.48 M allocations: 7.257 GiB, 4.09% gc time)
Testing rulesets/Base/fastmath_able.jl:  81.614611 seconds (172.61 M allocations: 8.414 GiB, 4.54% gc time)
Testing rulesets/Base/evalpoly.jl:       22.094328 seconds (47.41 M allocations: 2.402 GiB, 4.42% gc time)
Testing rulesets/Base/array.jl:   4.339693 seconds (10.15 M allocations: 513.666 MiB, 5.12% gc time)
Testing rulesets/Base/arraymath.jl:      97.615484 seconds (255.76 M allocations: 12.716 GiB, 6.45% gc time)
Testing rulesets/Base/indexing.jl:       10.097454 seconds (19.88 M allocations: 1018.051 MiB, 4.23% gc time)
Testing rulesets/Base/mapreduce.jl:      23.009607 seconds (51.07 M allocations: 2.580 GiB, 5.04% gc time)

Testing rulesets/Statistics/statistics.jl:        0.809056 seconds (1.82 M allocations: 93.169 MiB, 7.08% gc time)

Testing rulesets/LinearAlgebra/dense.jl:         46.138051 seconds (97.07 M allocations: 4.818 GiB, 4.73% gc time)
Testing rulesets/LinearAlgebra/structured.jl:    39.893027 seconds (76.39 M allocations: 3.834 GiB, 4.74% gc time)
Testing rulesets/LinearAlgebra/factorization.jl:          4.975585 seconds (13.26 M allocations: 715.600 MiB, 6.50% gc time)
Testing rulesets/LinearAlgebra/blas.jl:  25.684569 seconds (87.81 M allocations: 6.821 GiB, 11.84% gc time)

Testing rulesets/Random/random.jl:        0.483092 seconds (457.58 k allocations: 25.771 MiB)

Testing rulesets/packages/NaNMath.jl:     0.000101 seconds (76 allocations: 4.734 KiB)
Testing rulesets/packages/SpecialFunctions.jl:  ┌ Warning: `lgamma(x::Real)` is deprecated, use `(logabsgamma(x))[1]` instead.
│   caller = test_scalar(::typeof(lgamma), ::Float64; rtol::Float64, atol::Float64, fdm::FiniteDifferenceMethod{Array{Int64,1},Array{Float64,1},FiniteDifferences.var"#27#28"{FiniteDifferenceMethod{Array{Int64,1},Array{Float64,1},FiniteDifferences.var"#default_bound_estimator#19"{Int64}}}}, fkwargs::NamedTuple{(),Tuple{}}, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at testers.jl:120
<<GIANT PILE OF DEPRECATION WARNINGS>>
└ @ Core ~/.julia/packages/ChainRulesCore/LsNyE/src/rule_definition_tools.jl:194
 32.885779 seconds (64.71 M allocations: 3.122 GiB, 6.75% gc time)

Test Summary: |  Pass  Total
ChainRules    | 18881  18881
    Testing ChainRules tests passed 

@oxinabox oxinabox merged commit 1eccb5e into master Nov 18, 2020
@oxinabox oxinabox deleted the ox/loudtests branch November 18, 2020 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants