Run tests with and without Revise #83

timholy · 2021-07-28T09:55:35Z

Closes #57

codecov · 2021-07-28T09:58:44Z

Codecov Report

Merging #83 (8d45758) into master (79633d0) will increase coverage by 25.32%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master      #83       +/-   ##
===========================================
+ Coverage   55.02%   80.34%   +25.32%     
===========================================
  Files           3        3               
  Lines         229      229               
===========================================
+ Hits          126      184       +58     
+ Misses        103       45       -58

Impacted Files	Coverage Δ
src/pkgfiles.jl	`72.22% <0.00%> (+5.55%)`	⬆️
src/utils.jl	`72.52% <0.00%> (+24.17%)`	⬆️
src/CodeTracking.jl	`87.50% <0.00%> (+29.16%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 79633d0...8d45758. Read the comment docs.

KristofferC · 2021-07-28T10:10:17Z

.github/workflows/ci.yml

+      - name: Test while running Revise
+        if: ${{ matrix.os == 'ubuntu-latest' && matrix.version != '1.0' }}
+        run: julia -e 'using Pkg, CodeTracking;
+                       Pkg.activate(pkgdir(CodeTracking));


Would it be the same to just add --project ?

KristofferC · 2021-07-28T10:21:28Z

test/runtests.jl

+if !isempty(ARGS) && "revise" ∈ ARGS
+    # For running tests with and without Revise
+    Pkg.add("Revise")
+    Pkg.instantiate()


I doubt the instantiate should be needed after the add. Was it? Also, perhaps it is better to still do the add in the CI script?

Not tested, just paranoid. And the add script failed, see https://github.com/timholy/CodeTracking.jl/runs/3180687064

So I think I've finally figured out how to test this locally in a separate DEPOT so that my default environment doesn't confuse matters). I've added Pkg as a test dependency

[extras] ColorTypes = "3da002f7-5984-5a60-b8a6-cbb66c0b333f" LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e" Pkg = "44cfe95a-1eb2-52ea-b672-e2afdf69b78f" SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf" Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40" [targets] test = ["ColorTypes", "LinearAlgebra", "Pkg", "SparseArrays", "Test"]

and a using Pkg to the test script but when I run the tests here's what I get:

$ JULIA_DEPOT_PATH=/tmp/pkgs julia --project -e 'using Pkg, CodeTracking; Pkg.activate(pkgdir(CodeTracking)); Pkg.add("Revise"); Pkg.test("CodeTracking"; coverage=true, test_args=["revise"])' Activating environment at `/tmp/pkgs/dev/CodeTracking/Project.toml` Updating registry at `/tmp/pkgs/registries/General` Resolving package versions... Updating `/tmp/pkgs/dev/CodeTracking/Project.toml` [295af30f] + Revise v3.1.17 Updating `/tmp/pkgs/dev/CodeTracking/Manifest.toml` [aa1ae85d] + JuliaInterpreter v0.8.18 [6f1432cf] + LoweredCodeUtils v2.1.0 [bac558e1] + OrderedCollections v1.4.1 [ae029012] + Requires v1.1.3 [295af30f] + Revise v3.1.17 [0dad84c5] + ArgTools [56f22d72] + Artifacts [2a0f44e3] + Base64 [ade2ca70] + Dates [8ba89e20] + Distributed [f43a241f] + Downloads [7b1f6079] + FileWatching [b77e0a4c] + InteractiveUtils [b27032c2] + LibCURL [76f85450] + LibGit2 [8f399da3] + Libdl [56ddb016] + Logging [d6f4376e] + Markdown [ca575930] + NetworkOptions [44cfe95a] + Pkg [de0858da] + Printf [3fa0cd96] + REPL [9a3f8284] + Random [ea8e919c] + SHA [9e88b42a] + Serialization [6462fe0b] + Sockets [fa267f1f] + TOML [a4e569a6] + Tar [cf7118a7] + UUIDs [4ec0a83e] + Unicode [deac9b47] + LibCURL_jll [29816b5a] + LibSSH2_jll [c8ffd9c3] + MbedTLS_jll [14a3606d] + MozillaCACerts_jll [83775a58] + Zlib_jll [8e850ede] + nghttp2_jll [3f19e933] + p7zip_jll Testing CodeTracking Status `/tmp/jl_z5eLOK/Project.toml` [da1fd8a2] CodeTracking v1.0.5 `/tmp/pkgs/dev/CodeTracking` [3da002f7] ColorTypes v0.11.0 [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils` [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra` [2f01184e] SparseArrays `@stdlib/SparseArrays` [8dfed614] Test `@stdlib/Test` Status `/tmp/jl_z5eLOK/Manifest.toml` [da1fd8a2] CodeTracking v1.0.5 `/tmp/pkgs/dev/CodeTracking` [3da002f7] ColorTypes v0.11.0 [53c48c17] FixedPointNumbers v0.8.4 [aa1ae85d] JuliaInterpreter v0.8.18 [6f1432cf] LoweredCodeUtils v2.1.0 [bac558e1] OrderedCollections v1.4.1 [ae029012] Requires v1.1.3 [295af30f] Revise v3.1.17 [0dad84c5] ArgTools `@stdlib/ArgTools` [56f22d72] Artifacts `@stdlib/Artifacts` [2a0f44e3] Base64 `@stdlib/Base64` [ade2ca70] Dates `@stdlib/Dates` [8ba89e20] Distributed `@stdlib/Distributed` [f43a241f] Downloads `@stdlib/Downloads` [7b1f6079] FileWatching `@stdlib/FileWatching` [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils` [b27032c2] LibCURL `@stdlib/LibCURL` [76f85450] LibGit2 `@stdlib/LibGit2` [8f399da3] Libdl `@stdlib/Libdl` [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra` [56ddb016] Logging `@stdlib/Logging` [d6f4376e] Markdown `@stdlib/Markdown` [ca575930] NetworkOptions `@stdlib/NetworkOptions` [44cfe95a] Pkg `@stdlib/Pkg` [de0858da] Printf `@stdlib/Printf` [3fa0cd96] REPL `@stdlib/REPL` [9a3f8284] Random `@stdlib/Random` [ea8e919c] SHA `@stdlib/SHA` [9e88b42a] Serialization `@stdlib/Serialization` [6462fe0b] Sockets `@stdlib/Sockets` [2f01184e] SparseArrays `@stdlib/SparseArrays` [10745b16] Statistics `@stdlib/Statistics` [fa267f1f] TOML `@stdlib/TOML` [a4e569a6] Tar `@stdlib/Tar` [8dfed614] Test `@stdlib/Test` [cf7118a7] UUIDs `@stdlib/UUIDs` [4ec0a83e] Unicode `@stdlib/Unicode` [deac9b47] LibCURL_jll `@stdlib/LibCURL_jll` [29816b5a] LibSSH2_jll `@stdlib/LibSSH2_jll` [c8ffd9c3] MbedTLS_jll `@stdlib/MbedTLS_jll` [14a3606d] MozillaCACerts_jll `@stdlib/MozillaCACerts_jll` [83775a58] Zlib_jll `@stdlib/Zlib_jll` [8e850ede] nghttp2_jll `@stdlib/nghttp2_jll` [3f19e933] p7zip_jll `@stdlib/p7zip_jll` Testing Running tests... ERROR: LoadError: ArgumentError: Package Pkg not found in current path: - Run `import Pkg; Pkg.add("Pkg")` to install the Pkg package. Stacktrace: [1] require(into::Module, mod::Symbol) @ Base ./loading.jl:893 [2] include(fname::String) @ Base.MainInclude ./client.jl:444 [3] top-level scope @ none:6 in expression starting at /tmp/pkgs/dev/CodeTracking/test/runtests.jl:6 ERROR: Package CodeTracking errored during testing Stacktrace: [1] pkgerror(msg::String) @ Pkg.Types ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:55 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing) @ Pkg.Operations ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1693 [3] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Vector{String}, kwargs::Base.Iterators.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}}) @ Pkg.API ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:343 [4] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Iterators.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:coverage, :test_args), Tuple{Bool, Vector{String}}}}) @ Pkg.API ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:80 [5] #test#63 @ ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:76 [inlined] [6] #test#62 @ ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:75 [inlined] [7] top-level scope @ none:1 tim@diva:/tmp/pkgs/dev/CodeTracking$

The funny thing is that Pkg isn't listed in the environment when Pkg.test runs. The first 15 lines of the test script are

# Note: some of CodeTracking's functionality can only be tested by Revise using CodeTracking using Test, InteractiveUtils, LinearAlgebra, SparseArrays # Note: ColorTypes needs to be installed, but note the intentional absence of `using ColorTypes` using Pkg using CodeTracking: line_is_decl if !isempty(ARGS) && "revise" ∈ ARGS # For running tests with and without Revise println(Pkg.status()) using Revise end

and line 6 (where the error occurs) is the using Pkg statement.

If I get rid of the Pkg stuff then it complains about Revise, despite the fact that it shows up in the resolved environment:

$ JULIA_DEPOT_PATH=/tmp/pkgs julia --project -e 'using Pkg, CodeTracking; Pkg.activate(pkgdir(CodeTracking)); Pkg.add("Revise"); Pkg.test("CodeTracking"; coverage=true, test_args=["revise"])' Activating environment at `/tmp/pkgs/dev/CodeTracking/Project.toml` Updating registry at `/tmp/pkgs/registries/General` Resolving package versions... Updating `/tmp/pkgs/dev/CodeTracking/Project.toml` [295af30f] + Revise v3.1.17 No Changes to `/tmp/pkgs/dev/CodeTracking/Manifest.toml` Testing CodeTracking Status `/tmp/jl_UbCiHU/Project.toml` [da1fd8a2] CodeTracking v1.0.5 `/tmp/pkgs/dev/CodeTracking` [3da002f7] ColorTypes v0.11.0 [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils` [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra` [2f01184e] SparseArrays `@stdlib/SparseArrays` [8dfed614] Test `@stdlib/Test` Status `/tmp/jl_UbCiHU/Manifest.toml` [da1fd8a2] CodeTracking v1.0.5 `/tmp/pkgs/dev/CodeTracking` [3da002f7] ColorTypes v0.11.0 [53c48c17] FixedPointNumbers v0.8.4 [aa1ae85d] JuliaInterpreter v0.8.18 [6f1432cf] LoweredCodeUtils v2.1.0 [bac558e1] OrderedCollections v1.4.1 [ae029012] Requires v1.1.3 [295af30f] Revise v3.1.17 [0dad84c5] ArgTools `@stdlib/ArgTools` [56f22d72] Artifacts `@stdlib/Artifacts` [2a0f44e3] Base64 `@stdlib/Base64` [ade2ca70] Dates `@stdlib/Dates` [8ba89e20] Distributed `@stdlib/Distributed` [f43a241f] Downloads `@stdlib/Downloads` [7b1f6079] FileWatching `@stdlib/FileWatching` [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils` [b27032c2] LibCURL `@stdlib/LibCURL` [76f85450] LibGit2 `@stdlib/LibGit2` [8f399da3] Libdl `@stdlib/Libdl` [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra` [56ddb016] Logging `@stdlib/Logging` [d6f4376e] Markdown `@stdlib/Markdown` [ca575930] NetworkOptions `@stdlib/NetworkOptions` [44cfe95a] Pkg `@stdlib/Pkg` [de0858da] Printf `@stdlib/Printf` [3fa0cd96] REPL `@stdlib/REPL` [9a3f8284] Random `@stdlib/Random` [ea8e919c] SHA `@stdlib/SHA` [9e88b42a] Serialization `@stdlib/Serialization` [6462fe0b] Sockets `@stdlib/Sockets` [2f01184e] SparseArrays `@stdlib/SparseArrays` [10745b16] Statistics `@stdlib/Statistics` [fa267f1f] TOML `@stdlib/TOML` [a4e569a6] Tar `@stdlib/Tar` [8dfed614] Test `@stdlib/Test` [cf7118a7] UUIDs `@stdlib/UUIDs` [4ec0a83e] Unicode `@stdlib/Unicode` [deac9b47] LibCURL_jll `@stdlib/LibCURL_jll` [29816b5a] LibSSH2_jll `@stdlib/LibSSH2_jll` [c8ffd9c3] MbedTLS_jll `@stdlib/MbedTLS_jll` [14a3606d] MozillaCACerts_jll `@stdlib/MozillaCACerts_jll` [83775a58] Zlib_jll `@stdlib/Zlib_jll` [8e850ede] nghttp2_jll `@stdlib/nghttp2_jll` [3f19e933] p7zip_jll `@stdlib/p7zip_jll` Testing Running tests... ERROR: LoadError: ArgumentError: Package Revise not found in current path: - Run `import Pkg; Pkg.add("Revise")` to install the Revise package. Stacktrace: [1] require(into::Module, mod::Symbol) @ Base ./loading.jl:893 [2] top-level scope @ /tmp/pkgs/dev/CodeTracking/test/runtests.jl:11 [3] include(fname::String) @ Base.MainInclude ./client.jl:444 [4] top-level scope @ none:6 in expression starting at /tmp/pkgs/dev/CodeTracking/test/runtests.jl:9 ERROR: Package CodeTracking errored during testing Stacktrace: [1] pkgerror(msg::String) @ Pkg.Types ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:55 [2] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, julia_args::Cmd, test_args::Cmd, test_fn::Nothing) @ Pkg.Operations ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1693 [3] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Cmd, test_args::Vector{String}, kwargs::Base.Iterators.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}}) @ Pkg.API ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:343 [4] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Iterators.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:coverage, :test_args), Tuple{Bool, Vector{String}}}}) @ Pkg.API ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:80 [5] #test#63 @ ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:76 [inlined] [6] #test#62 @ ~/src/julia-1/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:75 [inlined] [7] top-level scope @ none:1

The difference from the comment above is that I deleted the lines in the test script that use Pkg.

If I get rid of the Pkg stuff then it complains about Revise, despite the fact that it shows up in the resolved environment:

It doesn't show up as a top-level dependency though:

Status `/tmp/jl_UbCiHU/Project.toml` [da1fd8a2] CodeTracking v1.0.5 `/tmp/pkgs/dev/CodeTracking` [3da002f7] ColorTypes v0.11.0 [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils` [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra` [2f01184e] SparseArrays `@stdlib/SparseArrays` [8dfed614] Test `@stdlib/Test`

I'll try figure out what is happening

Aha, this package is using a test/Project.toml file... I don't really know how that stuff works. I would just remove that file and things seem to work reasonably again.

Run tests with and without Revise

a0d355e

Closes #57

Add Revise to CodeTracking's env during testing

4cc910f

KristofferC reviewed Jul 28, 2021

View reviewed changes

Remove test/Project.toml, streamline ci script

8d45758

timholy force-pushed the teh/rpath2 branch from a3c96d1 to 8d45758 Compare July 28, 2021 14:27

timholy merged commit c490872 into master Jul 28, 2021

timholy deleted the teh/rpath2 branch July 28, 2021 14:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Run tests with and without Revise #83

Run tests with and without Revise #83

Uh oh!

timholy commented Jul 28, 2021

Uh oh!

codecov bot commented Jul 28, 2021 •

edited

Loading

Uh oh!

KristofferC Jul 28, 2021

Uh oh!

KristofferC Jul 28, 2021

Uh oh!

timholy Jul 28, 2021

Uh oh!

timholy Jul 28, 2021

Uh oh!

timholy Jul 28, 2021

Uh oh!

KristofferC Jul 28, 2021

Uh oh!

KristofferC Jul 28, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Run tests with and without Revise #83

Run tests with and without Revise #83

Uh oh!

Conversation

timholy commented Jul 28, 2021

Uh oh!

codecov bot commented Jul 28, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

KristofferC Jul 28, 2021

Choose a reason for hiding this comment

Uh oh!

KristofferC Jul 28, 2021

Choose a reason for hiding this comment

Uh oh!

timholy Jul 28, 2021

Choose a reason for hiding this comment

Uh oh!

timholy Jul 28, 2021

Choose a reason for hiding this comment

Uh oh!

timholy Jul 28, 2021

Choose a reason for hiding this comment

Uh oh!

KristofferC Jul 28, 2021

Choose a reason for hiding this comment

Uh oh!

KristofferC Jul 28, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Jul 28, 2021 •

edited

Loading