Run test suite in parallel

[cpaniaguam]

The test suite, when running serially, takes ~15 minutes to run. The following version of runtests.jl makes the tests run in parallel but the (partial) reports are produced in a random order.

using Distributed

addprocs(nprocs())

@everywhere begin
    using IceFloeTracker
    using Images
    using Test
    using DelimitedFiles
    using Dates
    using DataFrames
    using Random
    using ImageTransformations: imrotate
    using ArgParse: ArgParseSettings, @add_arg_table!, add_arg_group!, parse_args
    include("test_error_rate.jl")
    include("config.jl")

    """
    Run a test file.

    # Arguments
    - `test`: The test file to run.

    """
    function run_test(test)
    #     @testset verbose = true "IceFloeTracker.jl" begin
            include(test)
        # end
    end
end

## Get all test files filenames "test-*" in test folder and their corresponding names/label 
alltests = [f for f in readdir(@__DIR__) if startswith(f, "test-")]

pmap(run_test, alltests)

Here is what the output looks like:

      From worker 2:    --------------- Pad Image Tests -----------------
      From worker 2:    Test Summary:          | Pass  Total  Time
      From worker 2:    utils.jl pad utilities |    5      5  0.3s
      From worker 3:     56.325320 seconds (1.20 G allocations: 29.758 GiB, 9.63% gc time, 2.13% compilation time)
      From worker 3:    ┌ Info: Persisting image to ./test_outputs/normalized_test_image_2024-04-03-123911.png.
      From worker 3:    └ To load the persisted object use `Images.load(img_path)`
      From worker 3:    Test Summary:   | Pass  Total     Time
      From worker 3:    Normalize Image |    6      6  1m29.4s
      From worker 5:    140.003612 seconds (34.91 k allocations: 166.595 GiB, 0.58% gc time, 0.07% compilation time)
      From worker 5:    106.103786 seconds (6.79 k allocations: 119.681 GiB, 0.38% gc time)
      From worker 5:      0.000165 seconds (3 allocations: 1.439 MiB)
      From worker 5:    ┌ Info: Persisting image to ./test_outputs/watershed_ice_intersect-2024-04-03-123848.png.
      From worker 5:    └ To load the persisted object use `Images.load(img_path)`
      From worker 5:    ┌ Info: Persisting image to ./test_outputs/watershed_not_ice-2024-04-03-123848.png.
      From worker 5:    └ To load the persisted object use `Images.load(img_path)`
      From worker 5:    ┌ Info: Persisting image to ./test_outputs/watershed_intersect-2024-04-03-123848.png.
      From worker 5:    └ To load the persisted object use `Images.load(img_path)`
      From worker 5:    ┌ Info: Persisting image to ./test_outputs/matlab_not_ice_bit-2024-04-03-123848.png.
      From worker 5:    └ To load the persisted object use `Images.load(img_path)`
      From worker 5:    [ Info: Test passed with 0.11491094570131472 mismatch with threshold 0.12
      From worker 5:    [ Info: Test passed with 0.14431421917123957 mismatch with threshold 0.15
      From worker 5:    [ Info: Test passed with 0.03286686908954287 mismatch with threshold 0.033
      From worker 5:    Test Summary:          | Pass  Total     Time
      From worker 5:    Segmentation-Watershed |    6      6  4m08.1s
      From worker 4:    [ Info: Done filling segmented_ice
      From worker 4:    705.015198 seconds (2.28 G allocations: 164.438 GiB, 1.24% gc time, 0.04% compilation time)
      From worker 4:    ┌ Info: Persisting image to ./test_outputs/segmented_a-2024-04-03-123814.png.
      From worker 4:    └ To load the persisted object use `Images.load(img_path)`
      From worker 4:    [ Info: Done with reshape
      From worker 4:      4.313654 seconds (273 allocations: 1.979 GiB, 5.68% gc time)
      From worker 4:    ┌ Info: Persisting image to ./test_outputs/segmented_a_ice-2024-04-03-123814.png.
      From worker 4:    └ To load the persisted object use `Images.load(img_path)`
      From worker 4:    [ Info: Test passed with 0.03881857232375131 mismatch with threshold 0.039
      From worker 4:    [ Info: Test passed with 0.0500808514820341 mismatch with threshold 0.051
      From worker 4:    [ Info: Test passed with 8.808560561878439e-5 mismatch with threshold 0.001
      From worker 4:    Test Summary:  | Pass  Total      Time
      From worker 4:    Segmentation-A |    6      6  11m58.5s
     Testing IceFloeTracker tests passed