Improve stats gathering of mapping-tester

[fsimonis]

Main changes of this PR

This PR improves the gathering of stats in the mapping-tester by:

1. Making it more robust. Various files are now only loaded if they exist. So, post-processing is now optional.
2. Loads cases in parallel, which can help to improve the import speed of events via precice-profiling

Author's checklist

• I used the pre-commit hook and used pre-commit run --all to apply all available hooks.
• I added a test to cover the proposed changes in our test suite.
• I updated the documentation in docs/README.md.
• I added a changelog entry in ./changelog-entries/ (create if necessary).
• I updated potential breaking changes in the tutorial precice/tutorials/aste-turbine.

[davidscn]

I face the following error using the run script in the mapping_tester, not sure if this is related to the changes here

+ python3 ~/precice/aste/examples/mapping_tester/../../tools/mapping-tester//compare.py reference-statistics.csv test-statistics.csv
Comparing results with atol=1e-08 and rtol=1e-05
Given test-statistics.csv
Reference reference-statistics.csv
Traceback (most recent call last):
  File "~/precice/aste/examples/mapping_tester/../../tools/mapping-tester//compare.py", line 63, in <module>
    if not ref.ne_missing(a).any():
           ^^^^^^^^^^^^^^^^^
  File "~/precice/precice-venv/lib/python3.12/site-packages/polars/series/series.py", line 929, in ne_missing
    return self.to_frame().select(F.col(self.name).ne_missing(other)).to_series()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/precice/precice-venv/lib/python3.12/site-packages/polars/dataframe/frame.py", line 8097, in select
    return self.lazy().select(*exprs, **named_exprs).collect(_eager=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/precice/precice-venv/lib/python3.12/site-packages/polars/lazyframe/frame.py", line 1730, in collect
    return wrap_df(ldf.collect())
                   ^^^^^^^^^^^^^
polars.exceptions.ComputeError: cannot evaluate two Series of different lengths (2 and 3)

Error originated in expression: '[(col("median(abs)")) !=v (Series[median(abs)])]'

[fsimonis]

Not a pretty error message. I think the case directory still includes the NP mapping case which was removed by #212. There should only be 2 data lines in the file.

[davidscn]

Ah alright, I cleaned it and ran it again:

Part of the mapping-tester is also the plotting script. Using the plotting here gives me

  File "~/precice/aste/tools/mapping-tester/plotconv.py", line 203, in <module>
    sys.exit(main(sys.argv))
             ^^^^^^^^^^^^^^
  File "~/precice/aste/tools/mapping-tester/plotconv.py", line 187, in main
    df = pandas.read_csv(args.file)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/precice/precice-venv/lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 1026, in read_csv
    return _read(filepath_or_buffer, kwds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/precice/precice-venv/lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 626, in _read
    return parser.read(nrows)
           ^^^^^^^^^^^^^^^^^^
  File "~/precice/precice-venv/lib/python3.12/site-packages/pandas/io/parsers/readers.py", line 1923, in read
    ) = self._engine.read(  # type: ignore[attr-defined]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~/precice/precice-venv/lib/python3.12/site-packages/pandas/io/parsers/c_parser_wrapper.py", line 234, in read
    chunks = self._reader.read_low_memory(nrows)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "parsers.pyx", line 838, in pandas._libs.parsers.TextReader.read_low_memory
  File "parsers.pyx", line 905, in pandas._libs.parsers.TextReader._read_rows
  File "parsers.pyx", line 874, in pandas._libs.parsers.TextReader._tokenize_rows
  File "parsers.pyx", line 891, in pandas._libs.parsers.TextReader._check_tokenize_status
  File "parsers.pyx", line 2061, in pandas._libs.parsers.raise_parser_error
pandas.errors.ParserError: Error tokenizing data. C error: Expected 1 fields in line 3, saw 2

[davidscn]

Maybe we should add the plotconv to the testing pipeline (I thought it was actually part of it)?

[fsimonis]

True. Let's do that in a separate issue though.

The plotting scripts rely on the meshes being numerical values for the x axis, which currently doesn't work with our examples.

This crash looks very wild.

[davidscn]

True. Let's do that in a separate issue though.

The plotting scripts rely on the meshes being numerical values for the x axis, which currently doesn't work with our examples.

This crash looks very wild.

The problem is a bit that the current changes lead to the wild crash. So if I accept the changes, the script doesn't work anymore. The issue is an immediate consequence of the changes being made here. What do you think?

[fsimonis]

Can you remove the clean the case and rerun it?

I don't know what leads to this crash, works fine and triggers the expected warning and produces the expected graphs on my machine:

..../venv/lib/python3.12/site-packages/pandas/plotting/_matplotlib/core.py:822: UserWarning: Data has no positive values, and therefore cannot be log-scaled.

[davidscn]

Cleaned it and then got the same error:

/precice-venv/lib/python3.12/site-packages/pandas/plotting/_matplotlib/core.py:822: UserWarning: Data has no positive values, and therefore cannot be log-scaled.
  labels = axis.get_majorticklabels() + axis.get_minorticklabels()

[davidscn]

...which is however related to the axis names, so we are good to go 👍