Engine type changed in repeated output

[liangwang0734]

Main code, repeat_io.cxx:

#include <cstdio>
#include <string>
#include <mpi.h>
#include <adios2.h>

int main(int argc, char **argv)
{
  MPI_Init(&argc, &argv);

  adios2::ADIOS ad_("adios2cfg.xml", MPI_COMM_WORLD, adios2::DebugON);
  adios2::IO io_;
  adios2::Engine engine_;
  
  std::string io_name_ = "checkpoint";
  for (int c=0; c<3; c++) {
    io_ = ad_.DeclareIO(io_name_);
    std::string filename = "test.bp";
    engine_ = io_.Open(filename, adios2::Mode::Write);
    printf("+++ c=%d, type=%s\n", c, engine_.Type().c_str());
    engine_.Close();
    ad_.RemoveIO(io_name_);
  }

  MPI_Finalize();

  return 0;
}

adios2cfg.xml:

<?xml version="1.0"?>
<adios-config>
    <io name="checkpoint">
        <engine type="BP5">
            <parameter key="SubStreams" value="16"/>
        </engine>
    </io>
</adios-config>

Results (only the first iteration uses BP5; using adios2 release_28):

+++ c=0, type=BP5Writer
+++ c=1, type=BP4Writer
+++ c=2, type=BP4Writer

Question: How to make the subsequent IO use BP5?
One option is to move the declaration and removal of io_ out of the loop, but that requires some refactoring of our production code, so I prefer to avoid that.

[pnorbert]

Agree that this is a bug. We have not discovered it because we don't find a reason to delete the IO object and recreate it and redefine all variables again and again. The file name is not tied to the IO object until you call IO:Open(). So you could just create the IO object once and define the variables and attributes, outside the loop, and reuse it many times to write your checkpoint.

…

On Thu, Jul 27, 2023, 7:08 PM Liang Wang ***@***.***> wrote: Main code, repeat_io.cxx: #include <cstdio> #include <string> #include <mpi.h> #include <adios2.h> int main(int argc, char **argv) { MPI_Init(&argc, &argv); adios2::ADIOS ad_("adios2cfg.xml", MPI_COMM_WORLD, adios2::DebugON); adios2::IO io_; adios2::Engine engine_; std::string io_name_ = "checkpoint"; for (int c=0; c<3; c++) { io_ = ad_.DeclareIO(io_name_); std::string filename = "test.bp"; engine_ = io_.Open(filename, adios2::Mode::Write); printf("+++ c=%d, type=%s\n", c, engine_.Type().c_str()); engine_.Close(); ad_.RemoveIO(io_name_); } MPI_Finalize(); return 0; } adios2cfg.xml: <?xml version="1.0"?> <adios-config> <io name="checkpoint"> <engine type="BP5"> <parameter key="SubStreams" value="16"/> </engine> </io> </adios-config> Results (only the first iteration uses BP5): +++ c=0, type=BP5Writer +++ c=1, type=BP4Writer +++ c=2, type=BP4Writer Question: How to make the subsequent io use BP5? — Reply to this email directly, view it on GitHub <#3720>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYYYLPLB2ESO55GNUZLYVLXSLYH3ANCNFSM6AAAAAA22XQ4CQ> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[liangwang0734]

@pnorbert Yes, moving the IO creation out of the loop would solve the problem in this example. But for now, I prefer not to do this in our user code. Can we set the engine type manually at or after IO::Open?

[liangwang0734]

I now use IO::SetEngine to manually change the engine type.

If there is no plan/interest to fix this, I will close this issue.

[pnorbert]

IO:SetEngine() before the next Open, of course

…

On Thu, Jul 27, 2023, 9:30 PM Liang Wang ***@***.***> wrote: @pnorbert <https://github.com/pnorbert> Yes, moving the IO creation out of the loop would solve the problem in this example. But for now, I prefer not to do this in our user code. Can we set the engine type manually at or after IO::Open? — Reply to this email directly, view it on GitHub <#3720 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYYYLPCWPFOMNGZIKFMMELXSMIZ3ANCNFSM6AAAAAA22XQ4CQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[pnorbert]

It's a bug, so please keep it open

…

On Thu, Jul 27, 2023, 9:36 PM Liang Wang ***@***.***> wrote: I now use IO::SetEngine to manually change the engine type. If there is no plan/interest to fix this, I will close this issue. — Reply to this email directly, view it on GitHub <#3720 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAYYYLMZLCRYFI7FEXVIKCDXSMJQVANCNFSM6AAAAAA22XQ4CQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[germasch]

FWIW, this problem happens in master, too, though slightly differently in that if one sets "bp4" in the config file, it'll be bp4 the first time around, and then switch to bp5 in subsequent uses.

The reason is that the IO is created when the config file is read (when the ADIOS2 object is created), and the engine type is set correctly from the config file. But if one removes the IO and makes a new one, that one is created from scratch, unaware of the existing config for it, and so it defaults to a m_EngineType of "File", which turns into into BP5. I would consider it a bug, too, but I don't think there's an easy fix.

[spyridon97]

State of this issue:

In master if you run the code initially attached you get the following result:

~/Programming/Cpp/Test/cmake-build-relwithdebinfo > mpirun -np 2 ./test                                                      4s
+++ c=0, type=BP5Writer
+++ c=0, type=BP5Writer
+++ c=1, type=BP5Writer
+++ c=1, type=BP5Writer
+++ c=2, type=BP5Writer
+++ c=2, type=BP5Writer

What @germasch states is also true currently in master when using BP4 instead of BP5.

mpirun -np 2 ./test
+++ c=0, type=BP4Writer
+++ c=0, type=BP4Writer
+++ c=1, type=BP5Writer
+++ c=1, type=BP5Writer
+++ c=2, type=BP5Writer
+++ c=2, type=BP5Writer

I am looking into why this is happening.

[spyridon97]

After looking into the code my understanding is the following:

When using a config file, an ADIOS object is initialized and it defines a set of IO/Operator objects.
This is basically a convenience mechanism, instead of writing code to initialize the IO objects (and operators).

1. Beginning of 1st iteration:
   If you decide to declare an IO object with the same name as one defined in the config file, nothing is actually going to happen internally (1st iteration).

2. End of 1st iteration:
   If you remove the phenomenally newly created IO, you actually delete what the config file originally defined.

3. Beginning of 2nd iteration: If in the next iteration, you decide to define an IO object with the same name as one in the config file, the engine is not gonna be set to the same value as the one in the config file. because as we said the config file is only used for initialization and nothing else.

If the intended behavior of a config file is to initialize an ADIOS object with IO/Operators, then this is an expected behavior and not a bug.

If the intended behavior of a config file is to initialize an ADIOS object AND set the defaults (WHICH NEVER CHANGE) for initializing an ADIOS object, then this is a bug.

[williamfgc]

It's a bug and a very corner case. Config files are a convenience to move things to runtime, and Remove functions are part of the danger zone for very unique cases. We can find tons of cases like these if we combine the APIs in a way outside our targeted use-cases, which are always prioritized over corner cases with no real use-case.

[anagainaru]

Maybe we should re-discuss our policy to silently ignore invalid parameters / user input? This is all caused by the fact that we allow (and ignore) re-creating an IO object with the same name. A topic for next Tuesday.

[williamfgc]

The question is if the new IO with the same name as the removed one is a completely new entity (which is the current policy). Remove functions are a very special case we never paid much attention as it's a one off.

[spyridon97]

I know that ParaView is not related to ADIOS, but in my mind, the config file was the equivalent of a state file in ParaView. The state file is used to initialize something, but when you do any change (addition or removal of filters) to the paraview state, the previously loaded state, should not affect what happens next.

[spyridon97]

I know that ParaView is not related to ADIOS, but in my mind, the config file was the equivalent of a state file in ParaView. The state file is used to initialize something, but when you do any change (addition or removal of filters) to the ParaView state, the previously loaded state φιλε, should not affect what happens next, e.g. at the creation of a new filter.