BP5 Block selection on array of LocalValues reads wrong block and reads too much

[pnorbert]

Describe the bug
The examples/basics/values example creates a variable ProcessID, which is written as a LocalValue on each process, and which is presented as a global 1D array at reading. bpls dumps the value of each block properly:

$ ./bin/bpls values.bp/ -lDd ProcessID
  int32_t   ProcessID  {4} = 0 / 3
        step 0:
          block 0: [0:0] = 0 / 0
    (0)    0
          block 1: [1:1] = 1 / 1
    (0)    1
          block 2: [2:2] = 2 / 2
    (0)    2
          block 3: [3:3] = 3 / 3
    (0)    3

The following example reads this variable block-by-block, and the content read in is always the first block (0) instead of the expected block. Note that this approach is expected to work and this example works fine on LocalArrays and GlobalArrays constructed of actual arrays per process (not from LocalValues).

#include <algorithm>
#include <iostream>
#include <vector>

#include <adios2.h>

int main(int argc, char *argv[])
{
    std::string fname;
    adios2::ADIOS adios;
    if (argc > 1)
    {
        fname = argv[1];
    }
    else
    {
        std::cout << "Usage: bug_blocks values.bp" << std::endl;
        return 1;
    }
    adios2::IO io = adios.DeclareIO("Input");
    adios2::Engine reader = io.Open(fname, adios2::Mode::ReadRandomAccess);
    using T = int32_t;
    adios2::Variable<T> variable = io.InquireVariable<T>("ProcessID");
    for (size_t step = 0; step < variable.Steps(); ++step)
    {
        const std::vector<typename adios2::Variable<T>::Info> &blocks = reader.BlocksInfo(variable, step);
        size_t nblocks = blocks.size();
        std::cout << "Process step " << step << " nblocks = " << nblocks << std::endl;
        variable.SetStepSelection({step, 1});
        for (size_t b = 0; b < nblocks; ++b)
        {
            auto &bi = blocks[b];
            variable.SetBlockSelection(b);
            size_t nelems = bi.Count[0];
            std::vector<T> vec(nelems);
            reader.Get(variable, vec.data(), adios2::Mode::Sync);
            auto bounds = std::minmax_element(begin(vec), end(vec));
            T min = *bounds.first;
            T max = *bounds.second;

            std::cout << " Block " << b << " min = " << bi.Min << " max = " << bi.Max << " data min = " << min << " max = " << max
                      << std::endl;
            if (min != bi.Min || max != bi.Max)
            {
                std::cout << "ERROR: Variable " << variable.Name() << " step " << step << " block " << b
                          << " mismatch. Minimum write time = " << bi.Min << " read time = " << min
                          << " Maximum write time = " << bi.Max << " read time = " << max << std::endl;
                std::cout << "  [ ";
                for (auto e : vec)
                {
                    std::cout << e << " ";
                }
                std::cout << "]" << std::endl;
            }
        }
    }

    reader.Close();
    return 0;
}

The result is:

$ ./bug_blocks ~/ADIOS2/build.debug/values.bp/
Process step 0 nblocks = 4
 Block 0 min = 0 max = 0 data min = 0 max = 0
 Block 1 min = 1 max = 1 data min = 0 max = 0
ERROR: Variable ProcessID step 0 block 1 mismatch. Minimum write time = 1 read time = 0 Maximum write time = 1 read time = 0
  [ 0 ]
 Block 2 min = 2 max = 2 data min = 0 max = 0
ERROR: Variable ProcessID step 0 block 2 mismatch. Minimum write time = 2 read time = 0 Maximum write time = 2 read time = 0
  [ 0 ]
 Block 3 min = 3 max = 3 data min = 0 max = 0
ERROR: Variable ProcessID step 0 block 3 mismatch. Minimum write time = 3 read time = 0 Maximum write time = 3 read time = 0
  [ 0 ]

The bpls code treats ProcessID as a global array and uses SetSelection to read a single element from the array. This example is using SetBlockSelection to read a given block, processing the variable as a LocalArray (which should be working for all arrays).

The problem is that in BP5Deserializer::QueueGet() does not consider block selections, and gets misguided at line 1229 to process the first writer.

Even worse, then the BP5Deserializer::QueueGetSingle() call will read in all values/blocks in a loop at line 1315 and copies into the result buffer that is only allocated for a single element, not for all elements.

[eisenhauer]

Looking at this now...