Test failure with 1.8.0 (1300-awkward-to-cpp-converter-with-cling)

[kgizdov]

Version of Awkward Array

1.8.0

Description and code to reproduce

Test failures on Arch Linux x86_64, GCC 11.2.0, ROOT 6.26.00:

# test_1300-awkward-to-cpp-converter-with-cling.py
=================================== FAILURES ===================================
______________________ test_IndexedOptionArray_NumpyArray ______________________

    def test_IndexedOptionArray_NumpyArray():
        v2a = ak._v2.contents.indexedoptionarray.IndexedOptionArray(
            ak._v2.index.Index(np.array([2, 2, -1, 1, -1, 5, 4], np.int64)),
            ak._v2.contents.numpyarray.NumpyArray(np.array([0.0, 1.1, 2.2, 3.3, 4.4, 5.5])),
        )

        layout = v2a
        generator = ak._v2._connect.cling.togenerator(layout.form)
        lookup = ak._v2._lookup.Lookup(layout)
        generator.generate(compiler)

        ROOT.gInterpreter.Declare(
            f"""
    void roottest_IndexedOptionArray_NumpyArray_v2a(double* out, ssize_t length, ssize_t* ptrs) {{
      auto obj = {generator.entry()};
      out[0] = obj.size();
      out[1] = obj[0].has_value() ? obj[0].value() : 999.0;
      out[2] = obj[1].has_value() ? obj[1].value() : 999.0;
      out[3] = obj[2].has_value() ? obj[2].value() : 999.0;
      out[4] = obj[3].has_value() ? obj[3].value() : 999.0;
      out[5] = obj[4].has_value() ? obj[4].value() : 999.0;
      out[6] = obj[5].has_value() ? obj[5].value() : 999.0;
      out[7] = obj[6].has_value() ? obj[6].value() : 999.0;
    }}
    """
        )
        out = np.zeros(8, dtype=np.float64)
>       ROOT.roottest_IndexedOptionArray_NumpyArray_v2a(out, len(layout), lookup.arrayptrs)

generator  = <awkward._v2._connect.cling.IndexedOptionArrayGenerator object at 0x7fd3bc350fd0>
layout     = <IndexedOptionArray len='7'>
    <index><Index dtype='int64' len='7'>[ 2  2 -1  1 -1  5  4]</Index></index>
    <content><NumpyArray dtype='float64' len='6'>[0.  1.1 2.2 3.3 4.4 5.5]</NumpyArray></content>
</IndexedOptionArray>
lookup     = <awkward._v2._lookup.Lookup object at 0x7fd3bc350cd0>
out        = array([0., 0., 0., 0., 0., 0., 0., 0.])
v2a        = <IndexedOptionArray len='7'>
    <index><Index dtype='int64' len='7'>[ 2  2 -1  1 -1  5  4]</Index></index>
    <content><NumpyArray dtype='float64' len='6'>[0.  1.1 2.2 3.3 4.4 5.5]</NumpyArray></content>
</IndexedOptionArray>

tests/v2/test_1300-awkward-to-cpp-converter-with-cling.py:398:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>
name = 'roottest_IndexedOptionArray_NumpyArray_v2a'

    def _fallback_getattr(self, name):
        # Try:
        # - in the global namespace
        # - in the ROOT namespace
        # - in gROOT (ROOT lists such as list of files,
        #   memory mapped files, functions, geometries ecc.)
        # The first two attempts allow to lookup
        # e.g. ROOT.ROOT.Math as ROOT.Math

        if name == '__all__':
            self._handle_import_all()
            # Make the attributes of the facade be injected in the
            # caller module
            raise AttributeError()
        # Note that hasattr caches the lookup for getattr
        elif hasattr(gbl_namespace, name):
            return getattr(gbl_namespace, name)
        elif hasattr(gbl_namespace.ROOT, name):
            return getattr(gbl_namespace.ROOT, name)
        else:
            res = gROOT.FindObject(name)
            if res:
                return res
>       raise AttributeError("Failed to get attribute {} from ROOT".format(name))
E       AttributeError: Failed to get attribute roottest_IndexedOptionArray_NumpyArray_v2a from ROOT

name       = 'roottest_IndexedOptionArray_NumpyArray_v2a'
res        = <cppyy.gbl.TObject object at 0x(nil)>
self       = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>

/usr/lib/python3.10/site-packages/ROOT/_facade.py:195: AttributeError
----------------------------- Captured stderr call -----------------------------
input_line_88:7:18: error: no template named 'optional' in namespace 'std'
    typedef std::optional<double> value_type;
            ~~~~~^
input_line_88:25:28: error: type 'awkward::NumpyArray_float64_hzDE4rkLkhw::value_type' (aka 'double') cannot be narrowed to 'awkward::IndexedOptionArray_9nKQxf4wwdw::value_type' (aka 'int') in initializer list [-Wc++11-narrowing]
        return value_type{ NumpyArray_float64_hzDE4rkLkhw(index, index + 1, ptrs_[which_ + 2], ptrs_)[0] };
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input_line_88:25:28: note: insert an explicit cast to silence this issue
        return value_type{ NumpyArray_float64_hzDE4rkLkhw(index, index + 1, ptrs_[which_ + 2], ptrs_)[0] };
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                           static_cast<value_type>(                                                     )
input_line_88:28:21: error: no member named 'nullopt' in namespace 'std'
        return std::nullopt;
               ~~~~~^
input_line_89:3:23: error: no member named 'IndexedOptionArray_9nKQxf4wwdw' in namespace 'awkward'
  auto obj = awkward::IndexedOptionArray_9nKQxf4wwdw(0, length, 0, ptrs);
             ~~~~~~~~~^
_______________________ test_ByteMaskedArray_NumpyArray ________________________

    def test_ByteMaskedArray_NumpyArray():
        v2a = ak._v2.contents.bytemaskedarray.ByteMaskedArray(
            ak._v2.index.Index(np.array([1, 0, 1, 0, 1], np.int8)),
            ak._v2.contents.numpyarray.NumpyArray(np.array([1.1, 2.2, 3.3, 4.4, 5.5, 6.6])),
            valid_when=True,
        )

        layout = v2a
        generator = ak._v2._connect.cling.togenerator(layout.form)
        lookup = ak._v2._lookup.Lookup(layout)
        generator.generate(compiler)

        ROOT.gInterpreter.Declare(
            f"""
    void roottest_ByteMaskedArray_NumpyArray_v2a(double* out, ssize_t length, ssize_t* ptrs) {{
      auto obj = {generator.entry()};
      out[0] = obj.size();
      out[1] = obj[0].has_value() ? obj[0].value() : 999.0;
      out[2] = obj[1].has_value() ? obj[1].value() : 999.0;
      out[3] = obj[2].has_value() ? obj[2].value() : 999.0;
      out[4] = obj[3].has_value() ? obj[3].value() : 999.0;
      out[5] = obj[4].has_value() ? obj[4].value() : 999.0;
    }}
    """
        )
        out = np.zeros(6, dtype=np.float64)
>       ROOT.roottest_ByteMaskedArray_NumpyArray_v2a(out, len(layout), lookup.arrayptrs)

generator  = <awkward._v2._connect.cling.ByteMaskedArrayGenerator object at 0x7fd3bc5c2ad0>
layout     = <ByteMaskedArray valid_when='true' len='5'>
    <mask><Index dtype='int8' len='5'>[1 0 1 0 1]</Index></mask>
    <content><NumpyArray dtype='float64' len='6'>[1.1 2.2 3.3 4.4 5.5 6.6]</NumpyArray></content>
</ByteMaskedArray>
lookup     = <awkward._v2._lookup.Lookup object at 0x7fd3bc5c3cd0>
out        = array([0., 0., 0., 0., 0., 0.])
v2a        = <ByteMaskedArray valid_when='true' len='5'>
    <mask><Index dtype='int8' len='5'>[1 0 1 0 1]</Index></mask>
    <content><NumpyArray dtype='float64' len='6'>[1.1 2.2 3.3 4.4 5.5 6.6]</NumpyArray></content>
</ByteMaskedArray>

tests/v2/test_1300-awkward-to-cpp-converter-with-cling.py:428:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>
name = 'roottest_ByteMaskedArray_NumpyArray_v2a'

    def _fallback_getattr(self, name):
        # Try:
        # - in the global namespace
        # - in the ROOT namespace
        # - in gROOT (ROOT lists such as list of files,
        #   memory mapped files, functions, geometries ecc.)
        # The first two attempts allow to lookup
        # e.g. ROOT.ROOT.Math as ROOT.Math

        if name == '__all__':
            self._handle_import_all()
            # Make the attributes of the facade be injected in the
            # caller module
            raise AttributeError()
        # Note that hasattr caches the lookup for getattr
        elif hasattr(gbl_namespace, name):
            return getattr(gbl_namespace, name)
        elif hasattr(gbl_namespace.ROOT, name):
            return getattr(gbl_namespace.ROOT, name)
        else:
            res = gROOT.FindObject(name)
            if res:
                return res
>       raise AttributeError("Failed to get attribute {} from ROOT".format(name))
E       AttributeError: Failed to get attribute roottest_ByteMaskedArray_NumpyArray_v2a from ROOT

name       = 'roottest_ByteMaskedArray_NumpyArray_v2a'
res        = <cppyy.gbl.TObject object at 0x(nil)>
self       = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>

/usr/lib/python3.10/site-packages/ROOT/_facade.py:195: AttributeError
----------------------------- Captured stderr call -----------------------------
input_line_90:7:18: error: no template named 'optional' in namespace 'std'
    typedef std::optional<double> value_type;
            ~~~~~^
input_line_90:25:28: error: type 'awkward::NumpyArray_float64_hzDE4rkLkhw::value_type' (aka 'double') cannot be narrowed to 'awkward::ByteMaskedArray_w1nWecjWHw::value_type' (aka 'int') in initializer list [-Wc++11-narrowing]
        return value_type{ NumpyArray_float64_hzDE4rkLkhw(start_, stop_, ptrs_[which_ + 2], ptrs_)[at] };
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input_line_90:25:28: note: insert an explicit cast to silence this issue
        return value_type{ NumpyArray_float64_hzDE4rkLkhw(start_, stop_, ptrs_[which_ + 2], ptrs_)[at] };
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                           static_cast<value_type>(                                                   )
input_line_90:28:21: error: no member named 'nullopt' in namespace 'std'
        return std::nullopt;
               ~~~~~^
input_line_91:3:23: error: no member named 'ByteMaskedArray_w1nWecjWHw' in namespace 'awkward'
  auto obj = awkward::ByteMaskedArray_w1nWecjWHw(0, length, 0, ptrs);
             ~~~~~~~~~^
________________________ test_BitMaskedArray_NumpyArray ________________________

    def test_BitMaskedArray_NumpyArray():
        v2a = ak._v2.contents.bitmaskedarray.BitMaskedArray(
            ak._v2.index.Index(
                np.packbits(
                    np.array(
                        [
                            1,
                            1,
                            1,
                            1,
                            0,
                            0,
                            0,
                            0,
                            1,
                            0,
                            1,
                            0,
                            1,
                        ],
                        np.uint8,
                    )
                )
            ),
            ak._v2.contents.numpyarray.NumpyArray(
                np.array(
                    [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 1.1, 2.2, 3.3, 4.4, 5.5, 6.6]
                )
            ),
            valid_when=True,
            length=13,
            lsb_order=False,
        )

        layout = v2a
        generator = ak._v2._connect.cling.togenerator(layout.form)
        lookup = ak._v2._lookup.Lookup(layout)
        generator.generate(compiler)

        ROOT.gInterpreter.Declare(
            f"""
    void roottest_BitMaskedArray_NumpyArray_v2a(double* out, ssize_t length, ssize_t* ptrs) {{
      auto obj = {generator.entry()};
      out[0] = obj.size();
      out[1] = obj[0].has_value() ? obj[0].value() : 999.0;
      out[2] = obj[1].has_value() ? obj[1].value() : 999.0;
      out[3] = obj[2].has_value() ? obj[2].value() : 999.0;
      out[4] = obj[3].has_value() ? obj[3].value() : 999.0;
      out[5] = obj[4].has_value() ? obj[4].value() : 999.0;
      out[6] = obj[5].has_value() ? obj[5].value() : 999.0;
      out[7] = obj[6].has_value() ? obj[6].value() : 999.0;
      out[8] = obj[7].has_value() ? obj[7].value() : 999.0;
      out[9] = obj[8].has_value() ? obj[8].value() : 999.0;
      out[10] = obj[9].has_value() ? obj[9].value() : 999.0;
      out[11] = obj[10].has_value() ? obj[10].value() : 999.0;
      out[12] = obj[11].has_value() ? obj[11].value() : 999.0;
      out[13] = obj[12].has_value() ? obj[12].value() : 999.0;
    }}
    """
        )
        out = np.zeros(14, dtype=np.float64)
>       ROOT.roottest_BitMaskedArray_NumpyArray_v2a(out, len(layout), lookup.arrayptrs)

generator  = <awkward._v2._connect.cling.BitMaskedArrayGenerator object at 0x7fd3bc545ab0>
layout     = <BitMaskedArray valid_when='true' lsb_order='false' len='13'>
    <mask><Index dtype='uint8' len='2'>[240 168]</Index>...n='14'>
        [0.  1.  2.  3.  4.  5.  6.  7.  1.1 2.2 3.3 4.4 5.5 6.6]
    </NumpyArray></content>
</BitMaskedArray>
lookup     = <awkward._v2._lookup.Lookup object at 0x7fd3bc546170>
out        = array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
v2a        = <BitMaskedArray valid_when='true' lsb_order='false' len='13'>
    <mask><Index dtype='uint8' len='2'>[240 168]</Index>...n='14'>
        [0.  1.  2.  3.  4.  5.  6.  7.  1.1 2.2 3.3 4.4 5.5 6.6]
    </NumpyArray></content>
</BitMaskedArray>

tests/v2/test_1300-awkward-to-cpp-converter-with-cling.py:521:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>
name = 'roottest_BitMaskedArray_NumpyArray_v2a'

    def _fallback_getattr(self, name):
        # Try:
        # - in the global namespace
        # - in the ROOT namespace
        # - in gROOT (ROOT lists such as list of files,
        #   memory mapped files, functions, geometries ecc.)
        # The first two attempts allow to lookup
        # e.g. ROOT.ROOT.Math as ROOT.Math

        if name == '__all__':
            self._handle_import_all()
            # Make the attributes of the facade be injected in the
            # caller module
            raise AttributeError()
        # Note that hasattr caches the lookup for getattr
        elif hasattr(gbl_namespace, name):
            return getattr(gbl_namespace, name)
        elif hasattr(gbl_namespace.ROOT, name):
            return getattr(gbl_namespace.ROOT, name)
        else:
            res = gROOT.FindObject(name)
            if res:
                return res
>       raise AttributeError("Failed to get attribute {} from ROOT".format(name))
E       AttributeError: Failed to get attribute roottest_BitMaskedArray_NumpyArray_v2a from ROOT

name       = 'roottest_BitMaskedArray_NumpyArray_v2a'
res        = <cppyy.gbl.TObject object at 0x(nil)>
self       = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>

/usr/lib/python3.10/site-packages/ROOT/_facade.py:195: AttributeError
----------------------------- Captured stderr call -----------------------------
input_line_92:7:18: error: no template named 'optional' in namespace 'std'
    typedef std::optional<double> value_type;
            ~~~~~^
input_line_92:30:28: error: type 'awkward::NumpyArray_float64_hzDE4rkLkhw::value_type' (aka 'double') cannot be narrowed to 'awkward::BitMaskedArray_m2IqalSsoEE::value_type' (aka 'int') in initializer list [-Wc++11-narrowing]
        return value_type{ NumpyArray_float64_hzDE4rkLkhw(start_, stop_, ptrs_[which_ + 3], ptrs_)[at] };
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input_line_92:30:28: note: insert an explicit cast to silence this issue
        return value_type{ NumpyArray_float64_hzDE4rkLkhw(start_, stop_, ptrs_[which_ + 3], ptrs_)[at] };
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                           static_cast<value_type>(                                                   )
input_line_92:33:21: error: no member named 'nullopt' in namespace 'std'
        return std::nullopt;
               ~~~~~^
input_line_93:3:23: error: no member named 'BitMaskedArray_m2IqalSsoEE' in namespace 'awkward'
  auto obj = awkward::BitMaskedArray_m2IqalSsoEE(0, length, 0, ptrs);
             ~~~~~~~~~^
________________________ test_UnmaskedArray_NumpyArray _________________________

    def test_UnmaskedArray_NumpyArray():
        v2a = ak._v2.contents.unmaskedarray.UnmaskedArray(
            ak._v2.contents.numpyarray.NumpyArray(np.array([0.0, 1.1, 2.2, 3.3]))
        )

        layout = v2a
        generator = ak._v2._connect.cling.togenerator(layout.form)
        lookup = ak._v2._lookup.Lookup(layout)
        generator.generate(compiler)

        ROOT.gInterpreter.Declare(
            f"""
    void roottest_UnmaskedArray_NumpyArray_v2a(double* out, ssize_t length, ssize_t* ptrs) {{
      auto obj = {generator.entry()};
      out[0] = obj.size();
      out[1] = obj[1].has_value() ? obj[1].value() : 999.0;
      out[2] = obj[3].has_value() ? obj[3].value() : 999.0;
    }}
    """
        )
        out = np.zeros(3, dtype=np.float64)
>       ROOT.roottest_UnmaskedArray_NumpyArray_v2a(out, len(layout), lookup.arrayptrs)

generator  = <awkward._v2._connect.cling.UnmaskedArrayGenerator object at 0x7fd3bb031180>
layout     = <UnmaskedArray len='4'>
    <content><NumpyArray dtype='float64' len='4'>[0.  1.1 2.2 3.3]</NumpyArray></content>
</UnmaskedArray>
lookup     = <awkward._v2._lookup.Lookup object at 0x7fd3bb0305e0>
out        = array([0., 0., 0.])
v2a        = <UnmaskedArray len='4'>
    <content><NumpyArray dtype='float64' len='4'>[0.  1.1 2.2 3.3]</NumpyArray></content>
</UnmaskedArray>

tests/v2/test_1300-awkward-to-cpp-converter-with-cling.py:801:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>
name = 'roottest_UnmaskedArray_NumpyArray_v2a'

    def _fallback_getattr(self, name):
        # Try:
        # - in the global namespace
        # - in the ROOT namespace
        # - in gROOT (ROOT lists such as list of files,
        #   memory mapped files, functions, geometries ecc.)
        # The first two attempts allow to lookup
        # e.g. ROOT.ROOT.Math as ROOT.Math

        if name == '__all__':
            self._handle_import_all()
            # Make the attributes of the facade be injected in the
            # caller module
            raise AttributeError()
        # Note that hasattr caches the lookup for getattr
        elif hasattr(gbl_namespace, name):
            return getattr(gbl_namespace, name)
        elif hasattr(gbl_namespace.ROOT, name):
            return getattr(gbl_namespace.ROOT, name)
        else:
            res = gROOT.FindObject(name)
            if res:
                return res
>       raise AttributeError("Failed to get attribute {} from ROOT".format(name))
E       AttributeError: Failed to get attribute roottest_UnmaskedArray_NumpyArray_v2a from ROOT

name       = 'roottest_UnmaskedArray_NumpyArray_v2a'
res        = <cppyy.gbl.TObject object at 0x(nil)>
self       = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>

/usr/lib/python3.10/site-packages/ROOT/_facade.py:195: AttributeError
----------------------------- Captured stderr call -----------------------------
input_line_94:7:18: error: no template named 'optional' in namespace 'std'
    typedef std::optional<double> value_type;
            ~~~~~^
input_line_94:23:26: error: type 'awkward::NumpyArray_float64_hzDE4rkLkhw::value_type' (aka 'double') cannot be narrowed to 'awkward::UnmaskedArray_tgRoEa6pI4::value_type' (aka 'int') in initializer list [-Wc++11-narrowing]
      return value_type{ NumpyArray_float64_hzDE4rkLkhw(start_, stop_, ptrs_[which_ + 1], ptrs_)[at] };
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input_line_94:23:26: note: insert an explicit cast to silence this issue
      return value_type{ NumpyArray_float64_hzDE4rkLkhw(start_, stop_, ptrs_[which_ + 1], ptrs_)[at] };
                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                         static_cast<value_type>(                                                   )
input_line_95:3:23: error: no member named 'UnmaskedArray_tgRoEa6pI4' in namespace 'awkward'
  auto obj = awkward::UnmaskedArray_tgRoEa6pI4(0, length, 0, ptrs);
             ~~~~~~~~~^
__________________ test_nested_IndexedOptionArray_NumpyArray ___________________

    def test_nested_IndexedOptionArray_NumpyArray():
        v2a = ak._v2.contents.ListOffsetArray(
            ak._v2.index.Index64(np.array([0, 1, 8], dtype=np.int64)),
            ak._v2.contents.indexedoptionarray.IndexedOptionArray(
                ak._v2.index.Index(np.array([999, 2, 2, -1, 1, -1, 5, 4], np.int64)),
                ak._v2.contents.numpyarray.NumpyArray(
                    np.array([0.0, 1.1, 2.2, 3.3, 4.4, 5.5])
                ),
            ),
        )

        layout = v2a
        generator = ak._v2._connect.cling.togenerator(layout.form)
        lookup = ak._v2._lookup.Lookup(layout)
        generator.generate(compiler)

        ROOT.gInterpreter.Declare(
            f"""
    void roottest_nested_IndexedOptionArray_NumpyArray_v2a(double* out, ssize_t length, ssize_t* ptrs) {{
      auto obj = {generator.entry()}[1];
      out[0] = obj.size();
      out[1] = obj[0].has_value() ? obj[0].value() : 999.0;
      out[2] = obj[1].has_value() ? obj[1].value() : 999.0;
      out[3] = obj[2].has_value() ? obj[2].value() : 999.0;
      out[4] = obj[3].has_value() ? obj[3].value() : 999.0;
      out[5] = obj[4].has_value() ? obj[4].value() : 999.0;
      out[6] = obj[5].has_value() ? obj[5].value() : 999.0;
      out[7] = obj[6].has_value() ? obj[6].value() : 999.0;
    }}
    """
        )
        out = np.zeros(8, dtype=np.float64)
>       ROOT.roottest_nested_IndexedOptionArray_NumpyArray_v2a(
            out, len(layout), lookup.arrayptrs
        )

generator  = <awkward._v2._connect.cling.ListArrayGenerator object at 0x7fd3bc418340>
layout     = <ListOffsetArray len='2'>
    <offsets><Index dtype='int64' len='3'>[0 1 8]</Index></offsets>
    <content><IndexedOpt...loat64' len='6'>[0.  1.1 2.2 3.3 4.4 5.5]</NumpyArray></content>
    </IndexedOptionArray></content>
</ListOffsetArray>
lookup     = <awkward._v2._lookup.Lookup object at 0x7fd3bc419990>
out        = array([0., 0., 0., 0., 0., 0., 0., 0.])
v2a        = <ListOffsetArray len='2'>
    <offsets><Index dtype='int64' len='3'>[0 1 8]</Index></offsets>
    <content><IndexedOpt...loat64' len='6'>[0.  1.1 2.2 3.3 4.4 5.5]</NumpyArray></content>
    </IndexedOptionArray></content>
</ListOffsetArray>

tests/v2/test_1300-awkward-to-cpp-converter-with-cling.py:1253:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>
name = 'roottest_nested_IndexedOptionArray_NumpyArray_v2a'

    def _fallback_getattr(self, name):
        # Try:
        # - in the global namespace
        # - in the ROOT namespace
        # - in gROOT (ROOT lists such as list of files,
        #   memory mapped files, functions, geometries ecc.)
        # The first two attempts allow to lookup
        # e.g. ROOT.ROOT.Math as ROOT.Math

        if name == '__all__':
            self._handle_import_all()
            # Make the attributes of the facade be injected in the
            # caller module
            raise AttributeError()
        # Note that hasattr caches the lookup for getattr
        elif hasattr(gbl_namespace, name):
            return getattr(gbl_namespace, name)
        elif hasattr(gbl_namespace.ROOT, name):
            return getattr(gbl_namespace.ROOT, name)
        else:
            res = gROOT.FindObject(name)
            if res:
                return res
>       raise AttributeError("Failed to get attribute {} from ROOT".format(name))
E       AttributeError: Failed to get attribute roottest_nested_IndexedOptionArray_NumpyArray_v2a from ROOT

name       = 'roottest_nested_IndexedOptionArray_NumpyArray_v2a'
res        = <cppyy.gbl.TObject object at 0x(nil)>
self       = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>

/usr/lib/python3.10/site-packages/ROOT/_facade.py:195: AttributeError
----------------------------- Captured stderr call -----------------------------
input_line_143:7:13: error: unknown type name 'IndexedOptionArray_9nKQxf4wwdw'
    typedef IndexedOptionArray_9nKQxf4wwdw value_type;
            ^
input_line_143:25:14: error: excess elements in scalar initializer
      return value_type(start, stop, ptrs_[which_ + 3], ptrs_);
             ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input_line_144:3:23: error: no member named 'ListArray_K2QNKBAvj9k' in namespace 'awkward'
  auto obj = awkward::ListArray_K2QNKBAvj9k(0, length, 0, ptrs)[1];
             ~~~~~~~~~^
____________________ test_nested_ByteMaskedArray_NumpyArray ____________________

    def test_nested_ByteMaskedArray_NumpyArray():
        v2a = ak._v2.contents.ListOffsetArray(
            ak._v2.index.Index64(np.array([0, 1, 6], dtype=np.int64)),
            ak._v2.contents.bytemaskedarray.ByteMaskedArray(
                ak._v2.index.Index(np.array([123, 1, 0, 1, 0, 1], np.int8)),
                ak._v2.contents.numpyarray.NumpyArray(
                    np.array([999, 1.1, 2.2, 3.3, 4.4, 5.5, 6.6])
                ),
                valid_when=True,
            ),
        )

        layout = v2a
        generator = ak._v2._connect.cling.togenerator(layout.form)
        lookup = ak._v2._lookup.Lookup(layout)
        generator.generate(compiler)

        ROOT.gInterpreter.Declare(
            f"""
    void roottest_nested_ByteMaskedArray_NumpyArray_v2a(double* out, ssize_t length, ssize_t* ptrs) {{
      auto obj = {generator.entry()}[1];
      out[0] = obj.size();
      out[1] = obj[0].has_value() ? obj[0].value() : 999.0;
      out[2] = obj[1].has_value() ? obj[1].value() : 999.0;
      out[3] = obj[2].has_value() ? obj[2].value() : 999.0;
      out[4] = obj[3].has_value() ? obj[3].value() : 999.0;
      out[5] = obj[4].has_value() ? obj[4].value() : 999.0;
    }}
    """
        )
        out = np.zeros(6, dtype=np.float64)
>       ROOT.roottest_nested_ByteMaskedArray_NumpyArray_v2a(
            out, len(layout), lookup.arrayptrs
        )

generator  = <awkward._v2._connect.cling.ListArrayGenerator object at 0x7fd3cd2ffd30>
layout     = <ListOffsetArray len='2'>
    <offsets><Index dtype='int64' len='3'>[0 1 6]</Index></offsets>
    <content><ByteMasked... 1.1   2.2   3.3   4.4   5.5   6.6]
        </NumpyArray></content>
    </ByteMaskedArray></content>
</ListOffsetArray>
lookup     = <awkward._v2._lookup.Lookup object at 0x7fd3bcd07f40>
out        = array([0., 0., 0., 0., 0., 0.])
v2a        = <ListOffsetArray len='2'>
    <offsets><Index dtype='int64' len='3'>[0 1 6]</Index></offsets>
    <content><ByteMasked... 1.1   2.2   3.3   4.4   5.5   6.6]
        </NumpyArray></content>
    </ByteMaskedArray></content>
</ListOffsetArray>

tests/v2/test_1300-awkward-to-cpp-converter-with-cling.py:1290:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>
name = 'roottest_nested_ByteMaskedArray_NumpyArray_v2a'

    def _fallback_getattr(self, name):
        # Try:
        # - in the global namespace
        # - in the ROOT namespace
        # - in gROOT (ROOT lists such as list of files,
        #   memory mapped files, functions, geometries ecc.)
        # The first two attempts allow to lookup
        # e.g. ROOT.ROOT.Math as ROOT.Math

        if name == '__all__':
            self._handle_import_all()
            # Make the attributes of the facade be injected in the
            # caller module
            raise AttributeError()
        # Note that hasattr caches the lookup for getattr
        elif hasattr(gbl_namespace, name):
            return getattr(gbl_namespace, name)
        elif hasattr(gbl_namespace.ROOT, name):
            return getattr(gbl_namespace.ROOT, name)
        else:
            res = gROOT.FindObject(name)
            if res:
                return res
>       raise AttributeError("Failed to get attribute {} from ROOT".format(name))
E       AttributeError: Failed to get attribute roottest_nested_ByteMaskedArray_NumpyArray_v2a from ROOT

name       = 'roottest_nested_ByteMaskedArray_NumpyArray_v2a'
res        = <cppyy.gbl.TObject object at 0x(nil)>
self       = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>

/usr/lib/python3.10/site-packages/ROOT/_facade.py:195: AttributeError
----------------------------- Captured stderr call -----------------------------
input_line_145:7:13: error: unknown type name 'ByteMaskedArray_w1nWecjWHw'
    typedef ByteMaskedArray_w1nWecjWHw value_type;
            ^
input_line_145:25:14: error: excess elements in scalar initializer
      return value_type(start, stop, ptrs_[which_ + 3], ptrs_);
             ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input_line_146:3:23: error: no member named 'ListArray_V0gvVn41fs' in namespace 'awkward'
  auto obj = awkward::ListArray_V0gvVn41fs(0, length, 0, ptrs)[1];
             ~~~~~~~~~^
____________________ test_nested_BitMaskedArray_NumpyArray _____________________

    def test_nested_BitMaskedArray_NumpyArray():
        v2a = ak._v2.contents.ListOffsetArray(
            ak._v2.index.Index64(np.array([0, 1, 14], dtype=np.int64)),
            ak._v2.contents.bitmaskedarray.BitMaskedArray(
                ak._v2.index.Index(
                    np.packbits(
                        np.array(
                            [
                                0,
                                1,
                                1,
                                1,
                                1,
                                0,
                                0,
                                0,
                                0,
                                1,
                                0,
                                1,
                                0,
                                1,
                            ],
                            np.uint8,
                        )
                    )
                ),
                ak._v2.contents.numpyarray.NumpyArray(
                    np.array(
                        [
                            999,
                            0.0,
                            1.0,
                            2.0,
                            3.0,
                            4.0,
                            5.0,
                            6.0,
                            7.0,
                            1.1,
                            2.2,
                            3.3,
                            4.4,
                            5.5,
                            6.6,
                        ]
                    )
                ),
                valid_when=True,
                length=14,
                lsb_order=False,
            ),
        )

        layout = v2a
        generator = ak._v2._connect.cling.togenerator(layout.form)
        lookup = ak._v2._lookup.Lookup(layout)
        generator.generate(compiler)

        ROOT.gInterpreter.Declare(
            f"""
    void roottest_nested_BitMaskedArray_NumpyArray_v2a(double* out, ssize_t length, ssize_t* ptrs) {{
      auto obj = {generator.entry()}[1];
      out[0] = obj.size();
      out[1] = obj[0].has_value() ? obj[0].value() : 999.0;
      out[2] = obj[1].has_value() ? obj[1].value() : 999.0;
      out[3] = obj[2].has_value() ? obj[2].value() : 999.0;
      out[4] = obj[3].has_value() ? obj[3].value() : 999.0;
      out[5] = obj[4].has_value() ? obj[4].value() : 999.0;
      out[6] = obj[5].has_value() ? obj[5].value() : 999.0;
      out[7] = obj[6].has_value() ? obj[6].value() : 999.0;
      out[8] = obj[7].has_value() ? obj[7].value() : 999.0;
      out[9] = obj[8].has_value() ? obj[8].value() : 999.0;
      out[10] = obj[9].has_value() ? obj[9].value() : 999.0;
      out[11] = obj[10].has_value() ? obj[10].value() : 999.0;
      out[12] = obj[11].has_value() ? obj[11].value() : 999.0;
      out[13] = obj[12].has_value() ? obj[12].value() : 999.0;
    }}
    """
        )
        out = np.zeros(14, dtype=np.float64)
>       ROOT.roottest_nested_BitMaskedArray_NumpyArray_v2a(
            out, len(layout), lookup.arrayptrs
        )

generator  = <awkward._v2._connect.cling.ListArrayGenerator object at 0x7fd3bc7dbd90>
layout     = <ListOffsetArray len='2'>
    <offsets><Index dtype='int64' len='3'>[ 0  1 14]</Index></offsets>
    <content><BitMask...              3.3   4.4   5.5   6.6]
        </NumpyArray></content>
    </BitMaskedArray></content>
</ListOffsetArray>
lookup     = <awkward._v2._lookup.Lookup object at 0x7fd3bc7d80d0>
out        = array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
v2a        = <ListOffsetArray len='2'>
    <offsets><Index dtype='int64' len='3'>[ 0  1 14]</Index></offsets>
    <content><BitMask...              3.3   4.4   5.5   6.6]
        </NumpyArray></content>
    </BitMaskedArray></content>
</ListOffsetArray>

tests/v2/test_1300-awkward-to-cpp-converter-with-cling.py:1412:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>
name = 'roottest_nested_BitMaskedArray_NumpyArray_v2a'

    def _fallback_getattr(self, name):
        # Try:
        # - in the global namespace
        # - in the ROOT namespace
        # - in gROOT (ROOT lists such as list of files,
        #   memory mapped files, functions, geometries ecc.)
        # The first two attempts allow to lookup
        # e.g. ROOT.ROOT.Math as ROOT.Math

        if name == '__all__':
            self._handle_import_all()
            # Make the attributes of the facade be injected in the
            # caller module
            raise AttributeError()
        # Note that hasattr caches the lookup for getattr
        elif hasattr(gbl_namespace, name):
            return getattr(gbl_namespace, name)
        elif hasattr(gbl_namespace.ROOT, name):
            return getattr(gbl_namespace.ROOT, name)
        else:
            res = gROOT.FindObject(name)
            if res:
                return res
>       raise AttributeError("Failed to get attribute {} from ROOT".format(name))
E       AttributeError: Failed to get attribute roottest_nested_BitMaskedArray_NumpyArray_v2a from ROOT

name       = 'roottest_nested_BitMaskedArray_NumpyArray_v2a'
res        = <cppyy.gbl.TObject object at 0x(nil)>
self       = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>

/usr/lib/python3.10/site-packages/ROOT/_facade.py:195: AttributeError
----------------------------- Captured stderr call -----------------------------
input_line_147:7:13: error: unknown type name 'BitMaskedArray_m2IqalSsoEE'
    typedef BitMaskedArray_m2IqalSsoEE value_type;
            ^
input_line_147:25:14: error: excess elements in scalar initializer
      return value_type(start, stop, ptrs_[which_ + 3], ptrs_);
             ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input_line_148:3:23: error: no member named 'ListArray_5fHu1BwL0I' in namespace 'awkward'
  auto obj = awkward::ListArray_5fHu1BwL0I(0, length, 0, ptrs)[1];
             ~~~~~~~~~^
_____________________ test_nested_UnmaskedArray_NumpyArray _____________________

    def test_nested_UnmaskedArray_NumpyArray():
        v2a = ak._v2.contents.ListOffsetArray(
            ak._v2.index.Index64(np.array([0, 1, 5], dtype=np.int64)),
            ak._v2.contents.unmaskedarray.UnmaskedArray(
                ak._v2.contents.numpyarray.NumpyArray(np.array([999, 0.0, 1.1, 2.2, 3.3]))
            ),
        )

        layout = v2a
        generator = ak._v2._connect.cling.togenerator(layout.form)
        lookup = ak._v2._lookup.Lookup(layout)
        generator.generate(compiler)

        ROOT.gInterpreter.Declare(
            f"""
    void roottest_nested_UnmaskedArray_NumpyArray_v2a(double* out, ssize_t length, ssize_t* ptrs) {{
      auto obj = {generator.entry()}[1];
      out[0] = obj.size();
      out[1] = obj[1].has_value() ? obj[1].value() : 999.0;
      out[2] = obj[3].has_value() ? obj[3].value() : 999.0;
    }}
    """
        )
        out = np.zeros(3, dtype=np.float64)
>       ROOT.roottest_nested_UnmaskedArray_NumpyArray_v2a(
            out, len(layout), lookup.arrayptrs
        )

generator  = <awkward._v2._connect.cling.ListArrayGenerator object at 0x7fd3bca6f2e0>
layout     = <ListOffsetArray len='2'>
    <offsets><Index dtype='int64' len='3'>[0 1 5]</Index></offsets>
    <content><UnmaskedAr...      [999.    0.    1.1   2.2   3.3]
        </NumpyArray></content>
    </UnmaskedArray></content>
</ListOffsetArray>
lookup     = <awkward._v2._lookup.Lookup object at 0x7fd3bca6d300>
out        = array([0., 0., 0.])
v2a        = <ListOffsetArray len='2'>
    <offsets><Index dtype='int64' len='3'>[0 1 5]</Index></offsets>
    <content><UnmaskedAr...      [999.    0.    1.1   2.2   3.3]
        </NumpyArray></content>
    </UnmaskedArray></content>
</ListOffsetArray>

tests/v2/test_1300-awkward-to-cpp-converter-with-cling.py:1763:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>
name = 'roottest_nested_UnmaskedArray_NumpyArray_v2a'

    def _fallback_getattr(self, name):
        # Try:
        # - in the global namespace
        # - in the ROOT namespace
        # - in gROOT (ROOT lists such as list of files,
        #   memory mapped files, functions, geometries ecc.)
        # The first two attempts allow to lookup
        # e.g. ROOT.ROOT.Math as ROOT.Math

        if name == '__all__':
            self._handle_import_all()
            # Make the attributes of the facade be injected in the
            # caller module
            raise AttributeError()
        # Note that hasattr caches the lookup for getattr
        elif hasattr(gbl_namespace, name):
            return getattr(gbl_namespace, name)
        elif hasattr(gbl_namespace.ROOT, name):
            return getattr(gbl_namespace.ROOT, name)
        else:
            res = gROOT.FindObject(name)
            if res:
                return res
>       raise AttributeError("Failed to get attribute {} from ROOT".format(name))
E       AttributeError: Failed to get attribute roottest_nested_UnmaskedArray_NumpyArray_v2a from ROOT

name       = 'roottest_nested_UnmaskedArray_NumpyArray_v2a'
res        = <cppyy.gbl.TObject object at 0x(nil)>
self       = <module 'ROOT' from '/usr/lib/python3.10/site-packages/ROOT/__init__.py'>

/usr/lib/python3.10/site-packages/ROOT/_facade.py:195: AttributeError
----------------------------- Captured stderr call -----------------------------
input_line_149:7:13: error: unknown type name 'UnmaskedArray_tgRoEa6pI4'
    typedef UnmaskedArray_tgRoEa6pI4 value_type;
            ^
input_line_149:25:14: error: excess elements in scalar initializer
      return value_type(start, stop, ptrs_[which_ + 3], ptrs_);
             ^               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
input_line_150:3:23: error: no member named 'ListArray_d2MxjWTgr6k' in namespace 'awkward'
  auto obj = awkward::ListArray_d2MxjWTgr6k(0, length, 0, ptrs)[1];
             ~~~~~~~~~^

[jpivarski]

Oh: your ROOT wasn't compiled with C++17 support, so when this test tries to JIT-compile std::optional, it's not in ROOT.

(I'm only recognizing this as the error because it happened to me before I recompiled my ROOT, then it didn't occur to me that having these in as tests would run into the same problem for somebody else.)

Offhand, do you know of a way to determine from ROOT which C++ revision it uses?

[kgizdov]

I don't think that's possible - I enforce C++17 on the Arch Linux ROOT package - you can find the PKGBUILD here.

On your second question - I think you can ask the root compiler to spit out __cplusplus or get it from some ROOT header. Also, the CMake config files should contain the relevant info (at least they do on Arch). Otherwise, I'd have to dig in.

[jpivarski]

@kgizdov, in your ROOT environment (PyROOT, actually), what do you get when you try this:

>>> import ROOT
>>> hasattr(ROOT.std, "optional")
True

I don't know how to ask for __cplusplus. (I have to get this in Python to make a decision to skip those tests.)

[jpivarski]

So this line should ensure C++17:

https://github.com/archlinux/svntogit-community/blob/a6014d2660207c8a3dfba61c5df18404f90d4bf2/trunk/settings.cmake#L10

In mine, I had to pass -DCMAKE_CXX_STANDARD=17 as a CMake commandline argument, which looks like it's equivalent to your line.

Anyway, what's failing in the tests is that it can't find optional in namespace std, so I'm adding

@pytest.mark.skipif(not cpp17, reason="ROOT was compiled without C++17 support")

to those four tests with

cpp17 = hasattr(ROOT.std, "optional")

Even if it's a misnomer, if PyROOT doesn't find "optional" in ROOT.std, then it would have the same trouble finding optional in namespace std. At least it should be cutting on the symptom.

But I'd really like to know, on your system that doesn't find optional in namespace std in gInterpreter.Declare, whether "optional" is in ROOT.std in PyROOT. That would demonstrate that this fix is going to work.

[kgizdov]

Indeed, optional is not found, however, the reason for this is yet unknown - I can confirm that in all the places C++17 is selected and listed correctly at compile-time.

[jpivarski]

Then it might be a misnomer, but the important thing is whether std::optional exists.

I had thought that ROOT would take the latest C++ standard available by a compiler, because it's only limiting what code can be compiled with that ROOT (since C++ standards are backward-compatible, so far). I was surprised that I had to explicitly tell it.