Windows compilation error with MSVC >= 14.28

[k-dominik]

I've been tracking compilation issues with VS2019 compilers (I was using 14.29, but I've found that the problem was most likely introduced in MSVC 14.28.

I haven't found a solution (yet) but also don't want to forget everything the next time I get to it (or someone else looks into it).

The symptom:

following compilation error:

in the end the error occurs in MSVC STL: algorithm(6928): error C3892: '_First': you cannot assign to a variable that is const

more details here, note I shortened the paths a little for readability:

..\BuildTools\VC\Tools\MSVC\14.29.30133\include\algorithm(6928): error C3892: '_First': you cannot assign to a variable that is const
...\BuildTools\VC\Tools\MSVC\14.29.30133\include\algorithm(7050): note: see reference to function template instantiation '_BidIt std::_Insertion_sort_unchecked<_RanIt,_Pr>(const _BidIt,const _BidIt,_Pr)' being compiled
        with
        [
            _BidIt=vigra::StridedScanOrderIterator<1,vigra::UInt8,vigra::UInt8 &,vigra::UInt8 *>,
            _RanIt=vigra::StridedScanOrderIterator<1,vigra::UInt8,vigra::UInt8 &,vigra::UInt8 *>,
            _Pr=std::less<void>
        ]
...\BuildTools\VC\Tools\MSVC\14.29.30133\include\algorithm(7080): note: see reference to function template instantiation 'void std::_Sort_unchecked<_RanIt,_Fn>(_RanIt,_RanIt,__int64,_Pr)' being compiled
        with
        [
            _RanIt=vigra::StridedScanOrderIterator<1,vigra::UInt8,vigra::UInt8 &,vigra::UInt8 *>,
            _Fn=std::less<void>,
            _Pr=std::less<void>
        ]
...\BuildTools\VC\Tools\MSVC\14.29.30133\include\algorithm(7085): note: see reference to function template instantiation 'void std::sort<_RanIt,std::less<void>>(const _RanIt,const _RanIt,_Pr)' being compiled
        with
        [
            _RanIt=vigra::StridedScanOrderIterator<1,vigra::UInt8,vigra::UInt8 &,vigra::UInt8 *>,
            _Pr=std::less<void>
        ]
...\vigra\vigranumpy\src\core\segmentation.cxx(1176): note: see reference to function template instantiation 'void std::sort<vigra::StridedScanOrderIterator<1,T,T &,T *>>(const _RanIt,const _RanIt)' being compiled
        with
        [
            T=vigra::UInt8,
            _RanIt=vigra::StridedScanOrderIterator<1,vigra::UInt8,vigra::UInt8 &,vigra::UInt8 *>
        ]
...\vigra\vigranumpy\src\core\segmentation.cxx(1181): note: see reference to function template instantiation 'vigra::NumpyAnyArray vigra::pythonUnique<T,1>(vigra::NumpyArray<1,vigra::Singleband<T>,vigra::StridedArrayTag>,bool)' being compiled
        with
        [
            T=npy_uint8
        ]
...\vigra\vigranumpy\src\core\segmentation.cxx(1181): note: see reference to function template instantiation 'void vigra::pyUniqueImpl<T,1,1>::def<Args>(const char *,const Args &)' being compiled
        with
        [
            T=npy_uint8,
            Args=boost::python::detail::keywords<2>
        ]
...\vigra\vigranumpy\src\core\segmentation.cxx(1181): note: see reference to function template instantiation 'void vigra::pyUniqueImpl<T,1,1>::def<Args>(const char *,const Args &)' being compiled
        with
        [
            T=npy_uint8,
            Args=boost::python::detail::keywords<2>
        ]
...\vigra\vigranumpy\src\core\segmentation.cxx(1181): note: see reference to function template instantiation 'void vigra::pyUniqueImpl<T1,1,5>::def<Args>(const char *,const Args &,const char *)' being compiled
        with
        [
            T1=npy_uint8,
            Args=boost::python::detail::keywords<2>
        ]
...\vigra\vigranumpy\src\core\segmentation.cxx(1181): note: see reference to function template instantiation 'void vigra::pyUniqueImpl<T1,1,5>::def<Args>(const char *,const Args &,const char *)' being compiled
        with
        [
            T1=npy_uint8,
            Args=boost::python::detail::keywords<2>
        ]
...\vigra\include\vigra/numpy_array_converters.hxx(803): note: see reference to function template instantiation 'void vigra::pyUnique<1,5,npy_uint8,npy_uint32,npy_uint64,npy_int64,void,void,void,void,void,void,void,void>::def<Args>(const char *,const Args &,const char *) const' being compiled
        with
        [
            Args=boost::python::detail::keywords<2>
        ]
...\vigra\include\vigra/numpy_array_converters.hxx(803): note: see reference to function template instantiation 'void vigra::pyUnique<1,5,npy_uint8,npy_uint32,npy_uint64,npy_int64,void,void,void,void,void,void,void,void>::def<Args>(const char *,const Args &,const char *) const' being compiled
        with
        [
            Args=boost::python::detail::keywords<2>
        ]
...\vigra\vigranumpy\src\core\segmentation.cxx(1618): note: see reference to function template instantiation 'void boost::python::multidef<vigra::pyUnique<1,5,npy_uint8,npy_uint32,npy_uint64,npy_int64,void,void,void,void,void,void,void,void>,boost::python::detail::keywords<2>>(const char *,const Functor &,const Args &,const char *)' being compiled
        with
        [
            Functor=vigra::pyUnique<1,5,npy_uint8,npy_uint32,npy_uint64,npy_int64,void,void,void,void,void,void,void,void>,
            Args=boost::python::detail::keywords<2>
        ]

I found a discussion related to the above error here: https://developercommunity.visualstudio.com/t/VS2019-1684---c-const-issue:-error-C/1320890#T-N1325280

Bottom line seems to be that the vigra iterator implementation is "wrong" and de-referencing a const iterator should not return a const reference.

I followed the trail a bit to

vigra/include/vigra/multi_iterator.hxx

Lines 312 to 315 in 574de57

	const_reference operator*() const
	{
	return this->template get<1>();
	}

and

vigra/include/vigra/multi_iterator_coupled.hxx

Lines 661 to 666 in 574de57

	template<unsigned int TARGET_INDEX>
	typename ConstReference<TARGET_INDEX>::type
	get() const
	{
	return vigra::get<TARGET_INDEX>(handles_);
	}

, and finally to

vigra/include/vigra/multi_handle.hxx

Lines 932 to 940 in 574de57

	/** Returns a constant reference to the element in the band of the handle with index TARGET_INDEX.
	*/
	template <unsigned int TARGET_INDEX, class Handle>
	inline
	typename CoupledHandleCast<TARGET_INDEX, Handle>::const_reference
	get(Handle const & handle)
	{
	return *cast<TARGET_INDEX>(handle);
	}

.

As far as I understand the problem, the solution should be removing the const from the returned reference. However, I didn't manage to do it in a way that would compile.

[k-dominik]

Looks like on cf even the python 3.11 vigra package is built with MSVC 14.16. I'll propose this as a workaround.

[hmaarrfk]

chatgpt says to look at the usage of std:sort

https://github.com/ukoethe/vigra/blob/master/vigranumpy/src/core/segmentation.cxx#L1176

It says that begin and end may be returning const iterators, which would be stopping sort from swapping values, but I can't find anything about this.

[hmaarrfk]

i'm somewhat confused, but it feel like the iterators are defined in:
https://github.com/ukoethe/vigra/blob/master/include/vigra/multi_array.hxx#L1923

vigra/include/vigra/multi_array.hxx

Line 755 in 502d5bc

typedef StridedScanOrderIterator<actual_dimension, T, T &, T *> iterator;

[k-dominik]

bummer, when I saw you recompiling I had the vague memory of fixing this - but apparently I haven't :/

[hmaarrfk]

I think we have to introduce cbegin and cend

[h-vetinari]

Problem remains also with VS2022, see conda-forge/vigra-feedstock#129

[hmaarrfk]

@a-detiste if you have any ideas here that would be great to see!

[hmaarrfk]

I think i finally figured it out.

The MultiArrayView needs to implement the dereferencing pointer for the sort call in unique to work.

#573 passes, but breaks the call to unique