Fix/assertion error #182

sanjayankur31 · 2022-03-09T08:43:08Z

Fixes an assertion error thrown when _GLIBCXX_ASSERTIONS is defined, and enables the option by default

With `-Wp,-D_GLIBCXX_ASSERTIONS`, tests fail with an assertion error of the form: ``` /usr/include/c++/12/bits/stl_vector.h:1122: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](size_type) [with _T p = float; _Alloc = std::allocator<float>; reference = float&; size_type = long unsigned int]: Assertion '__n < this->size()' failed. ``` The assertion failure happens on line 473 in report_reader.cpp: ``` std::copy(&buffer[gid_start], &buffer[gid_end], &data_ptr[offset]); ``` The problem is that `gid_end` can equal the size of `buffer`, and `&buffer[gid_end]` triggers the assertion failure in that case. Diagnosed and fixed suggested by Jerry James in the Fedora package review: https://bugzilla.redhat.com/show_bug.cgi?id=2061077#c6

https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_macros.html

matz-e

Seems sensible from my side, wondering if this will work on all compilers though.

sanjayankur31 · 2022-03-09T09:25:49Z

I'm not sure what the corresponding flags for clang are, but I see that the CI builds completed fine so perhaps these extra definitions are simply ignored?

sergiorg-hpc · 2022-03-09T13:10:53Z

src/report_reader.cpp

@@ -470,7 +470,7 @@ DataFrame<T> ReportReader<T>::Population::get(const nonstd::optional<Selection>&
                data_ptr[offset] = buffer[gid_start];
            } else {  // Elements report
                uint64_t gid_end = range.second - min;
-                std::copy(&buffer[gid_start], &buffer[gid_end], &data_ptr[offset]);
+                std::copy(buffer.begin() + gid_start, buffer.begin() + gid_end, &data_ptr[offset]);


Hmmmm... Do you mind to clarify why the change, please? I don't mind, but if we do change this, we should also change the code in line 470 and we must verify that the performance of the copies remains the same (e.g., this operation is executed thousands of times in thousands of timesteps, it matters!).

On a side note, perhaps using something like std::next(buffer.begin(), gid_start) as well, but this is irrelevant.

Turning on the _GLIBCXX_ASSERTIONS makes doing the subscript on buffer look like &buffer[buffer.size()], which would be an out of bounds error and thus undefined behavior. AFAIK, this is ok in this specific case as long as std::copy doesn't deref the pointer.

The change means we use the std::copy overload that takes iterators, and is "safer", because there is no chance there is an OOB deref, as long as gid_end never is larger than the size of the array.

The ASM will look the same from both of them, I'd expect, at a reasonable optimization level.

Thanks @mgeplf for clarifying. My point was that accessing the array with [] in std::copy might avoid boundary checks or additional code, but if you feel that it should be the same, then no complains at all.

I'd still prefer the use of std::next(buffer.begin(), gid_start) and std::next(buffer.begin(), gid_end) with iterators, although I don't mind leaving the code as it is with the + operator.

but if you feel that it should be the same, then no complains at all.

Had a quick look on godbolt to confirm (which I never take as conclusive), but it seems that compiler sees through this:
https://godbolt.org/z/8dez5nEM3

I'd still prefer the use of std::next(buffer.begin(), gid_start)

Ok, I'll make those changes, and another one to remove the pointer usage altogether.

I've made the changes; any/all reviews welcome.

All tests pass in my Fedora build with the newest changes 👍

mgeplf · 2022-03-10T09:49:20Z

src/report_reader.cpp

@@ -470,7 +470,7 @@ DataFrame<T> ReportReader<T>::Population::get(const nonstd::optional<Selection>&
                data_ptr[offset] = buffer[gid_start];
            } else {  // Elements report
                uint64_t gid_end = range.second - min;
-                std::copy(&buffer[gid_start], &buffer[gid_end], &data_ptr[offset]);
+                std::copy(buffer.begin() + gid_start, buffer.begin() + gid_end, &data_ptr[offset]);


Turning on the _GLIBCXX_ASSERTIONS makes doing the subscript on buffer look like &buffer[buffer.size()], which would be an out of bounds error and thus undefined behavior. AFAIK, this is ok in this specific case as long as std::copy doesn't deref the pointer.

The change means we use the std::copy overload that takes iterators, and is "safer", because there is no chance there is an OOB deref, as long as gid_end never is larger than the size of the array.

The ASM will look the same from both of them, I'd expect, at a reasonable optimization level.

CMakeLists.txt

* move _GLIBCXX_ASSERTIONS behind CMake option SONATA_CXX_WARNINGS * don't use off_t, use size_t instead * use iterator style access instead of raw pointers

sergiorg-hpc

Sorry for the delay, looks good to me 👍. Thank you @mgeplf and @sanjayankur31 for the improvements!

sanjayankur31 added 2 commits March 9, 2022 08:40

feat: define _GLIBCXX_ASSERTIONS to enable additional bounds checking

3edcc79

https://gcc.gnu.org/onlinedocs/libstdc++/manual/using_macros.html

pramodk requested a review from matz-e March 9, 2022 08:44

matz-e reviewed Mar 9, 2022

View reviewed changes

matz-e requested a review from sergiorg-hpc March 9, 2022 09:58

sergiorg-hpc requested a review from mgeplf March 9, 2022 13:07

sergiorg-hpc reviewed Mar 9, 2022

View reviewed changes

mgeplf reviewed Mar 10, 2022

View reviewed changes

updates

f98f068

* move _GLIBCXX_ASSERTIONS behind CMake option SONATA_CXX_WARNINGS * don't use off_t, use size_t instead * use iterator style access instead of raw pointers

sanjayankur31 requested a review from sergiorg-hpc March 15, 2022 17:59

sergiorg-hpc approved these changes Mar 16, 2022

View reviewed changes

sergiorg-hpc mentioned this pull request Mar 16, 2022

Add support for IO-blocks #183

Merged

mgeplf merged commit bfdf6f5 into BlueBrain:master Mar 17, 2022

sergiorg-hpc mentioned this pull request Mar 22, 2022

Update AUTHORS.txt #186

Merged

sergiorg-hpc added a commit that referenced this pull request Mar 22, 2022

Merge and adapt changes from #182

93fbd60

sergiorg-hpc added a commit that referenced this pull request Mar 22, 2022

Merge and adapt changes from #182

fa0cc30

sergiorg-hpc added a commit that referenced this pull request Mar 30, 2022

Merge and adapt changes from #182

d6c2679

sergiorg-hpc added a commit that referenced this pull request Mar 30, 2022

Merge and adapt changes from #182

4e1612e

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix/assertion error #182

Fix/assertion error #182

sanjayankur31 commented Mar 9, 2022 •

edited

matz-e left a comment

sanjayankur31 commented Mar 9, 2022

sergiorg-hpc Mar 9, 2022 •

edited

mgeplf Mar 10, 2022

sergiorg-hpc Mar 10, 2022

mgeplf Mar 10, 2022

mgeplf Mar 11, 2022

sanjayankur31 Mar 15, 2022

mgeplf Mar 10, 2022

sergiorg-hpc left a comment

Fix/assertion error #182

Fix/assertion error #182

Conversation

sanjayankur31 commented Mar 9, 2022 • edited

matz-e left a comment

Choose a reason for hiding this comment

sanjayankur31 commented Mar 9, 2022

sergiorg-hpc Mar 9, 2022 • edited

Choose a reason for hiding this comment

mgeplf Mar 10, 2022

Choose a reason for hiding this comment

sergiorg-hpc Mar 10, 2022

Choose a reason for hiding this comment

mgeplf Mar 10, 2022

Choose a reason for hiding this comment

mgeplf Mar 11, 2022

Choose a reason for hiding this comment

sanjayankur31 Mar 15, 2022

Choose a reason for hiding this comment

mgeplf Mar 10, 2022

Choose a reason for hiding this comment

sergiorg-hpc left a comment

Choose a reason for hiding this comment

sanjayankur31 commented Mar 9, 2022 •

edited

sergiorg-hpc Mar 9, 2022 •

edited