-
Notifications
You must be signed in to change notification settings - Fork 61
Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 #92
Conversation
@@ -309,14 +309,14 @@ static PyObject* csv_chunk_reader(std::istream * f, size_t fsz, bool is_parallel | |||
std::vector<size_t> line_offset = count_lines(f, hpat_dist_get_node_portion(fsz, nranks, rank)); | |||
size_t no_lines = line_offset.size(); | |||
// get total number of lines using allreduce | |||
size_t tot_no_lines(0); | |||
int64_t tot_no_lines = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code readability and consistence. I think this is more common way to initialize variables. From the compiler point of view, in this particular place, they are equal.
size_t byte_first_line = hpat_dist_exscan_i8(no_lines); | ||
size_t byte_last_line = byte_first_line + no_lines; | ||
int64_t byte_first_line = hpat_dist_exscan_i8(no_lines); | ||
int64_t byte_last_line = byte_first_line + no_lines; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do change to signed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because later it participated in comparison with signed input argument. In this case tne compiler show the warning.
@@ -377,8 +370,6 @@ void decode_utf8(const char *s, Py_ssize_t size, int* kind, int *is_ascii, int* | |||
goto End; | |||
} | |||
errmsg = "invalid continuation byte"; | |||
startinpos = s - starts; | |||
endinpos = startinpos + ch - 1; | |||
break; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How close is this code to the original CPython version?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure about CPython but these variables used nowhere.
Could you please describe the bug this is fixing? We have binaries using boost which seem to work. How can I reproduce the bug? |
On Ubuntu 16.04 with default GCC 5.4 in newly installed Anaconda 3 I see following:
This symbol |
* Remove spark dependency (#102) Remove spark dependency from HPA; use pre-generated sdf_dt.pq * explicitly adding data-file (#104) * HPAT Build: Code style check for C and Python sources (#103) * HPAT Build: Code style check for C and Python sources * PR103. Comments partially addressed * Code style change part 1 (#106) * Style check config fo pystyle (#105) * Fix for pandas.merge wrong overload handling of 'on' args (#99) Problem description: merge_overload and merge_asof_overload functions use 'on' argument value to compute 'left_on' and 'right_on' arguments in a way that breaks type stability, causing compilation failure when 'on' is assigned a StringLiteral value. Error: File "../hpat/hiframes/dataframe_pass.py", line 202, in _run_assign return self._run_call(assign, lhs, rhs) File "../hpat/hiframes/dataframe_pass.py", line 522, in _run_call return self._run_call_join(assign, lhs, rhs) File "../hpat/hiframes/dataframe_pass.py", line 1488, in _run_call_join left_on = self._get_const_or_list(left_on_var) File "../hpat/hiframes/dataframe_pass.py", line 2135, in _get_const_or_list raise ValueError(err_msg) ValueError: Failed in hpat mode pipeline (step: typed dataframe pass) None Following tests should be fixed with this commit: test_join_cat1 (hpat.tests.test_join.TestJoin) test_join_cat2 (hpat.tests.test_join.TestJoin) test_join_cat_parallel1 (hpat.tests.test_join.TestJoin) test_join_datetime_seq1 (hpat.tests.test_join.TestJoin) test_join_left_seq1 (hpat.tests.test_join.TestJoin) test_join_left_seq2 (hpat.tests.test_join.TestJoin) test_join_outer_seq1 (hpat.tests.test_join.TestJoin) test_join_right_seq1 (hpat.tests.test_join.TestJoin) test_merge_asof_seq1 (hpat.tests.test_join.TestJoin) * [STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (#97) * pep8 style for 'test_strings.py'; flake8 check successful * pep8 style for 'test_utils.py' * pep8 style for 'test_series.py'; more readable * fixed 'test_string_series' * removed extra white spaces * deleted mention of flake8 * trigger build * Code style change part 2 (#107) * code_style_change_part_2 * Add more check in style configuration (#108) * code_style_part_3 (#109) * Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (#92) * Code style change part 4 (#110) * Revert "Code style change part 4 (#110)" This reverts commit dfc54ee. * Revert "Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (#92)" This reverts commit 231a76c. * Revert "code_style_part_3 (#109)" This reverts commit 4070ce3. * Revert "Add more check in style configuration (#108)" This reverts commit abf5bd0. * Revert "Code style change part 2 (#107)" This reverts commit 9076493. * Revert "[STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (#97)" This reverts commit 8641f7a. * Revert "Fix for pandas.merge wrong overload handling of 'on' args (#99)" This reverts commit a2a8ee5. * Revert "Style check config fo pystyle (#105)" This reverts commit 551c0e3. * Revert "Code style change part 1 (#106)" This reverts commit 6dae0b3. * Revert "HPAT Build: Code style check for C and Python sources (#103)" This reverts commit 1a30e4f. * Revert "explicitly adding data-file (#104)" This reverts commit 34a2260. * Revert "Remove spark dependency (#102)" This reverts commit 9e77fde. * Unskip passing dataframe tests Actually, following tests in test_dataframe.py are pass and can be unskipped: test_create1 test_len1 test_column_getitem1 test_df_apply test_df_apply_branch test_df_describe test_count1 test_append1 At the same time, test_sort_parallel and test_sort_parallel_single_col has some problems with __pycache__: - They are passed if execute the suite with -B - They are passed if execute them separate - They and failed when some tests above are unskipped. So, decide to skip them.
* Remove spark dependency (#102) Remove spark dependency from HPA; use pre-generated sdf_dt.pq * explicitly adding data-file (#104) * HPAT Build: Code style check for C and Python sources (#103) * HPAT Build: Code style check for C and Python sources * PR103. Comments partially addressed * Code style change part 1 (#106) * Style check config fo pystyle (#105) * Fix for pandas.merge wrong overload handling of 'on' args (#99) Problem description: merge_overload and merge_asof_overload functions use 'on' argument value to compute 'left_on' and 'right_on' arguments in a way that breaks type stability, causing compilation failure when 'on' is assigned a StringLiteral value. Error: File "../hpat/hiframes/dataframe_pass.py", line 202, in _run_assign return self._run_call(assign, lhs, rhs) File "../hpat/hiframes/dataframe_pass.py", line 522, in _run_call return self._run_call_join(assign, lhs, rhs) File "../hpat/hiframes/dataframe_pass.py", line 1488, in _run_call_join left_on = self._get_const_or_list(left_on_var) File "../hpat/hiframes/dataframe_pass.py", line 2135, in _get_const_or_list raise ValueError(err_msg) ValueError: Failed in hpat mode pipeline (step: typed dataframe pass) None Following tests should be fixed with this commit: test_join_cat1 (hpat.tests.test_join.TestJoin) test_join_cat2 (hpat.tests.test_join.TestJoin) test_join_cat_parallel1 (hpat.tests.test_join.TestJoin) test_join_datetime_seq1 (hpat.tests.test_join.TestJoin) test_join_left_seq1 (hpat.tests.test_join.TestJoin) test_join_left_seq2 (hpat.tests.test_join.TestJoin) test_join_outer_seq1 (hpat.tests.test_join.TestJoin) test_join_right_seq1 (hpat.tests.test_join.TestJoin) test_merge_asof_seq1 (hpat.tests.test_join.TestJoin) * [STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (#97) * pep8 style for 'test_strings.py'; flake8 check successful * pep8 style for 'test_utils.py' * pep8 style for 'test_series.py'; more readable * fixed 'test_string_series' * removed extra white spaces * deleted mention of flake8 * trigger build * Code style change part 2 (#107) * code_style_change_part_2 * Add more check in style configuration (#108) * code_style_part_3 (#109) * Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (#92) * Code style change part 4 (#110) * Cahnge tests execution Actually test suite should be executed via hpat.runtests: python -u -m hpat.runtests -v This resolve the issue with doulbe test suite execution which occurs due to the "python -u -m unittest -v" command import all files in tree including runtests.py and runtests.py triggers 1-st suite execution. Then unittest triggers 2-d. Add decorator to execute some tests (mostly parallel) 2 or more times (depending on existance of REPEAT_TEST_NUMBER environment variable) This should highlight issues like the test fails if executed twice because is corrupts memory during first execution (like test_series_head_index_parallel1) Skip test_series_head_index_parallel1 because it triggers memory corruption. This should be fixed. * Revert "Code style change part 4 (#110)" This reverts commit dfc54ee. * Revert "Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (#92)" This reverts commit 231a76c. * Revert "code_style_part_3 (#109)" This reverts commit 4070ce3. * Revert "Add more check in style configuration (#108)" This reverts commit abf5bd0. * Revert "Code style change part 2 (#107)" This reverts commit 9076493. * Revert "[STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (#97)" This reverts commit 8641f7a. * Revert "Fix for pandas.merge wrong overload handling of 'on' args (#99)" This reverts commit a2a8ee5. * Revert "Style check config fo pystyle (#105)" This reverts commit 551c0e3. * Revert "Code style change part 1 (#106)" This reverts commit 6dae0b3. * Revert "HPAT Build: Code style check for C and Python sources (#103)" This reverts commit 1a30e4f. * Revert "explicitly adding data-file (#104)" This reverts commit 34a2260. * Revert "Remove spark dependency (#102)" This reverts commit 9e77fde. * Wrap functions to be executed twice in runtests.py * Update runtests.py Execute every test specified times, which is set via the REPEAT_TEST_NUMBER environment variable. Skip test_series_list_str_unbox1 because is fails on the second launch with Segmentation fault * Apply comments from review Rename REPEAT_TEST_NUMBER to HPAT_REPEAT_TEST_NUMBER Use os.getenv to get value for HPAT_REPEAT_TEST_NUMBER
* Remove spark dependency (IntelPython#102) Remove spark dependency from HPA; use pre-generated sdf_dt.pq * explicitly adding data-file (IntelPython#104) * HPAT Build: Code style check for C and Python sources (IntelPython#103) * HPAT Build: Code style check for C and Python sources * PR103. Comments partially addressed * Code style change part 1 (IntelPython#106) * Style check config fo pystyle (IntelPython#105) * Fix for pandas.merge wrong overload handling of 'on' args (IntelPython#99) Problem description: merge_overload and merge_asof_overload functions use 'on' argument value to compute 'left_on' and 'right_on' arguments in a way that breaks type stability, causing compilation failure when 'on' is assigned a StringLiteral value. Error: File "../hpat/hiframes/dataframe_pass.py", line 202, in _run_assign return self._run_call(assign, lhs, rhs) File "../hpat/hiframes/dataframe_pass.py", line 522, in _run_call return self._run_call_join(assign, lhs, rhs) File "../hpat/hiframes/dataframe_pass.py", line 1488, in _run_call_join left_on = self._get_const_or_list(left_on_var) File "../hpat/hiframes/dataframe_pass.py", line 2135, in _get_const_or_list raise ValueError(err_msg) ValueError: Failed in hpat mode pipeline (step: typed dataframe pass) None Following tests should be fixed with this commit: test_join_cat1 (hpat.tests.test_join.TestJoin) test_join_cat2 (hpat.tests.test_join.TestJoin) test_join_cat_parallel1 (hpat.tests.test_join.TestJoin) test_join_datetime_seq1 (hpat.tests.test_join.TestJoin) test_join_left_seq1 (hpat.tests.test_join.TestJoin) test_join_left_seq2 (hpat.tests.test_join.TestJoin) test_join_outer_seq1 (hpat.tests.test_join.TestJoin) test_join_right_seq1 (hpat.tests.test_join.TestJoin) test_merge_asof_seq1 (hpat.tests.test_join.TestJoin) * [STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (IntelPython#97) * pep8 style for 'test_strings.py'; flake8 check successful * pep8 style for 'test_utils.py' * pep8 style for 'test_series.py'; more readable * fixed 'test_string_series' * removed extra white spaces * deleted mention of flake8 * trigger build * Code style change part 2 (IntelPython#107) * code_style_change_part_2 * Add more check in style configuration (IntelPython#108) * code_style_part_3 (IntelPython#109) * Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (IntelPython#92) * Code style change part 4 (IntelPython#110) * Revert "Code style change part 4 (IntelPython#110)" This reverts commit dfc54ee. * Revert "Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (IntelPython#92)" This reverts commit 231a76c. * Revert "code_style_part_3 (IntelPython#109)" This reverts commit 4070ce3. * Revert "Add more check in style configuration (IntelPython#108)" This reverts commit abf5bd0. * Revert "Code style change part 2 (IntelPython#107)" This reverts commit 9076493. * Revert "[STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (IntelPython#97)" This reverts commit 8641f7a. * Revert "Fix for pandas.merge wrong overload handling of 'on' args (IntelPython#99)" This reverts commit a2a8ee5. * Revert "Style check config fo pystyle (IntelPython#105)" This reverts commit 551c0e3. * Revert "Code style change part 1 (IntelPython#106)" This reverts commit 6dae0b3. * Revert "HPAT Build: Code style check for C and Python sources (IntelPython#103)" This reverts commit 1a30e4f. * Revert "explicitly adding data-file (IntelPython#104)" This reverts commit 34a2260. * Revert "Remove spark dependency (IntelPython#102)" This reverts commit 9e77fde. * Unskip passing dataframe tests Actually, following tests in test_dataframe.py are pass and can be unskipped: test_create1 test_len1 test_column_getitem1 test_df_apply test_df_apply_branch test_df_describe test_count1 test_append1 At the same time, test_sort_parallel and test_sort_parallel_single_col has some problems with __pycache__: - They are passed if execute the suite with -B - They are passed if execute them separate - They and failed when some tests above are unskipped. So, decide to skip them.
* Remove spark dependency (IntelPython#102) Remove spark dependency from HPA; use pre-generated sdf_dt.pq * explicitly adding data-file (IntelPython#104) * HPAT Build: Code style check for C and Python sources (IntelPython#103) * HPAT Build: Code style check for C and Python sources * PR103. Comments partially addressed * Code style change part 1 (IntelPython#106) * Style check config fo pystyle (IntelPython#105) * Fix for pandas.merge wrong overload handling of 'on' args (IntelPython#99) Problem description: merge_overload and merge_asof_overload functions use 'on' argument value to compute 'left_on' and 'right_on' arguments in a way that breaks type stability, causing compilation failure when 'on' is assigned a StringLiteral value. Error: File "../hpat/hiframes/dataframe_pass.py", line 202, in _run_assign return self._run_call(assign, lhs, rhs) File "../hpat/hiframes/dataframe_pass.py", line 522, in _run_call return self._run_call_join(assign, lhs, rhs) File "../hpat/hiframes/dataframe_pass.py", line 1488, in _run_call_join left_on = self._get_const_or_list(left_on_var) File "../hpat/hiframes/dataframe_pass.py", line 2135, in _get_const_or_list raise ValueError(err_msg) ValueError: Failed in hpat mode pipeline (step: typed dataframe pass) None Following tests should be fixed with this commit: test_join_cat1 (hpat.tests.test_join.TestJoin) test_join_cat2 (hpat.tests.test_join.TestJoin) test_join_cat_parallel1 (hpat.tests.test_join.TestJoin) test_join_datetime_seq1 (hpat.tests.test_join.TestJoin) test_join_left_seq1 (hpat.tests.test_join.TestJoin) test_join_left_seq2 (hpat.tests.test_join.TestJoin) test_join_outer_seq1 (hpat.tests.test_join.TestJoin) test_join_right_seq1 (hpat.tests.test_join.TestJoin) test_merge_asof_seq1 (hpat.tests.test_join.TestJoin) * [STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (IntelPython#97) * pep8 style for 'test_strings.py'; flake8 check successful * pep8 style for 'test_utils.py' * pep8 style for 'test_series.py'; more readable * fixed 'test_string_series' * removed extra white spaces * deleted mention of flake8 * trigger build * Code style change part 2 (IntelPython#107) * code_style_change_part_2 * Add more check in style configuration (IntelPython#108) * code_style_part_3 (IntelPython#109) * Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (IntelPython#92) * Code style change part 4 (IntelPython#110) * Cahnge tests execution Actually test suite should be executed via hpat.runtests: python -u -m hpat.runtests -v This resolve the issue with doulbe test suite execution which occurs due to the "python -u -m unittest -v" command import all files in tree including runtests.py and runtests.py triggers 1-st suite execution. Then unittest triggers 2-d. Add decorator to execute some tests (mostly parallel) 2 or more times (depending on existance of REPEAT_TEST_NUMBER environment variable) This should highlight issues like the test fails if executed twice because is corrupts memory during first execution (like test_series_head_index_parallel1) Skip test_series_head_index_parallel1 because it triggers memory corruption. This should be fixed. * Revert "Code style change part 4 (IntelPython#110)" This reverts commit dfc54ee. * Revert "Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (IntelPython#92)" This reverts commit 231a76c. * Revert "code_style_part_3 (IntelPython#109)" This reverts commit 4070ce3. * Revert "Add more check in style configuration (IntelPython#108)" This reverts commit abf5bd0. * Revert "Code style change part 2 (IntelPython#107)" This reverts commit 9076493. * Revert "[STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (IntelPython#97)" This reverts commit 8641f7a. * Revert "Fix for pandas.merge wrong overload handling of 'on' args (IntelPython#99)" This reverts commit a2a8ee5. * Revert "Style check config fo pystyle (IntelPython#105)" This reverts commit 551c0e3. * Revert "Code style change part 1 (IntelPython#106)" This reverts commit 6dae0b3. * Revert "HPAT Build: Code style check for C and Python sources (IntelPython#103)" This reverts commit 1a30e4f. * Revert "explicitly adding data-file (IntelPython#104)" This reverts commit 34a2260. * Revert "Remove spark dependency (IntelPython#102)" This reverts commit 9e77fde. * Wrap functions to be executed twice in runtests.py * Update runtests.py Execute every test specified times, which is set via the REPEAT_TEST_NUMBER environment variable. Skip test_series_list_str_unbox1 because is fails on the second launch with Segmentation fault * Apply comments from review Rename REPEAT_TEST_NUMBER to HPAT_REPEAT_TEST_NUMBER Use os.getenv to get value for HPAT_REPEAT_TEST_NUMBER
boost::regex
cause runtime symbol resolution issue.