Skip to content

Commit

Permalink
Unskip passing dataframe tests (IntelPython#113)
Browse files Browse the repository at this point in the history 
* Remove spark dependency (IntelPython#102)

Remove spark dependency from HPA; use pre-generated sdf_dt.pq

* explicitly adding data-file (IntelPython#104)

* HPAT Build: Code style check for C and Python sources (IntelPython#103)

* HPAT Build: Code style check for C and Python sources

* PR103. Comments partially addressed

* Code style change part 1 (IntelPython#106)

* Style check config fo pystyle (IntelPython#105)

* Fix for pandas.merge wrong overload handling of 'on' args (IntelPython#99)

Problem description: merge_overload and merge_asof_overload functions
use 'on' argument value to compute 'left_on' and 'right_on' arguments
in a way that breaks type stability, causing compilation failure
when 'on' is assigned a StringLiteral value.

Error:
  File "../hpat/hiframes/dataframe_pass.py", line 202, in _run_assign
    return self._run_call(assign, lhs, rhs)
  File "../hpat/hiframes/dataframe_pass.py", line 522, in _run_call
    return self._run_call_join(assign, lhs, rhs)
  File "../hpat/hiframes/dataframe_pass.py", line 1488, in
_run_call_join
    left_on = self._get_const_or_list(left_on_var)
  File "../hpat/hiframes/dataframe_pass.py", line 2135, in
_get_const_or_list
    raise ValueError(err_msg)
ValueError: Failed in hpat mode pipeline (step: typed dataframe pass)
None

Following tests should be fixed with this commit:
    test_join_cat1 (hpat.tests.test_join.TestJoin)
    test_join_cat2 (hpat.tests.test_join.TestJoin)
    test_join_cat_parallel1 (hpat.tests.test_join.TestJoin)
    test_join_datetime_seq1 (hpat.tests.test_join.TestJoin)
    test_join_left_seq1 (hpat.tests.test_join.TestJoin)
    test_join_left_seq2 (hpat.tests.test_join.TestJoin)
    test_join_outer_seq1 (hpat.tests.test_join.TestJoin)
    test_join_right_seq1 (hpat.tests.test_join.TestJoin)
    test_merge_asof_seq1 (hpat.tests.test_join.TestJoin)

* [STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (IntelPython#97)

* pep8 style for 'test_strings.py'; flake8 check successful

* pep8 style for 'test_utils.py'

* pep8 style for 'test_series.py'; more readable

* fixed 'test_string_series'

* removed extra white spaces

* deleted mention of flake8

* trigger build

* Code style change part 2 (IntelPython#107)

* code_style_change_part_2

* Add more check in style configuration (IntelPython#108)

* code_style_part_3 (IntelPython#109)

* Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (IntelPython#92)

* Code style change part 4 (IntelPython#110)

* Revert "Code style change part 4 (IntelPython#110)"

This reverts commit dfc54ee.

* Revert "Fix boost runtime issue on Ubuntu16.04 with gcc 5.4 (IntelPython#92)"

This reverts commit 231a76c.

* Revert "code_style_part_3 (IntelPython#109)"

This reverts commit 4070ce3.

* Revert "Add more check in style configuration (IntelPython#108)"

This reverts commit abf5bd0.

* Revert "Code style change part 2 (IntelPython#107)"

This reverts commit 9076493.

* Revert "[STL] PEP8 code style for 'test_strings.py', 'test_utils.py', 'test_series.py' (IntelPython#97)"

This reverts commit 8641f7a.

* Revert "Fix for pandas.merge wrong overload handling of 'on' args (IntelPython#99)"

This reverts commit a2a8ee5.

* Revert "Style check config fo pystyle (IntelPython#105)"

This reverts commit 551c0e3.

* Revert "Code style change part 1 (IntelPython#106)"

This reverts commit 6dae0b3.

* Revert "HPAT Build: Code style check for C and Python sources (IntelPython#103)"

This reverts commit 1a30e4f.

* Revert "explicitly adding data-file (IntelPython#104)"

This reverts commit 34a2260.

* Revert "Remove spark dependency (IntelPython#102)"

This reverts commit 9e77fde.

* Unskip passing dataframe tests

Actually, following tests in test_dataframe.py are pass and can be unskipped:
test_create1
test_len1
test_column_getitem1
test_df_apply
test_df_apply_branch
test_df_describe
test_count1
test_append1

At the same time, test_sort_parallel and test_sort_parallel_single_col
has some problems with __pycache__:
 - They are passed if execute the suite with -B
 - They are passed if execute them separate
 - They and failed when some tests above are unskipped.
So, decide to skip them.
  • Loading branch information
@Vyacheslav-Smirnov @kozlov-alexey
Vyacheslav-Smirnov authored and kozlov-alexey committed Oct 4, 2019
1 parent 6f39bbc commit 8394d83
Showing 1 changed file with 2 additions and 8 deletions.
10 changes: 2 additions & 8 deletions hpat/tests/test_dataframe.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ def inner_get_column(df):

class TestDataFrame(unittest.TestCase):

@unittest.skip('Error - fix needed\n')
def test_create1(self):
def test_impl(n):
df = pd.DataFrame({'A': np.ones(n), 'B': np.random.ranf(n)})
Expand Down Expand Up @@ -131,7 +130,6 @@ def test_impl(n):
np.testing.assert_allclose(dist_sum(hres.A.sum()), res.A.sum())
np.testing.assert_allclose(dist_sum(hres.B.sum()), res.B.sum())

@unittest.skip('Error - fix needed\n')
def test_len1(self):
def test_impl(n):
df = pd.DataFrame({'A': np.ones(n, np.int64), 'B': np.random.ranf(n)})
Expand All @@ -154,7 +152,6 @@ def test_impl(n):
self.assertEqual(count_array_REPs(), 0)
self.assertEqual(count_parfor_REPs(), 0)

@unittest.skip('Error - fix needed\n')
def test_column_getitem1(self):
def test_impl(n):
df = pd.DataFrame({'A': np.ones(n), 'B': np.random.ranf(n)})
Expand Down Expand Up @@ -472,7 +469,6 @@ def test_impl(n):
self.assertEqual(count_array_REPs(), 0)
self.assertEqual(count_parfor_REPs(), 0)

@unittest.skip('Error - fix needed\n')
def test_df_apply(self):
def test_impl(n):
df = pd.DataFrame({'A': np.arange(n), 'B': np.arange(n)})
Expand All @@ -483,7 +479,6 @@ def test_impl(n):
hpat_func = hpat.jit(test_impl)
np.testing.assert_almost_equal(hpat_func(n), test_impl(n))

@unittest.skip('Error - fix needed\n')
def test_df_apply_branch(self):
def test_impl(n):
df = pd.DataFrame({'A': np.arange(n), 'B': np.arange(n)})
Expand All @@ -494,7 +489,6 @@ def test_impl(n):
hpat_func = hpat.jit(test_impl)
np.testing.assert_almost_equal(hpat_func(n), test_impl(n))

@unittest.skip('Error - fix needed\n')
def test_df_describe(self):
def test_impl(n):
df = pd.DataFrame({'A': np.arange(0, n, 1, np.float32),
Expand Down Expand Up @@ -582,6 +576,7 @@ def test_impl(df):
hpat_func = hpat.jit(test_impl)
self.assertTrue((hpat_func(df) == sorted_df.B.values).all())

@unittest.skip('Error - fix needed; issue is related to __pycache__\n')
def test_sort_parallel_single_col(self):
# create `kde.parquet` file
ParquetGenerator.gen_kde_pq()
Expand All @@ -604,6 +599,7 @@ def test_impl():
# restore global val
hpat.hiframes.sort.MIN_SAMPLES = save_min_samples

@unittest.skip('Error - fix needed; issue is related to __pycache__\n')
def test_sort_parallel(self):
# create `kde.parquet` file
ParquetGenerator.gen_kde_pq()
Expand Down Expand Up @@ -789,7 +785,6 @@ def test_impl(n):
n = 11
pd.testing.assert_series_equal(hpat_func(n), test_impl(n))

@unittest.skip('Error - fix needed\n')
def test_count1(self):
# TODO: non-numeric columns should be ignored automatically
def test_impl(n):
Expand Down Expand Up @@ -948,7 +943,6 @@ def test_impl(df):
df = pd.DataFrame({'A': np.arange(n), 'B': np.arange(n)**2})
pd.testing.assert_frame_equal(hpat_func(df), test_impl(df))

@unittest.skip('Error - fix needed\n')
def test_append1(self):
def test_impl(df, df2):
return df.append(df2, ignore_index=True)
Expand Down

0 comments on commit 8394d83

Please sign in to comment.