Change tests execution

[Vyacheslav-Smirnov]

Actually test suite should be executed via hpat.runtests:
python -u -m hpat.runtests -v
This resolve the issue with doulbe test suite execution
which occurs due to the "python -u -m unittest -v" command
import all files in tree including runtests.py and runtests.py
triggers 1-st suite execution. Then unittest triggers 2-d.

Add decorator to execute some tests (mostly parallel)
2 or more times (depending on existance of REPEAT_TEST_NUMBER
environment variable)
This should highlight issues like the test fails if executed twice
because is corrupts memory during first execution
(like test_series_head_index_parallel1)

Skip test_series_head_index_parallel1 because it triggers memory
corruption. This should be fixed.

[Vyacheslav-Smirnov]

@kozlov-alexey

[shssf]

May be it is better to implement this in runtests.py instead using decorator for each test?

[shssf]

I just don't like the idea that all tests have to be decorated in the same manner.

[shssf]

Need to find a way to avoid decorator for each test but keep the functionality proposed

[Vyacheslav-Smirnov]

Ok, I think I can add parallel tests in suite twice or wrap them in decorator in test_load in runtests.py
I will try this.

[shssf]

Please add new line at the end of the file

[shssf]

Do you still think we need a decorator for this? I'not sure but it looks like it could be done simpler (just by a loop).

[Vyacheslav-Smirnov]

Here the load_tests function returns the suite to be executed by unittest.
So I can add the test to the list multiple times instead of decorator, or now I think that there is more right, but more complicated way - overload loadTestsFromModule unittest method to make it return desired test collection or something like that.
Right now I do not see the other way.
What do you think about it?

[shssf]

I don't think that "overload loadTestsFromModule unittest method" is a good way. We need simpler solution. Need to think.
@dmitrii-zagornyi any ideas?

[shssf]

Also, I don't think we need to have subset of the tests which can be run several times. I think we can implement an ability to control all tests execution times. No need to get a sub set of tests for this.

[Vyacheslav-Smirnov]

My idea was to execute some of tests (for example only parallel due to there is a suspicion they can fails on the second time) but not whole tests. But looks like it is some tricky to implement)
If we want to execute all tests twice - maybe this can be done by adding all tests in suite twice:
hpat_tests = loader.loadTestsFromModule(hpat.tests)
suite.addTests(hpat_tests )
suite.addTests(lhpat_tests)

[shssf]

Please, use HPAT_ prefix for external variables.

[shssf]

Also, it might be better to use https://github.com/IntelPython/hpat/blob/master/hpat/config.py#L45 approach? It has default value and throw exception if wrong parameter (non digit) provided.