Add CHAINER_TEST_PAIRWISE_PARAMETERIZATION and enable it only in Travis CI

[niboshi]

See #8210

Adding CHAINER_TEST_PAIRWISE_PARAMETERIZATION environment variable and only set it to always in Travis CI.

[toslunar]

Looks good for introducing the env variable.

On moving the code to tests/conftest.py,

• I confirmed that PyTest (4.1.1) reads multiple conftest.py files.
• Could you fix the TODO comment on _product_dict_orig in chainer/testing/parameterized.py? ("is patched by tests/conftest.py")
• I'd like to hear ChainerMN team's comment on the pairwise testing.

[niboshi]

• Could you fix the TODO comment on _product_dict_orig in chainer/testing/parameterized.py? ("is patched by tests/conftest.py")

Fixed in ef3c7e2.

• I'd like to hear ChainerMN team's comment on the pairwise testing.

I don't think we should enable pairwise testing in chainermn_tests.
It only takes 1 minute in Travis CI.

[emcastillo]

LGTM

[emcastillo]

Jenkins, test this please

[chainer-ci]

Jenkins CI test (for commit ef3c7e2, target branch master) failed with status FAILURE.

[emcastillo]

Jenkins error might be related

[toslunar]

• I'd like to hear ChainerMN team's comment on the pairwise testing.

I don't think we should enable pairwise testing in chainermn_tests.
It only takes 1 minute in Travis CI.

I don't think it's needed. I meant that tests/conftest.py will enable it.

chainer/tests/chainermn_tests/functions_tests/test_pseudo_connect.py

Lines 13 to 17 in 3a184c8

	@testing.parameterize(*testing.product({
	'shape_x': [[(4, 5), (3, 2)], [(3, 2)], [()]],
	'shape_delegate': [(3, 2), ()],
	'dtype': [numpy.float16, numpy.float32, numpy.float64, chainer.mixed16],
	}))

[niboshi]

@toslunar
No, it's disabled by default.

[niboshi]

@emcastillo
The error should be fixed by #8217.

[toslunar]

@toslunar
No, it's disabled by default.

I see. Could you fix step_chainer_tests, too?

[niboshi]

@toslunar

I see. Could you fix step_chainer_tests, too?

Hm, do we need that?
Currently (only chainerx_tests) Travis takes 44 minutes, which has enough margin of 6 minutes.

Also if it were applied to chainer_tests, we wouldn't be able to test #8216.

[toslunar]

48 min 41 sec with macOS https://travis-ci.org/chainer/chainer/builds/592878722 where pairwise testing is applied to chainer and chainerx. How about fixing #8073 first?

[niboshi]

@toslunar I see. That sounds good.

[chainer-ci]

@emcastillo This pull-request is marked as st:test-and-merge, but there were no activities for the last 3 days. Could you check?

[chainer-ci]

@emcastillo This pull-request is marked as st:test-and-merge, but there were no activities for the last 3 days. Could you check?

[chainer-ci]

@emcastillo This pull-request is marked as st:test-and-merge, but there were no activities for the last 3 days. Could you check?

[emcastillo]

Jenkins, test this please

[chainer-ci]

Jenkins CI test (for commit f5db68b, target branch master) succeeded!