ND4J tests + Intellij IDEA

[raver119]

Intellij IDEA 2018.x has issues with ND4J tests, and it looks like it's not a bug.

https://github.com/deeplearning4j/nd4j/blob/12d63943d76c5152192d979beed03e39d9b1ab82/nd4j-backends/nd4j-backend-impls/pom.xml#L45
https://github.com/deeplearning4j/nd4j/blob/34fec21757695246b8a18a32812054f95b290cb0/nd4j-backends/nd4j-backend-impls/nd4j-native/pom.xml#L114
https://github.com/deeplearning4j/nd4j/blob/2138328469fc07bad0d5eed95112d73d9c445b72/nd4j-backends/nd4j-backend-impls/nd4j-cuda/pom.xml#L23

These maven properties are parsed, and imported into Intellij IDEA modules properties, causing problems with IDE.

@saudet Can we reconsider that somehow?

[saudet]

Which Maven properties?

[raver119]

click the links. they all point to the suspected property

…

On Tue, May 8, 2018 at 23:56 Samuel Audet ***@***.***> wrote: Which Maven properties? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5098 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALru_x8RMMx5WPF_hILrNbbUUzd2IzXaks5twpMjgaJpZM4T3rdf> .

[saudet]

testSourceDirectory? We can't do anything about that unless we don't share the tests..

[raver119]

Ok, we should start looking for another IDE then. Because in 2018.x after any reimport you must manually fix Intellij project

…

On Wed, May 9, 2018 at 00:02 Samuel Audet ***@***.***> wrote: testSourceDirectory? We can't do anything about that unless we don't share the tests.. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5098 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALru__8-bKwZ5qJ45MsskqZTb_YB2eqeks5twpR_gaJpZM4T3rdf> .

[AlexDBlack]

@raver119 what does this look like for you when it fails?
It seems like I'm unable to run nd4j tests in intellij anymore, not sure if it's the same issue or not.
Previously we set "use classpath of module" to nd4j-native.
Now I do that and simply get Class not found: "org.nd4j.linalg.Nd4jTestsC"Empty test suite.

[AlexDBlack]

OK, seems like this is still super-flaky... but I can now run tests with the following is a workaround:
(a) install Nd4J
(b) import nd4j-tests ONLY into intellij (separate project -> forces ND4J to use maven dependencies)
(c) Select and run clean/validate/compile under Maven Projects -> Lifecycle

Running them directly (i.e., opening more than 1 project) means I get errors related to ND4J initialization (missing native libraries).
Without the clean/validate/compile I get a ton of failed imports.

[raver119]

It's getting worse guys, now on top of original issue i can't use test resources in Intellij, if nd4j is open as separate project.

We need to get these issues fixed somehow, because it seriously slows down work flow.

[raver119]

deeplearning4j/pom.xml

Lines 538 to 551 in 0e66ef0

	<profile>
	<id>testresources</id>
	<activation>
	<activeByDefault>false</activeByDefault>
	</activation>
	<dependencies>
	<dependency>
	<groupId>org.deeplearning4j</groupId>
	<artifactId>dl4j-test-resources</artifactId>
	<version>${dl4j-test-resources.version}</version>
	<scope>test</scope>
	</dependency>
	</dependencies>
	</profile>

I think since this is declared in top-level pom.xml, Intellij just can't see it if you open nd4j as separate project.

[saudet]

@rajeev3983 That'd be a bug in IntelliJ, but we can do some copy/paste there, and if it works, we can leave it like that. It wouldn't hurt other tools.

[raver119]

Ye, there's definitely something goes on in Intellij, they do some refinements... But since we have no other viable IDEs - we should adopt somehow 😄

[raver119]

deeplearning4j/nd4j/pom.xml

Lines 403 to 416 in e97090b

	<profile>
	<id>nd4j-testresources</id>
	<activation>
	<activeByDefault>false</activeByDefault>
	</activation>
	<dependencies>
	<dependency>
	<groupId>org.deeplearning4j</groupId>
	<artifactId>dl4j-test-resources</artifactId>
	<version>${dl4j-test-resources.version}</version>
	<scope>test</scope>
	</dependency>
	</dependencies>
	</profile>

This copypasta seems to work fine, nd4j-tests work again in latest Intellij. Is it acceptable workaround for CI/CD compatibility? @saudet @sshepel

[raver119]

We've switched to profiles to address this issue.