Add profile for skipping test compilation/resolution

[ryan-williams]

Would it be feasible to support a Maven profile that removes or omits test dependencies? I want to be able to build/install ADAM without resolving/building/installing the tests (while developing/iterating).

I have a workflow where I am mvn installing custom versions of ADAM (version "numbers" are actually git commit SHAs, using the codehaus versions-maven-plugin) in my local Maven cache and then depending on them in my Pageant POM.

I have patches to both ADAM and Spark that I need to use and this seems like the best way to do it.

I'm interested in maximally streamlining the process of building/installing ADAM when I make changes to ADAM, so that I can link against the newly-built version in other projects.

Right now, I run mvn install -DskipTests to install custom ADAM jars in my local Maven cache.

• (cf. the install-sha script in my fork, which uses -Plocal profile that cuts out the distribution module and the javadoc plugin because I don't care about those).

While this skips running of tests, it still resolves, compiles, and installs the tests / test JARs. This makes my process take noticeably longer, and feels wasteful/wrong on principle.

Attempting to skip compiling (and installing) the tests:

mvn install -Dmaven.test.skip -DskipTests

results in errors like:

[ERROR] Failed to execute goal on project adam-apis_2.10: Could not resolve dependencies for project org.bdgenomics.adam:adam-apis_2.10:jar:e3e7e41: Could not find artifact org.bdgenomics.adam:adam-core_2.10:jar:tests:e3e7e41 in spark-rc (https://repository.apache.org/content/repositories/orgapachespark-1112) -> [Help 1]

…because adam-apis depends on adam-core's tests JAR, even when it won't build/use any tests.

I was able to work-around this in Pageant:

• non-test deps are top-level,
• test deps live in a default profile, and
• a no-tests profile has empty <dependencies/>, effectively excluding the test dependencies.

I tried to do a similar thing in ADAM but dealing with the <module>s took me out of my Maven depth quickly; I don't really understand how the <version>-less dependency declarations in the modules' POMs relate to their corresponding definitions in the parent POM.

Anyone have thoughts on how this could be done, and whether it should?

[ryan-williams]

Bumping this as I still can't locally mvn install a -SNAPSHOT and sometimes want to test/use a forked version's functionality in a downstream application.

[fnothaft]

Fascinating! Totally missed this the first time around. I typically just run with -DskipTests. @heuermh can you take a look? If it's not hard to patch, might be reasonable to add a profile.

[ryan-williams]

hm mvn install -DskipTests seems to have just worked for me. I'm very confused by that; pretty sure that'd failed previously.

[ryan-williams]

OK I figured out the issue: mvn install -DskipTests works when you are on a SNAPSHOT version, because it can resolve test JARs that ADAM's existing SNAPSHOT-uploading infrastructure is uploading to Maven.

If you change the <version> of ADAM to be something new/unique (e.g. I'm currently using a git SHA 5bd2d7e of a branch I'm on), then this issue appears: it can't find the tests jar so it won't install, even when passing -DskipTests and/or -Dmaven.test.skip.

Maybe this is specific to my workflow, but what I'm trying to do seems pretty reasonable and I don't see any way to work around it; it seems like a weird bootstrapping problem: I can only mvn install ADAM core for versions where someone has already uploaded a distribution (including test jars) to Maven.

More about my use case:
I want to use a custom version instead of 0.19.1-SNAPSHOT because when I build Pageant it keeps fetching the latest 0.19.1-SNAPSHOT from Maven, overwriting the fork I'd purposely installed, due to Maven snapshot semantics, I guess? This seems to imply that I can tell it not to clobber snapshots; maybe I'll try that.

The other workaround is just to keep mvn installing 0.19.1-SNAPSHOT each day after I inadvertently clobber the one I'd installed the previous day.

[heuermh]

If you look in the Sonatype snapshots repository

https://oss.sonatype.org/content/repositories/snapshots/org/bdgenomics/adam/adam-core_2.10/0.19.1-SNAPSHOT/

you will see timestamped snapshot builds, e.g. adam-core_2.10-0.19.1-20160315.112717-201.jar. You can use these as dependencies, although they won't necessarily be there forever. The same goes for snapshot versions you mvn install to your local repository.

Another workaround is to tell Pageant to use offline mode mvn -o ... so that it won't check with remote snapshot repositories.

[fnothaft]

I'm going to close this as won't fix for now. Please reopen if this is still a pain point, @ryan-williams.