Skip to content
This repository has been archived by the owner on Jan 8, 2021. It is now read-only.

JACE benchmarking: Dockerfile update #12

Merged
merged 12 commits into from
Feb 8, 2017
Merged

Conversation

sbesson
Copy link
Member

@sbesson sbesson commented Feb 6, 2017

Follow-up of #5, this PR updates the Dockerfile in order to be able to build and execute the JACE bindings in the Ubuntu 16.04 environment.

A couple of changes are still in progress:

  • the base image should be replaced by the organization one once https://github.com/sbesson/ome-files-cpp-u1604-docker is updated to build the current OME Files
  • the benchmarking scripts will execute all 3 components (OME Files C++, JACE, Bio-Formats)
  • modifies javaTools.cpp to add the JAR folder to the CLASSPATH and detect the logging configuration file (could be replaced by using another signature of createJVM with extra classpath)

- Use temporary base image (with OME Files 0.3.0)
- Install required JDK7 for JACE bindings and update JAVA_HOME
- Build JACE bindings and add BF_JACE_HOME
- Update LD_LIBRARY_PATH for runtime benchmarking
@sbesson
Copy link
Member Author

sbesson commented Feb 6, 2017

@rleigh-codelibre with the last set of commits, I can build the various components but I encounter failures at runtime of type

root@af3155d9ad6b:/git/ome-files-performance# ./scripts/run_benchmarking 
rm: cannot remove '/data/results/*': No such file or directory
rm: cannot remove '/data/out/*': No such file or directory
Exception creating JVM: Assert failed: Unable to find the class, jace.util.ShutdownHook.
WARNING in native method: JNI call made with exception pending
terminate called after throwing an instance of 'jace::proxy::java::lang::Throwable'
  what():  java.lang.NoClassDefFoundError: loci/formats/services/OMEXMLServiceImpl
Aborted (core dumped)

Probable something related to the environment. Will investigate tomorrow unless you have some ideas.

@rleigh-codelibre
Copy link
Contributor

rleigh-codelibre commented Feb 6, 2017

@sbesson You do need to copy the jar directory from the jace build into the current working directory when running the tests. (There may be a cleaner way of doing things, but this is what I found to work.)

@sbesson
Copy link
Member Author

sbesson commented Feb 7, 2017

With the latest commits, I can successfully run the JACE tests in the run_benchmarking scripts against the two filesets (BBBC and mitocheck). Next todo: try to fix the logging initialization which is currently printed unconditionally at DEBUG level.

@sbesson
Copy link
Member Author

sbesson commented Feb 8, 2017

Last commit includes an update of the statistics notebook with the latest results of the Linux/Windows builds using a recent version of the development OME Files. A few notes:

  • this does not include the recent C++ performance improvements (Improve TIFF write performance ome-files-cpp#54 and ome-xml: Performance improvements ome-model#28)
  • in most cases, the JACE times are similar to the Java times. The only notable exception is the BBBC writing where JACE is significantly slower. Is that a scaling limitation related to the ByteArray handling?
  • for BBBC the raw JACE pixels table only includes 1 recorded point and for mitocheck the JACE generated OME-TIFF is 3 times larger than the Java/C++ equivalent. I suspect we need to delete the file at the beginning of the test similarly to 5fd90c5.

@@ -64,7 +64,7 @@ find_package(OMECompat 5.2.0 REQUIRED)
find_package(OMECommon 5.2.0 REQUIRED)
find_package(OMEXML 5.2.0 REQUIRED)
find_package(OMEFiles 0.2.3 REQUIRED)
find_package(BioFormatsJACE 5.2.4)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't we need to revert this to get the correct header versioning?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the issue was related to the JACE discovery and the CMAKE_PATH_PREFIX variable. After removing the sed workaround, I was able to rebuild everything and execute the JACE scripts.


WORKDIR /build2
COPY cmake/JACEPrerequisites.cmake /git/bio-formats-jace/target/cppwrap
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs renaming as well; the target is /git/bio-formats-jace/target/cppwrap/Prerequisites.cmake?

@rleigh-codelibre
Copy link
Contributor

Mostly looks fine I think; a couple of points noted.

@sbesson
Copy link
Member Author

sbesson commented Feb 8, 2017

Discussed with @rleigh-codelibre. A couple of issues noted:

  • JACE benchmarking: Dockerfile update #12 (comment) was mostly required for Windows (not working), I will remove the copy step in a follow-up PR
  • similar to Java, we need to remove the file before running the pixels performance tests if it exists. currently, the BBBC tests crash as the data is appended to the existing TIFF and overcomes the BigTIFF limit.
  • additionally I will add a set -e step in the benchmarking script to fail on error

@sbesson sbesson merged commit 1558548 into ome:master Feb 8, 2017
@sbesson sbesson deleted the jace_dockerfile branch February 8, 2017 10:42
@sbesson sbesson mentioned this pull request Feb 8, 2017
@sbesson sbesson modified the milestone: 0.1.0 Apr 21, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants