updating singularity container to run headlessly with xvfb

Signed-off-by: vsoch <vsochat@stanford.edu>

.dockerignore

@@ -1 +1,2 @@
 .git
+*.sif

Dockerfile

@@ -6,7 +6,7 @@ RUN xvfb-run winetricks -q vcrun2015 && \
     mkdir -p /APPS /PROFILES && \
     chmod 0777 /APPS /PROFILES
 
-COPY ["Prairie View/", "/Prairie View/"]
+COPY ["Prairie View/", "/APPS/Prairie View/"]
 
 ENV PATH /opt/conda/bin:$PATH
 

README.md

@@ -37,10 +37,12 @@ Run the ripper using the following command, using a directory containin the *RAW
 
 ## Singularity for ripping only
 
+### 1. Build from Docker
+
 To build the Singularity container, build the Docker container first:
 
 ```bash
-docker build -t two-photon .
+docker build -t dlab/two-photon:latest .
 ```
 
 and then build the [Singularity recipe](singularity/Singularity) that has a custom entrypoint. 
@@ -50,8 +52,10 @@ Note that this script assumes the context to be the root of the repository.
 sudo singularity build two-photon.sif singularity/Singularity
 ```
 
-The runscript (entrypoint) works by way of creating a fresh wineprefix in /tmp (where we have write) and then
-starting an interactive (bash) shell for the user to issue commands. For example, if we don't
+### 2. Run interactively 
+
+If you don't bind a data directory, then the runscript (entrypoint) will give you an interactive bash
+shell to issue commands. For example, if we don't
 have data and want to interact with wine in the container:
 
 ```bash
@@ -60,10 +64,7 @@ mkdir -p profiles
 
 # Run the container! Note that you can also build the container with any Windows
 # applications already added (instead of bound)
-singularity run \
-    --bind "${PWD}/Prairie View":"/APPS/Prairie View/" \
-    --bind ${PWD}/profiles:/PROFILES \
-    two-photon.sif
+singularity run --bind ${PWD}/profiles:/PROFILES two-photon.sif
 ```
 
 This is going to set up wine, and then start a bash shell for you to work with. For example,
@@ -77,22 +78,7 @@ and this is the Image Block Ripping Utility:
 
 ![singularity/img/ripping-utility.png](singularity/img/ripping-utility.png)
 
-If you want to run the ripper, you can bind your data folder to /data in the container. 
-
-```bash
-# create profiles directory to save profiles
-mkdir -p profiles
-
-# Run the container! Note that you can also build the container with any Windows
-# applications already added (instead of bound)
-singularity run \
-    --bind "${PWD}/Prairie View":"/APPS/Prairie View/" \
-    --bind ${PWD}/profiles:/PROFILES \
-    --bind ${PWD}/overview-23:/data \
-    two-photon.sif
-```
-
-Again when we are in the bash shell, we can prepare to run the script by first
+If you want to run the ripper script, we can prepare to run the script by first
 looking at its usage:
 
 ```bash
@@ -108,10 +94,20 @@ optional arguments:
   --ripper RIPPER       Location of Bruker Image Block Ripping Utility.
 ```
 
+You would need to have bound the data to some folder in the container other than
+/data (binding to data will run the script automatically). For example:
+
+```bash
+singularity run \
+    --bind ${PWD}/profiles:/PROFILES \
+    --bind ${PWD}/overview-23:/ripper-data \
+    two-photon.sif
+```
+
 So we would then run:
 
 ```bash
-$ /usr/bin/python3 /app/rip.py --directory /data
+$ /usr/bin/python3 /app/rip.py --directory /ripper-data
 2020-08-30 13:26:17.533 rip:50 INFO Ripping from:
  /data/Cycle00001_Filelist.txt
  /data/CYCLE_000001_RAWDATA_000025
@@ -133,9 +129,26 @@ $ /usr/bin/python3 /app/rip.py --directory /data
 2020-08-30 13:26:48.567 rip:88 INFO cleaned up!
 ```
 
-The above example is interactive, but you can modify this logic however needed.
-For example, you can customize the runscript to accept the data folder, and run the 
-python3 command directly and exit. You could also choose to add the Windows app to the
-container beforehand (and then not bind it) and this will only work
-if you don't require write in that folder. Finally, please be careful about specifying /usr/bin/python3
+### 3. Run headlessly
+
+If you want to run the same command but headlessly, you can bind your data folder to /data in the container,
+and be sure to export environment variables for xvfb-run (a virtual display).
+
+```bash
+# create profiles directory to save profiles
+mkdir -p profiles
+
+singularity run \
+    --env=XVFB_SERVER=:95 \
+    --env=XVFB_SCREEN=0 \
+    --env=XVFB_RESOLUTION=320x240x8 \
+    --env=DISPLAY=:95 \
+    --bind ${PWD}/profiles:/PROFILES \
+    --bind ${PWD}/overview-23:/data \
+    two-photon.sif
+```
+
+The command above should work headlessly, and exit the container when all is
+finished. Again, if you want to work headlessly without having it automated, just bind
+the data to another location. Also please be careful about specifying /usr/bin/python3
 directly, as likely a python from a host environment could also be found.

rip.py

@@ -125,7 +125,7 @@ def cleanup():
     parser = argparse.ArgumentParser(description='Preprocess 2-photon raw data into individual tiffs')
     parser.add_argument('--directory', type=pathlib.Path, required=True,
                         help='Directory containing RAWDATA and Filelist.txt files for ripping')
-    parser.add_argument('--ripper', default='/Prairie View/Utilities/Image-Block Ripping Utility.exe',
+    parser.add_argument('--ripper', default='/APPS/Prairie View/Utilities/Image-Block Ripping Utility.exe',
                         help='Location of Bruker Image Block Ripping Utility.')
     args = parser.parse_args()
     raw_to_tiff(args.directory, args.ripper)

singularity/Singularity

@@ -8,7 +8,6 @@ from: dlab/two-photon:latest
     sudo singularity build two-photon.sif Singularity
   And then run the container with binds to setup wine
     singularity run \
-      --bind "${PWD}/Prairie View":"/APPS/Prairie View/" \
       --bind ${PWD}/profiles:/PROFILES \
       --bind ${PWD}/overview-21:/data \
       two-photon.sif

singularity/runscript.sh

@@ -7,24 +7,18 @@ cd "$TEMPDIR"
 APPDIR="/APPS"
 PROFILEDIR="/PROFILES/${USER}@${HOSTNAME}"
 
+# Prepare temporary wine prefix directory
 echo "Setting up wine prefix..."
 export WINEPREFIX="$TEMPDIR/wineprefix"
 export WINEARCH="win64"
 
-if [[ -f "$APPDIR/wineprefix.tgz" ]]; then
-    echo "Found existing wineprefix - restoring it..."
-    mkdir -p "$WINEPREFIX"
-    cd "$WINEPREFIX"
-    tar xzf "$APPDIR/wineprefix.tgz"
+# If XVFB Server envars are not set, don't use it
+if [ -z "XVFB_SERVER" ]; then
+   wineboot --init
+   winetricks -q vcrun2015
 else
-  wineboot --init
-
-  echo "Installing DirectX9..."
-  winetricks dlls d3dx9
-
-  echo "Installing C++ libraries..."
-  winetricks -q vcrun2015
-
+   xvfb-run wineboot --init
+   xvfb-run winetricks -q vcrun2015
 fi
 
 echo "Containerizing apps directory..."
@@ -44,16 +38,33 @@ else
 fi
 ln -s "$PROFILEDIR" "$WINEPREFIX/drive_c/users/$USER"
 
-echo "Please install any software and use it! For an example"
-echo "To install Broken Sword 2.5 (download size ~700MB):"
-echo " wget http://server.c-otto.de/baphometsfluch/bs25setup.zip"
-echo " unzip bs25setup.zip"
-echo " wine ./bs25-setup.exe"
-echo
-echo "To run the two-photon ripper:"
-echo "/usr/bin/python3 /app/rip.py --directory /data"
 cd $TEMPDIR
-env WINEPREFIX="$WINEPREFIX" WINEARCH="$WINEARCH" /bin/bash
+
+# If a data directory is bound, process it
+if [ -d "/data" ]; then
+    echo "Detected /data folder bound!"
+    echo "/usr/bin/python3 /app/rip.py --directory /data"
+
+    # xvfb-run is needed for headless, otherwise, assume connection
+    if [ -z "XVFB_SERVER" ]; then
+        env WINEPREFIX="$WINEPREFIX" WINEARCH="$WINEARCH" /usr/bin/python3 /app/rip.py --directory /data    
+    else
+        env WINEPREFIX="$WINEPREFIX" WINEARCH="$WINEARCH" xvfb-run /usr/bin/python3 /app/rip.py --directory /data    
+    fi
+
+# Otherwise give an interactive terminal
+else
+    echo "Please install any software and use it! For an example"
+    echo "To install Broken Sword 2.5 (download size ~700MB):"
+    echo " wget http://server.c-otto.de/baphometsfluch/bs25setup.zip"
+    echo " unzip bs25setup.zip"
+    echo " wine ./bs25-setup.exe"
+    echo
+    echo "To run the two-photon ripper:"
+    echo "/usr/bin/python3 /app/rip.py --directory /data"
+    echo "You'll need to use xvfb-run if you don't have a display."
+    env WINEPREFIX="$WINEPREFIX" WINEARCH="$WINEARCH" /bin/bash
+fi
 
 wineboot --end-session