Dockerfile.RBuildEnv issues with dependencies

[famulare]

geojsonio isn't properly installing in the docker image. Multiple packages dependency libraries aren't coming through, despite having added them to the build environment file.

Called interactively:

> install.packages('geojsonio')

 installing *source* package ‘protolite’ ...
** package ‘protolite’ successfully unpacked and MD5 sums checked
Package protobuf was not found in the pkg-config search path.
Perhaps you should add the directory containing `protobuf.pc'
to the PKG_CONFIG_PATH environment variable
Package 'protobuf', required by 'world', not found
Using PKG_CFLAGS=
Using PKG_LIBS=-lprotobuf
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because protobuf was not found. Try installing:
 * deb: libprotobuf-dev (Debian, Ubuntu, etc)
 * rpm: protobuf-devel (Fedora, EPEL)
 * csw: protobuf_dev (Solaris)
 * brew: protobuf (OSX)
If protobuf is already installed, check that 'pkg-config' is in your
PATH and PKG_CONFIG_PATH contains a protobuf.pc file. If pkg-config
is unavailable you can set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
ERROR: configuration failed for package ‘protolite’
* removing ‘/usr/local/lib/R/site-library/protolite’
Warning in install.packages :
  installation of package ‘protolite’ had non-zero exit status
* installing *source* package ‘V8’ ...
** package ‘V8’ successfully unpacked and MD5 sums checked
Using PKG_CFLAGS=-I/usr/include/v8 -I/usr/include/v8-3.14
Using PKG_LIBS=-lv8 -lv8_libplatform
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because  was not found. Try installing:
 * deb: libv8-dev or libnode-dev (Debian / Ubuntu)
 * rpm: v8-devel (Fedora, EPEL)
 * brew: v8 (OSX)
 * csw: libv8_dev (Solaris)
To use a custom libv8, set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
ERROR: configuration failed for package ‘V8’
* removing ‘/usr/local/lib/R/site-library/V8’
Warning in install.packages :
  installation of package ‘V8’ had non-zero exit status
* installing *source* package ‘jqr’ ...
** package ‘jqr’ successfully unpacked and MD5 sums checked
Using PKG_CFLAGS=
Using PKG_LIBS=-ljq
------------------------- ANTICONF ERROR ---------------------------
Configuration failed because libjq was not found.
On Ubuntu 14.04 or 16.04 you can use the PPA:
  sudo add-apt-repository -y ppa:opencpu/jq
  sudo apt-get update
  sudo apt-get install libjq-dev
On other sytems try installing:
 * deb: libjq-dev (Debian, Ubuntu 16.10 and up).
 * rpm: jq-devel (Fedora, EPEL)
 * csw: libjq_dev (Solaris)
 * brew: jq (OSX)
If  is already installed set INCLUDE_DIR and LIB_DIR manually via:
R CMD INSTALL --configure-vars='INCLUDE_DIR=... LIB_DIR=...'
--------------------------------------------------------------------
ERROR: configuration failed for package ‘jqr’
* removing ‘/usr/local/lib/R/site-library/jqr’
Warning in install.packages :
  installation of package ‘jqr’ had non-zero exit status
ERROR: dependencies ‘protolite’, ‘jqr’ are not available for package ‘geojson’
* removing ‘/usr/local/lib/R/site-library/geojson’
Warning in install.packages :
  installation of package ‘geojson’ had non-zero exit status
ERROR: dependencies ‘V8’, ‘geojson’, ‘jqr’ are not available for package ‘geojsonio’
* removing ‘/usr/local/lib/R/site-library/geojsonio’
Warning in install.packages :
  installation of package ‘geojsonio’ had non-zero exit status

[famulare]

But if I install the dependencies from the terminal in RStudio, all is good:

terminal:

sudo apt install -y libproj-dev libv8-dev libjq-dev libprotobuf-dev protobuf-compiler

console:

> install.packages('geojsonio')
...
...
installing vignettes
** testing if installed package can be loaded
* DONE (geojsonio)

[famulare]

The most up-to-date build environment is in the connect-to-real-data branch: https://github.com/seattleflu/incidence-mapper/blob/connect-to-real-data/Dockerfile.RBuildEnv

[tsibley]

I believe I've fixed some of these package issues on my PR #35. Someone should check that it also works for them.

[famulare]

@devclinton I rebuilt the image on our machine from trs/simplify-db-connection. Seems mostly good (and better than before). One issue is this line doesn't work

incidence-mapper/Dockerfile.RBuildEnv

Line 46 in d090214

RUN Rscript install_local_packages.R

, but I added that and I assume it's my fault--something about how docker deals with relative paths I don't understand.

[tsibley]

@famulare Docker RUN commands happen inside the container, so you need to first COPY the install_local_packages.R file into the container somewhere and then run it using its path inside the container.

[tsibley]

(I didn't notice that failing for me, but it likely did!)

[famulare]

@devclinton I'll return this to your list, as I'm a little confused about copying in something that is at the same level as the dockerfile. Thanks!

incidence-mapper/Dockerfile.RBuildEnv

Line 56 in 9d31ed8

RUN Rscript install_local_packages.R

[devclinton]

@tsibley For now I removed the running of install_local_packages.R on the building on the container.

This is more because of how the future workflow will actually execute. At the moment, the container is being used as a development environment but later the workflow will be more along the lines you provided where we need to build each package. For now, let's make that a script the user can execute from within the R environment after they start the environment.

Later, I envision a workflow like the following

1. Build the package building/training container. This will be used to create the packages in the form of tar.gz file for now. I am working to get a local CRAN server up or alternatively maybe later we could publish to the general CRAN servers.

2. From There we build 2 new containers

   1. Training container with incidenceMapR will be installed. Most likely dbViewer as well. That container will then be used to train all the various iterations of the the models. Having a separate training container will be slightly lighter but is optional. The main thing I want to be able to do is begin to highly parallelize the training step and having it as a separate step makes that slightly easier. so as long as I can have a container that at runtime has the incidenceMapR available, I should be good.

   2. The second container will then just install the modelServ package. This will act as the execution worker for the API server. Really, this layer should rarely change since it is just loading a model and then running it. I suspect the training containers will change the most. Also, we could re-use the build container later in development process as the training container as well.