preffered way of installing Linuxbrew with rocker project?

[pgensler]

Hello,

I created an docker image based from the rocker project, and I'm not quite sure what the correct way of installing Linuxbrew is on the container. It seems like there is already a Debian build, and I'm a bit confused on the preferred way of installing: should I be using the Debian dockerfile(which is what my image is based on), the core image, or the instructions listed on the readme page? Should I just be trying to chain together the dockerfile's?

My image builds fine, but I've just had issues with getting Linuxbrew to install, so any help is appreciated.

I think my build does not like:
*** update-locale: Error: invalid locale settings: LANG=en_US.UTF-8
from: https://travis-ci.org/pgensler/sandboxr/builds/301806260
which really does not make any sense....is this necessary?

EDIT: After trying a multi-stage build, I'm not really sure what I need to be running after my FROM linuxbrew/debian...just simply a RUN command like:
docker run -it linuxbrew/debian ?

Thanks for your help.
My dockerfile is below:

##Combining this dockerfile:https://github.com/MethodsConsultants/tidyverse-h2o/blob/master/Dockerfile
##with this: https://hub.docker.com/r/andrewheiss/tidyverse-rstanarm/~/dockerfile/
#tidyverse image comes with devtools preinstalled
FROM rocker/tidyverse:latest
LABEL maintainer="Peter Gensler <peterjgensler@gmail.com>"

# ------------------------------
# Install rstanarm and friends
# ------------------------------
# Docker Hub (and Docker in general) chokes on memory issues when compiling
# with gcc, so copy custom CXX settings to /root/.R/Makevars and use ccache and
# clang++ instead

# prophet needs rstan pkg, and we need to set Makevars compiler flag to compile properly
# Make ~/.R
RUN mkdir -p $HOME/.R

# $HOME doesn't exist in the COPY shell, so be explicit
COPY R/Makevars /root/.R/Makevars

# Install ggplot extensions like ggstance and ggrepel
# rm rf for removing all temp files after install
# adding devtools for starting and ending of package installation
# we need to use single quotes for packages:
# https://stackoverflow.com/questions/47127594/multi-line-rscript-in-dockerfile/47128124?noredirect=1#comment81206386_47128124

##Linuxbrew
# Set up UTF-8
RUN apt-get update && \
    apt-get install -y apt-utils locales && \
    sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
    dpkg-reconfigure locales && \
    update-locale LANG=en_US.UTF-8
ENV LANG en_US.UTF-8

RUN apt-get update -qq \
    && apt-get -y --no-install-recommends install \
    liblzma-dev \
    libbz2-dev \
    clang  \
    ccache \
    default-jdk \
    default-jre \
    libpoppler-cpp-dev \
    libapparmor-dev \
    xsel \
    xclip \
    build-essential \
    curl \ #linuxbrew packages below
    file \
    git \
    build-essential \
    python-setuptools \
    && R CMD javareconf \
    && Rscript -e "devtools::install_cran(c('ggstance','ggrepel','ggthemes', \
           'tidytext','readtext','textclean','janitor','corrr','datapasta', \
           'tidyquant','timetk','tibbletime','sweep','broom','prophet', \
           'forecast','prophet','lime','sparklyr','h2o','rsparkling','unbalanced', \
           'formattable','httr','rvest','xml2','jsonlite', \
           'corrr','officer','devtools','pacman','naniar','writexl'))" \
    ##GitHub Packages
    && Rscript -e 'devtools::install_github(c("hadley/multidplyr","jeremystan/tidyjson","ropenscilabs/skimr"))' \
    && rm -rf /tmp/downloaded_packages/ /tmp/*.rds \
	&& rm -rf /var/lib/apt/lists/*

# Create a linuxbrew user
RUN  useradd -m -s /bin/bash linuxbrew \
     && echo 'linuxbrew ALL=(ALL) NOPASSWD:ALL' >>/etc/sudoers
USER linuxbrew
WORKDIR /home/linuxbrew
ENV PATH=/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:$PATH \
    SHELL=/bin/bash

# Install Linuxbrew from github
RUN git clone https://github.com/Linuxbrew/brew.git .linuxbrew

# Install portable-ruby by running brew for the first time
RUN brew doctor

[sjackman]

See this Dockerfile for Debian: https://github.com/Linuxbrew/docker/blob/master/debian/Dockerfile

[pgensler]

@sjackman I know about your dockerfile for Debian, I wanted to do a multi-stage build dockerfile with your debian image. My issue is, I think your base image most of the heavy lifting, so I'm not sure what I should be implementing in my script.

My question is around if I simply just use a multi build with a FROM at the end of my dockerfile, it works, but I have no idea what else I should be doing after that, which I imagine I should be running some sort of test. Shouldn't that dockerfile you linked be doing everything necessary to set it up, and hence it should just work?

[sjackman]

The base image of this Dockerfile is debian:stretch.
https://github.com/Linuxbrew/docker/blob/master/debian/Dockerfile

The following installs Linuxbrew using rocker/tidyverse as the base image:

docker run -it rocker/tidyverse /bin/bash
useradd -m -s /bin/bash linuxbrew
echo 'linuxbrew ALL=(ALL) NOPASSWD:ALL' >>/etc/sudoers
sudo -u linuxbrew /bin/bash -l
sudo apt-get install curl
sh -c "$(curl -fsSL https://raw.githubusercontent.com/Linuxbrew/install/master/install.sh)"
echo 'PATH="/home/linuxbrew/.linuxbrew/bin:$PATH"' >>~/.profile
PATH="/home/linuxbrew/.linuxbrew/bin:$PATH"

For tests, try…

brew doctor
brew install hello
brew test hello

[pgensler]

We don't need your script as above, but rather because docker supports multi-stage builds:

FROM rocker/tidyverse:latest
LABEL maintainer="Peter Gensler <peterjgensler@gmail.com>"

# ------------------------------
# Install rstanarm and friends
# ------------------------------
# Docker Hub (and Docker in general) chokes on memory issues when compiling
# with gcc, so copy custom CXX settings to /root/.R/Makevars and use ccache and
# clang++ instead

# prophet needs rstan pkg, and we need to set Makevars compiler flag to compile properly
# Make ~/.R
RUN mkdir -p $HOME/.R

# $HOME doesn't exist in the COPY shell, so be explicit
COPY R/Makevars /root/.R/Makevars

# Install ggplot extensions like ggstance and ggrepel
# rm rf for removing all temp files after install
# adding devtools for starting and ending of package installation
# we need to use single quotes for packages:
# https://stackoverflow.com/questions/47127594/multi-line-rscript-in-dockerfile/47128124?noredirect=1#comment81206386_47128124


#linuxbrew packages below curl
RUN apt-get update -qq \
    && apt-get -y --no-install-recommends install \
    liblzma-dev \
    libbz2-dev \
    clang  \
    ccache \
    default-jdk \
    default-jre \
    libpoppler-cpp-dev \
    libapparmor-dev \
    xsel \
    xclip \
    && R CMD javareconf \
    && Rscript -e "devtools::install_cran(c('ggstance','ggrepel','ggthemes', \
           'tidytext','readtext','textclean','janitor','corrr','datapasta', \
           'tidyquant','timetk','tibbletime','sweep','broom','prophet', \
           'forecast','prophet','lime','sparklyr','h2o','rsparkling','unbalanced', \
           'formattable','httr','rvest','xml2','jsonlite', \
           'corrr','officer','devtools','pacman','naniar','writexl'))" \
    ##GitHub Packages
    && Rscript -e 'devtools::install_github(c("hadley/multidplyr","jeremystan/tidyjson","ropenscilabs/skimr"))' \
    && rm -rf /tmp/downloaded_packages/ /tmp/*.rds \
	&& rm -rf /var/lib/apt/lists/*

##NOW ADD LINUXBREW
FROM linuxbrew/debian:latest
##Just run commands?
RUN brew doctor
RUN brew install hello
RUN brew test hello

So I just need some simple RUN commands, like above? I tried running Linuxbrew on this build, and I was unable to:
pgensler/sandboxr@eb62e30

[sjackman]

What error message are you seeing with this Dockerfile?

[pgensler]

Well I can't do anything with brew at the terminal so:

rstudio@36f78b8ed390:~$ brew install hello
bash: brew: command not found

[sjackman]

Try…

/home/linuxbrew/.linuxbrew/bin/brew --version

[sjackman]

For a multi-level Dockerfile, try something like…

FROM linuxbrew/linuxbrew as linuxbrew
FROM rocker/tidyverse:latest
COPY --from=linuxbrew /home/linuxbrew /home/linuxbrew
ENV PATH=/home/linuxbrew/.linuxbrew/bin:$PATH
RUN brew config

See for example https://github.com/Linuxbrew/docker/blob/master/scratch/Dockerfile#L12

[pgensler]

@sjackman yeah this did not do anything:
/home/linuxbrew/.linuxbrew/bin/brew --version
I'll try the code you mentioned above, and see if it works. I think that is exactly what needs to happen though.

[pgensler]

Hmm so after running that snippet of code, I get this in Travis:

Error parsing reference: "linuxbrew/linuxbrew as linuxbrew" is not a valid repository/tag: invalid reference format
Maybe as 'linuxbrew' ?

[sjackman]

The version of Docker provided by TravisCI is likely too old. Multi-stage Dockerfiles are a relatively new feature.

https://docs.docker.com/engine/userguide/eng-image/multistage-build/

Multi-stage builds are a new feature requiring Docker 17.05 or higher on the daemon and client.

[sjackman]

A single-stage build that uses FROM rocker/tidyverse:latest and the following script would be simpler:
#521 (comment)

[pgensler]

good catch, I assumed that Travis would stay up to date with Docker, I'll try that in my .yml file, and see what happens.
travis-ci/travis-ci#8181