Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

update dockerfile - maint/pre-hnn-core branch #338

Merged
merged 11 commits into from
Jun 4, 2024
Merged

update dockerfile - maint/pre-hnn-core branch #338

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
0516222
updated dockerfile and plist filename
jashlu Nov 7, 2023
34aec84
refactor: update README
jashlu Nov 7, 2023
8885d12
refactor: remove unused hnn-core
jashlu Nov 7, 2023
be1cd73
add configure_vcxsrv script and update README
jashlu Nov 30, 2023
8842dd9
update image placement
jashlu Nov 30, 2023
704c801
update
jashlu Nov 30, 2023
ada9ea9
Update README.md
jashlu Nov 30, 2023
f64049b
Update README.md
jashlu Dec 6, 2023
f581334
Update configure_vcxsrv.sh
jashlu Dec 6, 2023
d76933f
Update README.md
jashlu Dec 8, 2023
025a045
Update git command
jashlu Feb 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 23 additions & 15 deletions installer/docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
FROM caldweba/opengl-docker
FROM ubuntu:20.04

# avoid questions from debconf
ENV DEBIAN_FRONTEND noninteractive

# install sudo
RUN apt-get update && \
apt-get -y install sudo git

# create the group hnn_group and user hnn_user
# add hnn_user to the sudo group
RUN groupadd hnn_group && useradd -m -b /home/ -g hnn_group hnn_user && \
Expand All @@ -26,18 +30,21 @@ RUN apt-get update && \
python3-pip python3-setuptools openssh-server openmpi-bin lsof netcat xauth && \
apt-get clean


# get HNN python dependencies
# python3-dev and gcc needed for building psutil
RUN sudo pip3 install --no-cache-dir --upgrade pip && \
sudo apt-get update && \
sudo apt-get install --no-install-recommends -y \
RUN pip3 install --no-cache-dir --upgrade pip setuptools && \
apt-get update && \
apt-get install --no-install-recommends -y \
gcc python3-dev && \
sudo pip install --no-cache-dir matplotlib PyOpenGL \
pyqt5 pyqtgraph scipy numpy nlopt psutil && \
sudo apt-get -y remove --purge \
apt-get install --no-install-recommends -y \
cmake make build-essential python3-pyqt5 libnlopt-dev octave guile-2.2 && \
pip install --no-cache-dir matplotlib scipy numpy psutil swig matlab && \
pip install --no-cache-dir nlopt && \
apt-get -y remove --purge \
gcc python3-dev && \
sudo apt-get autoremove -y --purge && \
sudo apt-get clean
apt-get autoremove -y --purge && \
apt-get clean

COPY date_base_install.sh /usr/local/bin
RUN chmod +x /usr/local/bin/date_base_install.sh && \
Expand Down Expand Up @@ -83,17 +90,18 @@ LABEL org.label-schema.build-date=$BUILD_DATE \
org.label-schema.vcs-ref=$VCS_REF \
org.label-schema.schema-version=$VCS_TAG

# install NEURON
RUN sudo pip install NEURON
# install hnn-core
RUN pip install hnn-core

rythorpe marked this conversation as resolved.
Show resolved Hide resolved
RUN sudo apt-get update && \
sudo apt-get install -y neuron-dev
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of curiosity, is this preferred over doing a pip installation of NEURON?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just from experience I was having problems installing neuron with pip3, and so from some online resources (https://command-not-found.com/nrnivmodl) I tried an alternative and it worked

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I've never run into issues using pip except on Windows (without WSL). NEURON doesn't currently distribute a python wheel for windows, but it should work for WSL, native linux, and macOS.

Copy link
Collaborator Author

@jashlu jashlu Nov 7, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the error I run into

=> ERROR [19/28] RUN sudo apt-get update &&     sudo apt-get install --no-install-recommends -y         make gcc libc6-dev libtinfo-dev lib  11.0s 
...

9.505 Processing triggers for libc-bin (2.31-0ubuntu9.12) ...
9.545 Cloning into '/home/hnn_user/hnn_source_code'...
10.82 nrnivmodl mod
10.82 make: nrnivmodl: Command not found
10.82 make: *** [Makefile:11: x86_64/special] Error 127

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To further clarify, I run into this error attempting to build the image on my mac

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the "barebone" mac install in a docker is not the same as an actual mac computer ... there may be some compilers etc missing


# install HNN
RUN sudo apt-get update && \
sudo apt-get install --no-install-recommends -y \
make gcc libc6-dev libtinfo-dev libncurses-dev \
libx11-dev libreadline-dev g++ && \
git clone ${SOURCE_REPO} \
--depth 1 --single-branch --branch $SOURCE_BRANCH \
$HOME/hnn_source_code && \
git clone --single-branch --branch maint/pre-hnn-core ${SOURCE_REPO} $HOME/hnn_source_code &&\
cd $HOME/hnn_source_code && \
make && \
sudo apt-get -y remove --purge \
Expand All @@ -120,4 +128,4 @@ RUN sudo chown -R hnn_user:hnn_group $HOME && \
sudo chmod +x $HOME/start_hnn.sh $HOME/check_hnn_out_perms.sh && \
sudo chmod +x /check_sshd_port.sh /start_ssh.sh /check_x_port.sh && \
sudo chown root:root /check_sshd_port.sh /start_ssh.sh /check_x_port.sh && \
sudo chmod 666 $HOME/.config/QtProject.conf
sudo chmod 666 $HOME/.config/QtProject.conf
133 changes: 82 additions & 51 deletions installer/docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,81 +1,112 @@
# HNN Docker Container
### Windows Installation Process
**Part 1**
1. Open command Prompt
2. Run the following command to install Windows Subsystem for Linux (WSL)
```
wsl --install
```
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/f94c9ea6-1459-4082-8a01-af69d115368c)

This directory contains files for building the HNN Docker container. The container itself is running the Ubuntu 18.04 Linux distribution, but can run on any operating system assuming that [Docker](https://www.docker.com/) is installed.
3. Once the installation is done, you will see that an `Ubuntu` application is also downloaded. This can be found in Settings > Apps > Installed apps or simply by searching `Ubuntu` in the Windows search bar.
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/aa1635db-ccd7-4989-bdbd-7f89f5493225)

## Pulling the prebuilt Docker container from Docker Hub
4. Once WSL is installed, navigate to the Windows Start Menu and in the search bar, look up `Turn Windows Features on or off` and click on it
5. Go down the list until you see Windows Subsystem for Linux, and check the box so that it is enabled. Click ok so the changes can be applied.
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/d9c9580f-2082-4ac8-bdee-ca599e9b2da1)

The newest version of HNN is available as a prebuilt container posted on Docker Hub, which can be used instead of building the container and all of its prerequisites (NEURON) from scratch.
6. Then open up the Ubuntu application that was installed alongside WSL.
7. The very first time you open this application up, you will see instructions prompting you to create a new UNIX username and password. Afterwards, it will show you an installation successful message.
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/d6e26399-804d-4780-a451-492c1ccdf81f)

Linux container (built from release branches):

```bash
docker pull jonescompneurolab/hnn
```

Linux container (built from master):
**Part 2**
1. Please install Docker Desktop https://docs.docker.com/desktop/wsl/#download
2. Once installed, open up the Docker Desktop application.
3. Navigate to Settings, and click on general. Make sure that `Use the WSL 2 Based Engine` is checked.
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/84d0078a-ba4f-478c-8af1-e7771ba896b3)

```bash
docker pull jonescompneurolab/hnn:master
```
4. Next, click on Resources > WSL Integration
5. Inside here, make sure that all Ubuntu options are toggled. Then press apply & restart.
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/d01a0c14-37b2-456a-b7d8-3ed3bdd5283c)

Windows container:
6. To check if Docker is correctly installed, you can open up the Ubuntu terminal (application) and type `docker --version` this should display the version and build number of the software installed.

```bash
docker pull jonescompneurolab/hnn:win64
```

## Building HNN container
**Part 3**
1. Please install Vcxsrv https://sourceforge.net/projects/vcxsrv/
2. Run the installer, choosing "C:\Program Files\VcXsrv" as the destination folder.
3. Start the XLaunch desktop app from the VcXsrv folder in the start menu.
4. Choose "Multiple windows". Choose '0' for the "Display number". Click 'Next'.
5. Select "Start no client" and click 'Next'.
6. Important: under "Extra settings" make sure that "Disable access control" is checked.

The BUILD_DATE argument is important to build the container with the latest HNN source code. Without it, the build will reuse the docker build cache, which may have been with an old code version.
7. For the following step, you will need the IP address of your local machine. To find it, open up Settings > Network & internet > Wi-Fi > Hardware properties and look for `IPv4 address:`
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/45f51c4b-c2c5-48f4-9f7f-490ee423ebd3)

Optional arguments are SOURCE_BRANCH and SOURCE_REPO. If they are not specified, the image will be build from source at 'https://github.com/jonescompneurolab/hnn' on branch 'master'

```bash
docker build --tag jonescompneurolab/hnn --build-arg SOURCE_BRANCH=master --build-arg SOURCE_REPO="https://github.com/jonescompneurolab/hnn" --build-arg BUILD_DATE=$(date +%s) installer/docker
8. Now open up the ubuntu application terminal and run the following commands. Comments above each command help to provide additional details. Original source of instructions comes from https://sourceforge.net/p/vcxsrv/wiki/VcXsrv%20%26%20Win10/.
```
# in the following command, replace the NN.NN.NN.NN with the IPv4 addess that you've obtained in the previous step.
export DISPLAY=NN.NN.NN.NN:0

The windows container container can be built with the following command:
# you can ignore any errors that arise from running the following command.
sudo apt install -y xauth coreutils gawk gnome-terminal

```bash
docker build --tag jonescompneurolab/hnn:win64 -f installer/windows/Dockerfile installer/docker
```
# running the following command should display nothing, to show you that you do not have credentials set up yet.
xauth list

# run the following command, with {some-pass-phrase} replaced by any phrase. Ex. hnn-app
magiccookie=$(echo '{some-pass-phrase}'|tr -d '\n\r'|md5sum|gawk '{print $1}')

## Starting HNN container
# run the following
xauth add "$DISPLAY" . "$magiccookie"

The container is designed to be run from the `hnn_docker.sh` script, but in general, the following scheme will work for Linux/mac:
# running this command again, you will see that something shows up, displaying the new credentials you have set up.
xauth list

```bash
docker run -d -v "$HOME/hnn_out":"$HOME/hnn_out" --env XAUTHORITY=/tmp/.Xauthority --env SYSTEM_USER_DIR="$HOME" --name hnn_container jonescompneurolab/hnn
# run the following commands
userprofile=$(wslpath $(/mnt/c/Windows/System32/cmd.exe /C "echo %USERPROFILE%" | tr -d '\r\n'))
cp ~/.Xauthority "$userprofile
```

This will start the container in the background. In order to start the HNN GUI, the start_hnn.sh script must be run inside the container:

## Running HNN with Docker
**Part 4**
1. Open up a new Ubuntu terminal application.
2. We will now download the code that will run the program to start up the GUI.
3. Run this command to create a copy of the source code in your computer.
```
Git clone https://github.com/jonescompneurolab/hnn.git
```
4. Run the following command to navigate to inside the repository in your Ubuntu terminal.
```
cd hnn
```
5. Run the following command to start up the hnn GUI.
```
./hnn_docker.sh start
```

We recommend using the `hnn_docker.sh` script which includes checks for directory permissions, creating the necessary files, and adding the appropriate options for each host OS. The commands below give an outline of the process.
Your terminal should show the following logs if it successfully starts up the hnn GUI.
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/2e5942cb-100f-44cf-9c6f-fdee72576844)

Using SSH can be controlled by the environment variable USE_SSH
The resulting hnn GUI on initial startup.
![image](https://github.com/jonescompneurolab/hnn/assets/34087669/cb6f404c-d6df-45ef-ab80-cf85e293bf8b)

### Without SSH (only recommended for Linux)

```bash
export USE_SSH=0
docker exec --env SYSTEM_USER_DIR="$HOME" --env DISPLAY=":0" -u "$(id -u)" hnn_container /home/hnn_user/start_hnn.sh
```
**Part 4 Troubleshooting**
If you run into the following error while running `./hnn_docker.sh start`

* In order to access the hnn_out volume, the command is run as the current user on the host OS. For Windows container, the user hnn_user should be used (-u option can be omitted).
`
docker-machine could not be found.
`
- Please make sure you have gone over the previous steps for installing the docker desktop.
- If this error persists, please try closing and restarting the docker desktop application.

### Using SSH (stable X server connection)
If you run into the following error while running `./hnn_docker.sh start`

```bash
export USE_SSH=1
SSH_PRIVKEY="installer/docker/id_rsa_hnn"
ssh-keygen -f "$SSH_PRIVKEY" -t rsa -N ''
`
xuath: (argv):1: couldn't query Security extension on display “:0”
`
- Please double check that you have gone through the steps in Part 3 to configure the magic cookie credentials required for the VcXsrv server to accept connections.

export DISPLAY=127.0.0.1:0
export XAUTHORITY=/tmp/.Xauthority
export SYSTEM_USER_DIR="$HOME"
ssh -o SendEnv=DISPLAY -o SendEnv=XAUTHORITY -o SendEnv=SYSTEM_USER_DIR -o SendEnv=TRAVIS_TESTING -o PasswordAuthentication=no -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -v -i "SSH_PRIVKEY" -R 6000:127.0.0.1:6000 hnn_user@localhost -p 32791
```

* Commands should be run from the HNN source code directory
8 changes: 4 additions & 4 deletions scripts/docker_functions.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2140,13 +2140,13 @@ function config_xquartz_for_tcp {
local __xquartz_nolisten

# check if xquartz needs to be restarted to access preferences
__xquartz_output=$(output_run_command "defaults read org.macosforge.xquartz.X11.plist")
__xquartz_output=$(output_run_command "defaults read org.xquartz.X11.plist")
if [[ $? -ne 0 ]]; then
__restart_xquartz=1
return 1
elif [[ -z "$__xquartz_output" ]]; then
echo "*failed*" | tee -a "$LOGFILE"
echo -e "\nNo valid output from org.macosforge.xquartz.X11.plist" | tee -a "$LOGFILE"
echo -e "\nNo valid output from org.xquartz.X11.plist" | tee -a "$LOGFILE"
cleanup 2
fi

Expand All @@ -2167,13 +2167,13 @@ function config_xquartz_for_tcp {
if [[ "$__xquartz_nolisten" == "1" ]]; then
__restart_xquartz=1
print_header_message "Setting XQuartz preferences to listen for network connections... "
run_command_print_status_failure_exit "defaults write org.macosforge.xquartz.X11.plist nolisten_tcp 0"
run_command_print_status_failure_exit "defaults write org.xquartz.X11.plist nolisten_tcp 0"
fi

if [[ "$__xquartz_noauth" == "1" ]]; then
__restart_xquartz=1
print_header_message "Setting XQuartz preferences to use authentication... "
run_command_print_status_failure_exit "defaults write org.macosforge.xquartz.X11.plist no_auth 0"
run_command_print_status_failure_exit "defaults write org.xquartz.X11.plist no_auth 0"
fi
else
check_x_port_netcat
Expand Down