Adding support to get token in the URL

[vsoch]

This PR will address #138

Here are instructions for how to test! First, you can clone the branch and cd into the folder.

git clone -b add/url-token git@github.com:expfactory/expfactory
cd expfactory

Then you can add this Dockerfile to the root there:

FROM quay.io/vanessa/expfactory-builder:base

########################################
# Configure
########################################

ENV EXPFACTORY_STUDY_ID expfactory
ENV EXPFACTORY_SERVER localhost
ENV EXPFACTORY_CONTAINER true
ENV EXPFACTORY_DATA /scif/data
ENV EXPFACTORY_DATABASE filesystem
ENV EXPFACTORY_HEADLESS false
ENV EXPFACTORY_BASE /scif/apps
 
ADD startscript.sh /startscript.sh
RUN chmod u+x /startscript.sh

WORKDIR /opt/expfactory
COPY . /opt/expfactory
RUN cp script/nginx.gunicorn.conf /etc/nginx/sites-enabled/default && \
    cp script/nginx.conf /etc/nginx/nginx.conf
RUN mkdir -p /scif/data # saved data
RUN mkdir -p /scif/apps # experiments 
RUN mkdir -p /scif/logs # gunicorn logs 
RUN python3 -m pip install gunicorn
RUN cp expfactory/config_dummy.py expfactory/config.py && \
    chmod u+x /opt/expfactory/script/generate_key.sh && \
    /bin/bash /opt/expfactory/script/generate_key.sh /opt/expfactory/expfactory/config.py
RUN python3 setup.py install
RUN python3 -m pip install pyaml    pymysql psycopg2-binary
RUN apt-get clean          # tests, mysql,  postgres

########################################
# Experiments
########################################


LABEL EXPERIMENT_digit-span /scif/apps/digit-span
WORKDIR /scif/apps
RUN expfactory install https://github.com/expfactory-experiments/digit-span

LABEL EXPERIMENT_spatial-span /scif/apps/spatial-span
WORKDIR /scif/apps
RUN expfactory install https://github.com/expfactory-experiments/spatial-span

LABEL EXPERIMENT_tower-of-london /scif/apps/tower-of-london
WORKDIR /scif/apps
RUN expfactory install https://github.com/expfactory-experiments/tower-of-london

LABEL EXPERIMENT_test-task /scif/apps/test-task
WORKDIR /scif/apps
RUN expfactory install https://github.com/expfactory-experiments/test-task

ENTRYPOINT ["/bin/bash", "/startscript.sh"]
EXPOSE 5000
EXPOSE 80

And the startscript (startscript.sh)

#!/bin/bash

usage () {

    echo "Usage:


         docker run <container> [help|list|test-experiments|start]
         docker run -p 80:80 -v /tmp/data:/scif/data <container> start

         Headless Mode (requires token)

         docker run -p 80:80 -d --name experiments -v /tmp/data:/scif/data <container> --headless start
         docker exec experiments expfactory users 3


         Commands:

                help: show help and exit
                list: list installed experiments
                lib: list experiments in the library
                test: test experiments installed in container
                start: start the container to do the experiments*
                env: search for an environment variable set in the container
         
         *you are required to map port 80, otherwise you won't see the portal at localhost

         Options [start]:

                --db: specify a database url to override the default filesystem
                                 [sqlite|mysql|postgresql]:///


                --vars:  specify an experiment variables file
                --delim: specify a delimiter for the variables file (default is csv)

                --studyid:  specify a studyid to override the default
                --randomize: select experiment order at random
                --no-randomize: select experiment order manually in the GUI
                --experiments: a comma separated list of experiments (manual ordering) 

         Examples:

              docker run <container> test
              docker run <container> list
              docker run <container> start
              docker run -p 80:80 <container> --database mysql+pymysql://username:password@host/dbname start
              docker run -p 80:80 <container> --database sqlite start
              docker run -p 80:80 <container> --database postgresql://username:password@host/dbname start

         "
}

if [ $# -eq 0 ]; then
    usage
    exit
fi

EXPFACTORY_START="no"
EXPFACTORY_DATABASE="filesystem"

while true; do
    case ${1:-} in
        -h|--help|help)
            usage
            exit
        ;;
        --database|--db)
            shift
            EXPFACTORY_DATABASE=${1:-}
            export EXPFACTORY_DATABASE
            shift
        ;;
        --delim|delim)
            shift
            EXPFACTORY_RUNTIME_DELIM=${1:-}
            shift
            export EXPFACTORY_RUNTIME_DELIM
        ;;
        --env|env)
            shift
            env | grep ${1:-}
            exit
        ;;
        -e|experiments|--experiments)
            shift
            EXPFACTORY_EXPERIMENTS="${1:-}"
            shift
            export EXPFACTORY_EXPERIMENTS
        ;;
        --headless|headless)
            shift
            EXPFACTORY_HEADLESS="true"
            export EXPFACTORY_HEADLESS
        ;;
        --lib)
            echo "Experiments in the library:"
            expfactory list
            echo
            exit
        ;;
        -ls|--list|list)
            echo "Experiments in this image:"
            ls /scif/apps -1
            echo
            exit
        ;;
        --randomize)
            shift
            EXPFACTORY_RANDOM="true"
            export EXPFACTORY_RANDOM
        ;;
        --no-randomize)
            shift
            EXPFACTORY_RANDOM="false"
            export EXPFACTORY_RANDOM
        ;;
        -s|--start|start)
            EXPFACTORY_START="yes"
            shift
        ;;
        --studyid)
            shift
            EXPFACTORY_STUDY_ID=${1:-}
            echo "Study ID selected as ${EXPFACTORY_STUDYID}"
            export EXPFACTORY_STUDY_ID
            shift
        ;;
        -test-experiments|--te|test)
            cd /opt/expfactory/expfactory/templates/build
            exec python3 -m unittest tests.test_experiment
            exit
        ;;
        --vars|vars)
            shift
            EXPFACTORY_RUNTIME_VARS=${1:-}
            shift
            if [ -f "${EXPFACTORY_RUNTIME_VARS}" ]; then
                echo "Found Variable File: ${EXPFACTORY_RUNTIME_VARS}"
                export EXPFACTORY_RUNTIME_VARS
            else
                echo "WARNING: Cannot find ${EXPFACTORY_RUNTIME_VARS}"
            fi

        ;;
        -*)
            echo "Unknown option: ${1:-}"
            exit 1
        ;;
        *)
            break
        ;;
    esac
done

# Are we starting the server?

if [ "${EXPFACTORY_START}" == "yes" ]; then

    echo "Database set as ${EXPFACTORY_DATABASE}"
    export EXPFACTORY_DATABASE

    echo "Starting Web Server"
    echo
    service nginx start
    touch /scif/logs/gunicorn.log
    touch /scif/logs/gunicorn-access.log
    tail -n 0 -f /scif/logs/gunicorn*.log &

    exec  gunicorn expfactory.wsgi:app \
                  --bind 0.0.0.0:5000 \
                  --name expfactory_experiments
                  --workers 5
                  --log-level=info
                  --log-file=/scif/logs/gunicorn.log \
                  --access-logfile=/scif/logs/gunicorn-access.log \
            "$@" & service nginx restart

    # simple manual command could be
    # service nginx start
    # gunicorn --bind 0.0.0.0:5000 expfactory.wsgi:app
    # service nginx restart

    # Keep container running if we get here
    tail -f /dev/null
    exit
fi

You can modify the experiments in the Dockerfile as needed (e.g, delete some if you just need one) and then build an experiment container from it

$ docker build -t experiment .

Then start the container, detached headless, and generate some user tokens

$ docker run --name experiment -v -d -p 80:80 experiment start --headless
$ docker exec experiment expfactory users --new 3

First open your browser to http://127.0.0.1 to make sure the portal is there (the window asking to enter the token). Instead, copy one of the tokens (before [active] in the console) to the url http://127.0.0.1/login?token=<token> and that should work to login. if you need to logout of the session, just navigate to /logout.

I'm planning to do a refactor of the docs at expfactory.github.io soon, so they will have notes for this there as well!

Signed-off-by: vsoch vsoch@users.noreply.github.com

[vsoch]

And it's now mentioned in the docs (which are revamped and much better!), at the end of this section (so we should get this merged before someone realizes it's not deployed yet, hehe) https://expfactory.github.io/usage#use-tokens

[AlvaroAguilera]

Hello,

awesome! Unfortunately I don't get it to compile correctly. This is the output of the docker build:

Collecting psycopg2-binary
Downloading https://files.pythonhosted.org/packages/14/65/223a5b4146b1d5d5ab66f16ef194916a1ed9720da1f118d7bfb60b8f2bea/psycopg2-binary-2.9.1.tar.gz (380kB)
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2_binary.egg-info
writing pip-egg-info/psycopg2_binary.egg-info/PKG-INFO
writing dependency_links to pip-egg-info/psycopg2_binary.egg-info/dependency_links.txt
writing top-level names to pip-egg-info/psycopg2_binary.egg-info/top_level.txt
writing manifest file pip-egg-info/psycopg2_binary.egg-info/SOURCES.txt

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:

python setup.py build_ext --pg-config /path/to/pg_config build ...

or with the pg_config option in setup.cfg.

If you prefer to avoid building psycopg2 from source, please install the PyPI
psycopg2-binary' package instead.

For further information please check the 'doc/src/install.rst' file (also at
https://www.psycopg.org/docs/install.html).

----------------------------------------

Command python setup.py egg_info" failed with error code 1 in /tmp/pip-build-zhf5pz3r/psycopg2-binary/
The command /bin/sh -c python3 -m pip install pyaml pymysql psycopg2-binary returned a non-zero code: 1_

Note that I already have psycopg2 installed locally:

rpm -ql python-psycopg2 /usr/lib64/python2.7/site-packages/psycopg2 /usr/lib64/python2.7/site-packages/psycopg2-2.6-py2.7.egg-info /usr/lib64/python2.7/site-packages/psycopg2/__init__.py /usr/lib64/python2.7/site-packages/psycopg2/__init__.pyc /usr/lib64/python2.7/site-packages/psycopg2/_json.py /usr/lib64/python2.7/site-packages/psycopg2/_json.pyc /usr/lib64/python2.7/site-packages/psycopg2/_psycopg.so /usr/lib64/python2.7/site-packages/psycopg2/_range.py /usr/lib64/python2.7/site-packages/psycopg2/_range.pyc /usr/lib64/python2.7/site-packages/psycopg2/errorcodes.py /usr/lib64/python2.7/site-packages/psycopg2/errorcodes.pyc /usr/lib64/python2.7/site-packages/psycopg2/extensions.py /usr/lib64/python2.7/site-packages/psycopg2/extensions.pyc /usr/lib64/python2.7/site-packages/psycopg2/extras.py /usr/lib64/python2.7/site-packages/psycopg2/extras.pyc /usr/lib64/python2.7/site-packages/psycopg2/pool.py /usr/lib64/python2.7/site-packages/psycopg2/pool.pyc /usr/lib64/python2.7/site-packages/psycopg2/psycopg1.py /usr/lib64/python2.7/site-packages/psycopg2/psycopg1.pyc /usr/lib64/python2.7/site-packages/psycopg2/tz.py /usr/lib64/python2.7/site-packages/psycopg2/tz.pyc /usr/share/doc/packages/python-psycopg2 /usr/share/doc/packages/python-psycopg2/AUTHORS /usr/share/doc/packages/python-psycopg2/LICENSE /usr/share/doc/packages/python-psycopg2/NEWS /usr/share/doc/packages/python-psycopg2/README.rst

[vsoch]

What you have locally shouldn't matter... can you confirm this is what you are doing? I can't reproduce that error when I follow my steps above.

cd /tmp
docker pull quay.io/vanessa/expfactory-builder:base

git clone -b add/url-token git@github.com:expfactory/expfactory
cd expfactory

# Writing the Dockerfile and startscript.sh
gedit Dockerfile
gedit startscript.sh

# Building
docker build -t experiment .

Please try that above, exactly, and otherwise I need to be able to reproduce the error you are showing me.

[vsoch]

Also in the ouput I see that dependency is installed...

Finished processing dependencies for expfactory==3.2.0
Removing intermediate container 8a7673e82f4a
 ---> 06e56ffc4b8b
Step 20/36 : RUN python3 -m pip install pyaml    pymysql psycopg2-binary
 ---> Running in cf70cb522fdb
Collecting pyaml
  Downloading pyaml-21.8.3-py2.py3-none-any.whl (17 kB)
Collecting pymysql
  Downloading PyMySQL-1.0.2-py3-none-any.whl (43 kB)
Collecting psycopg2-binary
  Downloading psycopg2_binary-2.9.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB)
Collecting PyYAML
  Downloading PyYAML-5.4.1-cp38-cp38-manylinux1_x86_64.whl (662 kB)
Installing collected packages: PyYAML, pyaml, pymysql, psycopg2-binary
Successfully installed PyYAML-5.4.1 psycopg2-binary-2.9.1 pyaml-21.8.3 pymysql-1.0.2

As it's in the container

RUN python3 setup.py install
RUN python3 -m pip install pyaml    pymysql psycopg2-binary
RUN apt-get clean          # tests, mysql,  postgres

[AlvaroAguilera]

Now it seems to be working well, once I also pulled from quay.io. Great work and thank you.

[vsoch]

Great! So you tested the new token login with a GET request, and this is ready for merge?

[AlvaroAguilera]

Yes, go ahead!

[vsoch]

Ok great! I’ll merge here to get the new containers building, and do official releases later today (it’s still early morning here).

thank you for your help, and I hope this new feature is what you were looking for! Please deploy with https :)