Docker local changes

[ackagel]

Purpose

Allows users to update+reinstall ark codebase within docker with -u flag. closes #703

Implementation

Adds the ARK_UPDATE environment variable, which is set to 0 or 1 depending on if the -u or --update flag is set. If true, the docker will reinstall ark-analysis before running the jupyter lab command. This is done by replacing the CMD jupyter lab ... line in Dockerfile with a call to the new start_jupyter.sh script.

Remaining issues

I think I can neatly fit #710 in here too, but this is ready for testing in the mean time.

[ackagel]

@cliu72 here's the updatable docker stuff. There isn't a docker image uploaded for this one at the moment; I'll make one so you can just pull that instead of building this locally. I'll let you know when that's up so you can confirm this updates ark code with that -u flag (I tested it on my end and it worked ok, but just a double check).

[srivarra]

Oh my god.

[ackagel]

yes i endured pain

[cliu72]

lol

[cliu72]

Just tested it and it works for me!

[ngreenwald]

Will you also test modifying the ark codebase with jupyter launched, and see what %magic% command or whatever needs to be run within the notebook for the updated changes?

[ngreenwald]

Testing this is what I meant:
cd /opt/ark-analysis && pip install

[ackagel]

@cliu72 awesome! do you mind also running some R and/or pixel clustering stuff, just to make sure it works? I changed the way flowsom gets installed (for speed & less-"compiler error debugging" reasons), and just want to make sure everything works as intended.

Mostly made those changes because travis ran into a compiler error while trying to get this docker built that looked like it might bog down future docker builds on travis. Bonus from that adventure, the docker compilation time is down from ~1hr to ~20mins

[cliu72]

Will you also test modifying the ark codebase with jupyter launched, and see what %magic% command or whatever needs to be run within the notebook for the updated changes?

Yup tested this already. The specific commands are:
cd /opt/ark-analysis
pip install .

(double ampersand doesn't work)

Also @ackagel, I didn't actually need the -u tag to make this work. Without it, the above commands still work to make the code editable.

[ngreenwald]

I think the -u flag basically does that automatically. So for non technical users, they can just run that command if we've changed the code but not made a new release, and they'll get the updated version. For developers, they can run that command the first time to incorporate their changes, and then for subsequent changes with the docker still open, use that command within the notebook. Is that correct Adam?

How long did it take when you ran that in the notebook?

[cliu72]

Oh I see. So using the -u flag should pull the latest changes in ark, and running the cd/pip commands will pull any local changes the user makes. Is that right? If that's the case, I tested the second part of that (cd/pip part) and it works (not sure if you wanted me to test the first part of that too? not sure how to do that)

Running the pip install command in jupyter takes ~20 seconds.

Tested pixel clustering and it seems fine. Just a note - Alex created our own version of the FlowSOM package and edited it a bit. I don't think his changes are super obvious when testing, so I didn't confirm whether our version of FlowSOM is being used. Just wanted to confirm that with your changes, you're using our version of FlowSOM and not installing from Bioconductor.

[ackagel]

The -u flag doesn't pull changes; users still need to run git pull. The -u flag just runs the cd/pip commands automatically, before opening up jupyter.

I can confirm that we're using our version of FLowSOM, not Bioconductor's.

@ngreenwald sounds like we're all good here, so I'll update the branch and merge it in post-standup

[ackagel]

@ngreenwald just need you to approve to unblock merging

[ngreenwald]

Looks good