Is there a way to change where models will be downloaded?

[VasylVaskivskyi]

I have a problem with running Cellpose in a pipeline inside a container. The problem is that at each run of the pipeline Cellpose redownloads models, although they are already present in the container in the proper location. This happens because Cellpose looks for models in the $HOME directory, and $HOME directory is changed during the pipeline runtime.

So is there a way to change the directory in which Cellpose will look for model files?

[tibuch]

I have a similar issue. My home is very small since all my files are stored in another file-system. It would be great if we could provide a location for the downloaded models.

I guess this would be the place: https://github.com/MouseLand/cellpose/blob/master/cellpose/models.py#L45

[Prehani]

I also had this issue when trying to run cellpose on a distributed compute system. The workaround I found is to create a miniconda environment with the relevant packages that you need installed, run a basic script to have cellpose import models, pack with conda-pack, and pass that tarred conda environment into your container. This is far from optimal, but it gets around downloading models every time. There's a bug with the system environment variable export where it doesn't actually take that into account for the model paths.

[chrisroat]

If you are using a recent version (the released version on pypi does not have this), you can set this environmental variable:
CELLPOSE_LOCAL_MODELS_PATH

[Prehani]

Hi Chris--I saw your PR on this and looked through it: everything made sense, but I opened another issue on the Cellpose Github where I can't get that to work properly for me on distributed systems/containers. Not sure if that was tested for/considered, but just what I've seen.

[chrisroat]

Was it working prior to the PR?

For reference, the issue is at #334

[Prehani]

No, I don't think I've gotten it to work at all either pre or post PR.

[chrisroat]

This may not be a cellpose problem. It sounds like its likely an issue understanding how your HPC environment gives you access to a home directory (default place for models) or to the directory you specify with CELLPOSE_LOCAL_MODELS_PATH.

[Prehani]

Hmm, not 100% sure: did you see my latest comment on #334? There is no home directory on the execute side, and the directory I specify for the CELLPOSE_LOCAL_MODELS_PATH should allow for full permissions as I'm able to set the environment variable successfully and then extract everything I'd need from there. What additional permissions would I need? I will verify with our Condor group on the environment variable and permissions.

[Prehani]

Alright, I think I figured out (at least in part) where my issue came from: I believe the way I was setting whatever path I happened to be on was incorrect. For anybody else who happens to have this issue, here's what I ended up helping me: https://stackoverflow.com/questions/4774054/reliable-way-for-a-bash-script-to-get-the-full-path-to-itself

[chrisroat]

OK. This can be closed then, I believe.

[VasylVaskivskyi]

Hi @chrisroat , so if I want to use CELLPOSE_LOCAL_MODELS_PATH to which directory it should point /.../.cellpose or /.../.cellpose/models ?

[chrisroat]

It should point to the exact directory you want the models downloaded into. It does not add ".cellpose" or ".cellpose/models". This code snippet shows the logic -- either the value of CELLPOSE_LOCAL_MODELS_PATH is used, or it defaults to {home}/.cellpose/models

cellpose/cellpose/models.py

Lines 29 to 31 in ec85934

	_MODEL_DIR_ENV = os.environ.get("CELLPOSE_LOCAL_MODELS_PATH")
	_MODEL_DIR_DEFAULT = pathlib.Path.home().joinpath('.cellpose', 'models')
	MODEL_DIR = pathlib.Path(_MODEL_DIR_ENV) if _MODEL_DIR_ENV else _MODEL_DIR_DEFAULT

[VasylVaskivskyi]

I checked, this solution works for me. Thank you very much. The issue can be closed.

[VasylVaskivskyi]

Fixed in bddabe8