Docs: add info on env variables

[nick-youngblut]

paths.py lists env variables that are used to set paths (CAIMAN_DATA and CAIMAN_TEMP), but it appears that these env variables are not mentioned in the docs.

These variables are especially important when non-standard paths are needed, such as with integrating caiman into Nextflow or Snakemake pipeline.

[nick-youngblut]

Setting CAIMAN_DATA results in an error in cm.save_memmap:

Traceback (most recent call last):
  File "/home/nickyoungblut/dev/nextflow/lizard-wizard/bin/caiman_run.py", line 83, in <module>
    main(args)
  File "/home/nickyoungblut/dev/nextflow/lizard-wizard/bin/caiman_run.py", line 72, in main
    fname_new = cm.save_memmap([args.img_file], base_name='memmap_', order='C')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/checkpoint/multiomics/nextflow-work/nickyoungblut/conda/lizard_wizard-6c29c38f897e1b902748ec5eaa30c78c/lib/python3.11/site-packages/caiman/mmapping.py", line 537, in save_memmap
    os.rename(fname_tot, fname_new)
FileNotFoundError: [Errno 2] No such file or directory: 'caiman_data/temp/memmap_d1_684_d2_800_d3_1_order_C' -> 'caiman_data/temp/caiman_data/temp/memmap_d1_684_d2_800_d3_1_order_C_frames_408.mmap'

This is due to:

fname_new = os.path.join(caiman.paths.get_tempdir(), caiman.paths.fn_relocated(f'{fname_tot}_frames_{Ttot}.mmap'))

in mmapping.py in which caiman.paths.fn_relocated already includes the temp directory:

def fn_relocated(fn:str, force_temp:bool=False) -> str:
    """ If the provided filename does not contain any path elements, this returns what would be its absolute pathname
        as located in get_tempdir(). Otherwise it just returns what it is passed.

        The intent behind this is to ease having functions that explicitly mention pathnames have them go where they want,
        but if all they think about is filenames, they go under CaImAn's notion of its temporary dir. This is under the
        principle of "sensible defaults, but users can override them".
    """
    if os.path.split(fn)[0] == '': # No path stuff
        return os.path.join(get_tempdir(), fn)
    elif force_temp:
        return os.path.join(get_tempdir(), os.path.split(fn)[1])
    else:
        return fn

...so os.path.join(caiman.paths.get_tempdir(), is adding an extra temp directory.

[pgunn]

Thanks for spotting this; I'll fix it in the dev branch shortly and the fix will go out with the next release.

[nick-youngblut]

Thanks for the quick reply! I thought that I was going to have to patch the code, for the near future

[pgunn]

I do my best to keep it actively maintained.

[pgunn]

Fixed in dev; next release should be soonish (I have a few other smaller things to get in there before it goes out)

[nick-youngblut]

Fixed in dev; next release should be soonish (I have a few other smaller things to get in there before it goes out)

Thanks for the quick fix! It would be great to get the release soon so that the bioconda recipe is updated, but I definitely get that creating new releases is usually a ton of work (on top of your actual job that you get paid for).

[pgunn]

This actually is part of my job (I work at an academic institute that maintains Caiman). The release process isn't that bad, I just don't do releases too often because then they'd be very small and the changelogs would have too many tiny entries.

My ETA for the next release is less than a week; I might even do it tomorrow depending on what comes up.

[nick-youngblut]

That's great that it's part of your job. Sadly, many open source projects are maintained by people that have moved on in their jobs, or students/postdocs that already have published on the software, so no real compensation is provided for further working on the open source project. So, the reponse to issues can often be "I'm really busy right now (with my actual work) so I won't be able to address this issue in the near future".

Thanks again for the quick response!