Directory importer

[ximenesuk]

This PR adds a revised auto importer that uses a limited nested directory structure to create containers and initiate imports. An example directory is included though this should probably be changed to use fakes and add fake screens. A readme file describes the directory structure.

The intention here is to firstly allow different import scenarios by rsyncing data into place and then running the script, avoiding duplication of configuration files. Ultimately with proposed modifications to DropBox a simple rsync to the DropBox folder should have the same effect as running the script.

[pwalczysko]

First test done locally. It does what is says it would do. I certainly need more explanation about the Additionally the structure could be represented as a straight text file... (from Readme.txt) -> very interesting, just need to understand how. Will continue tomorrow.

[ximenesuk]

@pwalczysko Because of the way the script works it would be possible with minor changes to ask the script to create P/D and S containers based on a text file, similar to the way the previous version did using auto_import.users. This might be useful if you wanted a test set-up that didn't include any real imports but needed pre-created containers.

[pwalczysko]

Tested further locally.
The creation of new Projects and Datasets when the corresponding folders are empty works as expected.
The creation of new orphaned empty Datasets when the /orphans/... subfolders are empty works as expected.
The creation of Screens works as expected.
The creation of orphaned plates works as expected.
The creation of empty Screens when nothing in folder works as expected.
The creation of empty plates - does nothing, but this is expected (from the UI point of view we do not have empty plates - a plate must have data in it, otherwise it does not exist - although there is also a bug allowing user to delete a Run inside a plate with Insight crashing after open-attempt on the (now empty) plate). So I would say this is as expected (from the script point of view) as well.

[pwalczysko]

As discussed with @ximenesuk , the --no-imports and --file.... options do not work yet.

[ximenesuk]

@pwalczysko I've added these options in the latest commits.

[pwalczysko]

Both options (-f and -n) work fine.

Admittedly much less frequent case, but possible : how about creating a project with no datasets (and, obvously, no images) inside ? Possibly

• using no_datasets (analogous to the existing no_projects) option for the folder import (the normal and -n cases).
• more important (for this particular feature) is the -f case - possibly the same trick (no_datasets) ?

[joshmoore]

@ximenesuk : prefer to have this merged before the general cleanup (i.e. file deletions, etc)?

[ximenesuk]

@joshmoore I would prefer this to be merged now and then maybe let @pwalczysko do the clean-up so that he has the configuration he needs. I'll add the use case mentioned above in a separate PR.

[joshmoore]

Understood, and agreed. Merging.