[FEATURE] [Desktop] Run trained pipelines

[subdavis]

Ability to run new trained pipelines

[BryonLewis]

Just some thoughts and notes about the process forward for the trained pipelines being run on the system and how it could be done.

Possible Options:

• Recursive search for trained pipelines in the jobs folder
  • This seems annoying and bad
  • When do we do this (each request for pipelines, on load)?
• Storage of list of datasetIds that have trained pipelines and is verified by checking on system load (then when pipelines are requested it returns the default pipelines and the list of trained pipelines). This requires updating the list when a trained pipeline has been completed and adding it to the list.
  • something like trainedPiplines.json inside of the root of the project directory which contains similar data used for the web version.
  • 
  • the file is updated when training completes successfully and a valid .pipe and model file are generated
  • folderID is the job folder ID for the trained pipeline
  • On initial launch of app, the list is verified incase the user manually deleted projects/jobs
• Copying/Moving trained pipelines to a specific location and utilizing that as the location to search for pipelines
  • data duplication stinks and models can be quite large so copying is probably not the best
  • Don't necessarily have to duplicate data could move it instead. No reason why the result of a job needs to live solely in it's folder. We can keep the log and other information there but the trained pipeline and model could be moved
  • Trained Pipelines become independent of jobs making it easier to import trained pipes or export the trained pipes in the future
  • Similar behavior to the web version where the results are transferred to an indepedent location
  • Can remove trained pipelines without deleting job history logs
  • This could be combined with a trainedPipeline.json file which
  • This could also be used with the current structure for determining the pipelines in the configs/folder by just pointing it to this new folder and tagging the pipelines as 'trained'

Personally I'm leaning towards the third option with something like /VIAME_DATA/DIVE_Pipelines being the location.

Some additional notes:

• Trained pipelines will require the additional parameter that Matt has talked about to run properly.
• Will also need to modify the pipelineRelativeDir to point to the trained folder when running trained pipelines.
• Also include some filtering when specifying trained pipeline names so it doesn't duplicate an existing name. They are technically done by folderId, but this would make the list of pipelines prevent conflicts.

[subdavis]

Personally I'm leaning towards the third option with something like /VIAME_DATA/DIVE_Pipelines being the location.

Agree with your conclusion. This seems easiest to manage.