Desktop Using Trained Pipelines

[BryonLewis]

Pipeline outputs follow the way the web is done with a {type}.pipe file and a .zip file that are moved into a folder in the DIVE_Pipelines. This folder is spread onto the pipeline object with the category of 'trained' when the pipeline list is requested.

• After a pipeline is complete it will check the job folder for a category_models folder and then check the contents for the [type].pipe file. Then it will move that into the DIVE_Pipelines directory.
• Modified the exit function for training to include some errors if it can't find the .pipe file or directory. I wish I knew a bit more about what is expected in there for the different training configurations.

Question:

• Let me know if you think it should be a JSON file indexed with trained pipeline folder instead of utilizing the folder names. The one nice thing about folder names is that if a user wants to import or delete pipelines right now they can do it by going to the folder and either adding or removing the folder.

Note:
This has had the basic train_netharn_cascade.viame_csv.conf testing process done on it.

Future PRs:

• Deletion of trained pipelines through an interface. I don't know if this lives in "Training" or somewhere else but it may be nice to do that.
• As an aside to the above I think I want to implement dataset deletion from the recents folder
• Importing trained pipelines from the web to the desktop version (I've tested this manually by moving the files into a proper folder and it works well at least for train_netharn_cascade.viame_csv.conf pipelines). This could be done in a few ways:
  • I think either having them provide a folder and giving it a name and it moves/copies the items to that directory after some basic checking to make sure there is a .pipe and other data
  • Allowing a zip file to be uploaded and we use something like jszip or other utilities to check it.
  • Advanced?? - we provide them with a link directly to the folder and let them add/remove files. Obviously this has the potential for allowing to break stuff as well.

[subdavis]

Looks like u got some conflicts.

[subdavis]

One minor thing. This looks great!

[subdavis]

Suggested change

	const trainedPipelines = computed(() => {
	const list: string[] = [];
	Object.entries(unsortedPipelines.value).forEach(([, category]) => {
	category.pipes.forEach((pipe) => {
	if (pipe.type === 'trained') {
	list.push(pipe.name);
	}
	});
	});
	return list;
	});
	const trainedPipelines = computed(() => {
	return unsortedPipelines.value.trained || [];
	});

Untested, but doesn't this work? The pipes are already divided by category. You don't need to check their type, that's just a convenience of the schema.

[BryonLewis]

Needs to be slightly changed because I'm using an includes on an array and trained is an Category which contains an array of Pipes. So it's either modify the includes to be a find or something else or modify the source. I've adjusted to simplify the computed property with TS guaranteeing that if trained exists then pipes is an array..

interface Pipe {
  name: string;
  pipe: string;
  type: string;
}

interface Category {
  description: string;
  pipes: Pipe[];
}