Commands to find mal-formed workflow version and to create updates for Database

[aclum]

Implement 2 commands:

• find-affected

These are re-id studies where the there are two versions appended for read qc and reads based analysis

nmdc:sty-11-076c9980 -103 read_qc_analysis_activity_set
nmdc:sty-11-dcqce727 - 91 read_qc_analysis_activity_set

nmdc:sty-11-076c9980 - 103 read_based_taxonomy_analysis_activity_set
nmdc:sty-11-dcqce727 - 48 nmdc:sty-11-dcqce727

[ssarrafan]

Appears to be active. Moving to next sprint

[aclum]

@mbthornton-lbl update summary of the issue

read_qc_analysis_activity_set
194 workflow activity records in mongo with multiple versions IDs (.1.1).
Of those 194:
190 the data_object_set records do not match, they are .1.1.1 - URLs don't resolve
4 of the data_object_set records are consistent. - URLs do resolve
108 workflow activity records are .1 in mongo but the data_object_set record name and url are .1.1 - URLs don't resolve

read_based_taxonomy_analysis_activity_set
151 workflow activity records in monogo with multiple version IDs (.1.1).
Of those 151:
149 the data_object_set records do not match, they are .1.1.1 - URLs don't resolve
2 of the data_object_set records are consistent -URLs do resolve
108 workflow activity records are .1 in mongo but the data_object_set record name and url are .1.1 - URLs don't resolve

metagenome_assembly_set
no activity records in mongo have multiple versions
79 identifiers the ID with the blade that is in mongo doesn't exist on the file system - URLs don't resolve
180 workflow activity records are .1 in mongo but the data_object_set record name and url are .1.1 - URLs don't resolve

[mbthornton-lbl]

affected_workflow_records.json
find_affected_workflows.log

[mbthornton-lbl]

Summary of find-affected-workflows run on Prod:

Total of 346 Malformed Workflow ID versions found in the database
194 from reads QC
151 read based taxonomy
0 from metagenome assembly

Malformed data paths on the Filesystem:
180 malformed data file paths for metagenome assembly

[aclum]

I've spun out the ID blade issue to a new ticket #201. This ticket., #186 is higher priority to fix than #201, since it impacts more records.

[mbthornton-lbl]

This is a duplicate of
#198
and can be closed