Generate human-readable, flattened manifests

[glguy]

This changest the behavior of the rack manifest build command to always produce a single, flattened manifest that attempts to use names from the original directories for clarity. Example output from Turnstile:

manifest.yaml

footprint:
  data-graphs:
  - http://rack001/turnstiledata
  - http://rack001/do-178c
  model-graphs:
  - http://rack001/model
name: Turnstile
steps:
- model: 00_RACK-Ontology/import.yaml
- nodegroups: 01_queries
- model: 02_GE-Ontology/import.yaml
- data: 03_DO-178C/import.yaml
- data: 04_OwlModels/import.yaml
- data: 05_TurnstileDevelopmentPlanData/import.yaml
- data: 06_PlanningDocuments/import.yaml
- data: 07_HazardAssessment/import.yaml
- data: 08_TurnstileSystemDesign/import.yaml
- data: 09_TurnstileSystemRequirements/import.yaml
- data: 10_TurnstileHighLevelRequirements/import.yaml
- data: 11_TurnstileLowLevelRequirements/import.yaml
- data: 12_TurnstileRequirementModel/import.yaml
- data: 13_CounterApplicationReviews/import.yaml
- data: 14_CounterApplicationTesting/import.yaml
- data: 15_TurnstileSystemSpec/import.yaml
- data: 16_CounterApplicationRequirementSpec/import.yaml
- data: 17_CounterApplicationSoftwareDes/import.yaml
- data: 18_SystemVerificationReport/import.yaml
- data: 19_Objectives/import.yaml
- data: 20_TurnstileBaselines/import.yaml

[glguy]

@russell-d-e Does this match the kind of output you were aiming for?

[russell-d-e]

@glguy

What I was originally thinking I was think to just flatten so that there is a top-level manifest that pulls in other manifest correct order. This top level manifest would just have references into subfolders with models, nodegroups, and data.

What I was trying to remedy is was what I was seeing in the ARCOS data where it would be like something like (these aren't the exact name of the packages, but it will illustrate the issue I was try to address):

00_manifest
|__01_model
|__02_nodegroup
|__03_manifest
|____04_manifest
|______05_data
|____06_data
07_manifest
|__08_model
|__09_data

After adding human names for the manifests it was more understandable:

00_RACK-Ontology
|__01_model
|__02_nodegroup
|__03_DO-178C
|____04_CWEs
|______05_data
|____06_data
07_Boeing-Ontology
|__08_model
|__09_data

but this would still leave some strange nesting of manifests that didn't make sense (CWEs nested inside DO-178C, and DO-178C inside of RACK-Ontology), as well as some weird ordering quirks that the "stuff" in DO-178C is actually ingested after CWEs, eventhough the manifest is encountered first.
so what I then wanted to do was just make a top-level manifest that essentially creating the build order for manifests of actual "stuff":

00_RACK-Ontology
|__01_model
|__02_nodegroup
03_CWEs
|__04_data
05_DO-178C
|__06_data
07_Boeing-Ontology
|__08_model
|__09_data

In this way the ingestion order and data was really clear. I think the example you have fixes these things so I think it would work, I just worry that the top-level may be too busy as all the models, nodegroups, and data are at the same level but different folders. Its not shown in your example but you might get something like this in a manifest:

model: 00_PROJECT1/import.yaml
modegroups: 01_PROJECT1
data: 02_PROJECT1/import.yaml
model: 03_PROJECT2/import.yaml
data: 04_PROJECT2/import.yaml

while the zip file would look like:

00_PROJECT1
01_PROJECT1
02_PROJECT1
03_PROJECT2
04_PROJECT2
manifest.yaml

[glguy]

@russell-d-e I've updated the code to generate names closer to what you described. I also have tried using the name field from the manifest to label them, rather than to guess based on the path name. I'm hoping this is a more reliable solution to detecting duplication as well as giving the user more control over picking naming.

footprint:
  data-graphs:
  - http://rack001/turnstiledata
  - http://rack001/do-178c
  model-graphs:
  - http://rack001/model
name: Turnstile
steps:
- model: 00_RACK ontology/01_model/import.yaml
- nodegroups: 00_RACK ontology/02_nodegroups
- model: 03_Turnstile ingestion/04_model/import.yaml
- data: 03_Turnstile ingestion/05_data/import.yaml
- data: 06_Turnstile/07_data/import.yaml
- data: 06_Turnstile/08_data/import.yaml
- data: 06_Turnstile/09_data/import.yaml
- data: 06_Turnstile/10_data/import.yaml
- data: 06_Turnstile/11_data/import.yaml
- data: 06_Turnstile/12_data/import.yaml
- data: 06_Turnstile/13_data/import.yaml
- data: 06_Turnstile/14_data/import.yaml
- data: 06_Turnstile/15_data/import.yaml
- data: 06_Turnstile/16_data/import.yaml
- data: 06_Turnstile/17_data/import.yaml
- data: 06_Turnstile/18_data/import.yaml
- data: 06_Turnstile/19_data/import.yaml
- data: 06_Turnstile/20_data/import.yaml
- data: 06_Turnstile/21_data/import.yaml
- data: 06_Turnstile/22_data/import.yaml
- data: 06_Turnstile/23_data/import.yaml