Add modify-add feature #52

minhaj10p · 2019-01-17T11:35:22Z

Description

Attempts to process manipulation attributes in the profile modify section.
Linking to #12.

Feature set

Concurrent import chain traversal is enabled to fetch the root catalog for each import
Recursively searches the import chain for the controls/subcontrols referenced by the root profile for add manipulation attribute
Appends parts with the same class instead of adding a new part

generator/mapper.go

minhaj10p · 2019-01-17T14:59:59Z

generator/mapper.go

@@ -123,3 +123,72 @@ func getMappedCatalogControlsFromImport(importedCatalog *catalog.Catalog, profil
 	}
 	return newCatalog, nil
 }
+
+func getCatalogForImport(ctx context.Context, i profile.Import, c chan *catalog.Catalog, e chan error) {


@anweiss
This method traverses the entire import tree and finds the catalog concurrently. The first goroutine that finds it pushes it to the channel and all other goroutines get killed thanks to the context package.
Let me know if this approach works better in stead of the old fashioned LNR tree parsing

@heavypackets @justincormack for review

minhaj10p · 2019-01-19T10:40:55Z

Currently this PR is shutting down unit tests as import chain traversal isn't possible without file system.
If you can approve, we can commit mock profiles to pass those tests.

anweiss · 2019-01-19T18:42:47Z

@minhaj10p would it be possible to include the mocks in this PR?

minhaj10p · 2019-01-20T06:28:48Z

@minhaj10p would it be possible to include the mocks in this PR?

Sure. I can create a folder and put some profiles in it. Another thing just occurred to me is that we always put the manipulation attributes in the same profile so it never has to go in the find the manipulation attribute in the import chain. I can try and see if that works.

minhaj10p · 2019-01-20T06:40:39Z

@anweiss Tests updated. All are passing now with 73.0% code coverage

minhaj10p · 2019-01-20T18:28:50Z

@anweiss Let me know if theres anything you need changed in this pull request. Thanks.

anweiss

@minhaj10p please add a space after // in all comments.

anweiss · 2019-01-18T17:32:03Z

generator/mapper.go

@@ -123,3 +123,72 @@ func getMappedCatalogControlsFromImport(importedCatalog *catalog.Catalog, profil
 	}
 	return newCatalog, nil
 }
+
+func getCatalogForImport(ctx context.Context, i profile.Import, c chan *catalog.Catalog, e chan error) {


@heavypackets @justincormack for review

generator/manipulation.go

generator/mapper.go

generator/manipulation.go

anweiss · 2019-01-21T21:24:45Z

@minhaj10p I tested this on my end with the FedRAMP High profile, and it looks like it's appending some erroneous structs. For example:

Parts: []catalog.Part{
	catalog.Part{
		Id:    "ir-9_smt",
		Class: "statement",
		Title: "",
	},

	catalog.Part{
		Id:    "ir-9_gdn",
		Class: "guidance",
		Title: "",
	},

	catalog.Part{
		Id:    "ir-9_obj",
		Class: "objective",
		Title: "",
	},

	catalog.Part{
		Id:    "",
		Class: "assessment",
		Title: "",
	},

	catalog.Part{
		Id:    "",
		Class: "assessment",
		Title: "",
	},

	catalog.Part{
		Id:    "",
		Class: "assessment",
		Title: "",
	},

	catalog.Part{
		Id:    "",
		Class: "justification",
		Title: "",
	},
},

Note the empty "assessment" and "justification" parts.

Also as far as the Prose, you should be able to just include it in your template.

minhaj10p · 2019-01-22T06:19:09Z

@minhaj10p I tested this on my end with the FedRAMP High profile, and it looks like it's appending some erroneous structs. For example:

Parts: []catalog.Part{
	catalog.Part{
		Id:    "ir-9_smt",
		Class: "statement",
		Title: "",
	},

	catalog.Part{
		Id:    "ir-9_gdn",
		Class: "guidance",
		Title: "",
	},

	catalog.Part{
		Id:    "ir-9_obj",
		Class: "objective",
		Title: "",
	},

	catalog.Part{
		Id:    "",
		Class: "assessment",
		Title: "",
	},

	catalog.Part{
		Id:    "",
		Class: "assessment",
		Title: "",
	},

	catalog.Part{
		Id:    "",
		Class: "assessment",
		Title: "",
	},

	catalog.Part{
		Id:    "",
		Class: "justification",
		Title: "",
	},
},

Note the empty "assessment" and "justification" parts.

Also as far as the Prose, you should be able to just include it in your template.

@anweiss
These erroneous looking part's are coming from the NIST-80053 catalog where the parts have no ids and only classes

         <part class="assessment">
            <prop class="method">EXAMINE</prop>
            <part class="objects">
               <p>Incident response policy</p>
               <p>procedures addressing information spillage</p>
               <p>incident response plan</p>
               <p>records of information spillage alerts/notifications, list of personnel who should receive alerts of information spillage</p>
               <p>list of actions to be performed regarding information spillage</p>
               <p>other relevant documents or records</p>
            </part>
         </part>
         <part class="assessment">
            <prop class="method">INTERVIEW</prop>
            <part class="objects">
               <p>Organizational personnel with incident response responsibilities</p>
               <p>organizational personnel with information security responsibilities</p>
            </part>
         </part>
         <part class="assessment">
            <prop class="method">TEST</prop>
            <part class="objects">
               <p>Organizational processes for information spillage response</p>
               <p>automated mechanisms supporting and/or implementing information spillage response actions and related communications</p>
            </part>
         </part>

I haven't been able to implement nested part in our code generation.
The last part with justification class comes from the profile with no id in the profile.
They look erroneous probably because it doesn't have the Prose section populated.

It looks somewhat like this if just included in the template

	catalog.Part{
		Id:  "ir-9_obj",
		Class: "objective",
		Title: "",
		Prose: "{{ } [] [p] [{{http://csrc.nist.gov/ns/oscal/1.0 p} [] [] [] [] [] [] [] [] [] [] [] Determine if the organization:  }] [] [] []
        }",

anweiss · 2019-01-23T15:01:20Z

@minhaj10p ah ok. That makes sense. I'll try to see what I can come up with re. nested parts and prose to help you out on this.

generator/profile.go

minhaj10p · 2019-01-24T15:48:20Z

@anweiss if you merge this, let me know so I can rebase the other dependent branches.

anweiss · 2019-01-24T17:32:23Z

@minhaj10p after speaking internally with @justincormack, we can go ahead and merge this. Thanks again for your patience.

anweiss

LGTM

minhaj10p mentioned this pull request Jan 17, 2019

Profile processing for catalog code generation #12

Open

minhaj10p commented Jan 17, 2019

View reviewed changes

generator/mapper.go Outdated Show resolved Hide resolved

minhaj10p commented Jan 17, 2019

View reviewed changes

anweiss assigned anweiss, minhaj10p and asadullah-yousuf-10p Jan 17, 2019

anweiss added the enhancement New feature or request label Jan 17, 2019

anweiss added this to the OSCAL 1.0 milestone Jan 17, 2019

anweiss unassigned asadullah-yousuf-10p Jan 17, 2019

minhaj10p force-pushed the manipulation branch 4 times, most recently from 1ad414d to 56a14e9 Compare January 19, 2019 10:34

minhaj10p force-pushed the manipulation branch from 56a14e9 to c9cc98e Compare January 19, 2019 10:49

minhaj10p force-pushed the manipulation branch from c9cc98e to 6e4c915 Compare January 20, 2019 06:39

minhaj10p force-pushed the manipulation branch 2 times, most recently from 1b25402 to 13ebd1e Compare January 20, 2019 07:19

This was referenced Jan 20, 2019

Generate command doesn't properly handle relative profile imports #57

Closed

Import path #58

Merged

anweiss suggested changes Jan 21, 2019

View reviewed changes

anweiss requested review from justincormack and heavypackets January 21, 2019 21:01

anweiss mentioned this pull request Jan 21, 2019

Generate XML and JSON catalogs from processed profiles #61

Open

anweiss reviewed Jan 23, 2019

View reviewed changes

generator/profile.go Outdated Show resolved Hide resolved

minhaj10p added 3 commits January 24, 2019 00:37

Add modify-add feature

3447cfe

Add part parsing

f68e646

Add part addition and traverse import chain

90388ec

minhaj10p force-pushed the manipulation branch 2 times, most recently from 29d8b41 to aa73643 Compare January 23, 2019 19:43

Add spaces before comments and Fix spell checks

2618aa0

minhaj10p force-pushed the manipulation branch from aa73643 to 2618aa0 Compare January 23, 2019 20:00

minhaj10p mentioned this pull request Jan 24, 2019

Set param #64

Merged

Fix file close

1cfcf64

anweiss mentioned this pull request Jan 24, 2019

Create a new release for 0.2.0 #38

Closed

justincormack approved these changes Jan 24, 2019

View reviewed changes

anweiss removed the request for review from heavypackets January 24, 2019 17:32

anweiss approved these changes Jan 24, 2019

View reviewed changes

anweiss merged commit 415eaf4 into docker-archive:master Jan 24, 2019

minhaj10p deleted the manipulation branch January 24, 2019 18:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add modify-add feature #52

Add modify-add feature #52

minhaj10p commented Jan 17, 2019 •

edited

Loading

minhaj10p Jan 17, 2019

anweiss Jan 18, 2019

minhaj10p commented Jan 19, 2019

anweiss commented Jan 19, 2019

minhaj10p commented Jan 20, 2019

minhaj10p commented Jan 20, 2019

minhaj10p commented Jan 20, 2019

anweiss left a comment

anweiss Jan 18, 2019

anweiss commented Jan 21, 2019

minhaj10p commented Jan 22, 2019 •

edited

Loading

anweiss commented Jan 23, 2019

minhaj10p commented Jan 24, 2019

anweiss commented Jan 24, 2019

anweiss left a comment

Add modify-add feature #52

Add modify-add feature #52

Conversation

minhaj10p commented Jan 17, 2019 • edited Loading

Description

Feature set

minhaj10p Jan 17, 2019

Choose a reason for hiding this comment

anweiss Jan 18, 2019

Choose a reason for hiding this comment

minhaj10p commented Jan 19, 2019

anweiss commented Jan 19, 2019

minhaj10p commented Jan 20, 2019

minhaj10p commented Jan 20, 2019

minhaj10p commented Jan 20, 2019

anweiss left a comment

Choose a reason for hiding this comment

anweiss Jan 18, 2019

Choose a reason for hiding this comment

anweiss commented Jan 21, 2019

minhaj10p commented Jan 22, 2019 • edited Loading

anweiss commented Jan 23, 2019

minhaj10p commented Jan 24, 2019

anweiss commented Jan 24, 2019

anweiss left a comment

Choose a reason for hiding this comment

minhaj10p commented Jan 17, 2019 •

edited

Loading

minhaj10p commented Jan 22, 2019 •

edited

Loading