Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON schema #10

Closed
anweiss opened this issue Aug 8, 2017 · 28 comments
Closed

JSON schema #10

anweiss opened this issue Aug 8, 2017 · 28 comments

Comments

@anweiss
Copy link
Contributor

anweiss commented Aug 8, 2017

We should include a JSON schema as well.

Tracking WIP in #6

@anweiss
Copy link
Contributor Author

anweiss commented Sep 1, 2017

Adding further commentary to this topic. Looking through the current mechanisms for building both the schema and translating existing standards/publications to OSCAL, IMO it seems a bit convoluted and not the most well-suited to supporting JSON schema. It would be great if we could come up with generic approaches for both generating XSD and JSON schema and also for mapping/executing OSCAL translations. Understanding that automated conversion is not straightforward and each case will have to be done ad-hoc, if we can at least document a high-level process for standards bodies and catalog maintainers to follow, that would be incredibly helpful.

From an implementation perspective, given there is really no direct equivalent to XProc and XSLT in the JSON world, it would also be helpful to develop similar tools/mechanisms that can be used to create the necessary declarations from existing standards for use with JSON schema in the same manner.

@wendellpiez
Copy link
Contributor

wendellpiez commented Sep 14, 2017

Bridging between XML and JSON is not a trivial problem although it isn't impossible either, at least as long as we don't expect the same affordances on both sides. In fact I imagine that since "catalog semantics" are often not exposed directly in OSCAL (which offers a basis without constraining much), the naively-corresponding JSON might be especially bad.

But perhaps conversions may be only one way, for example -- even a one-way "reduction" could often be useful, if it makes OSCAL-tagged data available to JSON. So rather than trying to port over constraint sets (which are designed for very different data models) maybe our focus should be on producing JSON 'dumps' of OSCAL data using off-the-shelf tooling such as XPath 3.1 JSON serialization, etc. Even if the JSON so produced is ugly, it would be a place to start and a basis for more refined mappings.

I should think that JSON capable of supporting editing OSCAL, for example, would be very different from JSON for simply querying the data and getting values back. (Round-tripping inline markup etc.) But does there have to be one Rule Them All schema for OSCAL data represented in JSON? (Maybe we need not one JSON schema, but several, describing different forms of JSON. Not being a JSON developer. Cool problem though.)

@anweiss
Copy link
Contributor Author

anweiss commented Sep 16, 2017

Great points @wendellpiez! ... I agree with not taking the approach of porting over constraint sets, but instead getting JSON outputs of existing OSCAL data. I'm personally not as well-versed in XPath as I am with JSON Pointer, but am going to start playing with some extractions.

Having multiple JSON schemas representing various OSCAL uses (e.g. editing vs. querying) would certainly be interesting.

@wendellpiez
Copy link
Contributor

@anweiss that is cool. To pull into JSON using XPath, one could do worse than start with the newest specs at https://www.w3.org/TR/xpath-functions-31/#json (for example, as implemented in Saxon). I also think we will have more joy sooner with the profile format than with the catalog format - they are related, but distinct, and profiles will typically not have gobs of relatively unstructured text (at least as far as I've seen). When it comes to catalogs, there are probably going to have to be conventions such as where to flatten / rewrite as markdown, etc. etc. to make a representation one would want to live with.

When it comes to it, if/as catalogs are very stable (once published they are not likely to change much) it may be useful to define regular mappings not only to JSON 'filtered views' but also a fairly plain-and-simple HTML. Then those things could simply be resources accompanying the catalogs.

@anweiss
Copy link
Contributor Author

anweiss commented Sep 19, 2017

Are we locked in to the semantics of XSLT for defining the translations? Would it be appropriate to also incorporate alternative conversion mechanisms via a more traditional programming language (e.g. JavaScript, C#, Golang, etc)? Or are we reserving such tools as the responsibility of OSCAL consumers to build and maintain?

@wendellpiez
Copy link
Contributor

As a developer I'm using XSLT 'cause that's what I know :-) but I should think/hope/imagine that all the transformations should be implementation-independent wrt their definition. If the functional specs are locked into XSLT for now, that is a problem to be fixed -- everything should be documented and exposed so there are no mysteries. Given an explicit spec, any of the tools should be (re) implementable in another language/form i.e. the spec describes a process, mapping or functional relation, not an algorithm or approach.

The reality is however that as of now there is much thinking and many assumptions that are locked into the code at the moment awaiting documentation. Accordingly any help asking questions or documenting answers is appreciated. The code is XSLT so far but the specs do not have to be. (My $0.02 this question is actually way above my pay grade.)

FWIW most of the internals of OSCAL mirror HTML very closely and class is used ubiquitously, partly with an eye to CSS selectors --

@anweiss
Copy link
Contributor Author

anweiss commented Sep 19, 2017

Solid points! I've personally been playing with a couple of parsers (using both Node.js and Go) and methodologies in order to try and identify where the the specific transformation gaps lie.

@wendellpiez
Copy link
Contributor

@anweiss that is awesome. Warning: models are still not completely stable yet (though that doesn't have to be an impediment if you know about it).

@anweiss
Copy link
Contributor Author

anweiss commented Sep 24, 2017

Some updates. So since the source of truth for the schemas as they exist today seems to be the Relax NG Compact files, I've found that it makes more sense to use those .rnc schemas as the source of truth for the JSON schema translations as well. Conducting manual conversions from the Relax NG Compact schemas to JSON schema is far more effective than taking the various OSCAL implementations in XML and trying to reverse-engineer JSON schemas. The JSON schema is much more well-formed and closely aligned with the original constraints defined by Relax NG Compact.

I think this also encourages one to keep the RNC schemas simplistic and concise so as not to warrant the development and/or use of additional tooling simply to spit out JSON schema; in the near-term at least. This also keeps the diffs down to a minimum over time and makes it easier to maintain the JSON schemas by hand.

@anweiss
Copy link
Contributor Author

anweiss commented Sep 24, 2017

Another thing that could be interesting is to define the OSCAL schema in a higher level, abstracted DSL, from which both RNC, JSON schema, etc could be written. Kind of like putting OSCAL in the form of a "behavior driven development"-like style (e.g. Cucumber, Jasmine, etc) where the DSL (written in plain English) defines OSCAL's requirements and RNC, JSON schema, etc represent the implementations.

@wendellpiez
Copy link
Contributor

I am looking at https://dvcs.w3.org/hg/microxml/raw-file/tip/spec/microxml.html. It would be interesting to throw OSCAL data at a MicroXML parser and see if it complains.)

Also we might try a couple of easy tools such as Saxon's implementation of XPath xml-to-json(). I am in no position to evaluate the results but it's a low-cost exercise. Even contemplating the delta between such machine output, and an "optimal mockup" or two, could be interesting.

Also we might keep in mind that different applications or flavors (control and catalog types) of OSCAL may suggest different JSON serializations.

What is not clear to me is what application requirements are served by the JSON schema (either analogous or in contrast to what they do in/for XML). A schema that is optimal for one use may not be for another. (And the same thing is true of the data formats they respectively describe/constrain.)

Meanwhile (or eventually), the DSL you are describing could be defined as a use case of OSCAL with constraint languages embedded.

@anweiss
Copy link
Contributor Author

anweiss commented Sep 26, 2017

Nice! Will take a peek at MicroXML. I have actually started playing with Saxon-JS, but came across a bug with it's implementation of fn:parse-xml -> https://saxonica.plan.io/boards/5/topics/6978?r=6985#message-6985. Haven't touched any of the other SDKs yet but was planning on it.

Per an earlier comment, I've also already got a couple of simple XML-to-JSON serialization/deserialization code samples put together. In this case, the marshaling/unmarshaling has been customized to fit the JSON schema. At the end of the day, the conversion is still one-way. Will publish the samples and generated artifacts to the PR.

Where I'm coming from as far as JSON schema applicability and as a complement to the XML schema is in terms of software and tooling that consumes JSON. For example, JavaScript is a top 10 language (per TIOBE), and since JSON is just a subset of JavaScript, it's easy to consume and well-suited for that development community. The move towards REST has also seen JSON as a more common communication medium; although both XML and JSON can of course be used in that regard. JSON is also a bit more lightweight and ideal-suited for IoT/embedded devices which could possibly consume OSCAL in some capacity in the future.

To your point, each is used in a manner that is optimal for its intended targets. Some sort of high-level DSL that describes constraints could definitely be appealing. Although I'd have to defer to folks more well-versed in that space as it is not one of my areas of expertise lol

@wendellpiez
Copy link
Contributor

I would not be surprised if SaxonJS does not yet support the parse-xml() function as it would ordinarily not be required in its architecture. (In XSLT you would only need this function if your data were not already parsed by the calling application, which it generally is. Interestingly, until XPath 3.0 there was no standard way to parse XML from inside XSLT.)

As for JSON, what interests me is not the case for JSON (which is clear enough) as an interface to OSCAL data (both catalog and profile), but for how/why we want a JSON schema as such. For example, it seems to me that a schema that is useful as a gateway application to validate data across organizational boundary lines, is not necessarily the same as a schema that is used to configure tooling, etc. One might put the question another way: in use, when or why is a schema referenced and by whom, to what purpose and effect -- and is the binding to the data consequently a loose one (no formal schema validation or only at checkpoints) or tight (no application at all without the schema). In XML, it can be either/both - though for OSCAL it does appear that loose bindings should suffice. I don't wish to make too much of these questions however.

BTW it may be a myth that JSON is more lightweight in general ... it seems it can be more lightweight for some applications and the question is more whether/which applications of OSCAL (with its requirements for representing "documentary data") fall into that category. (See http://www.balisage.net/Proceedings/vol10/html/Lee01/BalisageVol10-Lee01.html for an actual study not just reporting what's "obvious".) I am not arguing that a lightweight OSCAL that happened to be in JSON does not have a use case; it definitely does. Just that modeling the optimal JSON for those purposes as well as aligning it with XML OSCAL, may entail questions that go beyond the schema alone - I bet you agree. 😈

@anweiss
Copy link
Contributor Author

anweiss commented Sep 26, 2017

Couldn't agree more! What might help is to simply highlight a list of use cases for the OSCAL JSON schema and include those in the initial documentation. Especially in the early phases, we can at least get something out the door and let the community decide how best to implement/incorporate JSON schema and contribute to the relevant feedback loops

@wendellpiez
Copy link
Contributor

Totally makes sense.

Yeah, was going to add, other use cases for schemas (esp under "loose binding" scenarios) include serving as specification of interfaces for us developers. In the extreme case the only reason to actually run validation is to test the schema against data known to be correct. The main reason to have it is not to run validation processes, but to (be able to) refer to it. Once tools are properly designed and configured, validation becomes only a formality.

To the extent that's the case, we might be able to get away with a period of playing-around-with-JSON before developing any actual validation model (architecturally) and/or runtime constraint model (in an implementation). In other words, experiment more with mappings into JSON than the modeling question head-on, then think about JSON schemas more bottom up. (I bet in doing this the actual mapping issues would emerge quickly.)

On the other hand, if a JSON schema is needed for more than documentation (i.e. explaining what the heck is going on) this approach might not work.

+1 to listing use cases

I bet there are also user stories for "exposing OSCAL data in JSON" not just for a JSON schema.

@anweiss
Copy link
Contributor Author

anweiss commented Sep 28, 2017

@wendellpiez I've started pushing some sample JSON generated from OSCAL-formatted XML. I built a little utility that does the conversions that we could include in the upstream once we start to come to a consensus on the JSON schemas.

@wendellpiez
Copy link
Contributor

Awesome! But I am getting a 404, is there a problem on my side? (Hrm do I have to be added to the repo to see that?)

But anyway that sounds perfect. The utility can be a place to "anchor" the XML-JSON mapping as requirements become clearer.

Of course, if turnabout is fair play you will also write up what the utility does so we can (also) reimplement it in XSLT. :-)

@anweiss
Copy link
Contributor Author

anweiss commented Sep 29, 2017

Whoops ... fixed the link. Give it a go now.

And indeed, will get it documented!

@anweiss
Copy link
Contributor Author

anweiss commented Oct 2, 2017

@anweiss
Copy link
Contributor Author

anweiss commented Oct 7, 2017

@wendellpiez see #60

@anweiss
Copy link
Contributor Author

anweiss commented Mar 5, 2018

@wendellpiez something I think we should revisit is that of a "lightweight", JSON-only variant of the catalog schema. As it is, the equivalent OSCAL-formatted JSON for the SP800-53-rev-4-catalog.xml is nearly 138k lines (which includes properly-formatted line breaks) and is 6.1MB in size. IMO, that is way too big for many use cases. I think it would be great if we could come up with a list of elements that we could strip from the full schema and in turn, create an alternative "lightweight" rendition that significantly reduces the size of OSCAL-formatted JSON. This allows us to retain the core model, but gives folks the ability to more easily build OSCAL-formatted JSON artifacts. Furthermore, communities like OpenControl can create and maintain YAML equivalents based on the "lightweight" JSON model.

@wendellpiez
Copy link
Contributor

@anweiss Totally agree. We might also want some help from a subject expert to come up with the list of elements at least for a plausible use case or two. (Unless this is a no-brainer of course: you tell me!)

But the solution might not be in the schema. I am taking you to mean "define ways of exposing catalog data in well-described slices or subsets", not just reducing granularity of data description (i.e. still exposing the entire data set but reducing "tag overhead" at the cost of machine readability) -- which is one thing that might be meant by "remove elements from the schema". In other words, we are talking about permitting catalogs or their representations to be optimized for low data footprint not just by selecting controls, but by 'semantic' filtering of the contents of controls and subcontrols, to pull only what we know we want. (This is what, in other domains, we might call a "profiling" mechanism. Heh.) FWIW, this is a capability that both the publishers/producers of catalogs, and their consumers, will be wanting.

@anweiss
Copy link
Contributor Author

anweiss commented Mar 7, 2018

You are correct. Not looking to remove elements from the original scheme. Just looking to select only a core set of elements that would be sufficient for a “lightweight” variant

@david-waltermire david-waltermire added the Discussion Needed This issues needs to be reviewed by the OSCAL development team. label Mar 30, 2018
@david-waltermire
Copy link
Contributor

We need to sort out what parity means between the XML and JSON versions of OSCAL. In my mind, we need a way to represent everything that can be represented in XML in JSON. Granted, not everything will be represented in XML and JSON schema, but the content models should be equivalent. On a related note, we also need to figure out a content integrity (signing) approach for both XML and JSON. We can use signing capabilities in XMLDSIG and JOSE for this purpose.

david-waltermire added a commit that referenced this issue Apr 3, 2018
# The first commit's message is:

# This is a combination of 54 commits.
# The first commit's message is:

# This is a combination of 4 commits.
# The first commit's message is:

# This is a combination of 2 commits.
# The first commit's message is:

# This is a combination of 2 commits.
# The first commit's message is:

# This is a combination of 6 commits.
# The first commit's message is:

# This is a combination of 3 commits.
# The first commit's message is:

# This is a combination of 111 commits.
# The first commit's message is:

Initial commit of docs branch.

# This is the commit message #2:

Create CONTRIBUTING.md
# This is the commit message #3:

Create ROADMAP.md
# This is the commit message #4:

Update README.md
# This is the commit message #5:

Create README.md
# This is the commit message #6:

Update README.md
# This is the commit message #7:

Update README.md
# This is the commit message #8:

Create OSCAL-PRODUCERS.md
# This is the commit message #9:

Create OSCAL-CONSUMERS.md
# This is the commit message #10:

Update and rename OSCAL-CONSUMERS.md to USERS.md
# This is the commit message #11:

Update and rename OSCAL-PRODUCERS.md to IMPLEMENTERS.md
# This is the commit message #12:

Rename CONTRIBUTING.md to CONTRIBUTORS.md
# This is the commit message #13:

Update README.md
# This is the commit message #14:

Update README.md
# This is the commit message #15:

Update USERS.md
# This is the commit message #16:

Update README.md
# This is the commit message #17:

Update IMPLEMENTERS.md
# This is the commit message #18:

Update README.md
# This is the commit message #19:

Update ROADMAP.md
# This is the commit message #20:

Update USERS.md
# This is the commit message #21:

Update CONTRIBUTORS.md
# This is the commit message #22:

Update README.md
# This is the commit message #23:

Update README.md
# This is the commit message #24:

Update IMPLEMENTERS.md
# This is the commit message #25:

Update IMPLEMENTERS.md
# This is the commit message #26:

Rename CONTRIBUTORS.md to CONTRIBUTING.md
# This is the commit message #27:

Create control.md
# This is the commit message #28:

Update control.md
# This is the commit message #29:

Update control.md
# This is the commit message #30:

Update control.md
# This is the commit message #31:

Update control.md
# This is the commit message #32:

Add files via upload
# This is the commit message #33:

Update control.md
# This is the commit message #34:

Create temp.md
# This is the commit message #35:

Delete NIST-SP-800-53-Rev4-AC1.png
# This is the commit message #36:

Add files via upload
# This is the commit message #37:

Delete temp.md
# This is the commit message #38:

Add files via upload
# This is the commit message #39:

Update control.md
# This is the commit message #40:

Add files via upload
# This is the commit message #41:

Add files via upload
# This is the commit message #42:

Update control.md
# This is the commit message #43:

Update CONTRIBUTING.md
# This is the commit message #44:

Update CONTRIBUTING.md
# This is the commit message #45:

Update USERS.md
# This is the commit message #46:

Update CONTRIBUTING.md
# This is the commit message #47:

Delete CONTRIBUTING.md
# This is the commit message #48:

Delete USERS.md
# This is the commit message #49:

Add files via upload
# This is the commit message #50:

Delete CSA-CCM-IAM02.png
# This is the commit message #51:

Update control.md
# This is the commit message #52:

Update control.md
# This is the commit message #53:

Update control.md
# This is the commit message #54:

Update control.md
# This is the commit message #55:

Update control.md
# This is the commit message #56:

Update control.md
# This is the commit message #57:

Update control.md
# This is the commit message #58:

Update control.md
# This is the commit message #59:

Update control.md
# This is the commit message #60:

Update control.md
# This is the commit message #61:

Delete NIST-SP-800-53-AC1-in-OSCAL-XML.png
# This is the commit message #62:

Update README.md
# This is the commit message #63:

Update control.md
# This is the commit message #64:

Update control.md
# This is the commit message #65:

Add files via upload
# This is the commit message #66:

Delete ISO-27001-Control-A9.png
# This is the commit message #67:

Update control.md
# This is the commit message #68:

Add files via upload
# This is the commit message #69:

Add files via upload
# This is the commit message #70:

Delete ISO-27002-Control-9.1.1-part1.png
# This is the commit message #71:

Delete ISO-27002-Control-9.1.1-part2.png
# This is the commit message #72:

Update control.md
# This is the commit message #73:

Update control.md
# This is the commit message #74:

Update control.md
# This is the commit message #75:

Update control.md
# This is the commit message #76:

Update control.md
# This is the commit message #77:

Update README.md
# This is the commit message #78:

Update IMPLEMENTERS.md
# This is the commit message #79:

Add files via upload
# This is the commit message #80:

Delete oscal-layers.png
# This is the commit message #81:

Add files via upload
# This is the commit message #82:

Delete oscal-layers.png
# This is the commit message #83:

Add files via upload
# This is the commit message #84:

Update IMPLEMENTERS.md
# This is the commit message #85:

Update control.md
# This is the commit message #86:

Update IMPLEMENTERS.md
# This is the commit message #87:

Update control.md
# This is the commit message #88:

Rename IMPLEMENTERS.md to docs/prose/IMPLEMENTERS.md
# This is the commit message #89:

Rename IMPLEMENTERS.md to implementers.md
# This is the commit message #90:

Rearranged and outlined catalog documentation based on the conversation with karen and Wendell.

# This is the commit message #91:

Create catalog-xml.md
# This is the commit message #92:

Rename control.md to catalog.md
# This is the commit message #93:

Update catalog.md
# This is the commit message #94:

Update catalog.md
# This is the commit message #95:

Update catalog.md
# This is the commit message #96:

Update catalog-xml.md
# This is the commit message #97:

Update catalog-xml.md
# This is the commit message #98:

Update catalog-xml.md
# This is the commit message #99:

Update catalog-xml.md
# This is the commit message #100:

Update catalog-xml.md
# This is the commit message #101:

Update catalog-xml.md
# This is the commit message #102:

Update catalog-xml.md
# This is the commit message #103:

Update catalog-xml.md
# This is the commit message #104:

Update catalog-xml.md
# This is the commit message #105:

Update catalog-xml.md
# This is the commit message #106:

Docset migration to Slate

# This is the commit message #107:

Removing unused file.

# This is the commit message #108:

Update README.md

Corrected a typo
# This is the commit message #109:

Add files via upload

Graphical representation of OSCAL schemas aligned with Risk Management Framework steps and tasks.
# This is the commit message #110:

Create CONTRIBUTING.md
# This is the commit message #111:

Create ROADMAP.md
# This is the commit message #2:

Create README.md
# This is the commit message #3:

Update README.md
# This is the commit message #2:

Create OSCAL-PRODUCERS.md
# This is the commit message #3:

Create OSCAL-CONSUMERS.md
# This is the commit message #4:

Update and rename OSCAL-CONSUMERS.md to USERS.md
# This is the commit message #5:

Update and rename OSCAL-PRODUCERS.md to IMPLEMENTERS.md
# This is the commit message #6:

Rename CONTRIBUTING.md to CONTRIBUTORS.md
# This is the commit message #2:

Update USERS.md
# This is the commit message #2:

Update IMPLEMENTERS.md
# This is the commit message #2:

Update ROADMAP.md
# This is the commit message #3:

Update USERS.md
# This is the commit message #4:

Update CONTRIBUTORS.md
# This is the commit message #2:

Update IMPLEMENTERS.md
# This is the commit message #3:

Update IMPLEMENTERS.md
# This is the commit message #4:

Rename CONTRIBUTORS.md to CONTRIBUTING.md
# This is the commit message #5:

Create control.md
# This is the commit message #6:

Update control.md
# This is the commit message #7:

Update control.md
# This is the commit message #8:

Update control.md
# This is the commit message #9:

Update control.md
# This is the commit message #10:

Add files via upload
# This is the commit message #11:

Update control.md
# This is the commit message #12:

Create temp.md
# This is the commit message #13:

Delete NIST-SP-800-53-Rev4-AC1.png
# This is the commit message #14:

Add files via upload
# This is the commit message #15:

Delete temp.md
# This is the commit message #16:

Add files via upload
# This is the commit message #17:

Update control.md
# This is the commit message #18:

Add files via upload
# This is the commit message #19:

Add files via upload
# This is the commit message #20:

Update control.md
# This is the commit message #21:

Update CONTRIBUTING.md
# This is the commit message #22:

Update CONTRIBUTING.md
# This is the commit message #23:

Update USERS.md
# This is the commit message #24:

Update CONTRIBUTING.md
# This is the commit message #25:

Delete CONTRIBUTING.md
# This is the commit message #26:

Delete USERS.md
# This is the commit message #27:

Add files via upload
# This is the commit message #28:

Delete CSA-CCM-IAM02.png
# This is the commit message #29:

Update control.md
# This is the commit message #30:

Update control.md
# This is the commit message #31:

Update control.md
# This is the commit message #32:

Update control.md
# This is the commit message #33:

Update control.md
# This is the commit message #34:

Update control.md
# This is the commit message #35:

Update control.md
# This is the commit message #36:

Update control.md
# This is the commit message #37:

Update control.md
# This is the commit message #38:

Update control.md
# This is the commit message #39:

Delete NIST-SP-800-53-AC1-in-OSCAL-XML.png
# This is the commit message #40:

Update README.md
# This is the commit message #41:

Update control.md
# This is the commit message #42:

Update control.md
# This is the commit message #43:

Add files via upload
# This is the commit message #44:

Delete ISO-27001-Control-A9.png
# This is the commit message #45:

Update control.md
# This is the commit message #46:

Add files via upload
# This is the commit message #47:

Add files via upload
# This is the commit message #48:

Delete ISO-27002-Control-9.1.1-part1.png
# This is the commit message #49:

Delete ISO-27002-Control-9.1.1-part2.png
# This is the commit message #50:

Update control.md
# This is the commit message #51:

Update control.md
# This is the commit message #52:

Update control.md
# This is the commit message #53:

Update control.md
# This is the commit message #54:

Update control.md
# This is the commit message #2:

Update IMPLEMENTERS.md
# This is the commit message #3:

Add files via upload
# This is the commit message #4:

Delete oscal-layers.png
# This is the commit message #5:

Add files via upload
# This is the commit message #6:

Delete oscal-layers.png
# This is the commit message #7:

Add files via upload
# This is the commit message #8:

Update IMPLEMENTERS.md
# This is the commit message #9:

Update control.md
# This is the commit message #10:

Update IMPLEMENTERS.md
# This is the commit message #11:

Update control.md
# This is the commit message #12:

Rename IMPLEMENTERS.md to docs/prose/IMPLEMENTERS.md
# This is the commit message #13:

Rename IMPLEMENTERS.md to implementers.md
# This is the commit message #14:

Rearranged and outlined catalog documentation based on the conversation with karen and Wendell.

# This is the commit message #15:

Create catalog-xml.md
# This is the commit message #16:

Rename control.md to catalog.md
# This is the commit message #17:

Update catalog.md
# This is the commit message #18:

Update catalog.md
# This is the commit message #19:

Update catalog.md
# This is the commit message #20:

Update catalog-xml.md
# This is the commit message #21:

Update catalog-xml.md
# This is the commit message #22:

Update catalog-xml.md
# This is the commit message #23:

Update catalog-xml.md
# This is the commit message #24:

Update catalog-xml.md
# This is the commit message #25:

Update catalog-xml.md
# This is the commit message #26:

Update catalog-xml.md
# This is the commit message #27:

Update catalog-xml.md
# This is the commit message #28:

Update catalog-xml.md
# This is the commit message #29:

Fixed typos, updated repo documentation, and migrated documentation for use in Slate.

Corrected a typo (+4 squashed commit)

Squashed commit:

[6ada57f] Removing unused file.

[503ad71] Docset migration to Slate

[351257e] Update catalog-xml.md

[aae1e8b] Add files via upload

Graphical representation of OSCAL schemas aligned with Risk Management Framework steps and tasks.
@wendellpiez
Copy link
Contributor

wendellpiez commented Apr 5, 2018 via email

@anweiss
Copy link
Contributor Author

anweiss commented Apr 5, 2018

To your last statement about schema design tooling, this is actually relatively sparse. In fact, there isn't much JSON schema design guidance out there altogether. Other than syntactical guidance (e.g. Google's JSON style guide), JSON schema modeling and design guidance is hard to come by and dependent on repetition and day-to-day usage in different scenarios. NSA's Information Assurance Directorate published some recommendations on JSON and JSON schema usage last year -> https://www.iad.gov/iad/library/supporting-documents/security-guidance-use-json-andjson-schemas.cfm#aboutMenu ... which contains some valuable tidbits but is more aligned with developing very explicit constraints on your JSON.

david-waltermire pushed a commit that referenced this issue Apr 5, 2018
# The first commit's message is:

Restructuring for new initiative

Moved everything into 'old' and added a new subdirectory for new
experiments.

More early iteration over names

More adjustments in names mostly

HTML page for Word import testing

Just touchups this time

Updates and misc improvements

Changed document element to 'catalog'

Improved and renamed mapping transformation

Adjustments to models, schema, Schematron

Managing gitignore

Same

Minor adjustments

First cut at 27K mapping into OSCAL

Renamed directories for consistency

Cleanup

Bit more cleanup

Adjustments to mapping

Same

Numerous enhancements, some reorg

Planning notes

Added starter FO XSLT

First cut at FO XSL

Minor improvements

Small adjustments

Many/much schema enhancements; proto XSD version

First cut at inline declarations with live validation

Implementing declarations; styling

Refinements and extensions; new assign, select, withdrawn elements

More support for parameters etc.

Now implementing parameters etc.

More touches

Misc improvements

Sundry adjustments

Many more small improvements

Much documentation

More touchups

Updates

Misc adjustments and notes

Significant rework following design session esp declarations

More improvements incl extending declarations to groups

More adjustments; now mapping SP800-53A also.

More polishing and maintenance

Much more including hierarchical number checking

Adjustments, extensions, corrections

Name changes in the model

More adjustments

Cleanup of obsolete artifacts

Minor alignment

Mostly CSS, also Schematron improvements

ISO27002 mapping document

Many enhancements; new COBIT 5 mappings

More on COBIT

Improvements to mapping docs

Adjustments to mappings incl HTML versions

Moved screenshot for documentation

More adjustments and docs

Mostly improvements to documentation

Refreshed 800-53 data with adjustments

More general improvements to models and mappings

Continuing refinements to validations in samples

Update README.md

Update README.md

Update README.md

Update README.md

Minor tweaks

Including initial cut at profiling (1st, rule-based control extraction) among other refinements and improvements

No longer extant

Added new control to COBIT5 example

Misc improvements including more COBIT5 support

File rename and further adjustments

Misc adjustment; SaxonJS demo files

Misc adjustments

Updated readme contents to reflect merge.

Fixed links.

Changed text to better reflect current state and to add more links.

Rearrangement and cleanup

Schematron and schema adjustment and refactoring

SP800-53 extraction improvement, enhancement, cleanup

Improvements and enhancements to ISO27002 extraction

OSCAL Documentation

COBIT5 example tweakage

Organizational notes

Adjustments

Removed old files directory.
Moved files from draft to root.

Updated README.md to reflect new pathing.

Updated readme to contain more documentation of the OSCAL layers.

Added documentation for the working directory.

Update README.md

Edited the text.

Update README.md

Edited the text.

Added Aug 10 tiger team slides with notes

Create a prose overview of OSCAL

Update OSCAL Overview.md

Update OSCAL Overview.md

Profile and catalog mapping: a trivial example

Replacing graphic with rescaled version

Update OSCAL Overview.md

Rescaled image

Update OSCAL Overview.md

Mods to oXygen project file

Removing 'demo' (till later)

Moved proprietary file

Refinements to ISO27002 mappings (new model)

Refining SP800-53 conversion/mappings

Updating lib support for new model

Top-level org

Draft readme docs

Further adjustment to directory readme.md

More adjustments to ISO 27002 and SP800-53 (new model etc.)

More cleanup to main sample subdirectories + readme tweakage

Mostly restructuring

More adjustments to extraction, parameter support

Includes more model refinements; draft implementation of parameter resolution

Adjustments for revised models

Adjustments to demos and conversion pipelines

Experimenting with profiles

Infrastructure adjustments

Schema modifications

# This is the commit message #2:

Minor correction to schema docs merge
# This is the commit message #3:

Updating XSD w/ docs
# This is the commit message #4:

More element renaming/adjustment
# This is the commit message #5:

Now a stable schema again? With SP800-53 extraction adjustments
# This is the commit message #6:

Profile-related mods

# This is the commit message #7:

Org stuff

# This is the commit message #8:

Moved mappind docs; added readme
# This is the commit message #9:

schema documentation
# This is the commit message #10:

Schema production and documentation pipeline
# This is the commit message #11:

More reorg; tweaked schema; more docs
# This is the commit message #12:

Improvements to tag library
# This is the commit message #13:

Another attempt at internal links in md
# This is the commit message #14:

trying again
# This is the commit message #15:

Gonna get this eventually
# This is the commit message #16:

More updates and tuneups
# This is the commit message #17:

More updates to schema and docs
# This is the commit message #18:

Lighter adjustments
# This is the commit message #19:

Similarly subtle adjustments
# This is the commit message #20:

Adjustments supporting parameter assignments
# This is the commit message #21:

More docs
# This is the commit message #22:

Enhancements to demo
# This is the commit message #23:

Moving distractions out of the way
# This is the commit message #24:

Slight readme tweakage
# This is the commit message #25:

Tag library tweaks
# This is the commit message #26:

More tag library adjustments
# This is the commit message #27:

New readme for Schematrons
# This is the commit message #28:

Adjustments to readme
# This is the commit message #29:

Adjustments to mapping documents
# This is the commit message #30:

Adjustments to readme
# This is the commit message #31:

More adjustments to mapping documents
# This is the commit message #32:

Updated punchlist
# This is the commit message #33:

Updates to mapping docs
# This is the commit message #34:


# This is the commit message #35:

CSS for local editing of OSCAL OSCAL
# This is the commit message #36:

Slight adjustments
# This is the commit message #37:

Light editing
# This is the commit message #38:

Light editing
# This is the commit message #39:

Light edits
# This is the commit message #40:

Light edits
# This is the commit message #41:

Light edits
# This is the commit message #42:

Light edits
# This is the commit message #43:

More work on profiles
# This is the commit message #44:

Revised directory descriptions
# This is the commit message #45:

Added an explanation of the demo directories
# This is the commit message #46:

More on profiling
# This is the commit message #47:

Editorial tweaks to oscal-oscal.xml
# This is the commit message #48:

Editorial tweaks to oscal-oscal.xml
# This is the commit message #49:

Edits to schema documentation
# This is the commit message #50:

More enhancements to tag library and its presentation
# This is the commit message #51:

Internal links?
# This is the commit message #52:

Slight adjustments
# This is the commit message #53:

Further touchups to schema docs and their HTML rendering
# This is the commit message #54:

Support for 'pre'
# This is the commit message #55:

Rearranged and touched up profile experiments
# This is the commit message #56:

More touchups to tag library (linking)
# This is the commit message #57:

More adjustments to copy
# This is the commit message #58:

More adjustments
# This is the commit message #59:

Correcting display bug failing to render chars in CSS
# This is the commit message #60:

Added a PDF with the notes.
@david-waltermire david-waltermire added this to the OSCAL 1.0 M1 milestone Apr 6, 2018
david-waltermire pushed a commit that referenced this issue Apr 13, 2018
# The first commit's message is:

# This is a combination of 60 commits.
# The first commit's message is:

# This is a combination of 17 commits.
# The first commit's message is:

Restructuring for new initiative

Moved everything into 'old' and added a new subdirectory for new
experiments.

More early iteration over names

More adjustments in names mostly

HTML page for Word import testing

Just touchups this time

Updates and misc improvements

Changed document element to 'catalog'

Improved and renamed mapping transformation

Adjustments to models, schema, Schematron

Managing gitignore

Same

Minor adjustments

First cut at 27K mapping into OSCAL

Renamed directories for consistency

Cleanup

Bit more cleanup

Adjustments to mapping

Same

Numerous enhancements, some reorg

Planning notes

Added starter FO XSLT

First cut at FO XSL

Minor improvements

Small adjustments

Many/much schema enhancements; proto XSD version

First cut at inline declarations with live validation

Implementing declarations; styling

Refinements and extensions; new assign, select, withdrawn elements

More support for parameters etc.

Now implementing parameters etc.

More touches

Misc improvements

Sundry adjustments

Many more small improvements

Much documentation

More touchups

Updates

Misc adjustments and notes

Significant rework following design session esp declarations

More improvements incl extending declarations to groups

More adjustments; now mapping SP800-53A also.

More polishing and maintenance

Much more including hierarchical number checking

Adjustments, extensions, corrections

Name changes in the model

More adjustments

Cleanup of obsolete artifacts

Minor alignment

Mostly CSS, also Schematron improvements

ISO27002 mapping document

Many enhancements; new COBIT 5 mappings

More on COBIT

Improvements to mapping docs

Adjustments to mappings incl HTML versions

Moved screenshot for documentation

More adjustments and docs

Mostly improvements to documentation

Refreshed 800-53 data with adjustments

More general improvements to models and mappings

Continuing refinements to validations in samples

Update README.md

Update README.md

Update README.md

Update README.md

Minor tweaks

Including initial cut at profiling (1st, rule-based control extraction) among other refinements and improvements

No longer extant

Added new control to COBIT5 example

Misc improvements including more COBIT5 support

File rename and further adjustments

Misc adjustment; SaxonJS demo files

Misc adjustments

Updated readme contents to reflect merge.

Fixed links.

Changed text to better reflect current state and to add more links.

Rearrangement and cleanup

Schematron and schema adjustment and refactoring

SP800-53 extraction improvement, enhancement, cleanup

Improvements and enhancements to ISO27002 extraction

OSCAL Documentation

COBIT5 example tweakage

Organizational notes

Adjustments

Removed old files directory.
Moved files from draft to root.

Updated README.md to reflect new pathing.

Updated readme to contain more documentation of the OSCAL layers.

Added documentation for the working directory.

Update README.md

Edited the text.

Update README.md

Edited the text.

Added Aug 10 tiger team slides with notes

Create a prose overview of OSCAL

Update OSCAL Overview.md

Update OSCAL Overview.md

Profile and catalog mapping: a trivial example

Replacing graphic with rescaled version

Update OSCAL Overview.md

Rescaled image

Update OSCAL Overview.md

Mods to oXygen project file

Removing 'demo' (till later)

# This is the commit message #2:

Moved proprietary file

# This is the commit message #3:

Refinements to ISO27002 mappings (new model)

# This is the commit message #4:

Refining SP800-53 conversion/mappings

# This is the commit message #5:

Updating lib support for new model

# This is the commit message #6:

Top-level org

# This is the commit message #7:

Draft readme docs

# This is the commit message #8:

Further adjustment to directory readme.md

# This is the commit message #9:

More adjustments to ISO 27002 and SP800-53 (new model etc.)

# This is the commit message #10:

More cleanup to main sample subdirectories + readme tweakage

# This is the commit message #11:

Mostly restructuring

# This is the commit message #12:

More adjustments to extraction, parameter support

# This is the commit message #13:

Includes more model refinements; draft implementation of parameter resolution

# This is the commit message #14:

Adjustments for revised models

# This is the commit message #15:

Adjustments to demos and conversion pipelines

# This is the commit message #16:

Experimenting with profiles

# This is the commit message #17:

Infrastructure adjustments

# This is the commit message #2:

Minor correction to schema docs merge
# This is the commit message #3:

Updating XSD w/ docs
# This is the commit message #4:

More element renaming/adjustment
# This is the commit message #5:

Now a stable schema again? With SP800-53 extraction adjustments
# This is the commit message #6:

Profile-related mods

# This is the commit message #7:

Org stuff

# This is the commit message #8:

Moved mappind docs; added readme
# This is the commit message #9:

schema documentation
# This is the commit message #10:

Schema production and documentation pipeline
# This is the commit message #11:

More reorg; tweaked schema; more docs
# This is the commit message #12:

Improvements to tag library
# This is the commit message #13:

Another attempt at internal links in md
# This is the commit message #14:

trying again
# This is the commit message #15:

Gonna get this eventually
# This is the commit message #16:

More updates and tuneups
# This is the commit message #17:

More updates to schema and docs
# This is the commit message #18:

Lighter adjustments
# This is the commit message #19:

Similarly subtle adjustments
# This is the commit message #20:

Adjustments supporting parameter assignments
# This is the commit message #21:

More docs
# This is the commit message #22:

Enhancements to demo
# This is the commit message #23:

Moving distractions out of the way
# This is the commit message #24:

Slight readme tweakage
# This is the commit message #25:

Tag library tweaks
# This is the commit message #26:

More tag library adjustments
# This is the commit message #27:

New readme for Schematrons
# This is the commit message #28:

Adjustments to readme
# This is the commit message #29:

Adjustments to mapping documents
# This is the commit message #30:

Adjustments to readme
# This is the commit message #31:

More adjustments to mapping documents
# This is the commit message #32:

Updated punchlist
# This is the commit message #33:

Updates to mapping docs
# This is the commit message #34:

# This is the commit message #35:

CSS for local editing of OSCAL OSCAL
# This is the commit message #36:

Slight adjustments
# This is the commit message #37:

Light editing
# This is the commit message #38:

Light editing
# This is the commit message #39:

Light edits
# This is the commit message #40:

Light edits
# This is the commit message #41:

Light edits
# This is the commit message #42:

Light edits
# This is the commit message #43:

More work on profiles
# This is the commit message #44:

Revised directory descriptions
# This is the commit message #45:

Added an explanation of the demo directories
# This is the commit message #46:

More on profiling
# This is the commit message #47:

Editorial tweaks to oscal-oscal.xml
# This is the commit message #48:

Editorial tweaks to oscal-oscal.xml
# This is the commit message #49:

Edits to schema documentation
# This is the commit message #50:

More enhancements to tag library and its presentation
# This is the commit message #51:

Internal links?
# This is the commit message #52:

Slight adjustments
# This is the commit message #53:

Further touchups to schema docs and their HTML rendering
# This is the commit message #54:

Support for 'pre'
# This is the commit message #55:

Rearranged and touched up profile experiments
# This is the commit message #56:

More touchups to tag library (linking)
# This is the commit message #57:

More adjustments to copy
# This is the commit message #58:

More adjustments
# This is the commit message #59:

Correcting display bug failing to render chars in CSS
# This is the commit message #60:

Added a PDF with the notes.

# This is the commit message #2:

Decided to risk it
# This is the commit message #3:

One more readme
david-waltermire added a commit that referenced this issue Apr 13, 2018
# The first commit's message is:

# This is a combination of 4 commits.
# The first commit's message is:

# This is a combination of 60 commits.
# The first commit's message is:

# This is a combination of 17 commits.
# The first commit's message is:

Restructuring for new initiative

Moved everything into 'old' and added a new subdirectory for new
experiments.

More early iteration over names

More adjustments in names mostly

HTML page for Word import testing

Just touchups this time

Updates and misc improvements

Changed document element to 'catalog'

Improved and renamed mapping transformation

Adjustments to models, schema, Schematron

Managing gitignore

Same

Minor adjustments

First cut at 27K mapping into OSCAL

Renamed directories for consistency

Cleanup

Bit more cleanup

Adjustments to mapping

Same

Numerous enhancements, some reorg

Planning notes

Added starter FO XSLT

First cut at FO XSL

Minor improvements

Small adjustments

Many/much schema enhancements; proto XSD version

First cut at inline declarations with live validation

Implementing declarations; styling

Refinements and extensions; new assign, select, withdrawn elements

More support for parameters etc.

Now implementing parameters etc.

More touches

Misc improvements

Sundry adjustments

Many more small improvements

Much documentation

More touchups

Updates

Misc adjustments and notes

Significant rework following design session esp declarations

More improvements incl extending declarations to groups

More adjustments; now mapping SP800-53A also.

More polishing and maintenance

Much more including hierarchical number checking

Adjustments, extensions, corrections

Name changes in the model

More adjustments

Cleanup of obsolete artifacts

Minor alignment

Mostly CSS, also Schematron improvements

ISO27002 mapping document

Many enhancements; new COBIT 5 mappings

More on COBIT

Improvements to mapping docs

Adjustments to mappings incl HTML versions

Moved screenshot for documentation

More adjustments and docs

Mostly improvements to documentation

Refreshed 800-53 data with adjustments

More general improvements to models and mappings

Continuing refinements to validations in samples

Update README.md

Update README.md

Update README.md

Update README.md

Minor tweaks

Including initial cut at profiling (1st, rule-based control extraction) among other refinements and improvements

No longer extant

Added new control to COBIT5 example

Misc improvements including more COBIT5 support

File rename and further adjustments

Misc adjustment; SaxonJS demo files

Misc adjustments

Updated readme contents to reflect merge.

Fixed links.

Changed text to better reflect current state and to add more links.

Rearrangement and cleanup

Schematron and schema adjustment and refactoring

SP800-53 extraction improvement, enhancement, cleanup

Improvements and enhancements to ISO27002 extraction

OSCAL Documentation

COBIT5 example tweakage

Organizational notes

Adjustments

Removed old files directory.
Moved files from draft to root.

Updated README.md to reflect new pathing.

Updated readme to contain more documentation of the OSCAL layers.

Added documentation for the working directory.

Update README.md

Edited the text.

Update README.md

Edited the text.

Added Aug 10 tiger team slides with notes

Create a prose overview of OSCAL

Update OSCAL Overview.md

Update OSCAL Overview.md

Profile and catalog mapping: a trivial example

Replacing graphic with rescaled version

Update OSCAL Overview.md

Rescaled image

Update OSCAL Overview.md

Mods to oXygen project file

Removing 'demo' (till later)

# This is the commit message #2:

Moved proprietary file

# This is the commit message #3:

Refinements to ISO27002 mappings (new model)

# This is the commit message #4:

Refining SP800-53 conversion/mappings

# This is the commit message #5:

Updating lib support for new model

# This is the commit message #6:

Top-level org

# This is the commit message #7:

Draft readme docs

# This is the commit message #8:

Further adjustment to directory readme.md

# This is the commit message #9:

More adjustments to ISO 27002 and SP800-53 (new model etc.)

# This is the commit message #10:

More cleanup to main sample subdirectories + readme tweakage

# This is the commit message #11:

Mostly restructuring

# This is the commit message #12:

More adjustments to extraction, parameter support

# This is the commit message #13:

Includes more model refinements; draft implementation of parameter resolution

# This is the commit message #14:

Adjustments for revised models

# This is the commit message #15:

Adjustments to demos and conversion pipelines

# This is the commit message #16:

Experimenting with profiles

# This is the commit message #17:

Infrastructure adjustments

# This is the commit message #2:

Minor correction to schema docs merge
# This is the commit message #3:

Updating XSD w/ docs
# This is the commit message #4:

More element renaming/adjustment
# This is the commit message #5:

Now a stable schema again? With SP800-53 extraction adjustments
# This is the commit message #6:

Profile-related mods

# This is the commit message #7:

Org stuff

# This is the commit message #8:

Moved mappind docs; added readme
# This is the commit message #9:

schema documentation
# This is the commit message #10:

Schema production and documentation pipeline
# This is the commit message #11:

More reorg; tweaked schema; more docs
# This is the commit message #12:

Improvements to tag library
# This is the commit message #13:

Another attempt at internal links in md
# This is the commit message #14:

trying again
# This is the commit message #15:

Gonna get this eventually
# This is the commit message #16:

More updates and tuneups
# This is the commit message #17:

More updates to schema and docs
# This is the commit message #18:

Lighter adjustments
# This is the commit message #19:

Similarly subtle adjustments
# This is the commit message #20:

Adjustments supporting parameter assignments
# This is the commit message #21:

More docs
# This is the commit message #22:

Enhancements to demo
# This is the commit message #23:

Moving distractions out of the way
# This is the commit message #24:

Slight readme tweakage
# This is the commit message #25:

Tag library tweaks
# This is the commit message #26:

More tag library adjustments
# This is the commit message #27:

New readme for Schematrons
# This is the commit message #28:

Adjustments to readme
# This is the commit message #29:

Adjustments to mapping documents
# This is the commit message #30:

Adjustments to readme
# This is the commit message #31:

More adjustments to mapping documents
# This is the commit message #32:

Updated punchlist
# This is the commit message #33:

Updates to mapping docs
# This is the commit message #34:

# This is the commit message #35:

CSS for local editing of OSCAL OSCAL
# This is the commit message #36:

Slight adjustments
# This is the commit message #37:

Light editing
# This is the commit message #38:

Light editing
# This is the commit message #39:

Light edits
# This is the commit message #40:

Light edits
# This is the commit message #41:

Light edits
# This is the commit message #42:

Light edits
# This is the commit message #43:

More work on profiles
# This is the commit message #44:

Revised directory descriptions
# This is the commit message #45:

Added an explanation of the demo directories
# This is the commit message #46:

More on profiling
# This is the commit message #47:

Editorial tweaks to oscal-oscal.xml
# This is the commit message #48:

Editorial tweaks to oscal-oscal.xml
# This is the commit message #49:

Edits to schema documentation
# This is the commit message #50:

More enhancements to tag library and its presentation
# This is the commit message #51:

Internal links?
# This is the commit message #52:

Slight adjustments
# This is the commit message #53:

Further touchups to schema docs and their HTML rendering
# This is the commit message #54:

Support for 'pre'
# This is the commit message #55:

Rearranged and touched up profile experiments
# This is the commit message #56:

More touchups to tag library (linking)
# This is the commit message #57:

More adjustments to copy
# This is the commit message #58:

More adjustments
# This is the commit message #59:

Correcting display bug failing to render chars in CSS
# This is the commit message #60:

Added a PDF with the notes.

# This is the commit message #2:

Decided to risk it
# This is the commit message #3:

One more readme
# This is the commit message #4:

Added timestamping to extractors

# This is the commit message #2:

Light edits to oscal-oscal.xml
# This is the commit message #3:

More edits to oscal-oscal.xml with regenerated materials
# This is the commit message #4:

Refining markdown export
# This is the commit message #5:

More edits to docs and rendering
# This is the commit message #6:

This time smaller adjustments
# This is the commit message #7:

More improvements to docs and rendering
@anweiss
Copy link
Contributor Author

anweiss commented Aug 2, 2018

@david-waltermire-nist we can probably close this issue

wendellpiez referenced this issue in wendellpiez/OSCAL Mar 11, 2019
* updated formatting for auto-generated HTML documentation

* minor updates per Wendell
@david-waltermire david-waltermire removed Discussion Needed This issues needs to be reviewed by the OSCAL development team. labels Jun 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants