Definition of Arrays/Lists: Null vs Empty

[StephenOTT]

Has there been any docs created around where/when a distinction should be made between a null and empty array for all of the objects in the spec?

Where a null could mean not provided, and empty means “none to report”.

Example being a Email Message COO, you have a List of to Recipients. Should this be returned as a empty array or considered null? Where null could indicate that “we don't know what the value was”, and an empty array can indicate that we are confirming that there are no values in the to property.

Is anyone currently implementing this in their spec build?

[rpiazza]

@StephenOTT - look at section 2.8 of Part 1 of the specification. There are three cases:

• the property is required - there must be a list with at least one entry
• the property is optional and not empty - a list with the entries
• the property is optional and empty - omit the property all together.

Empty lists are not permitted.

Please respond if you have additional questions.

[StephenOTT]

@rpiazza ignoring the complexity of finding this info, how does that spec support the issue case I raised above? If you have something like email addresses: which are option, but how do you define the difference between it was empty vs the data was omitted/redacted vs the data was not collected or did not exist.

[rpiazza]

@StephenOTT - First, there is no NULL value in STIX. I'm not sure how you would represent it in JSON anyway. The spec doesn't describe a way to discriminate between the two cases (no value vs not provided). Perhaps a trust group could agree upon a "special" value for each.

[StephenOTT]

I would also raise this becomes more complicated when you take data markings into account: if you actually use data markings to cleanse properties, based on the receiver of the data, you can end up with mandatory fields that are redacted. Or can have optional fields that had data but you are redacting the content vs hiding the existence of content. A null is a convinent way to show this.

If there is no way to show nulls in the spec (is no nulls defined anywhere? Could not find it with a search for "null"), I would suggest something like Per object comments/notes be used to document any variance such as "CCs in email were found but not collected"

[ejratl]

@StephenOTT rpiazza confirmed that NULL is not defined in STIX. Your follow-up suggestion sounds to me like something that would go in a best practices guide for using the spec rather than in the spec itself. Is it ok to close this issue?

[StephenOTT]

Why separate the info? If we add this as a note into the spec it's in the place that 99% will read when impl the spec.

[jordan2175]

We talked a lot about this way back when and the various intel communities often can not tell you one way or the other. They are the ones that are most likely to redact information. For trust groups the general idea is people would define something that works for them. Most will just use REDACTED, just like they do in other pieces of data. I agree with Emily, that what you are asking for really feels like a best practices / implementation guide that might be used within a trust group. We also need to remember that not every programming language can handle the difference between an empty item, a NULL item, or a not included item. I know I had to write my own JSON handlers in Go to address this.