Allow only 1 @list per property #92

Closed
dlongley opened this Issue Mar 28, 2012 · 5 comments

Projects

None yet

3 participants

@dlongley
JSON-LD Public Repositories member

Gregg, Manu, and I discussed what the syntax would look like if a property had multiple @lists and how it would behave with respect to coercion, compaction, and expansion. Note that this is different from @lists of @lists. From our discussion we decided that supporting multiple @lists would introduce complexity both for authors and processors.

If we were to support multiple @lists, how they collapse down or expand out may introduce ambiguities or may just be overly complex. For example, if a property had two @lists, and a @container rule, would this result in an array of arrays in the compacted output? Would this be confused with a @list of @lists by authors? Processors would need some kind of new coercion rule to be able to tell the difference. We also haven't previously permitted arrays of arrays in JSON-LD. What happens if you combine a @set within an array that also has a @list ... and then combine a @container rule? Might a decision conflict with potential future support of @lists of lists?

Another argument against using multiple lists is that it makes a list appear as a value rather than as a simple modifier of a set. Right now, it is easy for a JSON author to think of a @list as an ordered set. A JSON-LD property can have a single value, or a set of values that is either ordered or not ordered. Thinking of a @list as a value works in the graph model or if you're coming from an RDF background, but this may be a foreign or bizarre concept to JSON authors regardless of how much we talk about JSON-LD being based on graphs.

A lot of design work would need to take place to support multiple lists when it doesn't seem likely that we have many use cases for it for JSON-LD 1.0. However, order is an important concept in JSON so we should continue to support @list as a way to specify an ordered @set. The end proposal we came up with was to simply support only one @list per property -- and it must be the only value for that property. In addition to this, we suggest that processors throw an exception if a @list or @set are put into an array.

PROPOSAL: A JSON-LD property MUST NOT have more than one @list value. If a property has a @list value that uses the @list keyword, then the property value MUST be an object with a @list key. The @list key's value MUST be an array with the ordered values for the property. The property MAY use a rule in the @context to specify that its values are in a @list @container, in which case the property value will be an array containing the property's ordered values. A property value that is an array MUST NOT contain @list or @set keyword (this does not apply to deeply-embedded subjects). The expanded form for a @list value is a single object, NOT an array.

Here is the relevant IRC conversation:


<manu-db> ok
<dlongley> so ... multiple lists ...
<manu-db> example?
<dlongley> {"foo": [{"@list": ...}, {"@list": ...}]}
<manu-db> hmm
<dlongley> i find this to be a bit problematic for a few reasons.
<dlongley> first of all, what happens when you coerce?
<dlongley> you get:
<dlongley> {"foo": [[],[]]}
<dlongley> which was illegal up to now and leads to ambiguity.
<dlongley> also, i considered @list to be a sort of analog to @set
<dlongley> the only difference being order.
<dlongley> this would no longer be the case
<dlongley> and should we support multiple {"@set":...}s now too?
<dlongley> seems very odd to me.
<dlongley> is this a requirement to support RDF things?
<dlongley> what happens if you do: {"foo": [{"@list": ...}, {"@set": ...}]?
<manu-db> Well, I think we made a decision not to support lists of lists
<dlongley> or any other combination of things?
<manu-db> and that is a list of lists.
<dlongley> well, that isn't a list of lists.
<manu-db> The first example is.
<dlongley> well, actually no...
<dlongley> it's a property with 2 lists.
<dlongley> it's a set of lists.
<manu-db> oh, right
<gkellogg> Actually, foo: [{@list: []], {@list: []}] is not a list of lists ...
<manu-db> I think we do have to support having a property that has two lists.
<dlongley> not to mention that i think it's more of a pain to process [{"@list":...}] than just {"@list":...}
<gkellogg> the issue comes when doing coercion, which would require multiple term definitions.
<gkellogg> It's a pain to have one rule for @list and another one for everything else.
<manu-db> We could always take @list support out :P
<dlongley> =)
<dlongley> well, i can see people wanting to order their JSON.
<gkellogg> What did we say about [{@set: []}, {@set: []}], I guess that's optimized out in expansion?
<manu-db> yes, I believe that would be optimized out in expansion.
<dlongley> well, we've never allowed an array with a set in it before
<dlongley> what if we combine @set and @list?
<dlongley> under a single property?
<manu-db> we've never allowed [{"@set": [...]}] ?
<dlongley> i don't think so.
<dlongley> but @set is pretty new
<gkellogg> In expansion, @set is optimized out.
<gkellogg> We added it just to be consistent with @list.
<dlongley> does that actually help authors?
<dlongley> or make them more confused?
<gkellogg> ie., we wanted to allow {"@set": []}
<manu-db> we thought it would confuse them to have "@list", but not have "@set"
<gkellogg> Because we can say @container: @set or @list, and we allow {@list: []}, it makes sense to also allow {@set: []}
<manu-db> and we have "@container": "@set" so that compaction will create an [], right?
<dlongley> yes
<gkellogg> Right.
<dlongley> but we don't expand to @set
<gkellogg> Although, I don't think it creates an empty list in compaction.
<dlongley> instead we remove it for compaction or expansion
<dlongley> you have to preserve []
<dlongley> for @set and @list
<dlongley> (i think)
<dlongley> and i really hope.
<gkellogg> @set [] doesn't define any relations, and will go away in normalization.
<dlongley> because if not that messes up framing.
<dlongley> yes
<dlongley> but it *only* goes away during normalization.
<dlongley> because structure is important in JSON, not for triples.
<gkellogg> I believe we remove empty @set definitions in compaction.
<dlongley> let me check
<dlongley> http://json-ld.org/minutes/2012-03-13/#resolution-4
<gkellogg> We added it so that authors would know that if a property exists, it will always be an array of values, not just a single value.
<dlongley> "If an empty array ([]) used as a value is not subject to @list coercion, then the value MUST be removed from normalized output. The empty array MUST be preserved in compacted and expanded output."
<gkellogg> Hmm, yes I vaguely remember this.
<dlongley> IMO, we *must* preserve that.
<gkellogg> Difference between compaction and normalization.
<dlongley> if we don't, it really messes up framing.
<dlongley> framing already got all changed around because we started messing with structure in compaction/expansion
<gkellogg> But, framing (currently) depends on normalization, which removes it.
<dlongley> when we didn't before (array structure)
<dlongley> no
<dlongley> framing no longer depends on normalization
<dlongley> the latest code removes that dependency
<gkellogg> Okay, I didn't think it was necessary either.
<dlongley> (implemented last night)
<dlongley> but...
<dlongley> now framing relies on @context to handle array/not-array.
<gkellogg> In this case, we can probably remove the normalization API call from JSON-LD, and just have a generic implementation in the normalization spec.
<dlongley> framing itself only deals with tree positioning and subject filtering.
<dlongley> gkellogg: btw, my new normalization algorithm appears to work
<dlongley> which is good news for anyone who has to implement normalization.
<gkellogg> Glad to hear, I haven't gotten to try to implement that yet; it will be a different library, though.
<dlongley> because it is very simple now.
<dlongley> https://github.com/digitalbazaar/jsonld.js/blob/master/js/jsonld.js#L1194
<manu-db> (and it's much better time/space complexity)
<dlongley> yes.
<manu-db> going back to: [{@set: []}, {@set: []}]
<manu-db> I think it's clear what we do in that case.
<manu-db> (both sets just get collapsed)
<dlongley> yes
<manu-db> It's clear what this means - [{@list: []}, {@list: []}]
<gkellogg> Okay, and disallow [{@list}{@list}]
<manu-db> the property is associated with two lists.
<dlongley> wait ...
<dlongley> are those the same things you guys just typed?
<manu-db> note /means/ not "it's clear what we should do"
<manu-db> gkellogg was using abbreviated syntax, I think.
<gkellogg> Well, there are some issues with handling a property with two lists, but it wasn't the case I was originally worried about .
<dlongley> right... and gkellogg you think we should disallow that now?
<manu-db> note: I'm also fine w/ disallowing it - but then I think people might be annoyed.
<gkellogg> Not really my opinion, trying to summarize conversation.
<dlongley> ok, what is your opinion?
<gkellogg> Like I said, I think the problem comes when compacting with @list coercion.
<dlongley> i think we'd have to just say it can't be done.
<gkellogg> Or, we just interpret that [[],[]] isn't a list of a list, but two lists.
<dlongley> are there some cases where we can't tell what's going on?
<dlongley> like a list with a @set in it?
<manu-db> we could just say that if a list is within a list - you don't compact but keep the expanded form.
<dlongley> [[], [{@set}]]
<gkellogg> Doesn't make sense for @list to have a set in it.
<gkellogg> in RDF, anyway.
<manu-db> maybe if both @set and @list are in a bigger list, we don't compact?
<dlongley> why wouldn't a JSON-LD author try it?
<manu-db> (don't introduce the ambiguity)
<gkellogg> Set is really just a convenience to allow programmers to deal with arrays, and not branch on the form of the value.
<manu-db> don't we get around the issue by stating that if either expanded form is used for @set or @list and it is contained in a JSON array, the value is untouched?
<gkellogg> It doesn't mean anything in a graph sense.
<gkellogg> Best to get to a consistent representation. I think collapsing lists is a good idea.
<manu-db> but you can't do that w/o introducing ambiguity, no?
<gkellogg> s/list/set/
<manu-db> oh, collapsing sets is fine.
<dlongley> i'd like to see the possibilities enumerated here to ensure we're describing something that isn't ambiguous
<gkellogg> Representing lists as [{@list: []}] is consistent with other value representation.
<dlongley> it is just unfortunate to code for :)
<gkellogg> foo: {@list: ["bar"]} expands to http://example.com/foo: [{@list: ["bar"]}]
<gkellogg> IMO, consistency here is better.
<dlongley> i do like that everything is in an array in expanded form (vs. some things aren't)
<dlongley> but ... i originally liked the exception being a list ...
<gkellogg> How about foo: ["bar"], bar: ["baz"], where both foo and bar have the same ID and have list coercion?
<dlongley> because it's really not the same thing as a value (at least i didn't think of it that way)
<dlongley> i considered it a way of imposing order on the actual values.
<gkellogg> Sure, a list is a value. Think of it as RDF.
<dlongley> because the property points at values
<gkellogg> It's either rdf:nil or a BNode.
<dlongley> yes, but i'm not thinking that way looking at it as JSON-LD.
<dlongley> i *can* ... but that's not the mindset i think JSON people will be in.
<dlongley> they'll see @set as an unordered array of values...
<dlongley> and @list as an ordered array of values.
<dlongley> RDF will never enter the picture.
<gkellogg> We need to stay consistent. Think of it as a graph, and there must be a vertex for a list.
<gkellogg> JSON-LD is a graph representation.
<dlongley> i'm not disagreeing with you ... just saying where i think other peoples' minds will go.
<gkellogg> We say from the beginning that the data model is that of a graph.
<manu-db> gkellogg - I agree with dlongley - doesn't matter what we say, because most people won't care about reading the document or getting in the 'graph' mindset.
<manu-db> (and RDF Lists are awful things that are incredibly awkward to use)
<dlongley> w/respect to "foo" and "bar" properties both having @list coercion, i would expect that it would work like expansion does now ... they go into an array with 2 {@list}s
<gkellogg> So, you think that expanding a list to {"@list": []} rather than enclosing it in an array is the way to go? I won't stand in the way.
<dlongley> well, the only problem is ...
<dlongley> it doesn't support multiple @lists.
<manu-db> to be clear: I don't think anything that strongly right now...
<dlongley> if we're not going to support multiple @lists, i think it's the way to go.
<gkellogg> I think we need to start resolving corner cases to the simplest implementation
<dlongley> it certainly makes the code easier and more consistent ....
<dlongley> you loop through property containers ...
<manu-db> I'm just concerned about the different permutations of [{"@set": []}, {"@list": []}]s
<dlongley> and if its a list, you just preserve the order and go into its array just like you would going into a set.
<dlongley> but if the list itself is a value, you can't do that.
* gkellogg is it time to hit the "reset button" on JSON-LD? :P
<dlongley> ha
<dlongley> certainly not :)
<manu-db> yeah, these edge cases are becoming very screwy...
<dlongley> but @list is a mess, IMO.
* manu-db nods.
<manu-db> it's not just JSON-LD
<manu-db> RDF Lists are a mess
<dlongley> well, mixing it with RDF makes it more of a mess.
<gkellogg> We should start looking at sources of complexity and seeing how they can be eliminated.
<gkellogg> However, I advocated @list, and continue to think we need to support it.
<manu-db> gkellogg, I agree.
<manu-db> Yeah, I agree as well
<dlongley> i think we need a way to have ordered values.
<dlongley> so i agree there.
<manu-db> we may just not support certain types of things like arrays that contain more than 1 @list
<dlongley> it's a core concept in JSON.
<dlongley> we need to support it.
* manu-db wishes JSON supported both () and [].
<dlongley> if we're going to put a list in an array at all, i think we should try to support multiple lists.
<manu-db> that would solve our problems.
<dlongley> otherwise, don't put it in an array.
<gkellogg> More than one list per property is not important, IMO
<dlongley> if more than 1 list per property isn't important, then make @list never be in an array.
<dlongley> and it makes the code easy and the understanding of its purpose for ordering values easy.
<manu-db> yeah, I'm fine with that - only 1 list per property
<manu-db> (for JSON-LD 1.0, anyway)
<dlongley> so that means if a property is a list, it is {@list} (no array)
<gkellogg> Right, on expansion.
<dlongley> and it essentially means: this is an ordered set.
<dlongley> which i think authors will get easily.
<dlongley> at least, fairly easily.
<gkellogg> What about a property with a list and something else, other than an array? I presume that's not supported either.
<dlongley> i don't think we should support that.
<gkellogg> This could raise an exception on expansion.
<dlongley> i think this should be about what a "property" is like.
<dlongley> it's ordered or it isn't.
<dlongley> we should error out if you try to put a @list in an array.
<manu-db> well, when compacting or expanding...
<manu-db> we could support it in the RDF conversion algorithm?
@lanthaler
JSON-LD Public Repositories member

Funny, I stumbled across the same issue last night while implementing my processor but was too tired to write a mail :-)

Unfortunately I don't really agree with the conclusion you guys made. Multiple properties could resolve to the same IRI in expansion so you need to be able to merge values anyway - that was one of the reasons (at least for me) to expand every value to an array during expansion.

I see the problem differently. I think by doing what you propose we are introducing corner cases instead of eliminating them.

An array (@set) means nothing else for me than that a property has multiple values that are not sorted. If I have another property that expands to the same IRI, I just merge that properties value to that array. An @list-object is just another value.

I also think that @list values should expand to [ { "@list": [] } ] for consistency. All the complications you talked about could be easily solved by just checking whether an @list-array contains another object with an @list property. If so, throw an exception stating that lists of lists are not supported. I think sets of lists should, and have to be supported. So [ 2, 3, { "@list": [] }, "test", { "@list": [] } ] should be valid.

In compaction, all we do is to check whether the terms signature matches the data and remove empty arrays if no @list or @set container was set. So I don't see any problems there.

I don't know how complex list of lists are in RDF (and they are really just an RDF issue) but the best solution would be to just support them :-)

@lanthaler lanthaler added a commit that referenced this issue Mar 29, 2012
@lanthaler lanthaler Throw exception when multiple lists are found for a single property
Whether we just support one list per property or multiple is currently being discussed as part of issue #92. Gregg already modified the algorithm in 256734b to just support one. This might need to be reverted.
2fc868c
@lanthaler lanthaler added a commit that referenced this issue Apr 6, 2012
@lanthaler lanthaler Expand @list to arrays as everything else
@list objects are now expanded to arrays as everything else. It is still an open question whether we wanna just allow one list per property or not (see #92). Currently we don't restrict it and so the result should be an array.
8269764
@lanthaler
JSON-LD Public Repositories member

RESOLVED: When performing expansion properties that are coerced to a @container type of @list MUST be placed in an array in expanded form. For example, "prop-iri": [{"@list": [1, 2]}] is correct, "prop-iri": {"@list": [1, 2]} is not.

@gkellogg
JSON-LD Public Repositories member

Note that both syntax and API docs are consistent with this representation, so no change is necessary.

@lanthaler
JSON-LD Public Repositories member

RESOLVED: If the result of IRI compaction has an @container @list and there are multiple @list values, throw an exception. When doing IRI compaction do not select terms (other than absolute IRIs) that have @container @list if the value has more than one list.

@lanthaler
JSON-LD Public Repositories member

RESOLVED: JSON-LD allows sets of lists except where it conflicts with the previous resolution.

@lanthaler lanthaler added a commit that closed this issue Aug 1, 2012
@lanthaler lanthaler Specify what happens to multiple lists when compacted to one term
In this update I still use exceptions as they are still used in the rest of the spec. In a future update all exceptions should be replaced to calls of the error callback handler as decided in issue #100.

This closes #92.
8c15131
@lanthaler lanthaler closed this in 8c15131 Aug 1, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment