Improvement to the CUDS API: subscripting (i.e. cuds[...])

[kysrpex]

Hi all!

Due to a issues #416, #642, a new API for working with cuds based on the Python subscripting feature cuds[...] has been developed. The inner workings are ready, but I wanted to open a discussion to decide together with the community what design for using it would work best from a user's perspective.

To better introduce you to the topic, I would like to provide some background.

Adapted from #682:

Terminology

ObjectProperty: Is an OWL keyword. An ObjectProperty relates two individuals.
Example: :matthias foaf:knows :pablo --> foaf:knows is an ObjectProperty; matthias and pablo are individuals

DatatypeProperty: Is an OWL keyword. A DatatypeProperty relates an individual with data.
Example: :matthias foaf:nick "matze" --> DatatypeProperty foaf:nick relates the individual to his nickname
:matthias foaf:nick "fletcher" --> Second nickname is allowed

Functional Property: An OWL keyword. It is used when only one object is allowed for each subject.
Example: :matthias foaf:birthday "01.03.1995" --> Another triple matthias foaf:birthday "02.03.1995" would be not allowed.

Note that both object properties and datatype properties can be either functional or non-functional. In OSP-core, object properties are known as relationships and datatype properties are known as attributes.

The table below describes how to work with each type of property with the current CUDS API.

	functional DatatypeProperty	functional ObjectProperty	non-functional DatatypeProperty	non-functional ObjectProperty
add()		x		x
dot	x

What has been developed is a new way to interact with CUDS, based on the subscripting syntax cuds[...]. So the future CUDS API will look like this instead.

	functional DatatypeProperty	functional ObjectProperty	non-functional DatatypeProperty	non-functional ObjectProperty
add()		x		x
dot	x
[ ]	x	x	x	x

The goal of this discussion is to decide, what should be inside or around the brackets besides the OntologyRelationship or OntologyAttribute that one wishes to modify.

I will start by writing answers describing the solutions I have thought of. Please contribute your own ideas if you wish. You can upvote or downvote each solution using the buttons on the left. Please do not hesitate to use the reply buttons under the answers to comment on them or suggest amendments.

[kysrpex]

One solution is to always return a set-like object that contains all the attribute values or all the CUDS linked when using the subscripting API. See examples below.

In [6]: from osp.core.namespaces import city
   ...: paris = city.City(name='Paris')
   ...: marc = city.Citizen(name='Marc')
   ...: aimee = city.Citizen(name='Aimée')
   ...: 
   ...: paris[city.hasInhabitant] += marc
   ...: paris[city.hasInhabitant] += aimee

In [7]: paris[city.hasInhabitant]
Out[7]: {<city.Citizen: 44529dab-b4bf-434b-a196-693d638a84c4,  CoreSession: @0x7fee5562c1f0>, <city.Citizen: 4fa81a0a-1ab9-4801-931e-0b4665dec233,  CoreSession: @0x7fee5562c1f0>} <city.hasInhabitant of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [8]: del paris[city.hasInhabitant]

In [9]: paris[city.hasInhabitant]
Out[9]: set() <city.hasInhabitant of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [10]: paris[city.hasInhabitant] = marc

In [11]: paris[city.hasInhabitant]
Out[11]: {<city.Citizen: 4fa81a0a-1ab9-4801-931e-0b4665dec233,  CoreSession: @0x7fee5562c1f0>} <city.hasInhabitant of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [12]: paris[city.name]
Out[12]: {'Paris'} <city.name of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [13]: paris[city.hasInhabitant] = {marc, aimee}

In [14]: paris[city.hasInhabitant]
Out[14]: {<city.Citizen: 44529dab-b4bf-434b-a196-693d638a84c4,  CoreSession: @0x7fee5562c1f0>, <city.Citizen: 4fa81a0a-1ab9-4801-931e-0b4665dec233,  CoreSession: @0x7fee5562c1f0>} <city.hasInhabitant of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

Advantages

• Very simple.
• Very compact.
• One always expects a set-like object when using the [...] notation.
• Can be iterated, like for x in paris[city.hasInhabitant].

Disadvantages

• Confusing for newcomers, who may think: Why am I getting a set, if there is just one citizen assigned? Things are worse when the property is functional, such as the name of the city, as the user expects even more heavily to get an object and not a set-like object able to contain various things (even if it just contains one).
• Difficult and verbose to extract the single element in the result for functional properties. A possibility would be paris[city.hasInhabitant].pop(), but this removes a CUDS from the list of inhabitants. The only solution is to create a new container and extract things from there, like set(paris[city.hasInhabitant]).pop() or tuple(paris[city.hasInhabitant])[0].

[kysrpex]

Another solution is to use slices to decide whether to return a set-like object that contains all the attribute values or all the CUDS linked when using the subscripting API; or just one of the items (at random) in each case. See examples below.

In [6]: from osp.core.namespaces import city
   ...: paris = city.City(name='Paris')
   ...: marc = city.Citizen(name='Marc')
   ...: aimee = city.Citizen(name='Aimée')
   ...: 
   ...: paris[city.hasInhabitant, :] += marc
   ...: paris[city.hasInhabitant, :] += aimee

In [7]: paris[city.hasInhabitant, :]
Out[7]: {<city.Citizen: 44529dab-b4bf-434b-a196-693d638a84c4,  CoreSession: @0x7fee5562c1f0>, <city.Citizen: 4fa81a0a-1ab9-4801-931e-0b4665dec233,  CoreSession: @0x7fee5562c1f0>} <city.hasInhabitant of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [8]: del paris[city.hasInhabitant, :]

In [9]: paris[city.hasInhabitant, :]
Out[9]: set() <city.hasInhabitant of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [10]: paris[city.hasInhabitant, :] = marc

In [11]: paris[city.hasInhabitant, :]
Out[11]: {<city.Citizen: 4fa81a0a-1ab9-4801-931e-0b4665dec233,  CoreSession: @0x7fee5562c1f0>} <city.hasInhabitant of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [12]: paris[city.name, :]
Out[12]: {'Paris'} <city.name of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [13]: paris[city.hasInhabitant, :] = {marc, aimee}

In [14]: paris[city.hasInhabitant, :]
Out[14]: {<city.Citizen: 44529dab-b4bf-434b-a196-693d638a84c4,  CoreSession: @0x7fee5562c1f0>, <city.Citizen: 4fa81a0a-1ab9-4801-931e-0b4665dec233,  CoreSession: @0x7fee5562c1f0>} <city.hasInhabitant of CUDS city.City: 34bdf5b8-4757-4450-81f0-295a04aaa504>

In [15]: paris[city.hasInhabitant]
Out[15]: <city.Citizen: 44529dab-b4bf-434b-a196-693d638a84c4,  CoreSession: @0x7fee5562c1f0>

In [16]: paris[city.name]
Out[16]: 'Paris'

In [17]: paris[city.hasInhabitant] += marc  # Expected not to work. 
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-17-c57c7d04cd70> in <module>
----> 1 paris[city.hasInhabitant] += marc  # Expected not to work.

TypeError: unsupported operand type(s) for +=: 'Cuds' and 'Cuds'

In [26]: paris[city.hasInhabitant]
Out[26]: <city.Citizen: 4fa81a0a-1ab9-4801-931e-0b4665dec233,  CoreSession: @0x7fee5562c1f0>

In [28]: paris[city.hasInhabitant] = None

In [29]: paris[city.hasInhabitant]

In [30]: paris[city.hasInhabitant] = marc

In [31]: paris[city.hasInhabitant]
Out[31]: <city.Citizen: 4fa81a0a-1ab9-4801-931e-0b4665dec233,  CoreSession: @0x7fee5562c1f0>

In [32]: del paris[city.hasInhabitant]

In [33]: paris[city.hasInhabitant]

(the del, None, and assignment of single item functionality would also be available with this solution, I just did not showcase them there)

Advantages

• Very simple.
• Very compact.
• One always expects a set-like object when using the [..., :] notation and a single object when not including the slice.
• Can be iterated when sliced for x in paris[city.hasInhabitant, :].
• Very easy to get the single value of non-functional properties.

Disadvantages

• Less confusing for newcomers, but the support of non-functional properties is not discoverable: expect questions in this forum asking whether OSP-core supports non-functional attributes. Always returning a set-like object would make it clear that more than one value per property is possible. At least for attributes this may be fine, as most attributes that newcomers may be using may be meant to only hold one value. The situation is different for CUDS, where the most likely thing to happen is that one wants to link multiple CUDS through the same relationship. This may be remedied by having a good documentation where the answer can easily be found, and maybe such a question from this forum as one of the first results in search engines.
• If a user did not yet learn about the slicing syntax, they may always query without a slice, and get a single value, thinking that this is the only value that the property can have. This may lead to errors going unnoticed in the software they create.

[pablo-de-andres]

Notes from the discussion on 24.11.
Note that the == operator is used to shown what the output of the getter should be

from osp.core.namespaces import ns

paris = ns.City(name="Paris")

# Object properties
paris.get(rel=ns.hasInhabitant)  # A (result)set (possibly empty)
paris.get(rel=ns.hasInhabitant).one()  # Only if one, else exception
paris.get(rel=ns.hasInhabitant).any()  # Return one, if empty None
paris.get(rel=ns.hasInhabitant).all()  # A list/set of one or all

# Functional data properties for properties explicitly defined in the ontology, converted to datatype of the property
## Because it is functional, querying always returns the value (no set)
paris.name = "Paris"
paris.name == "Paris"
paris.name = {"Paris", "name2"}
paris.name == '{"Paris", "name2"}'


# Both for data and object properties, functional and non-functional
#   [] will return a ResultSet
#   ResultSet.one()    return element only if one, else exception
#   ResultSet.any()    return one if one or multiple, if empty None
#   ResultSet.all()    return all (no change)

## Object properties
paris[ns.hasInhabitant]
paris[ns.hasInhabitant].one()
paris[ns.hasInhabitant].any()
paris[ns.hasInhabitant].all()

## Data properties
paris[ns.nickname] = "City of love"
paris[ns.nickname] == ResultSet("City of love")
paris[ns.nickname].one() == "City of love"
paris[ns.nickname].any() == "City of love"
paris[ns.nickname].all() == ResultSet("City of love")

paris[ns.nickname] = {"City of love", "French capital"}
paris[ns.nickname] == ResultSet("City of love", "French capital")
paris[ns.nickname].one() == Exception
paris[ns.nickname].any() == "City of love"
paris[ns.nickname].any() == "French capital"
paris[ns.nickname].all() == ResultSet("City of love", "French capital")

paris[ns.nickname] = 8 # Converted to datatype, "8"

[kysrpex]

I want also to introduce and additional feature. For the bracket case I think it makes sense that with relationships it only refers to the relationship itself, and not to its subrelationships. See the following example.

# Create an example entity.
from osp.core.namespaces import materials, reductionistic, physical
atom = materials.Atom()
nucleus = materials.Nucleus()
atom.add(nucleus, rel=reductionistic.hasSpatialDirectPart)
# Remember that reductionistic.hasSpatialDirectPart ⊑ physical.hasSpatialPart.

# Behaviour of get.
atom.get(rel=reductionistic.hasSpatialDirectPart)  # -> {<materials.Nucleus: ad62d65a-423f-4c29-b6d7-f08968aa31fc,  CoreSession: @0x7f68179026d0>}
atom.get(rel=reductionistic.hasSpatialPart)  # -> {<materials.Nucleus: ad62d65a-423f-4c29-b6d7-f08968aa31fc,  CoreSession: @0x7f68179026d0>}

# Behaviour that I want to implement with brackets.
atom[reductionistic.hasSpatialDirectPart] -> {<materials.Nucleus: ad62d65a-423f-4c29-b6d7-f08968aa31fc,  CoreSession: @0x7f68179026d0>}
atom[reductionistic.hasSpatialPart] -> {}
atom[:reductionistic.hasSpatialPart] -> {<materials.Nucleus: ad62d65a-423f-4c29-b6d7-f08968aa31fc,  CoreSession: @0x7f68179026d0>}
atom[reductionistic.hasSpatialPart:] -> {}
# So here, the slice `:reductionistic.hasSpatialPart` means "this class and all subclasses", very similar to what happens with a list [1, 2, 3, 4][:2] -> [1, 2].
# The slice `reductionistic.hasSpatialPart:` means "this class and all superclasses" (this one is less useful and more confusing, but for coherence should be there).
atom[reductionistic.hasSpatialDirectPart] = nucleus  # OK
atom[reductionistic.hasSpatialPart] = nucleus  # OK
atom[:reductionistic.hasSpatialPart] = nucleus  # Exception

In this way we can now also query a single relationship without sub- and superclasses (this was not available before if I am not wrong). If I do the same with attributes then we gain the ability to query one attribute and its subattributes at once (definitely also a functionality that was not available before).

[pablo-de-andres]

I am not sure if having different behaviours might be confusing? On the other hand, I could see the benefit.

Why are the : back?

[kysrpex]

To differentiate between "this class and all subclasses" and "this class only". This time it has the same meaning as in a list, a sequential meaning.

[pablo-de-andres]

I should definitely read all the comments before I start asking 🙄
I again see the benefit, but wonder if it might be a bit obscure to a user. If it is used as an advanced feature it might be ok

[yoavnash]

Maybe we can spend some time on this in the next meeting