Push elements in attributes which value is a vector

[fgalan]

At the present moment (Orion 0.18.1), the only way of adding an element to an attribute which value is a vector is to update such attribute with a new vector containing the new element.

However, I think that pushing elements in attributes which value is a vector whitout needing to update the whole vector is an interesting feature to take into account.

[fgalan]

Raised in http://stackoverflow.com/questions/28347889/orion-context-broker-push-array-element-in-entities/28369017

[fgalan]

Not only pushing a single element, but a list of them.

[inter-coder]

My vision includes the insertion of Observer who will trigger the function on the basis of parameters. The script with functional examples is here https://github.com/inter-coder/Orion-Web-Proxy-for-Cross-Domain

[fgalan]

This needs to be re-considered under the light of NGSIv2 API.

[fgalan]

Out of the scope of NGSIv2, althouth it is a good idea that could (potentially) be included in a later version of the API (v2.1) in a backward compatible way.

CC: @jmcanterafonseca

[mkavipriya]

Hi Fermin,

Wanted to check if the above feature to push new array elements into attributes, without overwriting the existing values, has been implemented yet in latest NGSI v2 APIs? We have a similar requirement, where we want to keep adding new elements to an array without overwriting the previous values. But seems the current API does not allow this feature. Can you please help us on this. Thanks!

[fgalan]

Orion 3.3.0 supports the usage of update operators (doc here: https://fiware-orion.readthedocs.io/en/master/user/update_operators/index.html) as extension to "clasical" NGSIv2. The $push operator allows adding items. Eg:

POST /v2/entities/E/attrs/A
{
  "value": { "$push": 3 },
  "type": "Number"
}

With regards to:

Not only pushing a single element, but a list of them.

Taking advantage of $each MongoDB semantics, that could be done with something like this:

POST /v2/entities/E/attrs/A
{
  "value": { "$push": {"$each": [3, 4, 5]} },
  "type": "Number"
}

Could you test them and tell us how it goes, please? Thanks!

[mkavipriya]

Hi Fermin,

Thanks for your quick reply. We were able to successfully test the above mentioned update operators, with few observations:

1. POST method fails with error "Method not allowed", we had to replace it with PUT method and then it worked

PUT /v2/entities/E/attrs/A
{
"value": { "$push": 3 },
"type": "Number"
}

2. We also tested this with Batch Update operation and it worked fine. But one observation here is, if the attribute A does not already exist, it creates a new child attribute named "$push" (which is incorrect behaviour). But if attribute A already exists, then it works as expected by adding new array element.

In our case, we wouldn't know if attribute already exists or not when performing Batch operation, so this will be an overhead to first make sure Array attribute exists and only then use $push appropriately. Is there a solution/workaround to make it work the first time itself?

POST /v2/op/update
{
"actionType": "APPEND",
"entities": [
{
"id":"E",
"A": {
"value": { "$push": 4 },
"type": "Number"
}
}
]
}

3. Orion 3.3.0-next version: we are currently using 3.1.0-next in our production environment. Also noticed that fiware-orion:latest on docker-hub still pointing to 3.1.0-next version. We had to explicitly download 3.3.0-PRE-137 to test this functionality. Is there any workaround to get this "Update Attribute" to work in 3.1.0-next? Also when is the official release of 3.3.0-next please?

[fgalan]

Thanks for the feedback! Some comments:

• Your are right: the verb to use is PUT, not POST. Sorry for the typo :)
• At the present moment, update operators cannot be used at entity creation time. This is described in the documentation https://fiware-orion.readthedocs.io/en/master/user/update_operators/index.html#create-or-replace-entities. However, they can be used at attribute creation time if the entity already exists.
• Orion 3.3.0 has been already released (see https://github.com/telefonicaid/fiware-orion/releases/tag/3.3.0). Note that x.y.z-next doesn't means that x.y.z is to come, but that x.y.z has been released and that we are in the preparation of the next release to x.y.z.
• fiware-orion:latest shouldn't point to 3.1.0-next. It should point to latests commit in master branch. Could you provide more detail on how are you pulling and running this image, please?
• There is no possible workaround to use this functionality in 3.1.0-next. This functionality has been developed for 3.3.0. You may find it in some of the interim versions 3.2.0-next, but I'd strongly recommend to use 3.3.0.

[fgalan]

fiware-orion:latest shouldn't point to 3.1.0-next. It should point to latests commit in master branch. Could you provide more detail on how are you pulling and running this image, please?

I've done the test myself and you are right: there is some problem with fiware/orion:latest image. I have opened an issue here: #4028

In the meanwhile, you can use telefonicaiot/fiware-orion:latest, that works correctly

[fgalan]

I have tested

curl localhost:1026/v2/entities/E/attrs -s -S -H 'Content-Type: application/json' -d @- <<EOF
{
  "A": {
    "value": { "\$push": {"\$each": [2, 3, 4]} },
    "type": "Array"
  }
}
EOF

but it is not working as expected, as the CB replacing logic in dbEncode() is changing $each to >each before doing the findAndModify in DB.

I'll try to fix.

[fgalan]

PR #4029