Updating a nested field

[fifi98]

Hello,

I appreciate all the work you put into onetable development! I've been having some trouble executing update command on my model. My data in JSON format looks like this:

 "pk": "story:iVsIUKLk69s3dkwHs-S8z",
 "sk": "story:",
 "location": {},
 "slides": {
  "4SWys5q0BhHOiTxlRbBsK": {
   "id": "4SWys5q0BhHOiTxlRbBsK",
   "mediaType": "image",
   "mediaUri": "qmLisXFMD5z7g-e0fsdzb",
  },
  "FSzd9sjpenCAbZ_klOszZ": {
   "id": "FSzd9sjpenCAbZ_klOszZ",
   "mediaType": "image",
   "mediaUri": "jG45dB31jcDXXTYjRVMGX",
   "processed": true
  }
 }

What I want is to set processed to true on slides[4SWys5q0BhHOiTxlRbBsK].processed. Basically I have to execute the following update statement from aws-sdk using Onetable:

const params = {
  TableName: "table-name-dev",
  Key: {
    pk: "story:iVsIUKLk69s3dkwHs-S8z",
    sk: "story:",
  },
  UpdateExpression: "set slides.#slideId.#key = :r",
  ExpressionAttributeNames: {
    "#slideId": "hvhf1VCCBqsVI3KngYk3N",
    "#key": "processed",
  },
  ExpressionAttributeValues: {
    ":r": true,
  },
  ReturnValues: "ALL_NEW",
};

docClient.update(params, function (error, data) {
  if (err) {
    console.error("Unable to update item. Error:", error);
  } else {
    console.log(data.Attributes);
  }
});

Can you please point me into the right direction how can I execute that update command with Onetable? I know there is this example of updating nested fields:

user = await User.update({email: 'roadrunner@acme.com'}, {set: {'address.zip': '{"98034"}'}})

but I didn't have any luck with doing it like set: { 'slides.4SWys5q0BhHOiTxlRbBsK.processed': true }

(the reason why slides isn't an array but an object is because I wouldn't be able to update a specific field of a specific object inside the array as far as I know, please let me know if I'm wrong)

Thank you!

[mobsense]

This works for me:

Model:

export default {
    version: '0.0.1',
    indexes: {
        primary: { hash: 'pk', sort: 'sk' },
    },
    models: {
        User: {
            pk:          { type: String, value: '${_type}#' },
            sk:          { type: String, value: '${_type}#${email}' },
            id:          { type: String, uuid: true },
            email:       { type: String },
            slides:      { type: Object, schema: {
                            id:         { type: String },
                            mediaType:  { type: String },
                            mediaUrl:   { type: String },
                            processed:  { type: Boolean },
                        }
            },
        }
    }
}

Code:

    user = await User.create({
        email: 'rr@acme.com',
        slides: {
            id:         '4SWys5q0BhHOiTxlRbBsK',
            mediaType:  'video',
            mediaUrl:   'https://acme.com/',
            processed:  false,
        }
    })

    user = await User.update({email: 'rr@acme.com'}, {set: {'slides.4SWys5q0BhHOiTxlRbBsK.processed': true}})

[fifi98]

Thanks for the fast answer! Unfortunately I am getting this error with that code when trying to update the user:
TypeError: Cannot read property 'processed' of null

And besides that error, the code you provided would not allow me to store multiple slides under "slides" property.

[mobsense]

Try with the 1.9 just released. I tested with that.

[fifi98]

Unfortunately no luck. I have tested with the newly released 1.9, but processed isn't set to true after .update. Also, is it possible to have the slides property to contain multiple slide objects like this, but not only one:

 "pk": "story:iVsIUKLk69s3dkwHs-S8z",
 "sk": "story:",
 "location": {},
 "slides": {
  "4SWys5q0BhHOiTxlRbBsK": {
   "id": "4SWys5q0BhHOiTxlRbBsK",
   "mediaType": "image",
   "mediaUri": "qmLisXFMD5z7g-e0fsdzb",
  },
  "FSzd9sjpenCAbZ_klOszZ": {
   "id": "FSzd9sjpenCAbZ_klOszZ",
   "mediaType": "image",
   "mediaUri": "jG45dB31jcDXXTYjRVMGX",
   "processed": true
  }
 }

[mobsense]

Can you look at test/debug.ts to see if you can see a difference. I tested with jest debug.

Did you get the same error message? What happened?
Can you use {debug: true} in the params and see the actual command generated which will be traced to stdout.

[mobsense]

I need to see the actual command output. Add {debug: true} to the 2nd parameter to update.

[mobsense]

I mean the actual command issued to dynamodb

[fifi98]

await table
    .getModel("Testing")
    .update({ email: "rr@acme.com" }, { set: { "slides.4SWys5q0BhHOiTxlRbBsK.processed": true }, debug: true });

Did you mean like this? If yes, there wasn't any output.

[mobsense]

See the README section on the logger.

OneTable can log complete request parameters and responses to assist you in debugging and understanding how your API requests are being translated to DynamoDB.

You can set logger parameter to true for simple logging to the console. Alternatively, the logger may be set to logging callback that will be invoked as required to log data. The logger function has the signature:

const table = new Table({
    ...
    logger: (level, message, context) => {
        if (level == 'trace' || level == 'data') return
        console.log(`${new Date().toLocaleString()}: ${level}: ${message}`)
        console.log(JSON.stringify(context, null, 4) + '\n')
    }
})
Where level is set to info, error, warn, exception, trace or data. The trace level is for verbose debugging messages. The data level logs user data retrieved find and get API calls.

The message is a simple String containing a descriptive message. The context is a hash of contextual properties regarding the request, response or error.

If you use {log: true} in the various OneTable Model API options, the more verbose trace and data levels will be changed to info for that call before passing to the logging callback. In this way you can emit trace and data output on a per API basis.

[fifi98]

Got it, sorry. Here is the output of the update function:

30/09/2021, 22:55:23: info: OneTable "update" "Testing"
{
    "trace": {
        "model": "Testing",
        "cmd": {
            "ConditionExpression": "(attribute_exists(pk)) and (attribute_exists(sk))",
            "ExpressionAttributeNames": {
                "#_0": "slides",
                "#_1": "4SWys5q0BhHOiTxlRbBsK",
                "#_2": "email",
                "#_3": "updatedAt"
            },
            "ExpressionAttributeValues": {
                ":_0": {
                    "BOOL": true
                },
                ":_1": {
                    "S": "rr@acme.com"
                },
                ":_2": {
                    "N": "1633035323030"
                }
            },
            "TableName": "restory-dev",
            "ReturnValues": "ALL_NEW",
            "UpdateExpression": "set #_0.#_1 = :_0, #_2 = :_1, #_3 = :_2",
            "Key": {
                "pk": {
                    "S": "Testing#"
                },
                "sk": {
                    "S": "Testing#rr@acme.com"
                }
            }
        },
        "op": "update",
        "properties": {
            "pk": "Testing#",
            "sk": "Testing#rr@acme.com",
            "email": "rr@acme.com",
            "_type": "Testing",
            "updatedAt": 1633035323030
        }
    }
}

30/09/2021, 22:55:23: info: OneTable result for "update" "Testing"
{
    "cmd": {
        "ConditionExpression": "(attribute_exists(pk)) and (attribute_exists(sk))",
        "ExpressionAttributeNames": {
            "#_0": "slides",
            "#_1": "4SWys5q0BhHOiTxlRbBsK",
            "#_2": "email",
            "#_3": "updatedAt"
        },
        "ExpressionAttributeValues": {
            ":_0": {
                "BOOL": true
            },
            ":_1": {
                "S": "rr@acme.com"
            },
            ":_2": {
                "N": "1633035323030"
            }
        },
        "TableName": "restory-dev",
        "ReturnValues": "ALL_NEW",
        "UpdateExpression": "set #_0.#_1 = :_0, #_2 = :_1, #_3 = :_2",
        "Key": {
            "pk": {
                "S": "Testing#"
            },
            "sk": {
                "S": "Testing#rr@acme.com"
            }
        }
    },
    "items": [
        {
            "id": "493d93218fe2df508da923d883832739",
            "email": "rr@acme.com",
            "slides": {
                "4SWys5q0BhHOiTxlRbBsK": true,
                "id": "4SWys5q0BhHOiTxlRbBsK",
                "mediaType": "video",
                "mediaUrl": "https://acme.com/",
                "processed": false
            },
            "createdAt": "2021-09-30T20:55:22.439Z",
            "updatedAt": "2021-09-30T20:55:23.030Z"
        }
    ],
    "op": "update",
    "properties": {
        "pk": "Testing#",
        "sk": "Testing#rr@acme.com",
        "email": "rr@acme.com",
        "_type": "Testing",
        "updatedAt": 1633035323030
    },
    "params": {
        "exists": true,
        "parse": true,
        "high": true,
        "set": {
            "slides.4SWys5q0BhHOiTxlRbBsK.processed": true
        },
        "log": true,
        "checked": true
    }
}

[mobsense]

Thanks. Got it.

The reason is that your '4SWys5q0BhHOiTxlRbBsK' is not a property defined in the schema. i.e. not named. The current code supports named properties and one last trailing unnamed property.

I'll see if this can be fixed.

[mobsense]

Wait on.

I've got your create and schema code wrong. Should be slides just as an object with no nested schema.

export default {
    version: '0.0.1',
    indexes: {
        primary: { hash: 'pk', sort: 'sk' },
    },
    models: {
        User: {
            pk:          { type: String, value: '${_type}#' },
            sk:          { type: String, value: '${_type}#${email}' },
            id:          { type: String, uuid: true },
            email:       { type: String },
            slides:      { type: Object }
            },
        }
    }
}

Then the code should be

    user = await User.create({
        email: 'rr@acme.com',
        slides: {
            '4SWys5q0BhHOiTxlRbBsK': {
                mediaType:  'video',
                mediaUrl:   'https://acme.com/',
                processed:  false,
            }
        }
    })

    user = await User.update({email: 'rr@acme.com'}, {log: true, set: {'slides.4SWys5q0BhHOiTxlRbBsK.processed': true}})

Then there is a fix in Expression.js to make it work for mult-level set expressions like yours.

We'll apply this fix:

diff --git a/src/Expression.js b/src/Expression.js
index 0991938..a0b469e 100644
--- a/src/Expression.js
+++ b/src/Expression.js
@@ -384,7 +384,7 @@ export class Expression {
             } else {
                 subscript = ''
             }
-            let field = fields[prop]
+            let field = fields ? fields[prop] : null
             if (field) {
                 target.push(`#_${this.addName(field.attribute[0])}${subscript}`)
                 //  If nested schema, advance to the next level
@@ -394,9 +394,6 @@ export class Expression {
                 target.push(`#_${this.addName(prop)}${subscript}`)
                 fields = null
             }
-            if (fields == null) {
-                break
-            }
         }
         return target.join('.')
     }

[fifi98]

Awesome, thank you very much! I will try the fix and report back.

[onhate]

I have the following schema

const PermissionsSchema = {
  fields: {
    pk: { type: String, value: '${typename}#${resource}' },
    sk: { type: String, value: '${typename}#${subject}' },
    typename: { type: String },
    resource: { type: String },
    subject: { type: String },
    policies: { type: Object }
  }
};

and running the following update command, what I really want is that each time I call the set(res, sub, { docs: '1111' }) and later set(res, sub, { forums: '1111' }) I don't want the latest call to overwrite the already existing policies.

async set(resource: string, subject: string, policies: Policies) {
  const set = Object.fromEntries(
    Object.entries(policies).map(([key, value]) => ['policies.' + key, value])
  );
  return await this.model.update({ resource, subject }, { set, exists: null });
}

but this is not working, the response of this is an object without policies field, looks like it is completly ignoring the { set } param

this is the log for the update operation

console.log
   {
     "trace": {
       "model": "Permissions",
       "cmd": {
         "ExpressionAttributeNames": {
           "#_0": "createdAt",
           "#_1": "typename",
           "#_2": "resource",
           "#_3": "subject",
           "#_4": "updatedAt"
         },
         "ExpressionAttributeValues": {
           ":_0": "2022-01-17T19:27:03.412Z",
           ":_1": "Permissions",
           ":_2": "resource",
           ":_3": "subject"
         },
         "TableName": "BigTable-test",
         "ReturnValues": "ALL_NEW",
         "UpdateExpression": "set #_0 = if_not_exists(#_0, :_0), #_1 = :_1, #_2 = :_2, #_3 = :_3, #_4 = :_0",
         "Key": {
           "pk": "Permissions#resource",
           "sk": "Permissions#subject"
         }
       },
       "op": "update",
       "properties": {
         "pk": "Permissions#resource",
         "sk": "Permissions#subject",
         "typename": "Permissions",
         "resource": "resource",
         "subject": "subject",
         "updatedAt": "2022-01-17T19:27:03.412Z"
       }
     }
   } BigtableService

     at Logger.log (../../node_modules/@nestjs/common/services/logger.service.js:41:75)

 console.log
   [AWS dynamodb 200 0.016s 0 retries] updateItem({
     ExpressionAttributeNames: {
       '#_0': 'createdAt',
       '#_1': 'typename',
       '#_2': 'resource',
       '#_3': 'subject',
       '#_4': 'updatedAt'
     },
     ExpressionAttributeValues: {
       ':_0': { S: '2022-01-17T19:27:03.412Z' },
       ':_1': { S: 'Permissions' },
       ':_2': { S: 'resource' },
       ':_3': { S: 'subject' }
     },
     TableName: 'BigTable-test',
     ReturnValues: 'ALL_NEW',
     UpdateExpression: 'set #_0 = if_not_exists(#_0, :_0), #_1 = :_1, #_2 = :_2, #_3 = :_3, #_4 = :_0',
     Key: {
       pk: { S: 'Permissions#resource' },
       sk: { S: 'Permissions#subject' }
     }
   })

[onhate]

@mobsense got it, it has to do with timastamps and the exists: null property, when using exists null it overwrites the set param, I'll raise a PR fixing this

[onhate]

#251 here it is @mobsense