ObjectIDs are not returned as strings when .lean() is called with a cache

[bruce-bjorklund-sophos]

What is returned with a "findOne.cache().lean()" with no cache hydrated
{
_id: '660dd512e8532980bb5084e5'
}

What is returned with a "findOne.cache().lean()" with the cache hydrated:
{
_id: new ObjectId("660dd512e8532980bb5084e5"),
}

I would expect that we would get only JSON back if we called lean().

thoughts?

[ilovepixelart]

Hi @bruce-bjorklund-sophos, I'm concerned about your first statement, that you received a string.

Also .lean() !== .toJSON(), here is example in mongoose playground: https://playground.mongoosejs.io/#YXdhaXQgbW9uZ29vc2UuY29ubmVjdCgnbW9uZ29kYjovL2xvY2FsaG9zdDoyNzAxNy9tb25nb29zZV90ZXN0Jyk7CmNvbnN0IHNjaGVtYSA9IG5ldyBtb25nb29zZS5TY2hlbWEoeyBuYW1lOiBTdHJpbmcgfSk7CmNvbnN0IFVzZXIgPSBtb25nb29zZS5tb2RlbCgnVXNlcicsIHNjaGVtYSk7Cgpjb25zdCBjcmVhdGVkID0gYXdhaXQgVXNlci5jcmVhdGUoeyBuYW1lOiAnU3RldmUnIH0pCmNvbnN0IGxlYW4gPSBhd2FpdCBVc2VyLmZpbmRCeUlkKGNyZWF0ZWQuX2lkKS5sZWFuKCk7Cgpjb25zb2xlLmxvZyhsZWFuKQpjb25zb2xlLmxvZyh0eXBlb2YgbGVhbi5faWQp

So the question is under which circumstances you got string. Do you use in memory or redis driver?

If you want pure json you should use: https://mongoosejs.com/docs/api/document.html#Document.prototype.toJSON()

[bruce-bjorklund-sophos]

Yeah you are right, it is valid JSON, but what we are expecting is the _id to be a string, not an ObjectID.

I went through your code and it looks like in the Redis driver, and you do a thing where if the value is a string, and a UUID you cast it into a ObjectID.

In Mongoose, if we don't use lean() we get back UUIDs with ObjectID types... but if we use lean() we get back full JSON. So there are no methods like .toJSON(). TBH I think the decision to cast every ObjectID string into a objectID class breaks the implementation of lean()

[ilovepixelart]

Did you check example I gave you in playground mongoose returns _id as object when using lean. This is why I cast to object because lean not returning id as string.

[bruce-bjorklund-sophos]

Oh you know what I think it is?

We have the schema where the _id is a string... but we generate an ObjectID for it:

const UserSchema = new Schema<IUser>({ _id: String, organization_id: String });

const newUser = {_id: new ObjectId().toHexString()}

The redis engine will cast any string that is an ObjectID match regardless of the schema definition

[ilovepixelart]

Now thats makes total sense now! Honestly speaking don't do that :) store your id properly, but yeah interesting case

[bruce-bjorklund-sophos]

Yeah... Yeah I completely agree with you on that this is non-standard - I'm just like, the janitor.

I think the right thing to do would be migrate to the standard, but that LOE is too much. Would be cool if the engine used the schema for type casting but not sure what the LOE would be on that or if its even possible.

[ilovepixelart]

Yep I did this default casting due to aggregate inability to properly hydrate:
Automattic/mongoose#8784 (comment)

And for cases like distinct, will check what we can do, but it's not a trivial task (just removing casting won't cut it)

But if you have sufficient ram on pod/service/vm/metal you can switch to engine: 'memory', this will solve your issue since in memory engine stores object as is without stringify hydrate etc...

[ilovepixelart]

Will link here issue I created in mongoose official repo,
looks like proper hydration is not happening even on Query (only affects populated virtual objects)
Automattic/mongoose#14503

[ilovepixelart]

PR for serialization in redis engine: #198

[ilovepixelart]

Released in: https://github.com/ilovepixelart/ts-cache-mongoose/releases/tag/v1.6.0