Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Temporary (virtual) properties on model instance #2642

Open
wclr opened this issue Feb 3, 2015 · 29 comments
Open

Temporary (virtual) properties on model instance #2642

wclr opened this issue Feb 3, 2015 · 29 comments
Labels
discussion If you have any thoughts or comments on this issue, please share them! enhancement This issue is a user-facing general improvement that doesn't fix a bug or add a new feature

Comments

@wclr
Copy link

wclr commented Feb 3, 2015

Sometimes I need to have on a model instance some temporaty property that is not in DB but for example I want to send to the client.

The scenario:

say I a have a Parent Model and Child model, I store them in separeate collections and only Child model has reference to it's parent

Parent.findOne({..}, function(err, parent){
    Child.find({parent: parent.id}, function(){err, children}{
        // now I want to send to the client parent instance with children as nested array
          parent.children = children
          res.send(parent)
    })
})

the problem is that parent.children = children won't work if children property is not defined on Parent schema. But its only temporary property and should not be saved to DB, it is not reasonable (and just incorrect) to define it on Parent schema.

The workaround I found is to add the followin virtual peroperty to Parent model:

parentSchema.virtual('children').get(function(){
    return this.__children
}).set(function(val){
    this.__children = val
})

I thought maybe something like that could work for such proerties:

// that would allow setting ```children``` but do not store it in DB
parentSchema.virtual('children') 

What do you think?

@vkarpov15
Copy link
Collaborator

Virtuals are the correct way to do this. The method you described with .get() and .set() seems pretty concise, no?

@wclr
Copy link
Author

wclr commented Feb 6, 2015

Well I didn't try get/set. Will it work for non-schema properties, and won't it place "illegal" data in db by chance?

@vkarpov15
Copy link
Collaborator

I don't quite understand the last question. Can you please elaborate?

@wclr
Copy link
Author

wclr commented Feb 7, 2015

It seems set/get doesn't work for me.

One more time: I want to ``set/get` property that is NOT defined on schema. Its is only temporary. The property should be processed with toObject, toJSON, but should not be saved to DB. So it is like virtual or terporary. Look at my code example in OP.

@vkarpov15
Copy link
Collaborator

Both of the below options work for what you're trying to do as far as I can tell:

var Schema = mongoose.Schema;

mongoose.connect('mongodb://localhost:27017/gh2642');

var parentSchema = new Schema({ name: String });

var Parent = mongoose.model('gh2642', parentSchema, 'gh2642');

var p = new Parent({ name: 'test' });

p.children = { a: 1 };

console.log(JSON.stringify(p.children));

p.save(function(err, p) {
});

and

var Schema = mongoose.Schema;

mongoose.connect('mongodb://localhost:27017/gh2642');

var parentSchema = new Schema({ name: String });

parentSchema.virtual('children').
  get(function() {
    return this.__children;
  }).
  set(function(v) {
    this.__children = v;
  });

var Parent = mongoose.model('gh2642', parentSchema, 'gh2642');

var p = new Parent({ name: 'test' });

p.children = { a: 1 };

console.log(JSON.stringify(p.children));

p.save(function(err, p) {
});

In both cases children property isn't saved to the DB. If you want the children property included with toObject() and toJSON(), you need a virtual and pass virtuals: true to toObject() or toJSON().

@vkarpov15 vkarpov15 added the help This issue can likely be resolved in GitHub issues. No bug fixes, features, or docs necessary label Feb 7, 2015
@wclr
Copy link
Author

wclr commented Feb 8, 2015

Your first code example (where you do not define virtual property children) won't work with toJSON/toObject even if virtuals: true presents.

That is what about I'm talking. It only works if you have a definition for virutual property like this:

parentSchema.virtual('children').
    get(function() {
        return this.__children;
    }).
    set(function(v) {
        this.__children = v;
    });

The issue was about gettring rid of this exessive code.

@wclr
Copy link
Author

wclr commented Feb 14, 2015

Again about this issue I do not think it is resolved.

yes this code

p.children = { a: 1 };

works in 3.9 (though it seemd not to work in 3.8 p.children was undefined after assignment). But anyway with this approach children is not going to get into toObject or toJSON result. So the only way to make it getting there is to define the property as virtual field:

parentSchema.virtual('children').
    get(function() {
        return this.__children;
    }).
    set(function(v) {
        this.__children = v;
    });

So why not allow this with just one line, that would make all the stuff internaly

parentSchema.virtual('children')

?

@vkarpov15
Copy link
Collaborator

Too much magic under the hood IMO - what if you have a schema path called __children? You could utilize the fact that mongodb doesn't allow you to have field names that start with $ in >= 2.6, but mongoose should still support mongodb 2.4.

It would be pretty easy to implement this with a plugin though, if it really bugs you.

@wclr
Copy link
Author

wclr commented Feb 18, 2015

what if you have a schema path called __children

yes I undertand it, __children its just my local implementation, mongoose could implement it some how internaly with more correct approach. Any way if you don't like it, not a big problem.

@vkarpov15
Copy link
Collaborator

The general idea is good, but I think its tricky to get it right in the general case. Avoiding schema paths, etc. will make this more trouble than it's worth to trim 6 SLOC.

@vko-online
Copy link

Does virtual solves this problem?

@wclr
Copy link
Author

wclr commented Jul 17, 2015

only using such hack:

parentSchema.virtual('children').
    get(function() {
        return this.__children;
    }).
    set(function(v) {
        this.__children = v;
    });

@vkarpov15 vkarpov15 reopened this Jul 17, 2015
@techjeffharris
Copy link

@vkarpov15, is it safe to use __?

More specifically, do documents (or their prototypes) have any internal/non-enumerable properties that start with __ ?

@vko-online
Copy link

Is it possible to pass nodejs/express request req inside virtual property?

parentSchema.virtual('children').
    get(function() { //how can we invoke req here?
        return this.__children;
});

@techjeffharris
Copy link

@vko Sure it's possible, but probably not the best idea as you would need to use some scoping magic and define your schema within a particular express route handler or param middleware... You're asking a specific question that is more relevant to function scoping than mongoose virtuals...

http://ryanmorr.com/understanding-scope-and-context-in-javascript/

@vko-online
Copy link

@techjeffharris is there any demo? I really want this but not sure how it's done.

@vkarpov15
Copy link
Collaborator

@techjeffharris I think it should be safe

@vko-online not really possible, I would advise against trying to do that.

@techjeffharris
Copy link

@vkarpov15 sounds like it will work for now. Thank you!

I'll investigate what it would take to add a schema method called temporary or something that would essentially create virtual properties that are stored in a per-document property like __temp. This would allow for officially sanctioned (and anticipated) virtual properties while keeping the internal __ namespace clean.

@wclr
Copy link
Author

wclr commented Aug 26, 2015

@techjeffharris you can use ____ (4 underlines) :D

@techjeffharris
Copy link

@whitecolor Haha

@vkarpov15
Copy link
Collaborator

Just make sure you don't use $__ :p Hopefully ES6 symbols will be helpful for this...

@arindrajitb
Copy link

I found a alternate way to solve the problem. As its JavaScript adding property on the go is not a problem. The problem is here with the toJson or toObject. you can overcome that bu over riding those methods.
i used this ( as suggested in http://stackoverflow.com/questions/11160955/how-to-exclude-some-fields-from-the-document).
UserSchema.methods.toJSON = function() {
var obj = this.toObject()
delete obj.passwordHash
return obj
}

This is better over mongoose transform. I believe wont work in the scenario when you delete the backing field the virtual will lose the source of information. therefore will not show-up in JSON.

@wclr
Copy link
Author

wclr commented Nov 20, 2015

Hey, for thouse who intrested you can check my current implementation for virtual async fields api.

https://github.com/whitecolor/mongoose-fill

I think mongoose.js actually should have something like this in the core. Intresting in you oppinion.

@vkarpov15
Copy link
Collaborator

@whitecolor sweet module! Way to go, I really like the plugin's API and I'm looking forward to using it. But looks like it isn't up on npm yet? https://www.npmjs.com/package/mongoose-fill

@vko-online
Copy link

@whitecolor this __fill.fill.fill looks really weird, just sayin

@wclr
Copy link
Author

wclr commented Nov 21, 2015

@vkarpov15 published it.

@vko-online its in dev mode so convetions and terms in the code may seem not clear.

__fill - is actually is object that contains info about what should be filled in particular query
__fill.fill - is ref to filler rules that should be applied (that are determined with mySchema.fill API)
__fill.fill.fill - is fill method of filler rules (it is not mentioned in docs, as I don't use it at currently, and not sure if it is needed at all)

We use it in our projects and we like api that it provides and allows to fill/combine mongoose objects with any kind of data, not only form current mongoose DB, but also from any async service.

@techjeffharris
Copy link

@whitecolor perhaps my usage of virtual properties is deferent a bit unorthodox compared to yours, but this module seems IMHO a bit of a complicated workaround to simply not store an array of child IDs. If the parent had an array of refs to the child IDs, one could simply populate the array of children and only select the name and age properties.

I'm using virtual properties to attach data (for instance, socket.io "sockets" for a user—each page refresh end and creates a new socket) which is expires when the process closes (or sooner) and is thus worthless to store in the db, but is nonetheless useful to have associated with a user model.

I'm writing a glorified chat server that caches all users from db at startup, then adds references to each of the users sockets (open windows with the same user session) to the document instance. If he serve restarts or closes, the sockets close and new sockets are made when the sever is back up and each window reconnects.

I suppose it's a more SQL-like way of doing things, stirring the parent ID in the child rather than storing child IDs in the parent, but I feel like using refs and populating the parent is effectively the same. Though, admittedly, I may be missing something.

I'm glad that you made something that works for you, nonetheless 😄

@wclr
Copy link
Author

wclr commented Nov 21, 2015

@techjeffharris
Well storing IDs consistently is additional task. Some cases really do demand it, for example when you need to keep some scpecific order of nested items and have ability of changing that order (you just swap, insert, remove IDs), etc. Sometimes using array of refs probably it can be aslo justified by perfomace requirements (when stored IDs act like index).

But often such solid connections are really exessive and add additional potential breaking point that you really want to avoid. In my oppinion if you can avoid exessive linking (coupling) in you architecture - you shoud (or must) avoid it. Yes it is some kind of SQL normalized structure way, but it simplifies things by reducing coupling and makes design more flexible.

@vkarpov15 vkarpov15 removed the help This issue can likely be resolved in GitHub issues. No bug fixes, features, or docs necessary label Feb 15, 2019
@jloveridge
Copy link

jloveridge commented Aug 23, 2019

@techjeffharris Storing the parentId on the child perfectly fine in both SQL and NOSQL worlds. Especially since unbounded arrays is an anti-pattern. Eventually you could exceed the 16MB document limit for a MongoDB document. For your particular case I would use a virtual populate.

parentSchema.virtual('children', {
  ref: 'Child',
  localField: '_id',
  foreignField: 'parentId',
});

Then when you want the children for any given parent you would simply do:

Parent.findOne({..}).populate('children').exec(function(err, parent) {
    if (err) {
        next(err);
    } else {
      res.send(parent.toObject({virtuals: true});
    }
});
// or using promise syntax
Parent.findOne({...}).populate('children').exec()
  .then((parent) => res.send(parent.toObject({virtuals: true}))
  .catch(next)

Also note that you can omit the {virtuals: true} if you set the schema to default to including virtuals on toObject (default is false).

@vkarpov15 vkarpov15 added enhancement This issue is a user-facing general improvement that doesn't fix a bug or add a new feature and removed plugin labels Oct 14, 2021
@vkarpov15 vkarpov15 added this to the 6.1.0 milestone Oct 14, 2021
@vkarpov15 vkarpov15 added the discussion If you have any thoughts or comments on this issue, please share them! label Oct 14, 2021
@vkarpov15 vkarpov15 modified the milestones: 6.1.0, 6.2.0 Nov 17, 2021
@vkarpov15 vkarpov15 modified the milestones: 6.2.0, 6.x Unprioritized Jan 15, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion If you have any thoughts or comments on this issue, please share them! enhancement This issue is a user-facing general improvement that doesn't fix a bug or add a new feature
Projects
None yet
Development

No branches or pull requests

6 participants