Unable to filter response from 'find / findOne' within plugin

[a3957273]

Do you want to request a feature or report a bug?
Feature

What is the current behavior?
Unable to filter response from a .find() or .findOne() call.

If the current behavior is a bug, please provide the steps to reproduce.
Given a plugin:

(schema: Schema) => {
    schema.post('findOne', async function(doc) {
      if (await shouldRemove(doc)) return undefined
    
      return doc
    })
  }

I should be able to selectively return either nothing, or a document, from .find() and .findOne() calls.

What is the expected behavior?
If shouldRemove returns true, for no document to be returned from undefined.

What are the versions of Node.js, Mongoose and MongoDB you are using? Note that "latest" is not a version.
Mongoose v6.1.6, Node v16.13.2,

[Uzlopak]

Can you provide an mvce please?

[a3957273]

EDIT: This code has some typos in it, check out the code two comments below for an example that runs.

import { model, Mongoose } from 'mongoose'

const ExampleSchema = newSchema({
  hidden: Boolean
})

async function shouldRemove(doc) {
  return doc.hidden
}

ExampleSchema.plugin((schema) => {
  schema.post('findOne', async function(doc) {
    if (await shouldRemove(doc)) return undefined
  
    return doc
  })
})

const ExampleModel = model('Example', ExampleSchema)

Mongoose.connect('<SOME_MONGOOSE_INSTANCE')

async function main() {
  await ExampleModel.create({ hidden: true })

  const empty = await ExampleModel.findOne({ hidden: true })
  // empty = undefined

  await ExampleModel.create({ hidden: false })
  
  const found = await ExampleModel.findOne({ hidden: false })
  // found = Doc<{ hidden: false }>
}

main()

[a3957273]

The example is a little contrived / simplified. Here we could add { hidden: false } to the database query in a schema.pre('findOne') section. This isn't possible in our real code due to our checks requiring complex JavaScript that cannot be converted to a database query.

[a3957273]

Hey @IslandRhythms, you added 'confirmed-bug', is this meant to be an existing feature? Also, here's some functional reproduction code:

import mongoose, { model, Schema } from 'mongoose'

const ExampleSchema = new Schema({
  hidden: Boolean
})

async function shouldRemove(doc) {
  return doc.hidden
}

ExampleSchema.plugin((schema) => {
  schema.post('findOne', async function(doc) {
    if (await shouldRemove(doc)) return undefined
  
    return doc
  })
})

const ExampleModel = model('Example', ExampleSchema)
mongoose.connect('mongodb://localhost:27017/bailo')

async function main() {
  await ExampleModel.create({ hidden: true })

  const empty = await ExampleModel.findOne({ hidden: true })
  console.log('empty', empty)
  // empty = undefined

  await ExampleModel.create({ hidden: false })
  
  const found = await ExampleModel.findOne({ hidden: false })
  console.log('found', found)
  // found = Doc<{ hidden: false }>
}

main()

The 'empty' log should be empty, but it isn't!

[a3957273]

Do you not see this issue as happening @vkarpov15? The latest script should clearly show a lack of ability to remove a model from the results of findOne / find.

[Uzlopak]

It is not a bug but a feature request.

[a3957273]

Ah, fair fair. Would you accept outside contributions? Could you point out what would need to be done to implement this?

[Uzlopak]

@a3957273

Sure. I am not an official maintainer and I kind of spam the project with PRs :).

I dont know where precisely it needs to be patched. But the hooks are managed by the npm package named "kareem". So I would look into the files where kareem is required.

[vkarpov15]

I'm sorry it took me so long to get to this. We'll think more about how best to support this, but we'll have to add support in a minor release.

First, worth noting: you can modify the result of find() in post hooks, you just need to modify the array in place using splice()

ExampleSchema.plugin((schema) => {
  schema.post('find', async function(docs) {
    for (let i = docs.length - 1; i >= 0; --i) {
      if (await shouldRemove(docs[i])) {
        docs.splice(i, 1);
      }
    }
  })
})

A few notes about couple of issues to work around for implementing this:

1. We can't support return undefined because that would be backwards breaking, no way to tell the difference between return undefined and no return statement. Have to do this instead:

ExampleSchema.plugin((schema) => {
  schema.post('findOne', async function(doc) {
    if (await shouldRemove(doc)) return null
  
    return doc
  })
})

2. Allowing returning arbitrary values from post hooks would be a bit messy for TypeScript integrations. There's a workaround for developers because we support overriding the return type using Model.findOne<CustomType>(), but there isn't a good way for us to infer the findOne() return type at compile time if we support returning arbitrary values.