New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to fetch associations #382

Closed
MarcGodard opened this Issue Aug 7, 2016 · 29 comments

Comments

Projects
None yet
@MarcGodard

MarcGodard commented Aug 7, 2016

So I have developed an app with the feathers framework. I have a hook attached to a database service that I want to access data from another service. Is this possible, and easy? I want that service to be able to access several neDB files. How do I do this?

'use strict';

module.exports = function(options) {
  return function(hook) {

    // access other db files here and get data

    hook.data = {
        putDataHere: newData
    };
  };
};
@daffl

This comment has been minimized.

Member

daffl commented Aug 7, 2016

Yes. Have a look at the example how to fetch related items. Basically, in your hook you can access another service via hook.app.service('servicename') and call the method you need. For the hook to wait you can return a Promise (that resolves with the hook object) which conveniently is what all the service methods already do:

module.exports = function(options) {
  return function(hook) {
    return hook.app.service('otherservice').find({
      query: { name: 'David' }
    }).then(page => {
      const davids = page.data;

      hook.data.davids = davids;
      // Or you can replace all the data with
      hook.data = { davids };

      // IMPORTANT: always return the `hook` object in the end
      return hook;
    });
  };
};
@MarcGodard

This comment has been minimized.

MarcGodard commented Aug 8, 2016

@daffl Thank you very much. I figured it would be this easy, but couldn't find it in the documentation, I must have missed it.

@MarcGodard MarcGodard closed this Aug 8, 2016

@MarcGodard

This comment has been minimized.

MarcGodard commented Aug 8, 2016

Ok so this is a find call that I have this before hook (just if you need that info)

I am only getting 25 but there are hundreds. If I remove the limit, I only get 5 items.

What is going on and how do I get all?

module.exports = function(options) {
  return function(hook) {

    const dataSet = hook.params.query.dataSet;
    const userVector = hook.params.query.userVector;

    return hook.app.service('data').find({
      query: { dataSet: dataSet, $limit:2000 }
    }).then(page => {

      // lots of magic here

      //console.log(page.data);

      hook.result = page.data;

      return hook;
    });
  };
};

@MarcGodard MarcGodard reopened this Aug 8, 2016

@daffl

This comment has been minimized.

Member

daffl commented Aug 8, 2016

It depends on what option you set for paginate.max in the configuration. It's a safeguard so that someone can't just request millions of records remotely. In all newer adapters you can turn off pagination to get all record like this:

return hook.app.service('data').find({
      paginate: false,
      query: { dataSet: dataSet }
    }).then(data => {
      // data is an array
    });
@MarcGodard

This comment has been minimized.

MarcGodard commented Aug 8, 2016

When I do that I get 0 records. When I remove the pagination: false I get 25.

@daffl

This comment has been minimized.

Member

daffl commented Aug 8, 2016

Does npm ls feathers-nedb show v2.4.1? If not you may have to update your package.json accordingly.

@MarcGodard

This comment has been minimized.

MarcGodard commented Aug 8, 2016

└── feathers-nedb@2.4.1
@daffl

This comment has been minimized.

Member

daffl commented Aug 8, 2016

I just verified that paginate: false works with the following example:

const feathers = require('feathers');
const rest = require('feathers-rest');
const socketio = require('feathers-socketio');
const hooks = require('feathers-hooks');
const nedb = require('feathers-nedb');
const bodyParser = require('body-parser');
const handler = require('feathers-errors/handler');
const NeDB = require('nedb');

// A Feathers app is the same as an Express app
const app = feathers();
const db = new NeDB({
  filename: './data/messages.db',
  autoload: true
});

// Parse HTTP JSON bodies
app.use(bodyParser.json());
// Parse URL-encoded params
app.use(bodyParser.urlencoded({ extended: true }));
// Register hooks module
app.configure(hooks());
// Add REST API support
app.configure(rest());
// Configure Socket.io real-time APIs
app.configure(socketio());
// Register our memory "users" service
app.use('/todos', nedb({
  Model: db,
  paginate: {
    default: 20,
    max: 50
  }
}));
// Register a nicer error handler than the default Express one
app.use(handler());

const promises = [];

for(let i = 0; i < 700; i++) {
  promises.push(app.service('todos').create({ text: `Item #${i}`}));
}

Promise.all(promises).then(function() {
  app.service('todos').find({
    paginate: false,
    query: {}
  }).then(data => console.log(data));
});

// Start the server
app.listen(3333);

I am getting 700 items logged to the console. Did you accidentally add paginate: false into the query instead of the main parameters?

@MarcGodard

This comment has been minimized.

MarcGodard commented Aug 8, 2016

This is what I did:

'use strict';

module.exports = function(options) {
  return function(hook) {

    const dataSet = hook.params.query.dataSet;
    const userVector = hook.params.query.userVector;

    return hook.app.service('data').find({
      paginate: false,
      query: { dataSet: dataSet, $limit:2000 }
    }).then(page => {

      // lots of magic here
      console.log(page.data);

      hook.result = page.data;

      return hook;
    });
  };
};

Still get 0 results and 25 with pagination variable not there and 5 results without limit.

However, I got it working with this:

'use strict';

module.exports = function(options) {
  return function(hook) {

    const dataSet = hook.params.query.dataSet;
    const userVector = hook.params.query.userVector;

    return hook.app.service('data').find({
      paginate: false,
      query: { dataSet: dataSet }
    }).then(page => {

      // lots of magic here
      console.log(page);

      hook.result = page;

      return hook;
    });
  };
};

As you see the results are no longer in the page.data but is just in the page variable.

Lesson learned :)

@MarcGodard MarcGodard closed this Aug 8, 2016

@MarcGodard

This comment has been minimized.

MarcGodard commented Aug 8, 2016

Oh and thanks @daffl for your help.

@MarcGodard

This comment has been minimized.

MarcGodard commented Aug 8, 2016

On the client side it doesn't work. (the paginate: false, variable).

@daffl

This comment has been minimized.

Member

daffl commented Aug 8, 2016

Only query is passed between the client and the server. I'd try and avoid letting a client request all entires (if your database has a million records it will kill both, the server and the client). If there is no way around it you have to map a special query parameter (e.g. $paginate) from params.query into params in a hook on the server:

app.service('data').before({
  find(hook) {
    if(hook.params.query && hook.params.query.$paginate) {
      hook.params.paginate = hook.params.query.$paginate === 'false' || hook.params.query.$paginate === false;
      delete hook.params.query.$paginate;
    }
  }
});
@MarcGodard

This comment has been minimized.

MarcGodard commented Aug 8, 2016

I understand the warning of not allowing pagination on the client side, however, the tool I am making is just for me to run locally. Thanks again for everything.

@arkokoley

This comment has been minimized.

arkokoley commented Aug 28, 2016

@daffl Hi, I'm using the code you have posted to retrieve posts a user has.

module.exports = function(options) {
  return function(hook) {
    // hook.getPosts = true;
    const id = hook.result.id;
    console.log("id");

    return hook.app.service('posts').find({
      paginate: false,
      query: { postedBy: id }
    }).then(function(posts){
      console.log("posts");
      // Set the posts on the user property
      hook.result.posts = posts.data;
      // Always return the hook object or `undefined`
      return hook;
    });
  };
};

But this causes the server to go in an infinite loop and the node app ultimately crashes.

I am using MySQL as my DB connected by sequelize.

What am I doing wrong?

Update

I had setup a similar hook for posts to populate the user field based on the postedBy field. So it seems, the user hook would trigger the post hook which in turn would trigger the original user hook and the loop continues resulting in an infinite loop and memory overflow.

Any ideas how to populate only a shallow related item, i.e, the hook would pull only the related items on only the first level.

@fortunes-technology

This comment has been minimized.

fortunes-technology commented Mar 21, 2017

@daffl Your idea is awesome, though the code didn't work.
I had to change hook.paginate to hook.params.paginate to make it work.
And I made a little change so you can send whatever you want there.
so you can $paginate : {value: false} to disable pagination or
$paginate: { value: {default: 100, max: 2000}} to override pagination

app.service('data').before({ find(hook) { if(hook.params.query.$paginate) { hook.params.paginate = hook.params.query.$paginate.value; delete hook.params.query.$paginate; } } });

One more thing to consider is, when pagination disabled, the data is not in res.data but res itself.

@daffl

This comment has been minimized.

Member

daffl commented Mar 21, 2017

@fortunes-technology You are right. I updated my code snippet. This should work:

app.service('data').before({
  find(hook) {
    if(hook.params.query && hook.params.query.$paginate) {
      hook.params.paginate = hook.params.query.$paginate === 'false' || hook.params.query.$paginate === false;
      delete hook.params.query.$paginate;
    }
  }
});
@nathandial

This comment has been minimized.

nathandial commented Sep 12, 2017

I think if $paginate is set to false (which is the point of this hook, to disable it... ), the if is going to be false, too. Maybe

app.service('data').before({
  find(hook) {
    if(hook.params.query && hook.params.query.hasOwnProperty('$paginate')) {
      hook.params.paginate = hook.params.query.$paginate === 'false' || hook.params.query.$paginate === false;
      delete hook.params.query.$paginate;
    }
  }
});
@Alendorff

This comment has been minimized.

Alendorff commented Sep 19, 2017

Hi there. Could someone tell me if feathers services have findOne functionality or how to implement it? Thanks!
cc @daffl

@daffl

This comment has been minimized.

Member

daffl commented Sep 20, 2017

A findOne is just a find with a $limit of 1:

app.service('myservice').find({ query: { name: 'test', $limit: 1 } })
 .then(page => page.data[0])
 .then(result => console.log('Got', result);
@Alendorff

This comment has been minimized.

Alendorff commented Sep 20, 2017

@daffl I asked because mongoose has findOne and I suggested that it doesn't search over all rest collection if something already found. So, I thought about findOne as something smarter then search with limit=1... That's why I asked.

tuxrace added a commit to tuxrace/feathers-docs that referenced this issue Oct 11, 2017

Proposal to bring back the access another service from inside a hook
A common scenario is we need to access another service from inside the hook, I propose to revive a legacy documentation found on this issue feathersjs/feathers#382
@roelvan

This comment has been minimized.

roelvan commented Jan 18, 2018

I usually do this for a quick findOne:

const employee = (await context.app.service('employees').find({ query: { userId: user.id }, paginate: false }))[0];
@fridays

This comment has been minimized.

fridays commented Jan 21, 2018

I just made a plugin for a .findOne() method: feathers-findone

@joakimstrandell

This comment has been minimized.

joakimstrandell commented Jul 9, 2018

I couldn't find any example on how to add the $paginate workaround for the client. So I created a before hook that runs from the app.hooks.js file with some modifications:

module.exports = function () {
  return context => {
    if (context.params.query && hook.params.query.hasOwnProperty('$paginate')) {
      context.params.paginate = context.params.query.$paginate === 'false' || context.params.query.$paginate === false;
      delete context.params.query.$paginate;
    }
  };
};

bertho-zero added a commit to bertho-zero/feathers that referenced this issue Aug 23, 2018

Use new url parser (feathersjs#382)
Currently it issues the following warning:
(node:6686) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.

daffl added a commit that referenced this issue Aug 25, 2018

Use new url parser (#382)
Currently it issues the following warning:
(node:6686) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.

@daffl daffl changed the title from Noob question I am sure to Fetching associations Sep 18, 2018

@daffl daffl changed the title from Fetching associations to How to fetch associations Sep 18, 2018

@dtwilliams

This comment has been minimized.

dtwilliams commented Nov 20, 2018

How would you go about writing a unit or integration test for this?

@daffl

This comment has been minimized.

Member

daffl commented Nov 20, 2018

@dtwilliams

This comment has been minimized.

dtwilliams commented Nov 20, 2018

so if i have the following in my hook

const records = getItems(context);
records.authors = await context.app.service('users') .find({ query: { _id: { $in: records.authorIds } } }, context.params);

how do i "mock" the context.app.service in my unit test?

do i need to add it to my contextBefore object?

contextBefore = { type: 'before', params: { provider: 'socketio', user: { _id: 1 } }, data: { _id: 1, }, };

i'm using the unit-test auto generated from the feathers-plus generator.
https://generator.feathers-plus.com/get-started/#unit-test-for-a-hook

or should i be using an integration test instead?

@eddyystop

This comment has been minimized.

Member

eddyystop commented Nov 25, 2018

I have gone the mocking route some time ago. I think its a very bad idea. You will eventually have to mock more and more of Feathers. There is no guarentee your mocks function as feathers do and you can get false negatives, as I did.

Instanciating a Feathers app is very fast, so use Feathers instead of mocks. That's how the appraoch the cli+ test take.

I wrote an article that refers to this. https://medium.com/feathers-plus/automatic-tests-with-feathers-plus-cli-4844721a29cf

@dtwilliams

This comment has been minimized.

dtwilliams commented Nov 26, 2018

thanks eddyy, i've seen that, great article, really helped.

i was struggling on how to add the app to my context object but working it out eventually, i think!

    const app = feathers();

    contextBefore = {
      type: 'before',
      params: { provider: 'socketio', user: { _id: 1 } },
      data: {
        _id: 1,
      },
      app,
    };

Then i had to amend the test so it used async.

it('patch test', async () => {
    contextBefore.app.use('users', {
      async find() {
        return { data: [ { expectedResultObj1 }, { expectedResultObj2 } ] };
      }
    });
    contextBefore.data = {
      _id: 1,
    };
    assert(true);

    await hookToTest()(contextBefore);

    assert.deepEqual(contextBefore.data, {
      _id: 1,
      expectedResultObject1,
      expectedResultObject2,
    });
  });

Is that the right way to do it or is there a better way?

@eddyystop

This comment has been minimized.

Member

eddyystop commented Nov 26, 2018

Looks good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment