Description of running examples is confusing

[mjgs]

Looks like a great library, however I couldn't get either of the methods to import the example data to work. Am I doing something incorrectly?

/tmp $ git clone https://github.com/pkosiec/mongo-seeding
Cloning into 'mongo-seeding'...
remote: Counting objects: 838, done.
remote: Compressing objects: 100% (92/92), done.
remote: Total 838 (delta 63), reused 73 (delta 34), pack-reused 708
Receiving objects: 100% (838/838), 556.96 KiB | 87.00 KiB/s, done.
Resolving deltas: 100% (485/485), done.
/tmp $ cd mongo-seeding-test
/tmp/mongo-seeding-test $ cat package.json index.js
{
  "name": "mongo-seeding-test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "mongo-seeding": "^2.2.0"
  }
}
const { seedDatabase } = require('mongo-seeding');

const path = require('path');

const config = {
  database: {
    host: '127.0.0.1',
    port: 27017,
    name: 'mydatabase',
  },
  inputPath: path.resolve(__dirname, '../mongo-seeding/samples/example/data'),
  dropDatabase: true,
};

(async () => {
  try {
    await seedDatabase(config);
  } catch (err) {
    // Handle errors
    console.error('there was an error:');
    console.error(err);
  }
  // Do whatever you want after successful import
})()
/tmp/mongo-seeding-test $ node index.js
there was an error:
{ MongoSeedingError: Error: Cannot find module 'mongodb'
    at wrapError (/private/tmp/mongo-seeding-test/node_modules/mongo-seeding/dist/index.js:52:19)
    at Object.<anonymous> (/private/tmp/mongo-seeding-test/node_modules/mongo-seeding/dist/index.js:44:15)
    at Generator.next (<anonymous>)
    at /private/tmp/mongo-seeding-test/node_modules/mongo-seeding/dist/index.js:7:71
    at new Promise (<anonymous>)
    at __awaiter (/private/tmp/mongo-seeding-test/node_modules/mongo-seeding/dist/index.js:3:12)
    at exports.seedDatabase (/private/tmp/mongo-seeding-test/node_modules/mongo-seeding/dist/index.js:14:43)
    at __dirname (/private/tmp/mongo-seeding-test/index.js:17:11)
    at Object.<anonymous> (/private/tmp/mongo-seeding-test/index.js:24:3)
    at Module._compile (module.js:660:30) name: 'MongoSeedingError' }
/tmp/mongo-seeding-test $ cd ..
/tmp $ cd mongo-seeding
/tmp/mongo-seeding $ cd samples/example/data/
/tmp/mongo-seeding/samples/example/data $ seed -u 'mongodb://127.0.0.1:27017/mydb' -d .
  mongo-seeding Starting... +0ms
  mongo-seeding Closing connection... +5ms
Error MongoSeedingError: Error: Cannot find module 'mongodb'

[pkosiec]

Hello, the example directory contains helper functions, which requires module "mongo b". Just run 'npm install' in 'sample' directory :-) Tell mi if it works. Thanks!

[mjgs]

Yes both methods work after npm install (though there is a mongodb warning):

/tmp/mongo-seeding/samples/example/data $ seed -u 'mongodb://127.0.0.1:27017/mydb' -d .
  mongo-seeding Starting... +0ms
  mongo-seeding Connecting to mongodb://127.0.0.1:27017/mydb... +517ms
(node:4005) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.
  mongo-seeding Connection with database established. +23ms
  mongo-seeding Inserting 6 documents into collection categories... +1ms
  mongo-seeding Inserting 2 documents into collection posts... +69ms
  mongo-seeding Closing connection... +50ms
  mongo-seeding Finishing... +1ms

/tmp/mongo-seeding-test $ node index.js
(node:3856) DeprecationWarning: current URL string parser is deprecated, and will be removed in a future version. To use the new parser, pass option { useNewUrlParser: true } to MongoClient.connect.

Results in two databases being created with the data:

• mydatabase
• mydb

I like the library, though I will say I found the setup a big confusing. There are 4 package.json files in the project (which ones need to be installed, some of them, all of them?), I had to clone the project and create a separate test project to test the javascript method (an included sample project might be a better way to go). How the helper functions work isn't all that clear to me after reading the docs. Feels like there could be some simplifications/docs improvements.

Having said that, it's a neat library, mongodb data seeding is a pain point I've struggled with for a while.

[pkosiec]

Thanks for reporting, I've created a separate issue for depreciation warning.

Currently I'm on vacation without access to a computer, but 12 days later I will investigate the problem with creating two databases. Can anyone else confirm this?

Normally you don't need to clone the repository at all. CLI and JS library for using is available in npm, and the Docker Image is in Docker hub registry. So you need to clone the repository if you want to try out the examples.

All JS files for data import is pure Javascript - so you can do whatever you want, but you need to export an object of array of objects. If this is pure JS, it means that you can also import some dependencies in these files. Basically that's how the helper functions work - they are just dependencies stored in other directories (not evaluated for collections import).

Does the answer satisfy you? How would you simplify the docs? Any idea will be very helpful. Thank you!

[mjgs]

I don’t think that having two databases created is necessarily an issue, in fact having separate dbs created is sort of useful if you are trying each example import method one after the other. Is there a way to have the example data included in the files that get downloaded on npm install? I will have a think about doc ideas over the next few days.

…

On Sat, 1 Sep 2018 at 16:31, Paweł Kosiec ***@***.***> wrote: Thanks for reporting, I've created a separate issue for depreciation warning. Currently I'm on vacations, but 12 days later I will investigate the problem with creating two databases. Can anyone else confirm this? Normally you don't need to clone the repository at all. CLI and JS library for using is available in npm, and the Docker Image is in Docker hub registry. So you need to clone the repository if you want to try out the examples. All JS files for data import is pure Javascript - so you can do whatever you want, but you need to export an object of array of objects. If this is pure JS, it means that you can also import some dependencies in these files. Basically that's how the helper functions work - they are just dependencies stored in other directories (not evaluated for collections import). Does the answer satisfy you? How would you simplify the docs? Any idea will be very helpful. Thank you! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB07z-7yOtWMU2ItqGtIFKUY51pwNfOyks5uWlPvgaJpZM4WVHAk> .

[pkosiec]

Hmm, but wait - you probably configured a different name for MongoDB database in your test app. Is that right? So it isn't the issue of the library.

No, sorry, but in npm packages there shouldn't be other files that essential ones. So examples don't fit there.

[mjgs]

I copied and pasted the code from the core page installation instructions. It’s just a suggestion, from a library user perspective it would be nice to only have to npm install once and have all the necessary parts to seed data but also to run the examples. It felt like I had to do too many things to install and run the examples, and kind of confusing.

…

On Sun, 2 Sep 2018 at 12:10, Paweł Kosiec ***@***.***> wrote: Hmm, but wait - you probably configured a different name for MongoDB database in your test app. Is that right? No, sorry, but in npm packages there shouldn't be other files that essential ones. So examples don't fit there. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#19 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB07z5mBfYp8xvo1m85PAFNXX887kTs5ks5uW2hKgaJpZM4WVHAk> .

[pkosiec]

Oh, okay - so this is not an issue, you have different database names in your sample program and CLI config. So two databases have been created because you ran data imports twice with different configuration. Everything is fine then 🙂

Hmm, so you're saying that the samples should be included in CLI installation, right? Do you know other tools or libraries which provide examples with binaries?

[pkosiec]

Okay, I've just checked the docs for samples and there is nothing about running npm install in examples directory... Sorry for that! Once I return i will try to rewrite precisely the instructions for sample data import.
I'm still not convinced about putting samples in npm package(s) though. 😛 But I'm open for discussion of course.

[mjgs]

You have the output of all the commands I ran in the thread. The only thing I didn't include was that I installed the cli as the initial step.

• installed the cli
• cloned the repo - to get the examples
• created a test project - to test the javascript seed method
• ran npm install in the cloned repo examples directory

Did you expect me to do a different sequence of steps? Seems like there could be less steps just to take a test drive.

I'm not all that familiar with how you have structured your app, sounds like you have some shared code between the cli and the main app. Is the cli a separate repo?

As far as the binaries go I suppose you are referring to the docker images...I didn't go into the docker seeding method.

[pkosiec]

Hey @mjgs, sorry for a delay! I just wanted to reply you when I have access to my computer.

Did you expect me to do a different sequence of steps?

No, the sequence is fine, I just didn't get that correctly (reading it on mobile phone) and I assumed that after one single run of seeding you got 2 created databases. That would be a serious bug, but turns out that everything is OK 🙂

Seems like there could be less steps just to take a test drive.

It's not possible to reduce steps of running examples that much - cloning repository is a must have there.
Every NPM package should contain only essential files for them. Minimizing size of a library is very, very important - that's why usually the code is minified (without whitespaces and other unnecessary characters etc.). That's why samples must not be included in NPM package source.

But I will create a sample JS file that uses mongo-seeding library, just to be able to run it pretty easily, just after cloning repository.

I'm not all that familiar with how you have structured your app, sounds like you have some shared code between the cli and the main app. Is the cli a separate repo?

This repository contains code of CLI, library and Docker image. Docker image and CLI depends on core, so the JS library.

As far as the binaries go I suppose you are referring to the docker images...I didn't go into the docker seeding method.

I was referring to essential files in other NPM packages. Because I haven't encounter any case where examples were included in a package. But never mind - let's summary what has to be done 🙂

---

Summary

Here is a pretty long discussion - I would like to close this issue and create another one like Improve examples with the following tasks precised:

• Write instruction how to run samples (with all details like cloning repository etc.)
• Add example with basic app utilizing mongo-seeding library.
• Rename samples directory to examples (as it will contain example JS app as well)

What do you think? Will it satisfy you? Would you like to have another point in this list?

[mjgs]

Looks good, I can't think of anything else to add. I'll read and run through the instructions once they are updated.

[pkosiec]

Great. Thank you very much for your feedback and the discussion!
I'm closing this issue, but if you have any other comments, feel free to reopen it. I will ping you when the PR will be ready to review :-)

[pkosiec]

Hi again @mjgs! I've just finished work on improving examples in pull request #37. Could you please take a look and give some feedback on it? Thank you very much!