Running the example doesn't work

[joshghent]

Hi there,

The example found in the readme does not work when running it.
Here is the output when running it. Perhaps I'm being dumb but this is the result of running node app.js

throw new Err("There is no element selected for the '<option.name>' field. Please provide a selector, list item or use nested object structure.", {
                ^

Error: There is no element selected for the 'data' field. Please provide a selector, list item or use nested object structure.
    at Error (native)
    at new Err (/Users/josh/Projects/daystext/node_modules/err/lib/index.js:57:81)
    at /Users/josh/Projects/daystext/node_modules/scrape-it/lib/index.js:128:23
    at iterateObject (/Users/josh/Projects/daystext/node_modules/iterate-object/lib/index.js:25:17)
    at handleDataObj (/Users/josh/Projects/daystext/node_modules/scrape-it/lib/index.js:122:9)
    at Function.scrapeIt.scrapeHTML (/Users/josh/Projects/daystext/node_modules/scrape-it/lib/index.js:177:12)
    at /Users/josh/Projects/daystext/node_modules/scrape-it/lib/index.js:29:27
    at /Users/josh/Projects/daystext/node_modules/cheerio-req/lib/index.js:24:9
    at opt_callback (/Users/josh/Projects/daystext/node_modules/tinyreq/lib/index.js:57:9)
    at IncomingMessage.<anonymous> (/Users/josh/Projects/daystext/node_modules/tinyreq/lib/index.js:89:13)

[IonicaBizau]

Umm, I'm on it. Testing.

[IonicaBizau]

Cannot reproduce it. I npm installed and ran node example and it's working for me.

Can you post the output of node -v and tell me how you did it?

[IonicaBizau]

Closing due to no response.

[freshyill]

I get the same thing with both 6.9.2 and 7.2.1. All I'm doing is using the readme example. Even simplifying it results in the same thing.

[IonicaBizau]

@freshyill Hmm... Weird. If you do a curl -L ionicabizau.net do you get a bunch of HTML output in the terminal?

[freshyill]

Yep, I get the expected result with curl. Seems to work just fine.

[IonicaBizau]

What about running the tests? Are they passing?

[polo2ro]

Hi,
i began using scrap-it as a dependency, this work but i experienced problems with selectors. So i tried to clone the repository and got an error with npm install:

npm ERR! git clone git@github.com:github:chalk/ansi-styles Clonage dans le dépôt nu '/home/paul/.npm/_git-remotes/git-github-com-github-chalk-ansi-styles-0915ce26'
npm ERR! git clone git@github.com:github:chalk/ansi-styles Permission denied (publickey).
npm ERR! git clone git@github.com:github:chalk/ansi-styles fatal: Impossilble de lire le dépôt distant.
npm ERR! git clone git@github.com:github:chalk/ansi-styles
npm ERR! git clone git@github.com:github:chalk/ansi-styles Veuillez vérifier que vous avez les droits d'accès
npm ERR! git clone git@github.com:github:chalk/ansi-styles et que le dépôt existe.
npm ERR! notarget No compatible version found: ansi-styles@'github:chalk/ansi-styles#5dadc5c6333127f95dc0dc09ada4fbde8bf66e69'
npm ERR! notarget Valid install targets:
npm ERR! notarget ["0.1.0","0.1.1","0.1.2","0.2.0","1.0.0","1.1.0","2.0.0","2.0.1","2.1.0","2.2.1","3.0.0"]
npm ERR! notarget
npm ERR! notarget This is most likely not a problem with npm itself.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.

npm ERR! System Linux 4.9.0-2-amd64
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "install"
npm ERR! cwd /home/paul/git/scrape-it
npm ERR! node -v v4.7.2
npm ERR! npm -v 1.4.21
npm ERR! code ETARGET

[polo2ro]

A google search lead me to this issue:
vercel/vercel#65

this work:
npm install --production

here are the lines in npm-debug.log:

3119 verbose about to build /home/paul/git/scrape-it/node_modules/tester
3120 verbose node_modules/tester unbuild
3121 info preuninstall tester@1.3.7
3122 info uninstall tester@1.3.7
3123 verbose true,/home/paul/git/scrape-it/node_modules,/home/paul/git/scrape-it/node_modules unbuild tester@1.3.7
3124 info postuninstall tester@1.3.7
3125 error notarget No compatible version found: ansi-styles@'github:chalk/ansi-styles#5dadc5c6333127f95dc0dc09ada4fbde8bf66e69'

[mirkojotic]

Hi,

Just tried this package today and got similar issue. Error I'm getting is:
Error: There is no element selected for the 'data' field. Please provide a selector, list item or use nested object structure.

Even with first and most basic example on github.
node -v 7.4.0
npm -v 4.0.5

scrapeIt(url, { listItem: '.articles', // this retrieves articles data: { // but this throws an error selector: '.title' }, cb)

[IonicaBizau]

@mirkojotic Do the tests pass? I'm not sure.

[powlo]

I think the issue is that the syntax has changed for the {opts} parameter and the example posted here does not work.

This works:

// Fetch the articles on the page (list)
scrapeIt("http://ionicabizau.net", {
    articles : {
        listItem: ".article"
      , data: {
            createdAt: {
                selector: ".date"
              , convert: x => new Date(x)
            }
          , title: "a.article-title"
          , tags: {
                selector: ".tags"
              , convert: x => x.split("|").map(c => c.trim()).slice(1)
            }
          , content: {
                selector: ".article-content"
              , how: "html"
            }
        }
    }
}, (err, page) => {
    console.log(err || page);
});

This does not work (taken from blog post):

// Fetch the articles on the page (list)
scrapeIt("http://ionicabizau.net", {
    listItem: ".article"
  , name: "articles"
  , data: {
        createdAt: {
            selector: ".date"
          , convert: x => new Date(x)
        }
      , title: "a.article-title"
      , tags: {
            selector: ".tags"
          , convert: x => x.split("|").map(c => c.trim()).slice(1)
        }
      , content: {
            selector: ".article-content"
          , how: "html"
        }
    }
}, (err, page) => {
    console.log(err || page);
});

Note, the change in structure for {opts}; each key is now the "name" of the scraped list.

[IonicaBizau]

@powlo Phew! 🙈 Good point! Updated the example in that post too!