Export data from the evaluatePage() serialized data

[maticrivo]

Can the exporter receive fields from the evaluatePage serialized data?

[yujiosaka]

@maticrivo
Here you go!

const HCCrawler = require('headless-chrome-crawler');
const CSVExporter = require('headless-chrome-crawler/exporter/csv');

const FILE = './tmp/result.csv';

const exporter = new CSVExporter({
  file: FILE,
  fields: ['response.url', 'result.title'],
});

HCCrawler.launch({
  evaluatePage: (() => {
    return { title: window.document.title }
  }),
  maxDepth: 2,
  exporter,
})
  .then(crawler => {
    crawler.queue('https://example.com/');
    crawler.onIdle()
      .then(() => crawler.close());
  });

result.xxx is what you are looking for.

[maticrivo]

Thanks, for some reason it didn't work for me before, probably something wrong i was doing in the evaluatePage callback..
works now.