Add custom resource saver #201
Conversation
README.md
Outdated
| Scrape function resolves with array of [Resource](https://github.com/s0ph1e/node-website-scraper/blob/master/lib/resource.js) objects which contain `metadata` property from `httpResponseHandler`. | ||
|
|
||
| #### resourceStorage | ||
| Class which saves [Resources](https://github.com/s0ph1e/node-website-scraper/blob/master/lib/resource.js), should have methods `saveResource` and `removeSavedResources` which return Promises. Use it to save files where you need: to dropbox, amazon S3, existing directory, etc. By default all files are saved in local file system to new directory passed in `directory` option (see [lib/resource-storage.js](https://github.com/s0ph1e/node-website-scraper/blob/master/lib/resource-storage.js)). |
There was a problem hiding this comment.
Use relative links instead of absoulte
There was a problem hiding this comment.
Relative links will work on github but not on npm site
I use relative links only to anchors in readme file, but for other files - absolute
README.md
Outdated
| directory: '/path/to/save', | ||
| resourceStorage: class MyResourceStorage { | ||
| saveResource (resource) {/* code to save file where you need */} | ||
| removeSavedResources() {/* code to remove all previously saved files in case of error */} |
There was a problem hiding this comment.
It's not clear what remove all previously saved files in case of error means.
What the error? Why we have to remove files?
removeSavedResources no any mentions about error. Is it used in any other cases?
There was a problem hiding this comment.
If error happens and scraper can't continue downloading - it removes previously saved files (used here). The reason is to have completely downloaded site or nothing (I don't want to leave partially downloaded site in the system)
No description provided.