Adds an option to locally fetch site website content

[agallou]

Some websites like https://www.arretsurimages.net are Single Page Applications behind a Paywall.
Those websites cannot be fetched by wallabag using the credentials API.

This PR adds an option where you can list the websites that you want to fetch locally

The tab's page DOM will be used as content and the tab's title will be used as title.

[Simounet]

Hi Adrien,
First of all, thanks for this useful feature. Could you rebase please? You should have some CS linting to do. Let me know if I can do anything to help you.

[agallou]

@Simounet i've just rebased and fixed most of the linter errors.
There is still two errors :

/sources/wallabagger/js/options.js
  145:45  error  document.body() is not supported in Firefox 52  compat/compat
  147:9   error  document.body() is not supported in Firefox 52  compat/compat

Do you know what's the best way to handle them ?

[Simounet]

Thanks a lot!

[Simounet]

@Simounet i've just rebased and fixed most of the linter errors. There is still two errors :

/sources/wallabagger/js/options.js
  145:45  error  document.body() is not supported in Firefox 52  compat/compat
  147:9   error  document.body() is not supported in Firefox 52  compat/compat

Do you know what's the best way to handle them ?

That's weird because I fixed it on the master branch with this commit: 32cb230

[agallou]

That's weird because I fixed it on the master branch with this commit: 32cb230

Sorry, I did not fetched the latest changes before rebasing. That's fixed.

I've fixed the typos/conventions errors, improved the filtering of the pages and used an object to avoid duplication of the function.

If all of that is ok for you I can then squash those changes.

[nicosomb]

The tab's page DOM will be used as content

Do you send the tab's page DOM to wallabag server? (to clean it)

[Simounet]

I think we should be done after that. Thanks again for your time.

[agallou]

@nicosomb yes, the innerHTML of the body is retrived here : https://github.com/wallabag/wallabagger/pull/300/files#diff-bfbd252c728bc45cc06ffc1d7f8eb7b9d5489941ad1c9183d8708245cc257e1eR624
then it is send to wallabag's API via the content parameter here : https://github.com/wallabag/wallabagger/pull/300/files#diff-45a26550cb91696cb8ce6bab9815b6ed166a4b3901d68c64354d79887a171501R201

[agallou]

@Simounet i've just applied changes about the function/this and variable naming, then rebased/squashed the branch.

[Simounet]

@agallou Everything looks good to me. Thanks a lot!
@nicosomb We are posting the content to /api/entries.json. Is it enough for wallabag to clean it?

[nicosomb]

I think it's enough.

If I understand, you only do this new thing for selected websites?

[agallou]

I think it's enough.

If I understand, you only do this new thing for selected websites?

Yes, its only done for the sites listed in the configuration. It's usefull if the site is and SPA and/or if the site is behind a paywall that is not managed/can't be managed with wallabag's site credentials.

[Simounet]

I added a commit to bypass the cache for your local fetch and updated the translations. It's now on the master branch. Thanks a lot for this feature!

[nicosomb]

But why this feature is not enabled for all the websites?

[Simounet]

It could be but at the moment, submitting an entry to wallabag through the right click context menu on a link action is not supported. You have to open it on a new tab before.