New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send Webmentions #535

Closed
wants to merge 6 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@NinjaTrappeur

NinjaTrappeur commented Aug 22, 2018

Hi!

This PR implements the feature discussed here.

The general idea was to be as little invasive a possible.

Changelog

I made some changes to the initial plan.

I ended up not storing the webmentions endpoint in the database: we won't have the webmention endpoint synchronously as we need to download and parse the link page. We also can't put this webmention discovery in the fetchAttributes function as I first intended to do: there's no way to retrieve the content downloaded by this function when saving the story.

So instead, everything is done in a cron-triggered task. I think this is a better design, everything stays compartmentalized.

The task:

  • For any new story.
    • Download the web-page using Sponge.
    • Parse the web-page and look for a web-mention endpoint. The regexes used here are extracted from the official webmention-client gem. I did not directly used this gem: it was impossible to use the parser without the whole client.
    • Send a webmention to this endpoint (if any).

I also added some markup necessary to make the repost parseable by the webmention endpoint.

The markup has been tested against the official validator. The parser has been manually tested against the webmentions.rocks examples. The post method has been manually tested.

Potential Improvement

  • We could markup the comments, it would allow the mentioned websites to retrieve them.
  • We could send a new webmention on every comment, it would let know the mentioned websites it needs to parse again the comment page in order to retrieve the new comments.

I tried to markup the comments but did not find an immediate way to do so. The current template structure makes this tricky.

Fixes #383

I plan to maintain the changes introduced by this PR.

We'll also need to add this cron task to the ansible playbook. Should I open another PR there?

[Edit] Looks like the linter is angry. I'm fixing it.

NinjaTrappeur added some commits Aug 9, 2018

Add some microformat tags to stories.
This commit is part of the webmentions support. These tags help the website
receiving the webmention parsing the mention. Only a minimal amount of
informations have been tagged: I did not wanted to disrupt a lot of views.

We still could extend the h-entry by including the stories comment in them. It
would however spread this whole tagging across several views instead of being
contained in a single one.
Send webmentions using specific script.
We add a cron triggered script which send webmention for each new story submitted.
@pushcx

This really isn't very much code, that's great. I like how well you've contained it, especially the way you've broken out the parsing utility functions from the main logic. I've got some small tweaks, but this is overall solid and I look forward to merging it soon.

Show resolved Hide resolved app/views/stories/_listdetail.html.erb
Show resolved Hide resolved extras/sponge.rb
Show resolved Hide resolved script/send_new_webmentions
Show resolved Hide resolved script/send_new_webmentions
Show resolved Hide resolved script/send_new_webmentions
Show resolved Hide resolved script/send_new_webmentions
Show resolved Hide resolved script/send_new_webmentions
Show resolved Hide resolved script/send_new_webmentions
Show resolved Hide resolved script/send_new_webmentions

NinjaTrappeur added some commits Sep 3, 2018

Refactor sponge.fetch to also return http headers
Sponge.fetch will now return the http headers as well as the response
body. The existing codebase has been refactored according to this
change.
@NinjaTrappeur

This comment has been minimized.

Show comment
Hide comment
@NinjaTrappeur

NinjaTrappeur Sep 8, 2018

Alright, it should be good now.

Could you please review the sponge calls change, this is the only "dangerous" part of this PR, I really don't want to screw this up.

Other than that, Travis still fails on the "next unless" pattern. I guess we don't really care.

Sorry for the delay :)

NinjaTrappeur commented Sep 8, 2018

Alright, it should be good now.

Could you please review the sponge calls change, this is the only "dangerous" part of this PR, I really don't want to screw this up.

Other than that, Travis still fails on the "next unless" pattern. I guess we don't really care.

Sorry for the delay :)

pushcx added a commit that referenced this pull request Oct 3, 2018

@pushcx

This comment has been minimized.

Show comment
Hide comment
@pushcx

pushcx Oct 3, 2018

Member

GitHub's confused because I made some style tweaks and squash-merged this manually.

Thanks for doing this, @NinjaTrappeur. I'm deploying it now. Hopefully it'll help connect us to the wider web.

Member

pushcx commented Oct 3, 2018

GitHub's confused because I made some style tweaks and squash-merged this manually.

Thanks for doing this, @NinjaTrappeur. I'm deploying it now. Hopefully it'll help connect us to the wider web.

@pushcx pushcx closed this Oct 3, 2018

@pushcx pushcx referenced this pull request Oct 3, 2018

Closed

Feature: Send Webmentions #383

pushcx added a commit that referenced this pull request Oct 3, 2018

pushcx added a commit that referenced this pull request Oct 3, 2018

pushcx added a commit that referenced this pull request Oct 3, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment