Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Narrow webmention discovery #423

Closed
snarfed opened this issue Feb 14, 2023 · 5 comments
Closed

Narrow webmention discovery #423

snarfed opened this issue Feb 14, 2023 · 5 comments

Comments

@snarfed
Copy link
Owner

snarfed commented Feb 14, 2023

Right now we run webmention discovery liberally on incoming activities' "targets," eg inReplyTo, mentions, etc. In practice, these are almost all fediverse/Mastodon URLs that don't accept webmentions. We should probably do one or both of:

@snarfed
Copy link
Owner Author

snarfed commented Mar 8, 2023

Simpler start here is just to cache wm discovery. Ideally we'd move Bridgy's existing caching into webutil.webmention, opt in somehow, maybe via kwarg.

@snarfed
Copy link
Owner Author

snarfed commented Mar 8, 2023

As for a blocklist of all existing AP instances, fediverse.observer and the-federation.info both have APIs over their node lists, easy to export all the domains. They have 46k and 41k respectively, 36k in common, 13k in just one or the other.

https://api.fediverse.observer/ :

query MyQuery {
  thefederation_node {
    host
  }
}

and https://cloud.hasura.io/public/graphiql?endpoint=https%3A%2F%2Fthe-federation.info%2Fv1%2Fgraphql

query MyQuery {
  nodes {
    domain
  }
}

@snarfed
Copy link
Owner Author

snarfed commented Mar 11, 2023

Added wm discovery caching. It's ok, but not great, since right now we churn instances pretty frequently. Eventually the full fediverse instance blocklist would be useful.

@snarfed
Copy link
Owner Author

snarfed commented Jun 18, 2023

#552 would help here.

@snarfed
Copy link
Owner Author

snarfed commented Jul 21, 2023

This shouldn't be happening now that we do protocol discovery.

@snarfed snarfed closed this as completed Jul 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant