Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Bridgy pulls comments and likes from social networks back to your web site. You can also use it to publish your posts to those networks.
branch: master

replace handlers.SOURCE with models.source populated by Source metaclass

props to @bslatkin and https://github.com/bslatkin/effectivepython for the inspiration. my first use of metaclasses (not counting dog science copypasta), definitely not my last!
latest commit feceb8dd72
Ryan Barrett authored
Failed to load latest commit information.
activitystreams @ 5c438d7 replace handlers.SOURCE with models.source populated by Source metaclass
html2text_module @ 1203f62 switch from html2text/html2text to Alir3z4/html2text, part 2/2
scripts script that generated the graphs in https://snarfed.org/2014-11-06_ha…
static minor html embed tweaks
templates add faq for death of publishing facebook likes/comments. for #350
webmention-tools @ 5cc3578 test for webmention discovery bug on pages without Content-Type header
.gitignore blog webmentions: start to add superfeedr for sending webmentions
.gitmodules switch from html2text/html2text to Alir3z4/html2text, part 2/2
README.md noop readme language tweak
admin.py noop: remove unused imports, other minor tweaks
alltests.py fully qualify imports of oauth-dropins submodules
app.py replace handlers.SOURCE with models.source populated by Source metaclass
app.yaml drop style.css cache expiration from 1d to 1h, also set it on bridgy.js
app_test.py don't show user pages for users who've disabled all features
appengine_config.py ereporter exception for one known case (nytimes.com) of #237
backends.yaml disable twitter-streaming backend. switching to scraping html. :(
blog_webmention.py replace handlers.SOURCE with models.source populated by Source metaclass
blog_webmention_test.py replace handlers.SOURCE with models.source populated by Source metaclass
blogger.py blogger signup: if they have no blogs, show error and redirect to fro…
blogger_test.py blogger signup: if they have no blogs, show error and redirect to fro…
cloud_storage_lifecycle.json datastore backup: document in readme, add cloud storage lifecycle pol…
cron.py replace handlers.SOURCE with models.source populated by Source metaclass
cron.yaml twitter: drop cron job that updates profile picture URLs
cron_test.py replace handlers.SOURCE with models.source populated by Source metaclass
domain_blacklist.txt blacklist blog.mahabali.me; its comment spam filter rejects us
dos.yaml jkphl fixed his cron job!
facebook.py after oauth dance for interactive publish, check that it's the right …
facebook_test.py update tests for snarfed/activitystreams-unofficial@3745ffa
googleplus.py G+: follow redirects when canonicalizing urls to find +MyName nicknames
googleplus_test.py G+: follow redirects when canonicalizing urls to find +MyName nicknames
handlers.py replace handlers.SOURCE with models.source populated by Source metaclass
handlers_test.py replace handlers.SOURCE with models.source populated by Source metaclass
html2text switch from html2text/html2text to Alir3z4/html2text, part 2/2
index.yaml Added a new auto-generated index for Publish
instagram.py make instagram interactive publish use the existing oauth callback. for
instagram_test.py publish: allow users to sign up for Instagram/publish and publish likes
logs.py logs: link datastore keys to admin console datastore viewer
mapreduce.yaml add mapreduce to prune the Response.activity_json field. fixes #68
mapreduces.py noop: remove unused imports, other minor tweaks
models.py replace handlers.SOURCE with models.source populated by Source metaclass
models_test.py replace handlers.SOURCE with models.source populated by Source metaclass
original_post_discovery.py use custom user agent in external HTTP fetches. fixes #358
original_post_discovery_test.py port tasks.get_webmention_targets() to util.dedupe_urls()
publish.py facebook publish: RIP comments/likes. :(
publish_test.py facebook publish: RIP comments/likes. :(
queue.yaml minor task queue config tweaks, mostly noops, for #364
superfeedr.py use custom user agent in external HTTP fetches. fixes #358
superfeedr_test.py noop: remove unused imports, other minor tweaks
tasks.py port tasks.get_webmention_targets() to util.dedupe_urls()
tasks_test.py use custom user agent in external HTTP fetches. fixes #358
testutil.py replace handlers.SOURCE with models.source populated by Source metaclass
tumblr.py use custom user agent in external HTTP fetches. fixes #358
tumblr_test.py noop: remove unused imports, other minor tweaks
twitter.py after oauth dance for interactive publish, check that it's the right …
twitter_streaming.py Response.get_or_save(): re-propagate when an existing response has ch…
twitter_streaming_test.py update tests to match snarfed/activitystreams-unofficial@bd15523 and …
twitter_test.py update twitter test profile picture URLs to match snarfed/activitystr…
util.py use custom user agent in external HTTP fetches. fixes #358
util_test.py use custom user agent in external HTTP fetches. fixes #358
webmention.py use custom user agent in external HTTP fetches. fixes #358
wordpress_rest.py dedupe Source.domain_urls across http/https scheme and trailing slash…
wordpress_rest_test.py handle when wordpress.com API call returns non-json

README.md

Bridgy Bridgy

Got a web site? Want social network replies and likes on your site? Want to post and tweet from your site? Bridgy is for you.

http://brid.gy/

Bridgy pulls comments and likes from social networks back to your web site. You can also use it to publish your posts to those networks. See the docs for more details.

License: This project is placed in the public domain.

Development

Most dependencies are in git submodules. Be sure to run git submodule update --init --recursive after you clone the repo.

Requires the App Engine SDK and looks for it in the GAE_SDK_ROOT environment variable, /usr/local/google_appengine, or ~/google_appengine, in that order.

You can run the unit tests with alltests.py. If you send a pull request, please include (or update) a test for the new functionality if possible!

This command runs the tests, pushes any changes in your local repo(s), and deploys to App Engine:

./alltests.py && cd activitystreams && ./alltests.py && \
  cd oauth_dropins && ./alltests.py && cd webutil && ./alltests.py && \
  cd ../../.. && git push --recurse-submodules=on-demand && \
  ~/google_appengine/appcfg.py --oauth2 update .

Most dependencies are clean, but we've made patches to some that we haven't (yet) tried to push upstream. If we ever switch submodule repos for those dependencies, make sure the patches are included!

  • snarfed/gdata-python-client@fabb6227361612ac4fcb8bef4438719cb00eaa2b
  • snarfed/gdata-python-client@8453e3388d152ac650e22d219fae36da56d9a85d

Misc

The datastore is automatically backed up by a cron job that runs Datastore Admin backup and stores the results in Cloud Storage, in the brid-gy.appspot.com bucket. It backs up all entities weekly, and all entities except Response and SyndicatedPost daily, since they make up 92% of all entities by size and they aren't as critical to keep.

We use this command to set a Cloud Storage lifecycle policy on that bucket that deletes all files over 30 days old:

gsutil lifecycle set cloud_storage_lifecycle.json gs://brid-gy.appspot.com

So far, this has kept us within the 5GB free quota. Run this command to see how much space we're currently using:

gsutil du -hsc gs://brid-gy.appspot.com/\*

Here are remote_api_shell and shell commands for generating the statistics published at brid.gy/about#stats:

# remote_api_shell
from models import Response
cursor = None
with open('sent_urls', 'w') as sent, open('unsent_urls', 'w') as unsent:
  while True:
    results, cursor, _ = Response.query(
#      projection=[Response.sent,Response.skipped,Response.error,Response.failed]
      ).fetch_page(100, start_cursor=cursor)
    if not results:
      break
    for r in results:
      print >> sent, '\n'.join(r.sent)
      print >> unsent, '\n'.join(r.skipped + r.error + r.failed)

# shell
sort sent_urls  | uniq > sent_uniq
cut -f3 -d/ sent_uniq | sed 's/^www\.//' | sort --ignore-case | uniq -i > sent_domains
wc sent_urls sent_uniq sent_domains

Related projects and docs

Something went wrong with that request. Please try again.