New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

not seeing some brid.gy source URLs #14

Closed
snarfed opened this Issue Jan 2, 2014 · 9 comments

Comments

Projects
None yet
2 participants
@snarfed
Contributor

snarfed commented Jan 2, 2014

not sure of the root cause yet, but webmention.io returns 400 'The source URI does not exist' on at least some brid.gy webmentions. here's an example:

source: https://www.brid.gy/like/twitter/aaronpk/418251719303495680/14447132
target: http://ttk.me/t4Tq3
full log: https://www.brid.gy/log?start_time=1388621265&key=aglzfmJyaWQtZ3lySwsSCFJlc3BvbnNlIj10YWc6dHdpdHRlci5jb20sMjAxMzo0MTgyNTE3MTkzMDM0OTU2ODBfZmF2b3JpdGVkX2J5XzE0NDQ3MTMyDA

a couple guesses:

  • does webmention.io cache source fetches? if so, maybe brid.gy choked the first time webmention.io asked for it, and it hasn't retried?
  • the brid.gy URL is served over SSL but has mixed content due to the profile picture. (i'll fix that soon.) maybe webmention.io is picky?
@snarfed

This comment has been minimized.

Show comment
Hide comment
@snarfed

snarfed Jan 2, 2014

Contributor

just fyi, i fixed the SSL mixed mode error, but the webmention is still failing, so it wasn't that.

Contributor

snarfed commented Jan 2, 2014

just fyi, i fixed the SSL mixed mode error, but the webmention is still failing, so it wasn't that.

@snarfed

This comment has been minimized.

Show comment
Hide comment
@snarfed

This comment has been minimized.

Show comment
Hide comment
@snarfed

snarfed Jan 4, 2014

Contributor

happening again, fwiw: https://www.brid.gy/#twitter-t . the accepted one may have been a fluke.

Contributor

snarfed commented Jan 4, 2014

happening again, fwiw: https://www.brid.gy/#twitter-t . the accepted one may have been a fluke.

@aaronpk

This comment has been minimized.

Show comment
Hide comment
@aaronpk

aaronpk Jan 4, 2014

Owner

This is bizarre! A source_not_found error means webmention.io couldn't retrieve the source URL. (This is before it even looks for the link to target.) Here is the source where it returns that error.

I have been able to reproduce this by manually creating the scraper object and making a request for the page. It seems that the first time I make the request it's fine, and any later requests to www.brid.gy fail with OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server hello A

The only ref to this I was able to find is rubygems/rubygems#522 (comment) which says "It seems the "hello A" errors also occur on request timeouts and slow links."

At the very least I should update webmention.io to respond with SSL failures with a separate code so you at least get a little better idea of what's wrong.

Owner

aaronpk commented Jan 4, 2014

This is bizarre! A source_not_found error means webmention.io couldn't retrieve the source URL. (This is before it even looks for the link to target.) Here is the source where it returns that error.

I have been able to reproduce this by manually creating the scraper object and making a request for the page. It seems that the first time I make the request it's fine, and any later requests to www.brid.gy fail with OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server hello A

The only ref to this I was able to find is rubygems/rubygems#522 (comment) which says "It seems the "hello A" errors also occur on request timeouts and slow links."

At the very least I should update webmention.io to respond with SSL failures with a separate code so you at least get a little better idea of what's wrong.

@snarfed

This comment has been minimized.

Show comment
Hide comment
@snarfed

snarfed Jan 4, 2014

Contributor

huh! bizarre indeed. thanks for investigating. for my part, i have some low hanging fruit for speeding up those requests on brid.gy's side. i'll do some benchmarks and try those.

Contributor

snarfed commented Jan 4, 2014

huh! bizarre indeed. thanks for investigating. for my part, i have some low hanging fruit for speeding up those requests on brid.gy's side. i'll do some benchmarks and try those.

@snarfed

This comment has been minimized.

Show comment
Hide comment
@snarfed

snarfed Jan 4, 2014

Contributor

fwiw, i'm seeing median client side response times of around .5s, with spikes up to .9s. server side is around 150ms. not great, but not horrible. maybe you can increase the timeout? i'll still plan to optimize on my end.

$ foreach i (`seq 0 10`)
foreach? time curl -so /dev/null https://www.brid.gy/like/twitter/t/419207701747761152/284933
foreach? end
0.033u 0.005s 0:00.49 6.1%  0+0k 0+0io 0pf+0w
0.032u 0.005s 0:00.47 6.3%  0+0k 0+0io 0pf+0w
0.034u 0.006s 0:00.53 5.6%  0+0k 0+0io 0pf+0w
0.034u 0.005s 0:00.61 4.9%  0+0k 0+0io 0pf+0w
0.034u 0.006s 0:00.44 6.8%  0+0k 0+0io 0pf+0w
0.034u 0.006s 0:00.47 6.3%  0+0k 0+0io 0pf+0w
0.035u 0.006s 0:00.44 6.8%  0+0k 0+0io 0pf+0w
0.033u 0.006s 0:00.49 6.1%  0+0k 0+0io 0pf+0w
0.034u 0.006s 0:00.89 3.3%  0+0k 0+0io 0pf+0w
0.039u 0.006s 0:00.44 6.8%  0+0k 0+0io 0pf+0w
0.033u 0.006s 0:00.45 6.6%  0+0k 0+0io 0pf+0w
Contributor

snarfed commented Jan 4, 2014

fwiw, i'm seeing median client side response times of around .5s, with spikes up to .9s. server side is around 150ms. not great, but not horrible. maybe you can increase the timeout? i'll still plan to optimize on my end.

$ foreach i (`seq 0 10`)
foreach? time curl -so /dev/null https://www.brid.gy/like/twitter/t/419207701747761152/284933
foreach? end
0.033u 0.005s 0:00.49 6.1%  0+0k 0+0io 0pf+0w
0.032u 0.005s 0:00.47 6.3%  0+0k 0+0io 0pf+0w
0.034u 0.006s 0:00.53 5.6%  0+0k 0+0io 0pf+0w
0.034u 0.005s 0:00.61 4.9%  0+0k 0+0io 0pf+0w
0.034u 0.006s 0:00.44 6.8%  0+0k 0+0io 0pf+0w
0.034u 0.006s 0:00.47 6.3%  0+0k 0+0io 0pf+0w
0.035u 0.006s 0:00.44 6.8%  0+0k 0+0io 0pf+0w
0.033u 0.006s 0:00.49 6.1%  0+0k 0+0io 0pf+0w
0.034u 0.006s 0:00.89 3.3%  0+0k 0+0io 0pf+0w
0.039u 0.006s 0:00.44 6.8%  0+0k 0+0io 0pf+0w
0.033u 0.006s 0:00.45 6.6%  0+0k 0+0io 0pf+0w
@aaronpk

This comment has been minimized.

Show comment
Hide comment
@aaronpk

aaronpk Jan 4, 2014

Owner

I think it's actually a timeout in negotiating the SSL connection, before it even tries to make the HTTP request.

Owner

aaronpk commented Jan 4, 2014

I think it's actually a timeout in negotiating the SSL connection, before it even tries to make the HTTP request.

@aaronpk

This comment has been minimized.

Show comment
Hide comment
@aaronpk

aaronpk Jan 4, 2014

Owner

We tracked this down to what appears to be an issue with the SSL cert on www.brid.gy. I'll leave this ticket open until @snarfed installs the updated cert and we verify that it's working.

I also created a new issue #15 to return better errors to help debug this kind of thing in the future.

Owner

aaronpk commented Jan 4, 2014

We tracked this down to what appears to be an issue with the SSL cert on www.brid.gy. I'll leave this ticket open until @snarfed installs the updated cert and we verify that it's working.

I also created a new issue #15 to return better errors to help debug this kind of thing in the future.

@snarfed

This comment has been minimized.

Show comment
Hide comment
@snarfed

snarfed Jan 9, 2014

Contributor

this turned out to be an SNI issue. updating one of the ruby gems fixed it, so our debugging was cut short, but we captured some info in IRC and on the wiki.

Contributor

snarfed commented Jan 9, 2014

this turned out to be an SNI issue. updating one of the ruby gems fixed it, so our debugging was cut short, but we captured some info in IRC and on the wiki.

@snarfed snarfed closed this Jan 9, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment