Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mastodon instance not connecting to other instances, error 503 #6916

Closed
2 tasks done
bliekp opened this issue Mar 26, 2018 · 12 comments
Closed
2 tasks done

Mastodon instance not connecting to other instances, error 503 #6916

bliekp opened this issue Mar 26, 2018 · 12 comments

Comments

@bliekp
Copy link

bliekp commented Mar 26, 2018

Running 2.3.2

I have created a fresh and shiny instance on https://nicepeople.social
All works. New users can signup, emails are sent, people can toot, follow, etc.
But... I cannot toot to users outside of my instance. This throws the following errors:

Mar 26 08:37:01 nicepeople bundle[3489]: 2018-03-26T06:37:01.873Z 3489 TID-gs3lf9s9g ActivityPub::DeliveryWorker JID-7e32f6acd39df0171c2dad42 INFO: start Mar 26 08:37:02 nicepeople bundle[3489]: 2018-03-26T06:37:02.021Z 3489 TID-gs3lf9s9g ActivityPub::DeliveryWorker JID-7e32f6acd39df0171c2dad42 INFO: fail: 0.148 sec Mar 26 08:37:02 nicepeople bundle[3489]: 2018-03-26T06:37:02.024Z 3489 TID-gs3lf9s9g WARN: {"context":"Job raised exception","job":{"class":"ActivityPub::DeliveryWorker","args":["{\"@context\":[\"https://www.w3.org/ns/activitystreams\",\"https://w3id.org/security/v1\",{\"manuallyApprovesFollowers\":\"as:manuallyApprovesFollowers\",\"sensitive\":\"as:sensitive\",\"movedTo\":\"as:movedTo\",\"Hashtag\":\"as:Hashtag\",\"ostatus\":\"http://ostatus.org#\",\"atomUri\":\"ostatus:atomUri\",\"inReplyToAtomUri\":\"ostatus:inReplyToAtomUri\",\"conversation\":\"ostatus:conversation\",\"toot\":\"http://joinmastodon.org/ns#\",\"Emoji\":\"toot:Emoji\",\"focalPoint\":{\"@container\":\"@list\",\"@id\":\"toot:focalPoint\"},\"featured\":\"toot:featured\"}],\"id\":\"https://nicepeople.social/users/pimbliek/statuses/99748775179981571/activity\",\"type\":\"Create\",\"actor\":\"https://nicepeople.social/users/pimbliek\",\"published\":\"2018-03-26T06:25:19Z\",\"to\":[\"https://www.w3.org/ns/activitystreams#Public\"],\"cc\":[\"https://nicepeople.social/users/pimbliek/followers\",\"https://mastodon.xyz/users/TheKinrar\"],\"object\":{\"id\":\"https://nicepeople.social/users/pimbliek/statuses/99748775179981571\",\"type\":\"Note\",\"summary\":null,\"content\":\"\\u003cp\\u003ehey \\u003cspan class=\\\"h-card\\\"\\u003e\\u003ca href=\\\"https://mastodon.xyz/@TheKinrar\\\" class=\\\"u-url mention\\\"\\u003e@\\u003cspan\\u003eTheKinrar\\u003c/span\\u003e\\u003c/a\\u003e\\u003c/span\\u003e I have trouble adding my instance to the list.. What can I do? \\u003ca href=\\\"https://nicepeople.social/tags/nicepeople\\\" class=\\\"mention hashtag\\\" rel=\\\"tag\\\"\\u003e#\\u003cspan\\u003enicepeople\\u003c/span\\u003e\\u003c/a\\u003e.social\\u003c/p\\u003e\",\"inReplyTo\":null,\"published\":\"2018-03-26T06:25:19Z\",\"url\":\"https://nicepeople.social/@pimbliek/99748775179981571\",\"attributedTo\":\"https://nicepeople.social/users/pimbliek\",\"to\":[\"https://www.w3.org/ns/activitystreams#Public\"],\"cc\":[\"https://nicepeople.social/users/pimbliek/followers\",\"https://mastodon.xyz/users/TheKinrar\"],\"sensitive\":false,\"atomUri\":\"https://nicepeople.social/users/pimbliek/statuses/99748775179981571\",\"inReplyToAtomUri\":null,\"conversation\":\"tag:nicepeople.social,2018-03-26:objectId=27:objectType=Conversation\",\"attachment\":[],\"tag\":[{\"type\":\"Mention\",\"href\":\"https://mastodon.xyz/users/TheKinrar\",\"name\":\"@TheKinrar@mastodon.xyz\"}]},\"signature\":{\"type\":\"RsaSignature2017\",\"creator\":\"https://nicepeople.social/users/pimbliek#main-key\",\"created\":\"2018-03-26T06:25:21Z\",\"signatureValue\":\"d/R2hQ24yV0VMF7tas1VMg+tUNAVM2iWYh3CxN/2lCUpDTXyXq+bEYqSYhYBpKyxB2qSo+5n694yOf02kQ5L3qE3b9ppNC1FVlG91Z8DrCc1bmpmox2/+EXW+RWcEkjSfAjEV967Bl7UWkNtOa69wHRrnDeW3oYsL1gQ3r/psR/jd8sWgMoDP5bVGciv0uj0guYjnt8AD+/f0jUgKaJC3NKPKm6Gaj+/JGK8LmI6YDzSLPM+B4Y3MqTmiTwlgABPN3qvkQGB4Pc7Tp39FWAdcscsmCpfEwdhyAXuGPRH09BvhVn9nZoSkMoFZ31aa/HxaI6pKH3GX648IFh8CXmDOQ==\"}}",2,"https://mastodon.xyz/users/TheKinrar/inbox"],"retry":16,"queue":"push","dead":false,"jid":"7e32f6acd39df0171c2dad42","created_at":1522045521.1356204,"enqueued_at":1522046221.872532,"error_message":"Delivery failed for https://mastodon.xyz/users/TheKinrar/inbox: https://mastodon.xyz/users/TheKinrar/inbox returned code 503","error_class":"Mastodon::UnexpectedResponseError","failed_at":1522045521.2696996,"retry_count":4,"retried_at":1522045876.6536572},"jobstr":"{\"class\":\"ActivityPub::DeliveryWorker\",\"args\":[\"{\\\"@context\\\":[\\\"https://www.w3.org/ns/activitystreams\\\",\\\"https://w3id.org/security/v1\\\",{\\\"manuallyApprovesFollowers\\\":\\\"as:manuallyApprovesFollowers\\\",\\\"sensitive\\\":\\\"as:sensitive\\\",\\\"movedTo\\\":\\\"as:movedTo\\\",\\\"Hashtag\\\":\\\"as:Hashtag\\\",\\\"ostatus\\\":\\\"http://ostatus.org#\\\",\\\"atomUri\\\":\\\"ostatus:atomUri\\\",\\\"inReplyToAtomUri\\\":\\\"ostatus:inReplyToAtomUri\\\",\\\"conversation\\\":\\\"ostatus:conversation\\\",\\\"toot\\\":\\\"http://joinmastodon.org/ns#\\\",\\\"Emoji\\\":\\\"toot:Emoji\\\",\\\"focalPoint\\\":{\\\"@container\\\":\\\"@list\\\",\\\"@id\\\":\\\"toot:focalPoint\\\"},\\\"featured\\\":\\\"toot:featured\\\"}],\\\"id\\\":\\\"https://nicepeople.social/users/pimbliek/statuses/99748775179981571/activity\\\",\\\"type\\\":\\\"Create\\\",\\\"actor\\\":\\\"https://nicepeople.social/users/pimbliek\\\",\\\"published\\\":\\\"2018-03-26T06:25:19Z\\\",\\\"to\\\":[\\\"https://www.w3.org/ns/activitystreams#Public\\\"],\\\"cc\\\":[\\\"https://nicepeople.social/users/pimbliek/followers\\\",\\\"https://mastodon.xyz/users/TheKinrar\\\"],\\\"object\\\":{\\\"id\\\":\\\"https://nicepeople.social/users/pimbliek/statuses/99748775179981571\\\",\\\"type\\\":\\\"Note\\\",\\\"summary\\\":null,\\\"content\\\":\\\"\\\\u003cp\\\\u003ehey \\\\u003cspan class=\\\\\\\"h-card\\\\\\\"\\\\u003e\\\\u003ca href=\\\\\\\"https://mastodon.xyz/@TheKinrar\\\\\\\" class=\\\\\\\"u-url mention\\\\\\\"\\\\u003e@\\\\u003cspan\\\\u003eTheKinrar\\\\u003c/span\\\\u003e\\\\u003c/a\\\\u003e\\\\u003c/span\\\\u003e I have trouble adding my instance to the list.. What can I do? \\\\u003ca href=\\\\\\\"https://nicepeople.social/tags/nicepeople\\\\\\\" class=\\\\\\\"mention hashtag\\\\\\\" rel=\\\\\\\"tag\\\\\\\"\\\\u003e#\\\\u003cspan\\\\u003enicepeople\\\\u003c/span\\\\u003e\\\\u003c/a\\\\u003e.social\\\\u003c/p\\\\u003e\\\",\\\"inReplyTo\\\":null,\\\"published\\\":\\\"2018-03-26T06:25:19Z\\\",\\\"url\\\":\\\"https://nicepeople.social/@pimbliek/99748775179981571\\\",\\\"attributedTo\\\":\\\"https://nicepeople.social/users/pimbliek\\\",\\\"to\\\":[\\\"https://www.w3.org/ns/activitystreams#Public\\\"],\\\"cc\\\":[\\\"https://nicepeople.social/users/pimbliek/followers\\\",\\\"https://mastodon.xyz/users/TheKinrar\\\"],\\\"sensitive\\\":false,\\\"atomUri\\\":\\\"https://nicepeople.social/users/pimbliek/statuses/99748775179981571\\\",\\\"inReplyToAtomUri\\\":null,\\\"conversation\\\":\\\"tag:nicepeople.social,2018-03-26:objectId=27:objectType=Conversation\\\",\\\"attachment\\\":[],\\\"tag\\\":[{\\\"type\\\":\\\"Mention\\\",\\\"href\\\":\\\"https://mastodon.xyz/users/TheKinrar\\\",\\\"name\\\":\\\"@TheKinrar@mastodon.xyz\\\"}]},\\\"signature\\\":{\\\"type\\\":\\\"RsaSignature2017\\\",\\\"creator\\\":\\\"https://nicepeople.social/users/pimbliek#main-key\\\",\\\"created\\\":\\\"2018-03-26T06:25:21Z\\\",\\\"signatureValue\\\":\\\"d/R2hQ24yV0VMF7tas1VMg+tUNAVM2iWYh3CxN/2lCUpDTXyXq+bEYqSYhYBpKyxB2qSo+5n694yOf02kQ5L3qE3b9ppNC1FVlG91Z8DrCc1bmpmox2/+EXW+RWcEkjSfAjEV967Bl7UWkNtOa69wHRrnDeW3oYsL1gQ3r/psR/jd8sWgMoDP5bVGciv0uj0guYjnt8AD+/f0jUgKaJC3NKPKm6Gaj+/JGK8LmI6YDzSLPM+B4Y3MqTmiTwlgABPN3qvkQGB4Pc7Tp39FWAdcscsmCpfEwdhyAXuGPRH09BvhVn9nZoSkMoFZ31aa/HxaI6pKH3GX648IFh8CXmDOQ==\\\"}}\",2,\"https://mastodon.xyz/users/TheKinrar/inbox\"],\"retry\":16,\"queue\":\"push\",\"dead\":false,\"jid\":\"7e32f6acd39df0171c2dad42\",\"created_at\":1522045521.1356204,\"enqueued_at\":1522046221.872532,\"error_message\":\"Delivery failed for https://mastodon.xyz/users/TheKinrar/inbox: https://mastodon.xyz/users/TheKinrar/inbox returned code 503\",\"error_class\":\"Mastodon::UnexpectedResponseError\",\"failed_at\":1522045521.2696996,\"retry_count\":4,\"retried_at\":1522045876.6536572}"} Mar 26 08:37:02 nicepeople bundle[3489]: 2018-03-26T06:37:02.025Z 3489 TID-gs3lf9s9g WARN: Mastodon::UnexpectedResponseError: Delivery failed for https://mastodon.xyz/users/TheKinrar/inbox: https://mastodon.xyz/users/TheKinrar/inbox returned code 503 Mar 26 08:37:02 nicepeople bundle[3489]: 2018-03-26T06:37:02.025Z 3489 TID-gs3lf9s9g WARN: /home/mastodon/live/app/workers/activitypub/delivery_worker.rb:17:in perform'
Mar 26 08:37:18 nicepeople bundle[3472]: [a4f3e520-d24f-4628-8bf5-e20ce48e1e24] [active_model_serializers] Rendered ActiveModel::Serializer::CollectionSerializer with ActiveModelSerializers::Adapter::Attributes (0.08ms)
`

Maybe this is also related to the fact that I cannot reach the instance with an iOS app like amaroq (just says it can't connect) or that I cannot add my instance to https://instances.social/admin (tells me 'something' went wrong...).

Any help greatly appreciated!


  • I searched or browsed the repo’s other issues to ensure this is not a duplicate.
  • This bug happens on a tagged release and not on master (If you're a user, don't worry about this).
@bliekp
Copy link
Author

bliekp commented Mar 26, 2018

I wonder if it could be DNS related. DNS entries were added around 13:00 CET yesterday (18 hrs ago) and reverse DNS was only added some 10 hours later...

@bliekp bliekp changed the title WARN: Mastodon::UnexpectedResponseError: Delivery failed for https://mastodon.social/users/eurasierboy/inbox: https://mastodon.social/users/eurasierboy/inbox returned code 503 Mastodon instance not connecting to other instances, error 503 Mar 26, 2018
@bliekp
Copy link
Author

bliekp commented Mar 27, 2018

bump no response? This is a real showstopper for deployment of Mastodon on new instances. We cannot connect to the rest of the Mastodon Universe...

@bliekp
Copy link
Author

bliekp commented Mar 27, 2018

I have narrowed it down a bit... I tried tooting from another instance to my own (@pimbliek@nicepeople.social) and it threw an error: 503 Remote SSL certificate could not be verified.

So it seems the SSL cert is either wrong, or Mastodon is way too strict about it.

I have an officiel SSL certificate from my provider issued by Comodo RSA. Google Chrome is totally happy with it and tells me it is valid.

The error is thrown from this file it seems in the code: mastodon/app/controllers/api/base_controller.rb

@Gargron
Copy link
Member

Gargron commented Mar 27, 2018

In your nginx/ssl configuration, is your certificate chain complete? Might be missing some intermediary certificates.

@bliekp
Copy link
Author

bliekp commented Mar 28, 2018

Yeah it seems it is something like this. It would be nice if Mastodon gave a bit more information in the logs about this...
I sent the question to my hosting company since they provided me with the certificates and their documentation on how to combine is clearly wrong.. because whatever order I put them in, the SSL check fails... complaining about the order of the certificates..
I assume I will have more clarity on this issue within 24hrs.

@bliekp
Copy link
Author

bliekp commented Mar 28, 2018

It has been solved by putting the certificates in the right order.

I still think Mastodon could / should give a easier to understand error.

@albjeremias
Copy link

@bliekp how did you ended up diagnosing this? and fix it? Can you please share? thanks...

@bliekp
Copy link
Author

bliekp commented May 8, 2018

Like I said @albjeremias I created a working SSL certificate. The error was that my ISP gave me wrong instructions on how to create the SSL-chain. After contacting the provider of the SSL certificate this was resolved.
Also, @Gargron 's comment helped a lot to point me in the right direction. This was specially helpful since Mastodon itself doesn't give you any real hints on where to search.. See above.

@bliekp bliekp closed this as completed May 8, 2018
@albjeremias
Copy link

no, I think my problem is that I'm using a subdomain social.coletivos.org but the users in my instance are user@coletivos.org and somehow the remote instance is failling to request because its probably requesting to coletivos.org instead of social.coletivos.org so the mismatch?
I'm a bit lost on this one... :(

@Gargron
Copy link
Member

Gargron commented May 15, 2018

Check the documentation for serving mastodon from a subdomain. You probably missed a step

@albjeremias
Copy link

@A1vinSmith
Copy link

Not sure it's related. But when I searching my server from mastodon's mobile app, I got 503 not available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants