interpret Mastodon ActivityPub HTTP 202 response as failure

[snarfed]

mastodon's AP code returns HTTP 202 to inbox POSTs that fail signature verification:

• https://github.com/tootsuite/mastodon/blob/33ea042decc03c494c78a896934455ef79abc16b/app/controllers/activitypub/inboxes_controller.rb#L12
• https://github.com/tootsuite/mastodon/blob/33ea042decc03c494c78a896934455ef79abc16b/app/controllers/concerns/signature_verification.rb#L16-L68

TODO:

• interpret this as a failure instead of success, and return non-2xx ourselves
• figure out why production instances like https://mastodon.technology/ and https://mastodon.social/ are currently failing to verify our signatures

[snarfed]

cc @mblaney

[snarfed]

aha, looks like mastodon changed this 202 to a 401 in head (2.0.0): mastodon/mastodon@dfaa219#diff-b71581bf00ae84fbac757c5373533ebb

[snarfed]

tentatively reopening. i'm seeing 202 from mastodon again on Create but the reply isn't showing up, e.g. https://snarfed.org/2018-01-26_chris-aldrich-%f0%9f%8d%8d-anarchivistchaos-social-edsudigipres-club-sn-mastodon to https://mastodon.social/@chrisaldrich/99418039176997784 .

https://github.com/tootsuite/mastodon/blob/master/app/controllers/activitypub/inboxes_controller.rb at master shows that mastodon evidently returns 202 for success, so probably something else is going on that prevents it from appearing.

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "object": {
    "inReplyTo": "https://mastodon.social/users/chrisaldrich/statuses/99418039176997784",
    "cc": [
      "https://www.w3.org/ns/activitystreams#Public",
      "https://mastodon.social/users/chrisaldrich",
      "https://mastodon.social/users/chrisaldrich/followers",
      "https://mastodon.technology/users/snarfed",
      "https://digipres.club/users/edsu",
      "https://chaos.social/users/anarchivist"
    ],
    "attributedTo": [
      {
        "preferredUsername": "snarfed.org",
        "name": "Ryan Barrett",
        "url": "https://snarfed.org/",
        "image": [
          {
            "url": "https://secure.gravatar.com/avatar/947b5f3f323da0ef785b6f02d9c265d6?s=96&d=blank&r=g",
            "type": "Image"
          }
        ],
        "type": "Person",
        "id": "https://fed.brid.gy/snarfed.org",
        "icon": [
          {
            "url": "https://secure.gravatar.com/avatar/947b5f3f323da0ef785b6f02d9c265d6?s=96&d=blank&r=g",
            "type": "Image"
          }
        ]
      }
    ],
    "tag": [
      {
        "name": "reply"
      }
    ],
    "id": "https://snarfed.org/2018-01-26_chris-aldrich-%f0%9f%8d%8d-anarchivistchaos-social-edsudigipres-club-sn-mastodon",
    "name": "reply \n\nJanuary 26, 2018 Ryan Barrett\nLeave a comment\n\n\nthanks chris! bridgy fed is definitely open and ready for (more) users. it\u2019s not perfect yet, but it is usable. i\u2019m using it to post this reply right now! \n\nStandard",
    "url": "https://snarfed.org/2018-01-26_chris-aldrich-%f0%9f%8d%8d-anarchivistchaos-social-edsudigipres-club-sn-mastodon",
    "content": " <p>thanks chris! bridgy fed is definitely open and ready for (more) users. it\u2019s not perfect yet, but it is usable. i\u2019m using it to post this reply right now! <a class=\"u-in-reply-to\" href=\"https://mastodon.social/@chrisaldrich/99418039176997784\"></a> <cite class=\"via\"><a href=\"https://fed.brid.gy/\"></a></cite></p> ",
    "published": "2018-01-26T12:50:25-08:00",
    "@context": "https://www.w3.org/ns/activitystreams",
    "type": "Note"
  },
  "type": "Create"
}

[snarfed]

i suspect this is due to settings on mastodon.social specifically (currently 2.2.0rc2), since replies and likes still work on lgbt.io (2.1.2), chaos.social (2.1.2), and others.

[swentel]

Some feedback re: 202 responses.

Been playing around with integrating Drupal. I've done 3 tests, of which 2 work fine. Test url is https://mastodon.technology/@snarfed/3194674

What works is like (https://realize.be/like/1468) and repost (https://realize.be/repost/1469)
Interestingly enough, I got "202 response! If this is Mastodon 1.x, their signature verification probably failed. :(" back for both responses.

When trying to send a reply, I get back the same response, but nothing shows up on the post on mastodon itself.

(It might be that I'm missing something, currently, I have the .htaccess and atom feed, but nothing done yet with WebSub. But AFAICT, that shouldn't matter for sending a webmention.)

[valpackett]

mastodon 2.4.2, replies still don't show up (log), and likes do still seem to work

[swentel]

So out of curiosity, I tried a reply again today :) (to https://jawns.club/@timplunkett/100606109220664799)

The latest reply I tested now gives something back like 'Public key not found for key acct:realize.be@realize.be'. I haven't checked yet what that means and/or if it's relevant, but just wanted to leave it here.

[snarfed]

@swentel thanks for the update! that looks like #29, but the conclusion there was that they needed a representative h-card (#29 (comment)), which isn't your problem. https://fed.brid.gy/.well-known/webfinger?resource=realize.be@realize.be works fine.

...ah, but your site isn't actually set up for bridgy fed. https://realize.be/.well-known/host-meta and https://realize.be/.well-known/webfinger currently 404. they should redirect to fed.brid.gy instead. https://fed.brid.gy/#setup

[swentel]

Right - my bad, I commented out the lines in my .htaccess .. still the same outcome now.

[snarfed]

thanks! it actually gets a 202 from mastodon now. so we're back at the original issue here. whee!

https://fed.brid.gy/log?start_time=1536433273&key=https%3A%2F%2Frealize.be%2Freply%2Fcontent%2F1541+https%3A%2F%2Fjawns.club%2F%40timplunkett%2F100606109220664799

[swentel]

So, I checked mastodon, in app/controllers/activitypub/inboxes_controller.rb it will return 202 when the request is signed valid. But process_payload can still well basically do nothing. Kind of weird behavior if you tell me ..

[swentel]

So I finally found out why replies don't show up. The problem is that mastodon compares the id on the object with the actor url.

See https://github.com/tootsuite/mastodon/blob/master/app/lib/activitypub/activity/create.rb#L8
See https://github.com/tootsuite/mastodon/blob/master/app/lib/activitypub/activity/create.rb#L269

Since fed.brid.gy sends the url of the post on the website as id, and the actor url is something like https://fed.brid.gy/{sitename} this will now fail. Also, the 'url' property in an object gets the same check, but is ignored when it fails, the reply will still show up if the id check is fine.

When sending an id which looks like 'https://fed.brid.gy/realize.be?unique_id_here', the reply will show up. This has an impact on the canonical URL of the post being fed.brid.gy and not the website url. As a consequence, I guess round trips can't work anymore. One workaround could be to create the id in the form of https://fed.brid.gy/realize.be?url={original_url} - that works (see the 'Welcome to the club! AGAIN' canonical url). Of course, if at some point mastodon decides to start ignore get params, then it's back to the drawing table.

(I guess this also has impact on discovery of posts as they basically will have the same problem)

I often wonder whether it would be possible to set the id of https://fed.brid.gy/realize.be' to realize.be, but I'm afraid that ain't possible at all ?

Working replies:
https://mastodon.social/users/swentel/statuses/100776214865184909
https://mastodon.technology/@snarfed/3194674