Conflict with XRay's parsing of microformats

[ngm]

Not sure if this should be logged here or with XRay - but when ActivityPub plugin is enabled, XRay detects the content-type of posts as activity+json, and then doesn't parse the microformats in the post correctly.

e.g. this post - https://doubleloop.net/2020/01/03/6502/

with AP plugin disabled:

{
    "data": {
        "type": "entry",
        "published": "2020-01-03T21:02:30+00:00",
        "like-of": [
            "https://calumryan.com/blog/my-2010s/"
        ],
        "author": {
            "type": "card",
            "name": "Neil Mather",
            "url": "https://doubleloop.net/",
            "photo": "https://secure.gravatar.com/avatar/525ddf1ae0f87a3bc6ddf8be6cc4b50c?s=100&d=default&r=g"
        },
        "post-type": "like",
        "refs": {
            "https://calumryan.com/blog/my-2010s/": {
                "type": "entry",
                "url": "https://calumryan.com/blog/my-2010s/",
                "name": "My 2010s",
                "author": {
                    "type": "card",
                    "name": "Calum Ryan",
                    "url": false,
                    "photo": null
                },
                "post-type": "article"
            }
        },
        "rels": {
            "canonical": "https://doubleloop.net/2020/01/03/6502/"
        }
    },
    "url": "https://doubleloop.net/2020/01/03/6502/",
    "code": 200,
    "source-format": "mf2+html"
}

with AP plugin enabled:

{
    "data": {
        "type": "entry",
        "url": "https://doubleloop.net/2020/01/03/6502/",
        "published": "2020-01-03T21:02:30+00:00",
        "summary": "",
        "content": {
            "text": "Liked My 2010s by Calum Ryan (Calum Ryan)\nA decade-end review of my life to date seems almost too much to achieve in a digestible short read but here we go!https://doubleloop.net/2020/01/03/6502/",
            "html": "Liked <a href=\"https://calumryan.com/blog/my-2010s/\" class=\"p-name u-url\">My 2010s</a> by Calum Ryan (Calum Ryan)\n<blockquote class=\"e-summary\"><p>A decade-end review of my life to date seems almost too much to achieve in a digestible short read but here we go!</p></blockquote><p><a href=\"https://doubleloop.net/2020/01/03/6502/\">https://doubleloop.net/2020/01/03/6502/</a></p>"
        },
        "author": {
            "type": "card",
            "name": "Neil Mather",
            "url": "https://doubleloop.net/author/neil/",
            "photo": "https://doubleloop.net/wp-content/uploads/avatar-privacy/cache/user/8/5/8538d26ce878a6e2589b56516ab6d5333283f785d1b85009044b8aa617b81fba-120.png",
            "nickname": "neil"
        },
        "post-type": "note"
    },
    "url": "https://doubleloop.net/2020/01/03/6502/",
    "code": 200,
    "source-format": "activity+json"
}

[dshanske]

I see the problem...the accept header has multiple values. This plugin isn't following precedence

[dshanske]

https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.1

[dshanske]

If there is no quality score, then it is in order in the string

[dshanske]

The problem is how would it do precedence in its current implementation, at the least, it probably should only return the activitypub version if it is only that mime type, not multiple ones.

[pfefferle]

Hey @ngm after a short quick check, I do not think that it is a problem of this plugin or of precedence. XRay is explicitly looking for an as2 file (https://github.com/aaronpk/XRay/blob/fb3af995a68598cf1ca05498fea42a5a43a13bed/lib/XRay/Formats/HTML.php#L110) with the correct Accept headers, so if you do not want XRay to parse the files, I would recommend to file an issue on the XRay repo.

[pfefferle]

@dshanske because XRay is searching for an as2 file it would be a hack to not deliver that searched file and it might cause other issues and side effects.

[dshanske]

@pfefferle Look at https://github.com/aaronpk/XRay/blob/master/lib/XRay/Fetcher.php#L79

[dshanske]

You are correct that it is, but it is looking for multiple items...your code always returns activity+json even if something else is in the accept string.

[dshanske]

So no other plugin could trigger, even if we wanted the mf2-feed plugin to return mf2+json, which comes before it in precedence.

[pfefferle]

Ok, haven’t seen the Fetcher on my first quick look. But to solve THIS problem, the mf2-feed plugin has to act in the same way (load only if there is no other accept header) and we have an even bigger issue. Then it would load none of both plugin and if the theme does not support mf2, we have an other issue. To really solve the issue, we would have to build some kind of registry where plugins can have a look, which accept headers are registered.

[pfefferle]

The only quick fix I can think of, is to look at the position of the header and if activity+json is not on the first place, it will reload the hook on a way lower priority to give other plugins the possibility to load first. I could also check if html has a higher prio and skip the response if so... But even then, in our case it will load before html and therefore before mf2.

[dshanske]

Yes, that's why I chimed in... I had been thinking about this problem independently

[ngm]

Thanks @pfefferle and @dshanske

Just for some extra info - the reason I noticed this was because https://2020.indieweb.org/london wasn't processing my RSVP. I'm guessing that it probably uses the XRay library :)

Also, https://micrometa.jkphl.is/ and http://pin13.net both appeared to parse the raw microformats fine with the plugin enabled - I guess that's perhaps because they don't do any special handling of activity+json, whereas XRay does?

[dshanske]

@ngm All correct. Xray supports activity streams and none of the other services do.

[bekopharm]

I run into this again today because webmention.io uses Xray (afair) and so far I have to disable the plugin every time on Post or the Webmention won't make it.

This usually means that the Post doesn't make it to e.g. Mastodon but I could live with that since the usual Webmention post types are not really of importance for the Fediverse anyway.

[pfefferle]

But isn't that an XRay issue then? If XRay prefers AP over Microformats, we should report the "problem" there... I have no idea how I could change that on the WordPress end...

/cc @aaronpk

[bekopharm]

Almost a year and frankly I still have no idea if this has or can be resolved by XRay / @aaronpk or @pfefferle.

This is still a killer for anyone who wants Webmention and ActivityPub.

So many people use XRay and not everybody knows it's XRay by manually parsing the answer and do another Mention-flight with disabled ActivityPub to make it work - or recognize that this is an issue with this plugin in combination with the target at all.

What about the idea to disable ActivityPub automatically during Send Mention (and especially when the target is Webmention.io - since this is the most common case for XRay)?

What about scheduling another doPings with disabled ActivityPub when the first Mention was not accepted (hard, I know, Webmention.io does queuing as it is described in the spec)?

I know it's not a clean solution but this is what the user has to do manually every time currently.

(Raised aaronpk/XRay#101)

[dshanske]

@bekopharm @ngm https://gist.github.com/dshanske/e2659241f1c8c064441737afbd89654c Try adding this gist somewhere, such as in a plugin or otherwise in the interim. Should work for your use case.