Posts: we aren't showing post-counts for Jetpack sites

[hoverduck]

We turned the post-counts filtering feature (i.e., show the post-counts in the status selector and don't show statuses with a zero post-count) off for jetpack sites because our current implementation only supports post-counts on public posts (because that's all that gets synced right now).

So we somehow need to enable this for jetpack sites as well. This could be done by implementing the post-counts endpoint in the jetpack plugin itself (versus the post_process_jetpack_response we are currently using). It's already implemented to some point, but isn't up-to-date or working correctly.

I noticed that the post-counts endpoint code in the jetpack plugin wasn't updated with the latest changes. So I started a branch to make the relevant changes here.

https://github.com/Automattic/jetpack/compare/update/reshape-post-count-response

Notice that I also removed the force: true, which effectively forces the public-api.wordpress.com to run the code against our shadow blog (which only contains public posts), versus forwarding on the request to the jetpack site's API. I also changed the shape of the post-counts response because for some reason cast_and_filter was not happy with the double-nesting of arrays. These jetpack changes also need to be coordinated with corresponding updates in wpcom, which I have set up in a phabricator instance here: D181-code

With these two changes together, I can get the correct data for post-counts for a jetpack site when I query the post-counts endpoint directly. Yay!

However, we don't query the post-counts endpoint directly, we do it via meta-expansion from the post-lists endpoint. And if I am obtaining post-counts data via meta expansion from the post-lists endpoint (i.e., /v1.1/sites/:site/posts?meta=counts), the meta query is run against the shadow blog, so we still only get partial data (i.e., only counts for published posts). I think this may be a limitation of how meta expanded queries are processed. I haven't figured that out yet.

So we either need to get the meta expanded query to hit the jetpack site directly, or we need to separate our API queries for posts and post-counts (at least for jetpack sites).

And of course all of this will need to be coordinated with corresponding changes to wpcom.js and jetpack.

[lancewillett]

I believe post counts work just fine for Jetpack sites in Calypso now. If there are any other bugs or improvements, let's open as new issues in GitHub.