RFC: External Analytics Support

[gabestein]

tl;dr — we'd love your feedback on retiring our analytics platform and instead supporting a limited number of external analytics services (with all privacy and compliance options enabled).

Since the beginning of PubPub, we've taken a strong (some might say extreme) stance on user privacy. One side-effect of this stance has been that we've never allowed Communities to use third-party analytics packages like Google Analytics, because they often are not GDPR/CCPA compliant, and even when they are, use technologies that have the potential to be abused by the companies that maintain them. Instead, we've maintained our own analytics package that gives Communities insights into their community's web performance.

But this approach has some significant downsides:

• One of the most common requests we receive remains using Google Analytics and other third-party services on PubPub, and we know some people's lives are made more difficult by having to use a different analytics platform than they're used to, or use in other parts of their organization.
• Running our own analytics platform is very expensive, both in terms of outright cost and time spent maintaining it, and unless we were able to devote significant engineering resources towards it, the cost will scale poorly as PubPub continues to grow.
• Though our platform provides common web metrics, those metrics were mostly designed for e-commerce and advertising. We'd much rather focus our analytics efforts on developing new metrics that more effectively and specifically measure the impact of knowledge produced and consumed.
• It makes the app less self-hostable, because the analytics platform relies on Heap and Redshift, which is proprietary and expensive.

So, we'd like your feedback on an idea we've been thinking a little about internally. We are considering retiring our analytics platform and replacing it with the ability to use Google Analytics or, preferably, privacy-respecting next-gen services like Fathom Analytics. We would, of course, make historical data available for consumption and download for a significant period of time after making this decision. And, to continue to protect user privacy, we'd enable analytics options in a controlled way, automatically turning the most stringent GDPR/CCPA-compliant, cookie-less, no-cross-device tracking options available for any supported platform — and not supporting platforms that do not have those options. Since we last considered supporting Google Analytics in particular, the platform has made considerable strides towards making privacy-respecting, compliant options available, largely due to regulatory pressure in the EU.

The upsides are that it would give communities more flexibility and choice in analytics platforms, save the PubPub team considerable resources and time, decrease the automatic focus on traditional web analytics for impact measurement on PubPub, and allow us to focus our impact measurement efforts on novel metrics tailored for knowledge, rather than e-commerce and advertising. In our ideal world, taking approach would give us the space, resources, and focus to work with other knowledge organizations on developing an open, privacy-respective, compliant analytics platform, without sacrificing community's needs to understand their impact with existing tools.

But, of course, there would be downsides. The analytics platforms wouldn't be quite as tailored to PubPub as our current one is. Communities would not have analytics by default, making PubPub slightly less end-to-end, and requiring admins to do a bit more work to understand community performance. And, of course, although their practices have improved considerably in recent years, we would be tacitly endorsing the use of platforms that have historically encouraged some of the worst abuses of surveillance capitalism, not to mention reliance on often-irrelevant metrics for judging success.

Thanks for reading! We'd love your comments, feedback, and ideas.

[kyledenlinger]

Great questions, Gabe.

My lens is mostly EdTech, not as much publishing/scholcomm. So take that into account.

I'm sympathetic to all of the reasons you cite for wanting to retire the current analytics platform, but PubPub's unambiguous commitment to privacy is one of the primary reasons I'm supporting it on my campus. Telling students and faculty without hesitation that PubPub is not going to track you or sell your data to advertisers isn't something I can say about most tools. I worry that creating a more fractured analytics environment by allowing, at the community level, the integration of Google Analytics et al. -- as controlled as those integrations might be -- would introduce enough ambiguity so as to require a community-by-community privacy assessment, rather than what I consider to be the blanket assurance we have now.

Are there other options? Could analytics (either PubPub's current platform or one of a few vetted privacy-forward options) be a premium feature for KFG members? That seems reasonable and more scalable.

[writingpoli]

Rather than get embroiled in Google's unlovable ecosystem, what about something like Cloudflare Web Analytics?

I echo @kyledenlinger's comment on maintaining the attractiveness of PubPub's privacy focus, which is one reason I'd like to stay away from Google altogether.

[gabestein]

Thanks to both of you for the feedback. I'd like to dig into your opposition to GA a little bit (which I share, but I'm trying to argue on behalf of the many users who want it, so if you'll indulge me...). Is it mostly based on specific features, general distrust of Google, something else?

I ask because if we were to do something like this, we would still have extremely high standards for services we would support — no personal or device data collection or fingerprinting, no data sharing or selling, and we actually want to go beyond what we offer now in eliminating cookies altogether so we can get rid of the banner. So, services like Fathom, Cloudflare (thanks for the recommendation!), etc. And to be clear, we'd never allow a user to add their own tracking pixels. They'd enter their required IDs and we would run the code itself, with all the strictest options that accomplish the above set.

The problem is that I think, parsing their docs, it's at least technically possible to accomplish all of the above with the new version of GA. So, if we made the shift and laid out our standards, it might be hard to not support GA, especially with it being the main one folks request. If we upheld those standards but allowed GA, would it still introduce that amgibuity you mentioned, Kyle, and if so, why? And are there any services which would not introduce any ambiguity?

Some specific comments:

Cloudflare Web Analytics

Thanks, this is a great option to send many people to. I do think it's a little limited for many advanced uses, though.

PubPub is not going to track you or sell your data to advertisers

As I hope I made clear above, no matter what we did, we would make sure this was always still the case.

Could analytics (either PubPub's current platform or one of a few vetted privacy-forward options) be a premium feature for KFG members

We're hesitant to make a feature we've always offered to everyone members-only. And sadly, it wouldn't dent the cost very much, especially when it comes to maintenance. It's more likely that we'd always offer a much more basic analytics service to everyone, and point folks who need more advanced ones to third parties (with all the restrictions above).

[writingpoli]

For me, it's a general distrust of Google and not knowing what they'll change/do in the future. One question for y'all to kick around, too, is whether limiting what users are accustomed to getting from GA will satisfy them or if you're still going to get bombarded with requests for more GA-provided data (device data, etc.). Just something to game out amongst yourselves. Overall, I trust y'all to keep PubPub secure and adhere to your/our privacy principles . . . and I'd love to get rid of the pop-up, too. Those things are like the new banner ads on the web. And I also hate for needed, limited resources to be spent maintaining such a small piece of the platform -- though I do like being able to see how/if we're growing readership. :)

Thanks, Gabe!

EDIT: 'Cause it's not just about PubPub sharing or selling data. It's also about communities selling or sharing that data.

[gabestein]

Thanks, it's definitely worth gaming out internally.

To be clear: we would never allow communities to implement tools that would collect data they could use to identify users (and thus sell), either. And I do think we would try to find a way to provide a more basic set of analytics inexpensively, as has been suggested, so you'd still be able to get a basic sense of metrics.

[simonleandergrimm]

As discussed via e-mail, I'd support an option to use external analytics platforms!

[tosteiner]

Hi all, just very briefly, I'd also be in support of external analytics, but also want to voice my clear preference of open-source, privacy-friendly alternatives to the dreaded Google Analytics such as Matomo (formerly known as Piwik, see 1, 2), which can be tailored to one's own needs with regards to privacy and would fit squarely with the our general committment to open, values-led infrastructures ...

[simonleandergrimm]

Just to be sure, would different people be able to use different external analytics? If so, and if there are no implications for others when some use Google Analytics, I'd think there should be the option to use Google.

[gabestein]

We would likely consider supporting a range of services, with a preference for open and privacy-respective ones. But again, as above, I want to reiterate two key points:

1. We would not offer support for analytics tools that do not meet our standards (as spelled out in our terms and privacy policy).
2. We would not allow communities to add their own tracking code; instead, we would ask users to enter the IDs needed for their chosen service and add tracking code for each tool restricted to meet no. 1 This is why I feel confident that, if we did decide to support GA, we could do it without compromising on our principles. But it does mean that our support for it, and frankly all external services, even the more open ones, would be limited.

[gabestein]

Thanks again for all your feedback. It's been a while, but we're excited to announce a follow-up that, we think, satisfies base user needs in an extremely privacy-friendly way, is much more cost-effective than our current solution, and also provides options for communities who need them.

Long story short, we're retiring our current, extremely expensive analytics setup and replacing it with three new options:

1. Cookie and fingerprint-free basic analytics. Thanks to the hard work of @tefkah, we've developed a new analytics pipeline that we fully own and control, that's much more cost effective, and that uses no cookies or any other fingerprinting or PII-based tracking at all. Instead, it uses the Simple Analytics strategy for unique visitors and simple client-side event tracking for everything else. We'll be rolling this out automatically to PubPub in the next few weeks, starting by providing two dashboards -- one for new data and one for old -- and then ultimately migrating old data into the new format. The new data won't be perfectly apples-to-apples, but it will be close, and most of the features of the current dashboard will be present in the new ones. Best of all, this change will eliminate cookie banners for those using it, because there are no cookies and no GDPR/CCPA-violating third-party processing.

2. Simple Analytics. We'll enable (and encourage) communities who can to use Simple Analytics if they have needs that go beyond our basic offering. Simple Analytics is an awesome, privacy-first analytics solution, and using it will not require a cookie banner.

3. Google analytics. Communities will be able to use Google Analytics by adding their ID into settings. Using GA will require displaying the cookie banner, and we've taken all the precautions available to us to limit the ability of GA to do invasive tracking on PubPub, including the cookie banner and implementing DoNotTrack and limiting IP address-based tracking.

4. Future Options. Behind the scenes, we're using the excellent and popular https://getanalytics.io/ library for all analytics services. As more plugins for privacy-respecting options become available, we'll be able to quickly add them as analytics options for communities who want them.

[writingpoli]

Thank you for the update!