RFC: INP Collection Strategy in Browser SDKs

[0Calories]

I'm currently working on a proof of concept for collecting INP (Interaction to Next Paint) from the browser SDKs. I'd like to share with you all my current proposed solution for implementing this, but there are also some nuances with the implementation that I'd like to get some opinions on for how to deal with them.

Current Proposal

• Every time an element is clicked on a page, we create an interaction transaction. These transactions automatically have an interaction span that is provided by the browser API.
  

An example of what one of these transactions with an interaction span looks like

• This transaction is not immediately sent to Ingest, it is stored in memory on the browser.
• Each time an interaction transaction is detected with a slower interaction span, it replaces the current stored transaction. This is because INP is classified as the worst interaction on a specific page.
• When the user navigates away to a new page, this cached transaction is sent to Sentry with INP attached and cleared from memory.

Problems With This Approach

• In order for this to work, we need to send another transaction after the user navigates away from the page, which inevitably will impact transaction quota
• If you click on an element that updates the current page's query params via browserHistory.push, the SDK currently creates a new navigation transaction. In this case, an interaction span will still be added to the navigation transaction, but since these transactions are IdleTransactions, they will automatically be sent to Ingest after they finish.
• If a navigation transaction is detected as the 'worst' transaction in a page, it will have to be sent again once the user navigates to a new page. This will result in this same transaction being sent twice, but the second time will have INP attached.

Kapture.2023-04-05.at.14.24.00.mp4

Each of these tab clicks are classified as navigation transactions

Proposed Solutions to the Above Problems

• If the query parameters change but not the actual page, we should create an interaction transaction instead of a navigation. The problem with this is that it may affect customer's queries and the way they use the UI in the Performance product, since the TPM of these transactions will change.

Would love to hear if anyone has better ideas for solutions to the above problems. In the meantime I will continue to work on the PoC as described, but these problems will need to eventually be addressed!

[gggritso]

No suggestions yet, but two questions:

this cached transaction is sent to Sentry with INP attached

I'm curious about this part (and everything before it), it seems like this overlaps pretty heavily with web-vitals built-in onINP callback. Am I misunderstanding? The current proposal logic is to listen to all interactions, create temporary transactions, keep the longest one and report it when the page changes. Is that different from what onINP does under the hood?

which inevitably will impact transaction quota

Does this double the transaction usage, since there's an INP for every page load, or are there thresholds that prevent INP from being reported in some cases?

[0Calories]

Great questions @gggritso ,

You're right about the similarities with onINP, but there are some key distinctions that make this Sentry specific:

onINP only tracks interaction entries, which contain very limited data (see PerformanceEventTiming spec). My proposed implementation aims to Sentrify these interactions by creating an entire transaction for them. This transaction will contain the interaction as a span, along with any operations sequentially triggered by the interaction. This will not only provide more context around what happens after interactions, but we'll also be able to display INP on the timeline, similar to how we do it for our other web vitals.

onINP also includes some extra logic around BFCache which we don't use in our SDK and conflicts with the behaviour we have.

Does this double the transaction usage?

Yep. This is certainly not ideal, I imagine at the very least we should sample them at a lower rate compared to pageload and navigation, and maybe have some guardrails around sending low quality transactions (i.e a transaction with no spans), but this will probably require further brainstorming and perhaps discussion with DS.

[gggritso]

@0Calories gotcha, thanks, that makes sense!

I think long-term you might get some resistance to this approach. One reason is that (as far as I can tell) we have a top-level goal of having highly accurate metrics. Your approach is tricky firstly because INP becomes special, it's the one web vital that we don't collect in a standard way. I think for all other vitals we defer to the web-vitals library, right? It's tricky secondly because it'll definitely need its own sampling rules, which is a little at odds with DS. It's tricky thirdly because people are pretty sensitive to transaction name pricing. So, we might not be able to deliver accurate INP this way.

Maybe that's okay? Might be worth a conversation with Product!

Two loose ideas, in case any of this is helpful/useful:

• maybe it's possible to work with Ingest to make a special transaction type that is always sampled at 0%. This way INP measurements can come in and get indexed along with the most recent pageload transaction, but not use up any volume
• INP might get easier with the incoming trace model that has much longer-lived tracing. Unsure, though!

[0Calories]

@gggritso

We actually have our own implementations for the other vitals, based on what is in the web-vitals library, but we don't have it as an actual dependency. The problem is that INP is a special web vital, so it can't be treated otherwise 😛 . It's the sole web vital that is calculated after a page is done loading, so we can't simply calculate it and attach it to an existing transaction since they all are finalized after pageload.

The underlying mechanism for calculating INP is still the same as what is used by web-vitals, we just need a medium to transport the calculated metric.

I agree with your other points about sampling and pricing, that's certainly something I can discuss with Product!

I also had the same idea as you regarding making this transaction special and not count towards quota, but I'd imagine it's not entirely feasible since it still costs us money to process these transactions 😔

[philipphofmann]

On Mobile, we decided against implementing INP at the moment because we can't detect when the OS is drawing a new frame with new content, but also because of the limitations of our current lifespan of transactions. Keeping transactions in memory, waiting until a user navigates to the next screen, and then sending an almost empty transaction with only INP seems wrong. You can read more about our discussion on Notion. Furthermore, we have slow and frozen frames on Mobile, which work for every transaction. On React Native, we also have stall tracking, which I think you could maybe port to JavaScript browser.

Maybe you already noticed that the web-vitals library has support for INP. Also worth mentioning is that INP doesn't always send the slowest interaction on pages with plenty of interactions. For these, INP should pick the 98th or 99th percentile, see INP docs.