Buffered replays create a replay without an error occuring

[billyvg]

If you query for replay_type:buffer count_errors:0 you would expect to see no replays, but we have a lot: https://sentry.sentry.io/replays/?project=11276&query=replay_type%3Abuffer+count_errors%3A0

It looks like for buffered sessions, if a new session starts, we end up flushing and creating a replay.

[mydea]

hmm is this still revelant? Replay type would now be buffer, so we'd need to check for that instead, which seems to return nothing for me? 🤔

[billyvg]

Oops, I updated the query to be buffer, and we have a couple https://sentry.sentry.io/replays/?project=11276&query=replay_type%3Abuffer+count_errors%3A0

Seems firefox only?

[mwdiaz]

We have Sentry configured to only capture replays on errors and after upgrading to 7.52.1 yesterday (but not changing our configuration related to replays), we're now seeing tons of replays with no errors and subsequently we've burned through nearly our entire monthly allotment of Replays in a little over 24 hours (~20K and counting). It does not appear to be browser-specific.

Reading the release notes, I assume this was introduced in 7.50.0 (where error-based sampling behavior was changed), and this assumption seems to hold since I'm unable to reproduce the issue once I downgrade to 7.49.0. Unfortunately, that isn't really a viable option for us since we're also relying on #7967 which was in 7.50.0 as well.

[billyvg]

We have Sentry configured to only capture replays on errors and after upgrading to 7.52.1 yesterday (but not changing our configuration related to replays), we're now seeing tons of replays with no errors and subsequently we've burned through nearly our entire monthly allotment of Replays in a little over 24 hours (~20K and counting). It does not appear to be browser-specific.

Reading the release notes, I assume this was introduced in 7.50.0 (where error-based sampling behavior was changed), and this assumption seems to hold since I'm unable to reproduce the issue once I downgrade to 7.49.0. Unfortunately, that isn't really a viable option for us since we're also relying on #7967 which was in 7.50.0 as well.

Sorry about that, can you send in a support ticket so that we can get you squared away when we get this fixed? Thanks!

[billyvg]

@mwdiaz Can you confirm if this happens if you have

replaysSessionSampleRate: 0,
replaysOnErrorSampleRate: 1.0,

[mwdiaz]

@billyvg correct, those are precisely the settings we're using.

[billyvg]

@mwdiaz This should now be fixed in 7.53.1, can you upgrade and confirm? Thanks!

[mwdiaz]

Hi @billyvg, I deployed 7.53.1 to our two frontend web codebases about 45 minutes ago and it seems that things are working as expected now. It's a bit tough to tell since our quota is exceeded and I'm relying on the dropped replay count on the Stats page, so I can't confirm with 100% certainty (at least as far as I know?) that these dropped replays all actually contain errors, but so far in the 1:00-2:00pm window we've only seen ~200 dropped replays whereas the previous periods are more in the 2-3K range. A few hundred per hour is within the realm of what I would expect to see when the replays are correctly only being captured on errors.

I am working with our account rep to get some gifted quota to cover the rest of our usage period, and once I give a bit more time for old sessions to age out and clients to pick up the new versions, I'll let her know that things look fixed.