Autoplay visually jumps to next object on fast streams due to incorrect key up handling

[forteus19]

Type

Game behaviour

Bug description

On fast streams (300 BPM), hitcircles appear not to get hit while playing back in the editor. It also seems to happen only on certain maps, like Put an end and isogu. Other 300 BPM streams maps don't appear to be affected.

Screenshots or videos

2023-01-22.08-44-11.mp4

Versions

2023.121.0-lazer
2023.123.0-lazer

Logs

runtime.log
session.log

[ekrctb]

Clicks are skipped because FrameStablePlayback is always set to false for an editor even when it is not seeking the time.

osu/osu.Game/Rulesets/UI/FrameStabilityContainer.cs

Lines 194 to 197 in d695214

	// when stability is disabled, we don't really care about accuracy.
	// looping over the replay will allow it to catch up and feed out the required values
	// for the current time.
	while ((newTime = ReplayInputHandler.SetFrameFromTime(proposedTime)) != proposedTime)

I think frame stability should be ensured while playback / mouse wheel scrolling and only use the unstable playback while explicitly seeking. Notes are still missed after a seek, but it is much less prevalent.
Ultimately, some other solutions are needed like recording judgement results in replays.

[peppy]

So, a few things:

Yes, this is fixed by frame stability, but only because frame stability allows replay frames to exist in a non-linear time scale. As in there are frames in the past, and frame stability will rewind to handle them correctly.

The base issue is that hitobjects are closer than 50 ms in these beatmaps, but auto generator adds a 50 ms minimum window for key-down-to-key-up:

osu/osu.Game.Rulesets.Osu/Replays/OsuAutoGenerator.cs

Line 293 in e182acf

double hEndTime = h.GetEndTime() + KEY_UP_DELAY;

I'd argue that this should be solved by fixing the auto generator to infer the next object and not hold for as long, or just reducing the key down period altogether to a value which will not cause issues like this.

Side note that DelayedMovements in autogenerator is currently never used and a dead code path...

Attempted solution here but why it doesn't work is explained in the commit message.