New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Massive sound delay and jitter using sounddevice backend #2065

Open
stfnrpplngr opened this Issue Oct 15, 2018 · 8 comments

Comments

Projects
None yet
3 participants
@stfnrpplngr
Contributor

stfnrpplngr commented Oct 15, 2018

I have already posted in the psychopy forum: Massive Sound lag using PsychoPy. I am stuck. Trying to activate exclusive mode as described in the documentation of sounddevice results in PaErrorCode -9996.

Traceback (most recent call last):
  File "D:\Experimente\task1.py", line 85, in <module>
    sound_1 = sound.Sound('A', secs=-1, stereo=True)
  File "C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\sound\backend_sounddevice.py", line 296, in __init__
    hamming=self.hamming)
  File "C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\sound\backend_sounddevice.py", line 343, in setSound
    blockSize=self.blockSize)
  File "C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\sound\backend_sounddevice.py", line 84, in getStream
    blockSize=blockSize)
  File "C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\sound\backend_sounddevice.py", line 128, in _getStream
    device=defaultOutput)
  File "C:\Program Files (x86)\PsychoPy3\lib\site-packages\psychopy\sound\backend_sounddevice.py", line 159, in __init__
    callback=self.callback)
  File "C:\Program Files (x86)\PsychoPy3\lib\site-packages\sounddevice.py", line 1373, in __init__
    **_remove_self(locals()))
  File "C:\Program Files (x86)\PsychoPy3\lib\site-packages\sounddevice.py", line 779, in __init__
    'Error opening {0}'.format(self.__class__.__name__))
  File "C:\Program Files (x86)\PsychoPy3\lib\site-packages\sounddevice.py", line 2571, in _check
    raise PortAudioError(errormsg, err)
sounddevice.PortAudioError: Error opening OutputStream: Invalid device [PaErrorCode -9996]

Do you have an idea on what is going wrong? Is there a bug? @peircej @mgeier @bastibe

@stfnrpplngr

This comment has been minimized.

Contributor

stfnrpplngr commented Oct 15, 2018

After intense testing, it appears that the problem is not the exclusive mode setting. We believe that the problem is related to the play() function of sounddevice. We reduced the delay to about 10 ms, but there was a jitter of about 8 ms. For our measurements this is unfortunatley not acceptable. Probably the variation in sound delay is introduced invoking the sound stream using play().

@mgeier

This comment has been minimized.

Contributor

mgeier commented Oct 15, 2018

AFAICT, the play() function isn't even used in PsychoPy. At least it doesn't appear in https://github.com/psychopy/psychopy/blob/b6f17b791170e33d839df1a4f21b0bd91b96406b/psychopy/sound/backend_sounddevice.py.

About the "Invalid device" error: this sounds similar: spatialaudio/python-sounddevice#52.

@stfnrpplngr

This comment has been minimized.

Contributor

stfnrpplngr commented Oct 24, 2018

I checked the sampling rates. In the Windows sound setting, system's default for the inbuilt speakers was 48000Hz. Changing that to 44100Hz did not change anything. For the Steinberg system the value cannot be changes but it preset to 44100Hz. But the field is grey, so I don't know whether this has any meaning. Its highest possible sampling rate is 192kHz.

In PsychoPy, I generate an 'A' with the inbuilt function sound.Sound('A'). Default sampling rate for this is 44100Hz. Playing around with different settings regarding driver and API, I ended up with various latencies but I was never as good as Psychtoolbox or Presentation. Since they are able to present a jitter-free acoustic stimulus I know it is possible.

I reached the best timing using the ASIO drivers. Unfortunately, I did not manage (unitl now) to work in WASAPI exclusive mode. Interestingly there appears to be a wandering of the signal. The latency starts high and is reduced with every loop. Then, it jumps back to the high latency value repeating the process.

That is where it starts using ASIO:
asio_wandering_sound_highlatency
After decreasing its latency with every loop (it needs 6 steps), that is where it ends before jumping to the high latency again:
asio_wandering_sound_lowlatency
It appears not as a simple random jitter, it is more systematic. Do you have any idea what could be the cause?

@stfnrpplngr

This comment has been minimized.

Contributor

stfnrpplngr commented Oct 25, 2018

Today, I tried to present the sound directly via sounddevice, intentionally not using the PsychoPy backend. As a sound I generated a numpy sine wave:

import sounddevice as sd
sound1 = np.sin([np.arange(1, 3000), np.arange(1, 3000)]).transpose()
sd.default.device = 'UR22mkII Windows WASAPI'
wasapi_exclusive = sd.WasapiSettings(exclusive=True)
#sd.default.extra_settings = wasapi_exclusive

I commented the change of the default settings to wasapi_exclusive, because the leads to the Invalid device [PaErrorCode -9996].
The sound is played using sd.play(sound1). In the following pictures, you can see how the sound delay decreases before it jumps back. There is a corridor of about 10 ms.

sound_delay_wasapi_sharedmode_start
sound_delay_wasapi_sharedmode_intermediate
sound_delay_wasapi_sharedmode_stop

@lindeloev

This comment has been minimized.

Contributor

lindeloev commented Oct 26, 2018

@stfnrpplngr, great that you are testing audio latency and jitter. This is certainly a point where PsychoPy could use some love, so feel free to test out more options and let us know if you find a solution.

Latency and jitter using the sounddevice module has been discussed a few times on Discourse:

You could try out some of those tips and tricks. Some suggest setting up a stream, and then passing sounds to that rather than using .play(). Also, the sounddevice documentation mention a latency argument.

As for the jitter you observe, it is likely caused by a buffer in your soundcard dispatching sounds every 10 ms or so on top of a latency of around 34 ms. Just guessing, though.

@stfnrpplngr

This comment has been minimized.

Contributor

stfnrpplngr commented Oct 26, 2018

By setting up a stream, I solved the problem with our previous stimulation setup. But this necessitates using the coder for experiment conception. I was trying to solve it using the builder, so that other lab members could draft their experiments without extensive python proficiency.

@lindeloev

This comment has been minimized.

Contributor

lindeloev commented Oct 26, 2018

@stfnrpplngr, could you be a bit more specific about what you did to succeed and perhaps even post the trigger/waveform image? Perhaps the PsychoPy team could implement this in Builder. I know this doesn't solve your problems here and now, but it would be immensely useful information to help move PsychoPy forward in this respect.

As a temporary workaround, you can create the Python code in Builder by pressing F5; then replace the relevant sections with the streaming solution. This, of course, is a one-way street (it won't feed back into your Builder file), so just do it before data collection, when all other aspects of your experiment are settled.

@mgeier

This comment has been minimized.

Contributor

mgeier commented Oct 26, 2018

The sounddevice integration of PsychoPy does not use the sd.play() function, it already uses a stream, which is good, see:

self._sdStream = sd.OutputStream(samplerate=self.sampleRate,
blocksize=self.blockSize,
latency='low',
device=device,
channels=self.channels,
callback=self.callback)

However, I don't exactly know the timing details when a sound is played.

Anyhow, independent of the audio backend, there is typically a trade-off between lowest possible latency and lowest possible jitter. If a sound is played as soon as possible, there will be jitter related to the used block sizes in the audio backend/drivers/hardware (as suggested above).

If you know the desired playback time in advance, you can avoid the jitter. You can get the current stream time with the time attribute of the stream. You can then add the appropriate offset to this time value and communicate to the callback function to start playback at that later time. The callback has the same clock information in its time parameter (more specifically time.outputBufferDacTime).

If you don't know the playback time in advance but still want to minimize jitter, you'll have to intentionally add some latency in order to make it possible that the callback can start the playback on time.

If you want to investigate this even further, you should have a look at the sounddevice spin-off project rtmixer: https://python-rtmixer.readthedocs.io/. This is still work-in-progress, but it tries to minimize the latency as far as possible by implementing the callback function in C (while the rest remains in Python).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment