Note playing ends almost instantly

[gildurao]

Hello,

I tried using your library today (huge thanks btw for providing a clean way to just play sounds off .sf2 files) and when playing a note, it almost ends instantly (less than a second). Setting a duration for the note also does not fix this. Same thing happens when playing a chord. I wonder if this is because of the .sf2 file itself or because I'm running in on Ubuntu.

Python version is 3.9.5

Here's the code I used:

import sf2_loader as sf

sf2 = loader = sf.sf2_loader(r'pathToSf2File/piano.sf2')

sf2.play_note('C5', duration=10)

when running I do get a FluidSynth warning, don't know if it's related:

fluidsynth: warning: SDL2 not initialized, SDL2 audio driver won't be usable

Let me know if you need more info. Thanks again.

[gildurao]

Just tested in a Windows machine and the same thing happens without the SDL2 warning being logged.

[Rainbow-Dreamer]

Thanks for your feedback. This project is currently only tested on windows, and I am going to test it on Linux and MacOS recently. According to the FluidSynth warning, this might be some libraries in FluidSynth not fully compatible with Ubuntu. If your code isn't working on windows, I think that should be the problem of the .sf2 file you are using.

[Rainbow-Dreamer]

Could you provide a link to download this .sf2 file and I will test it on windows now?

[gildurao]

Yeah, it's right here via this link: https://filesend.standardnotes.com/send/1HEhj9ozmjuSSEfJ9p3v#MTFhNzEzNWJmMDVkMzhhZTM0MmM5

I've tried with 2 .sf2 files and both of them behave the same way

[Rainbow-Dreamer]

Thanks, I have tested both of them on windows 10, both works with no problems, the play_note function with duration 10 will play for 10 seconds, and with short duration like 1/8 will ends very quickly. Do you see any warnings when install this package using pip?
Or maybe it is because the python version you are using is old, I recommend using >= 3.7.

[gildurao]

Python version is 3.9.5, I don't think I had any warnings when installing with pip

EDIT: just reinstalled the package and no warnings showed up

[Rainbow-Dreamer]

I developed this package in python 3.7.9, so I think maybe some required python packages of this project is not very compatible for newer python versions. But now I have installed python 3.9.5, pip install this package, and then do the same test, it still works for both sf2 files. So maybe tries to upgrade numpy to the newest version via pip because numpy is mainly used as handling raw audio data in this package.

[gildurao]

I'm already with the most recent version of numpy unfortunately (1.21.2). Do you use ffmpeg or avconv? I'm using ffmpeg. If you rely on avconv that could potentially be the issue

[Rainbow-Dreamer]

That's a good reminder, I'm using pydub for audio rendering and exporting in this package, and pydub requires ffmpeg or libav to be installed correctly. Maybe it needs some configuration for the system environment variables, add the path to ffmpeg/bin to it, which is on windows, on Linux it might be something different. You can refer to this link which is pydub's github main page readme to see how to do it.
(I will add this as reminder to readme of this project)

[gildurao]

I believe I installed FFmpeg correctly in Windows since I can use it from the command prompt (on Linux I'm sure it is correctly installed because I had already used it). I don't think I had libav installed. I will recheck these steps and the link you provided and will reply promptly.

[Rainbow-Dreamer]

That's good, wait for your update :)

[gildurao]

@Rainbow-Dreamer everything is installed properly I think. I have pydub installed already so I assume libav exists somewhere too. Is there anything I can write on the command prompt to test libav?

[gildurao]

Just tested on another Windows machine, did all the necessary configs (installing ffmpeg, numpy, etc.), same issue persists.

[Rainbow-Dreamer]

Installed pydub does not install libav or ffmpeg automatically, the pydub readme says you need to install ffmpeg or libav manually besides of pip install pydub, and add the /bin folder of ffmpeg or libav to system environment variable PATH (on windows), and according to my memory, you will need to restart the computer then to make this PATH configuration works on some windows versions. If you are using ffmpeg, then I think you don't need to install libav, because pydub only requires one of them. I am using ffmpeg and there are no libav installed on my computer, and it works due to testing.
If there are no warnings from pydub says something like RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work pop up in IDE's shell (or cmd/terminal if you are using it) when you run the testing code for playing notes, this might not be a pydub's problem. If there is, I suggest put the ffmpeg folder in C:/, and put C:\\ffmpeg\\bin in PATH and then restart the computer, there are many issues about pydub not recognizing installed ffmpeg path actually except it is on C:/. Hope this helps you.

[gildurao]

I will try restarting the computer and see if that works, but I don't have any warnings whatsoever @Rainbow-Dreamer

[gildurao]

Did not work. I wonder if it's the ffmpeg version causing issues?

@Rainbow-Dreamer what's your output of ffmpeg -version ?

ffmpeg version 2021-09-01-git-c500dc7cca-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.3.0 (Rev5, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil      57.  4.101 / 57.  4.101
libavcodec     59.  6.100 / 59.  6.100
libavformat    59.  4.102 / 59.  4.102
libavdevice    59.  0.101 / 59.  0.101
libavfilter     8.  7.100 /  8.  7.100
libswscale      6.  0.100 /  6.  0.100
libswresample   4.  0.100 /  4.  0.100
libpostproc    56.  0.100 / 56.  0.100

[Rainbow-Dreamer]

This is mine.

ffmpeg version 4.4-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 10.2.0 (Rev6, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libglslang --enable-vulkan --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil      56. 70.100 / 56. 70.100
libavcodec     58.134.100 / 58.134.100
libavformat    58. 76.100 / 58. 76.100
libavdevice    58. 13.100 / 58. 13.100
libavfilter     7.110.100 /  7.110.100
libswscale      5.  9.100 /  5.  9.100
libswresample   3.  9.100 /  3.  9.100
libpostproc    55.  9.100 / 55.  9.100

[Rainbow-Dreamer]

This might be the problem, but I think you can try export_note function to export the audio file for a note using the .sf2 file, if it is successfully exported and resulted in correct content, I think it is not pydub's problem.

[gildurao]

@Rainbow-Dreamer export_note behaves correctly from what I tested. I tried exporting notes with several durations (1 second, 2, 10, 15, 20, 30) and the generated files were all with the right sound and duration.

[Rainbow-Dreamer]

Good to hear that, so this shows there are no problems with fluidsynth and pydub on windows when using this package, so this should be the problem of simpleaudio, which is the python package used to play audio data handled by pydub in this project. In pydub's readme, it is strongly recommended to use simpleaudio to play pydub's audio objects (which has not been exported yet), so I add simpleaudio to the requirements of this project when pip install, and it should works well on windows. Maybe I can add a global function for users to choose which python package used to play pydub's audio objects. You can have a look at this link which shows some python packages that could be used to play audio in pydub's readme.
For your current issues, could you try to change the read_sf2.py file line 10 from pydub.playback import _play_with_simpleaudio as play_sound to from pydub.playback import play as play_sound in read_sf2 subpackage (and read_sf2_32bit as well) in sf2_loader package in Lib/site-packages in your python installation path? And then uninstall simpleaudio via pip. It will try to play the audio using other python packages, you could try to install pyaudio first.
(you will also need to delete the line import simpleaudio in read_sf2.py, and comment all the lines starts with simpleaudio)
I am thinking of switching to other methods for playback, maybe let pydub export the audio file first and then use python packages that could play audio files to play it.

[gildurao]

Getting some compilation issues related to fluidsynth now 🤔

Traceback (most recent call last):
  File "C:\Users\GilDurao\Documents\GitHub\flutterpy-lmusic\flutterpy_lmusic\lib\yourscript.py", line 8, in <module>
    sf2 = loader = sf.sf2_loader(str(filePath))
  File "C:\Users\GilDurao\AppData\Local\Programs\Python\Python39\lib\site-packages\sf2_loader\read_sf2\read_sf2.py", line 180, in __init__
    self.synth = fluidsynth.Synth()
AttributeError: module 'fluidsynth' has no attribute 'Synth'

Any ideas?

Yeah, for now I will use the export functions and then playback the audio file. Will definitely keep assisting you here if you want.

[Rainbow-Dreamer]

Thanks for your support, I am glad to hear if there are any other problems when using this package from you in the future.
The simpleaudio package should work as playing audio immediately on windows (which is for me), but maybe it is not that compatible with some different windows versions from what I am using.
For the compilation issues related to fluidsynth, could you post the codes you are running resulted in this?
(and you can undo the changes you have done to the read_sf2.py)

[Rainbow-Dreamer]

By the way, just to mention that you don't need to write sf2 = loader = sf.sf2_loader, because loader = sf.sf2_loader is just an example of assigning the sf2_loader object to some variables in readme, you can directly write sf2 = sf.sf2_loader and it will works fine.
And for the simpleaudio issues, I think you can try to replace your ffmpeg folder with my version of ffmpeg, which could be downloaded as a rar file at the release page of one of my projects, the link is here, and check if this folder's bin path is in PATH, and restart the computer. Since different ffmpeg versions might result in different audio raw data that pydub objects contains, simpleaudio might works differently with it.

[gildurao]

Thanks for your support, I am glad to hear if there are any other problems when using this package from you in the future.
The simpleaudio package should work as playing audio immediately on windows (which is for me), but maybe it is not that compatible with some different windows versions from what I am using.
For the compilation issues related to fluidsynth, could you post the codes you are running resulted in this?
(and you can undo the changes you have done to the read_sf2.py)

The fluidsynth issues happened when I removed simpleaudio and uninstalled the package. So reverting that should fix it.

[gildurao]

By the way, just to mention that you don't need to write sf2 = loader = sf.sf2_loader, because loader = sf.sf2_loader is just an example of assigning the sf2_loader object to some variables in readme, you can directly write sf2 = sf.sf2_loader and it will works fine.
And for the simpleaudio issues, I think you can try to replace your ffmpeg folder with my version of ffmpeg, which could be downloaded as a rar file at the release page of one of my projects, the link is here, and check if this folder's bin path is in PATH, and restart the computer. Since different ffmpeg versions might result in different audio raw data that pydub objects contains, simpleaudio might works differently with it.

Will try this tomorrow and will let you know. Thanks for your patience and guidance!

[Rainbow-Dreamer]

You're welcome :)

[gildurao]

Switching to your ffmpeg version did not solve issue @Rainbow-Dreamer . I am also going to take a look at your musicpy package on its own, it may be better suited for my use case. Basically what I am trying to achieve is generating music via Lindenmayer Systems. You generate a string according to symbolic rules and you parse the final result to music.

EDIT: Interesting, I tried to use sample code of musicpy and I cannot hear any sound at all.

[Rainbow-Dreamer]

That sounds very cool, and I'm glad that you choose musicpy, since musicpy is a package that is very easy to implement mathematical and statistical algorithms in python on music theory stuffs, I recommended you to read the wiki I wrote to get full details and usages of musicpy. Thanks for your support :)

[Rainbow-Dreamer]

Hi, could you try to play some notes using this package on another computer that does not use a Focusrite Audio interface and could play a mp3 file using pygame normally (on windows)? (And don't forget to set up ffmpeg for pydub) If there are no issues with the note playing, we can make sure that this issue is only a conflict of audio driver and python itself.
By the way, the Linux and macOS compatible version of sf2_loader are coming out, you can see the readme to download them and see the necessary configuration steps for Linux and macOS.

[gildurao]

Hi, could you try to play some notes using this package on another computer that does not use a Focusrite Audio interface and could play a mp3 file using pygame normally (on windows)? (And don't forget to set up ffmpeg for pydub) If there are no issues with the note playing, we can make sure that this issue is only a conflict of audio driver and python itself.
By the way, the Linux and macOS compatible version of sf2_loader are coming out, you can see the readme to download them and see the necessary configuration steps for Linux and macOS.

Will try playing the note without having my Focusrite interface plugged in. I will give you feedback about sf2_loader on Linux for sure. Unfortunately, don't have access to a macOS machine.

[Rainbow-Dreamer]

It's fine, just mention it in case you want to use it on a Mac :)

[Rainbow-Dreamer]

@gildurao By the way, you actually don't need to construct a C major chord in musicpy like what you did in another issue, there are many convenient and advanced syntax for building complex chords in musicpy, you will see them in basic syntax sections of the wiki.
Here I will provide 3 ways of constructing a C major chord using musicpy as examples:

# using chord name
myGChord = sf.mp.C('C', 5, duration=1, volume=100)
# construct a C major chord starts at C5 with all notes have duration of 1 bar and volume of 100
myGChord = sf.mp.C('Cmaj7', 5, duration=1, volume=100) # build a Cmaj7 chord

# using a string of notes
myGChord = sf.mp.chord('C5, E5, G5', duration=1, volume=100)
myGChord = sf.mp.chord('c5, e5, g5', duration=1, volume=100) # the note names could also be lowercase
myGChord = sf.mp.chord('C, E, G', duration=1, volume=100, rootpitch=5) # if no octave numbers are specified,
# then it will automatically builds up with smallest intervals possible
myGChord = sf.mp.chord('C5[1;0;100], E5[1;0;100], G5[1;0;100]') # this is an advanced syntax,
# I also recommend "translate" syntax which is even more convenient than this

# using a list of notes (could be note instances or strings representing notes)
myGChord = sf.mp.chord(['C5', 'E5', 'G5'], duration=1, volume=100)
myGChord = sf.mp.chord([sf.mp.N('C5'), sf.mp.N('E5'), sf.mp.N('G5')], duration=1, volume=100)

[gildurao]

@gildurao By the way, you actually don't need to construct a C major chord in musicpy like what you did in another issue, there are many convenient and advanced syntax for building complex chords in musicpy, you will see them in basic syntax sections of the wiki.
Here I will provide 3 ways of constructing a C major chord using musicpy as examples:

# using chord name
myGChord = sf.mp.C('C', 5, duration=1, volume=100)
# construct a C major chord starts at C5 with all notes have duration of 1 bar and volume of 100
myGChord = sf.mp.C('Cmaj7', 5, duration=1, volume=100) # build a Cmaj7 chord

# using a string of notes
myGChord = sf.mp.chord('C5, E5, G5', duration=1, volume=100)
myGChord = sf.mp.chord('c5, e5, g5', duration=1, volume=100) # the note names could also be lowercase
myGChord = sf.mp.chord('C, E, G', duration=1, volume=100, rootpitch=5) # if no octave numbers are specified,
# then it will automatically builds up with smallest intervals possible
myGChord = sf.mp.chord('C5[1;0;100], E5[1;0;100], G5[1;0;100]') # this is an advanced syntax,
# I also recommend "translate" syntax which is even more convenient than this

# using a list of notes (could be note instances or strings representing notes)
myGChord = sf.mp.chord(['C5', 'E5', 'G5'], duration=1, volume=100)
myGChord = sf.mp.chord([sf.mp.N('C5'), sf.mp.N('E5'), sf.mp.N('G5')], duration=1, volume=100)

Thanks for the examples! I just did that small example to understand how your library worked 😁

[Rainbow-Dreamer]

You're welcome :)

[Rainbow-Dreamer]

Well, it's been a long time, I think I'm gonna close this issue, if you have any issues of this package on Linux or without having Focusrite interface plugged in, welcome to open a new issue, thanks :)