-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Apple Silicon support #19
Comments
this seems to run, a bit faster, but way hotter. add the following after the each of the device definitions on line 274, 292, 375, 545, 665, 743
(i'd recommend to put that into a function in the long term, but i have no idea about ai systems...) i also use the nightly build of https://pytorch.org/get-started/locally/ but i am not sure wether thats needed. be aware that it gets really hot when cpu and gpu are running. istats said gpu temps up to 100 degrees c on gpu. so maybe have a tool to turn up the fans. it stil took 10 mins to run, and i dont know where i can find the generated content... btw. i dont know if the result is anything usable, because i cant find it yet... |
Re Apple Silicon, definitely something we welcome input/suggestions on. Re where results end up, they start in memory. For example saving to wav/mp3 check out #13 |
there is a problem
my "run.py" looks like this now:
it generated something, but i gotta test with other texts now. |
something is definitely off. The sound file is not usable in any way. But the code runs without errors and the length of the file does make sense. Could be something about conversion. But it might be a deeper problem as well. Maybe somebody else has an idea. |
maybe related to precision? can you try running with float32? |
Saw the same issue as you @ludos1978, but I don't see an output file like I do when I don't change the 'device' definition to mps.
If there's a way you get past this issue, lmk! |
@gkucsko : i dont really understand where you mean to try running it in float32? I did figure out how to write to a file if you meant that. @sagardesai90 : cant really describe it any better then i did in my script or in the error message. With the code below, the audio generates fine on cpu. However on mps it only wushes and cracks all the time.
|
i meant hacking the autocast setting ontop of generation.py, but was a bit of a random guess. hard without access to your setup. |
autocast does not seem to be available on mps. The code in the beginning of #22 is at least not working for me with pytorch 2.0.0 on a M2 Max. |
I am using a mac pro with m2. I put this code
This is the output info. No GPU being used. Careful, inference might be extremely slow! I don't have a computer with GPU. So I can't compare. But if delete this code |
@QKJIN just declaring a |
From my experience the time it takes to generate sounds varies a lot between running, because there is no fixed seed we can use (this would be a nice addition for testing and comparison), so the voices can be really slow or fast and thus take longer or shorter to generate. |
@ludos1978 Yes, it's right. I generate several sounds. The speeds are all different. |
I have been modifying the code a bit so i can use mps on different parts of the generation process. I actually added to generation.py to have fixed seeds (generates the same sound every time). The fixed seeds however do not generate the same sound if you change the devices.
i am a bit lazy in my free time to use a versioning tool, but if anybody wants to have a look at my code, it's attached. edit: i am actually not completely sure it's accelerated using mps, but it makes a difference when i switch the devices, so i assume it must change something under the hood. |
added in most recent commit as experimental. has to be enabled via:
|
when I enable MPS I get this error: Traceback (most recent call last): just with the basic example, any ideas on how to fix? |
it's cause encodec doesn't work on mps i think, but technically that shouldn't happen, i can't reproduce. can you try to find the bug? |
im sorry but after reading everything here i still havent been able to run bark on apple silicon. issue here:
does nobody else see this? i tried searching "arm64" and didnt find anyone else with this issue |
hmm seems to be an issue with tokenizers from hugging face. maybe just try |
after using mps it has become slow |
RuntimeError: Placeholder storage has not been allocated on MPS device! |
Apple MacBook M2 #You may have to install anaconda, ffmpeg
#I also had to install soundfile
#start your file |
➜ bark git:(main) ✗ ~/venv/bin/python3 -m bark --text "Hello, my name is Suno." --output_filename "example.wav" |
Hey guys, thanks for releasing this as open-source!
Is there any plan to add Apple Silicon support and use MPS with PyTorch if available or is CUDA a "strict" requirement?
The text was updated successfully, but these errors were encountered: