Unable to run gpt-oss 20b on 32GB machine

[CuriousLocky]

In version 0.9.17 it says:

Improved memory management for gpt-oss:20b. Increase the chance to run on 32GB system (note: NPU can access <50% of total RAM)

However, when testing on my machine, it fails to run with the following error message:

[XRT] ERROR: Failed to submit command to hw queue (0xc01e0200):
Even after the video memory manager split the DMA buffer, the video 
memory manager could not page-in all of the required allocations into
video memory at the same time. The device is unable to continue.

FastflowLM version: 0.9.19
Driver version: 32.0.203.314
Total System RAM: 32GB
GPU UMA buffer size: 512MB
NPU accessible Shared memory size: 15.5GB

@CuriousLocky It increases the likelihood that this model will load, but it’s not guaranteed. I’m on a 32 GB system as well, and in practice I’d say it works only about 50% of the time right now. Sorry~

[CuriousLocky]

@CuriousLocky It increases the likelihood that this model will load, but it’s not guaranteed. I’m on a 32 GB system as well, and in practice I’d say it works only about 50% of the time right now. Sorry~

Sadly it never worked once on my machine 🥲

@CuriousLocky It increases the likelihood that this model will load, but it’s not guaranteed. I’m on a 32 GB system as well, and in practice I’d say it works only about 50% of the time right now. Sorry~

Sadly it never worked once on my machine 🥲

Sorry to hear that ... Other models should work fine

We tried hard to trim some running ram for gpt-oss .. but it is still to close the to upper limit for 32GB system (only less than 50% can be accessed by npu)

[CuriousLocky]

@CuriousLocky It increases the likelihood that this model will load, but it’s not guaranteed. I’m on a 32 GB system as well, and in practice I’d say it works only about 50% of the time right now. Sorry~

Sadly it never worked once on my machine 🥲

Sorry to hear that ... Other models should work fine

We tried hard to trim some running ram for gpt-oss .. but it is still to close the to upper limit for 32GB system (only less than 50% can be accessed by npu)

Just out of curiosity, is it in theory possible to map only the activated expert weights to NPU, and swap the mapping dynamically? There should be no data movement anyway.

You are right ... but slow that way, since you need to move the weight from non-npu mem region to npu mem region, that impose some overhead ... hope that makes sense ~

@CuriousLocky It increases the likelihood that this model will load, but it’s not guaranteed. I’m on a 32 GB system as well, and in practice I’d say it works only about 50% of the time right now. Sorry~

Sadly it never worked once on my machine 🥲

Sorry to hear that ... Other models should work fine

We tried hard to trim some running ram for gpt-oss .. but it is still to close the to upper limit for 32GB system (only less than 50% can be accessed by npu)

Just out of curiosity, is it in theory possible to map only the activated expert weights to NPU, and swap the mapping dynamically? There should be no data movement anyway.

[superjojo79]

Hi,

I get the same problem with my omen 16 Ryzen AI 9 365 with 32 gb ram.

I hope a soution can be found. I'm ready to help as far as I can.