StrixHalo: what is aie2 in rocminfo Agent 3, and is it good for anything?

[ddpasa]

When I runn rocminfo, I get 3 agents. First one is the CPU, the second one is the GPU. There is also a third agent (see below), which I don't understand what it is?

Is it useful for anything? Can I use it with PyTorch or llama.cpp?

*******                  
Agent 3                  
*******                  
  Name:                    aie2                               
  Uuid:                    AIE-XX                             
  Marketing Name:          AIE-ML                             
  Vendor Name:             AMD                                
  Feature:                 AGENT_DISPATCH                     
  Profile:                 BASE_PROFILE                       
  Float Round Mode:        NEAR                               
  Max Queue Number:        1(0x1)                             
  Queue Min Size:          64(0x40)                           
  Queue Max Size:          64(0x40)                           
  Queue Type:              SINGLE                             
  Node:                    0                                  
  Device Type:             DSP                                
  Cache Info:              
    L2:                      2048(0x800) KB                     
    L3:                      32768(0x8000) KB                   
  Chip ID:                 0(0x0)                             
  ASIC Revision:           0(0x0)                             
  Cacheline Size:          0(0x0)                             
  Max Clock Freq. (MHz):   0                                  
  BDFID:                   0                                  
  Internal Node ID:        0                                  
  Compute Unit:            0                                  
  SIMDs per CU:            0                                  
  Shader Engines:          0                                  
  Shader Arrs. per Eng.:   0                                  
  WatchPts on Addr. Ranges:0                                  
  Memory Properties:       
  Features:                AGENT_DISPATCH
  Pool Info:               
    Pool 1                   
      Segment:                 GLOBAL; FLAGS: KERNARG, COARSE GRAINED
      Size:                    65463060(0x3e6e314) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 2                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    65536(0x10000) KB                  
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:0KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
    Pool 3                   
      Segment:                 GLOBAL; FLAGS: COARSE GRAINED      
      Size:                    65463060(0x3e6e314) KB             
      Allocatable:             TRUE                               
      Alloc Granule:           4KB                                
      Alloc Recommended Granule:4KB                                
      Alloc Alignment:         4KB                                
      Accessible by all:       TRUE                               
  ISA Info:                
*** Done ***  

[Djip007]

it is the 50 TOPs AMD NPU ...
some element : https://github.com/amd/RyzenAI-SW
but did not try anything with it ;)

[ddpasa]

it is the 50 TOPs AMD NPU ... some element : https://github.com/amd/RyzenAI-SW but did not try anything with it ;)

I am guessing neither pytorch, nor llama.cpp support the NPU? Is there any way to use the NPU right now?

[astrelsky]

it is the 50 TOPs AMD NPU ... some element : https://github.com/amd/RyzenAI-SW but did not try anything with it ;)

I am guessing neither pytorch, nor llama.cpp support the NPU? Is there any way to use the NPU right now?

https://github.com/FastFlowLM/FastFlowLM

[Djip007]

there is some work here to:

• https://github.com/amd/RyzenAI-SW/tree/main/LLM-examples
• https://github.com/ypapadop-amd/ggml/tree/hsa-backend/src/ggml-hsa