Skip to content

[Bug]: Cannot get cloud GPU offers #2208

@jvstme

Description

@jvstme

Steps to reproduce

Apply this configuration:

type: dev-environment
ide: vscode
resources:
  gpu: 1

Actual behaviour

No offers.

> dstack apply
 Project          ilya                                  
 User             admin                                 
 Configuration    .dstack.yml                           
 Type             dev-environment                       
 Resources        2..xCPU, 8GB.., 1xGPU, 100GB.. (disk) 
 Max price        -                                     
 Max duration     6h                                    
 Spot policy      on-demand                             
 Retry policy     -                                     
 Creation policy  reuse-or-create                       
 Idle duration    5m                                    
 Reservation      -                                     

Submit a new run? [y/n]

Expected behaviour

Cloud offers suggested.

dstack version

master

Server logs

ERROR    dstack._internal.server.services.backends:435 Got exception when requesting offers from backend BackendType.RUNPOD                        
                    Traceback (most recent call last):                                                                                                        
                      File "/home/jvstme/git/dstack/dstack/src/dstack/_internal/utils/common.py", line 20, in run_async                                       
                        return await asyncio.get_running_loop().run_in_executor(None, func_with_args)                                                         
                      File "/usr/lib64/python3.9/concurrent/futures/thread.py", line 58, in run                                                               
                        result = self.fn(*self.args, **self.kwargs)                                                                                           
                      File "/home/jvstme/git/dstack/dstack/venv/lib64/python3.9/site-packages/cachetools/__init__.py", line 832, in wrapper                   
                        k = key(self, *args, **kwargs)                                                                                                        
                      File "/home/jvstme/git/dstack/dstack/src/dstack/_internal/core/backends/base/compute.py", line 191, in _get_offers_cached_key           
                        return hash(json.dumps(requirements.dict(), sort_keys=True))                                                                          
                      File "/usr/lib64/python3.9/json/__init__.py", line 234, in dumps                                                                        
                        return cls(                                                                                                                           
                      File "/usr/lib64/python3.9/json/encoder.py", line 199, in encode                                                                        
                        chunks = self.iterencode(o, _one_shot=True)                                                                                           
                      File "/usr/lib64/python3.9/json/encoder.py", line 257, in iterencode                                                                    
                        return _iterencode(o, 0)                                                                                                              
                      File "/usr/lib64/python3.9/json/encoder.py", line 179, in default                                                                       
                        raise TypeError(f'Object of type {o.__class__.__name__} '                                                                             
                    TypeError: Object of type AcceleratorVendor is not JSON serializable

Additional information

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingmajor

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions