Skip to content

[Bug]: Run fails on GCP if the run name contains underscores ("_") #902

@r4victor

Description

@r4victor

dstack version

master

Python version

3.11

Host OS

macOS 13.4

Host Arch

ARM

What happened?

GCP does not allow underscores in instance names. Since dstack allows underscores in run names and uses the run name as a part of instance name, running jobs on GCP fails.

CLI logs

(venv) ➜  my_dstack_public git:(master) ✗ dstack run . -b gcp --name test_run
 Configuration  .dstack.yml    
 Project        main           
 User           admin          
 Pool name      default-pool   
 Min resources  2..xCPU, 8GB.. 
 Max price      -              
 Max duration   6h             
 Spot policy    auto           
 Retry policy   no             

 #  BACKEND  REGION       INSTANCE       RESOURCES            SPOT  PRICE       
 1  gcp      us-central1  e2-standard-2  2xCPU, 8GB, 100GB    yes   $0.020103   
                                         (disk)                                 
 2  gcp      us-east1     e2-standard-2  2xCPU, 8GB, 100GB    yes   $0.020103   
                                         (disk)                                 
 3  gcp      us-central1  e2-highmem-2   2xCPU, 16GB, 100GB   yes   $0.02712    
                                         (disk)                                 
    ...                                                                         
 Shown 3 of 326 offers, $55.7395 max

Continue? [y/n]: y

Server logs

DEBUG 2024-02-16T10:26:02.450 dstack._internal.server.background.tasks.process_submitted_jobs (fcee04)test_run-0: trying e2-standard-2 in gcp/us-central1 for $0.0201 per hour
DEBUG 2024-02-16T10:26:04.239 dstack._internal.server.app Processed request POST http://127.0.0.1:8000/api/project/main/runs/get in 0.016545s
ERROR 2024-02-16T10:26:04.587 apscheduler.executors.default Job "process_submitted_jobs (trigger: interval[0:00:02], next run at: 2024-02-16 10:26:05 +05)" raised an exception
Traceback (most recent call last):
  File "/Users/r4victor/Projects/dstack/venv/lib/python3.11/site-packages/apscheduler/executors/base_py3.py", line 30, in run_coroutine_job
    retval = await job.func(*job.args, **job.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/server/background/tasks/process_submitted_jobs.py", line 64, in process_submitted_jobs
    await _process_job(job_id=job_model.id)
  File "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/server/background/tasks/process_submitted_jobs.py", line 73, in _process_job
    await _process_submitted_job(
  File "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/server/background/tasks/process_submitted_jobs.py", line 161, in _process_submitted_job
    job_provisioning_data, offer = await _run_job(
                                   ^^^^^^^^^^^^^^^
  File "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/server/background/tasks/process_submitted_jobs.py", line 239, in _run_job
    launched_instance_info: LaunchedInstanceInfo = await run_async(
                                                   ^^^^^^^^^^^^^^^^
  File "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/server/utils/common.py", line 13, in run_async
    return await asyncio.get_running_loop().run_in_executor(None, func_with_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/r4victor/.pyenv/versions/3.11.2/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/core/backends/gcp/compute.py", line 167, in run_job
    launched_instance_info = self.create_instance(instance_offer, instance_config)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/r4victor/Projects/dstack/dstack/src/dstack/_internal/core/backends/gcp/compute.py", line 127, in create_instance
    operation = self.instances_client.insert(request=request)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/r4victor/Projects/dstack/venv/lib/python3.11/site-packages/google/cloud/compute_v1/services/instances/client.py", line 3890, in insert
    response = rpc(
               ^^^^
  File "/Users/r4victor/Projects/dstack/venv/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__
    return wrapped_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/r4victor/Projects/dstack/venv/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 75, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/r4victor/Projects/dstack/venv/lib/python3.11/site-packages/google/cloud/compute_v1/services/instances/transports/rest.py", line 3163, in __call__
    raise core_exceptions.from_http_response(response)
google.api_core.exceptions.BadRequest: 400 POST https://compute.googleapis.com/compute/v1/projects/dstack/zones/us-central1-a/instances: Invalid value for field 'resource.name': 'main-test_run-0'. Must be a match of regex '(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?)'

Runner logs

No response

Additional Information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions