Possible uninterruptible hang when installing deprecated package

[Matthew-Whitlock]

Steps to reproduce

Reproducing:
Using the following .spack/packages.yaml

packages:
  cmake:
    require: "@3.27.7"
    externals:
      - spec: "cmake@3.27.7%clang@16.0.2"
        prefix: "/any/directory"

Run spack install cmake@3.27.7%gcc

This seems a little silly, so here's the realistic scenario being mimicked:

• I want to avoid rebuilding cmake inside every develop environment, so I add the cmake requirement @3.27.7 to my .spack/packages.yaml.
• Some time later, cmake 3.27.7 is deprecated.
• I want to be sure a package I'm testing is using dependencies with minimal features enabled, so I add reuse: false to my environment's concretizer
• It begins installing a cmake package with gcc instead of clang, so I decide to instead manually install cmake@3.27.7%gcc outside of the environment so I can use it in the future.
• I run into the problem described here

The problem arises from both requiring cmake@3.27.7 and having an existing install of it. You can mimic having an existing install by listing an external package; it doesn't matter if the prefix points to a valid install or not.

Error message

There is no output at all when run without debug, and running with debug does not give anything helpful:

...
==> [2024-06-24-11:09:51.301098] [BOOTSTRAP MODULE clingo] The installed spec "clingo-bootstrap@spack+python %gcc platform=linux target=x86_64 ^python@3.11/55qeu52pkt5shpwd7ulugv7wzt5j7vqd" provides the "clingo" Python module
==> [2024-06-24-11:09:51.301730] Reading config from file .../spack/defaults/concretizer.yaml
==> [2024-06-24-11:09:51.318318] Reading config from file .../spack/defaults/packages.yaml
==> [2024-06-24-11:09:51.353954] Reading config from file .../.spack/packages.yaml
==> [2024-06-24-11:09:52.125664] Reading config from file .../.spack/linux/compilers.yaml
==> [2024-06-24-11:09:52.133007] '/usr/bin/gcc' '-v' '/tmp/spack-implicit-link-infoghax35gq/main.c' '-o' '/tmp/spack-implicit-link-infoghax35gq/output'
==> [2024-06-24-11:09:52.201991] Reading config from file .../spack/defaults/config.yaml

At this point it just hangs indefinitely, and ignores Ctrl-C (though you can background the process with Ctrl-Z to escape and kill it). What I'm reporting as bugs:

• Silent hang instead of any error reporting
• Not responding to user interrupt

Running the same command without anything in packages.yaml:

==> Error: concretization failed for the following reasons:

   1. Cannot satisfy 'cmake@3.27.7'
   2. Cannot satisfy 'cmake@3.27.7'
        required because cmake@3.27.7%gcc requested explicitly

With just require: "@3.27.7":

==> Error: Version requirement 3.27.7 on cmake for cmake cannot match any known version from package.py or externals

With just the external package (and debug):

==> [2024-06-24-11:45:59.511500] SolverError: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.                                                          
    cmake@3.27.7%gcc is unsatisfiable, errors are:                                                                                                                                                                                                      
    internal_error("External weight used for built package")                                                                                                                                                                                            
    internal_error("must choose a single version to satisfy version constraints")                                                                                                                                                                       
    internal_error("version weights must exist and be unique")                                                                                                                                                                                          
==> [2024-06-24-11:45:59.511642] SolverError: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.                                                          
    cmake@3.27.7%gcc is unsatisfiable, errors are:                                                                                                                                                                                                      
    internal_error("External weight used for built package")                                                                                                                                                                                            
    internal_error("must choose a single version to satisfy version constraints")                                                                                                                                                                       
    internal_error("version weights must exist and be unique")                                                                                                                                                                                          
==> [2024-06-24-11:45:59.511701] Error: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.                                                                
    cmake@3.27.7%gcc is unsatisfiable, errors are:                                                                                                                                                                                                      
    internal_error("External weight used for built package")                                                                                                                                                                                            
    internal_error("must choose a single version to satisfy version constraints")                                                                                                                                                                       
    internal_error("version weights must exist and be unique")                                                                                                                                                                                          
Traceback (most recent call last):                                                                                                                                                                                                                      
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/main.py", line 1069, in main                                                                                                                                                                      
    return _main(argv)                                                                                                                                                                                                                                  
           ^^^^^^^^^^^                                                                                                                                                                                                                                  
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/main.py", line 1022, in _main                                                                                                                                                                     
    return finish_parse_and_run(parser, cmd_name, args, env_format_error)                                                                                                                                                                               
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                               
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/main.py", line 1052, in finish_parse_and_run                                                                                                                                                      
    return _invoke_command(command, parser, args, unknown)                                                                                                                                                                                              
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                                                              
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/main.py", line 649, in _invoke_command                                                                                                                                                            
    return_val = command(parser, args)                                                                                                                                                                                                                  
                 ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/cmd/install.py", line 354, in install
    install_without_active_env(args, install_kwargs, reporter_factory)
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/cmd/install.py", line 468, in install_without_active_env
    concrete_specs = concrete_specs_from_cli(args, install_kwargs) + concrete_specs_from_file(args)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/cmd/install.py", line 435, in concrete_specs_from_cli
    concrete_specs = spack.cmd.parse_specs(
                     ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/cmd/__init__.py", line 177, in parse_specs
    spec.concretize(tests=tests)
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/spec.py", line 3089, in concretize
    self._new_concretize(tests)
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/spec.py", line 3062, in _new_concretize
    result = solver.solve([self], tests=tests, allow_deprecated=allow_deprecated)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/solver/asp.py", line 3852, in solve
    result, _, _ = self.driver.solve(
                   ^^^^^^^^^^^^^^^^^^
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/solver/asp.py", line 931, in solve
    result.raise_if_unsat()
  File "/home/mwhitlo/installs/spack2/lib/spack/spack/solver/asp.py", line 474, in raise_if_unsat
    raise SolverError(constraints, conflicts=conflicts)
spack.solver.asp.SolverError: Spack concretizer internal error. Please submit a bug report and include the command, environment if applicable and the following error message.
    cmake@3.27.7%gcc is unsatisfiable, errors are:
    internal_error("External weight used for built package")
    internal_error("must choose a single version to satisfy version constraints")
    internal_error("version weights must exist and be unique") 

Information on your system

• Spack: 0.23.0.dev0 (27e28b3)
• Python: 3.11.6
• Platform: linux-rhel8-haswell
• Concretizer: clingo

General information

• I have run spack debug report and reported the version of Spack/Python/Platform
• I have searched the issues of this repo and believe this is not a duplicate
• I have run the failing commands in debug mode and reported the output