Add tests for amdgpu lldb-server plugin

[dmpots]

This commit adds some basic unit tests for the AMDGPU plugin.
It modifies the test configuration to expose the HIPCC compiler
to the make file when the AMDGPU plugin is enabled. It also modifies
the dotest framework so that we can compile .hip files using the
Makefile.rules like we do for other tests.

The unit tests compile a simple hip program and validate that we do
create the gpu target succesfully and that we can hit a breakpoint
in the gpu code.

[walter-erquinigo]

I have realized that I need to change this file as well for NVIDIA. I tried to pass the path of my compiler to the python test file when running with lldb-dotest but it didn't work. It seems that the path is only correctly passed when running check-lldb.

[clayborg]

We should have agnostic GPU settings and allow users to specify a GPU compiler path, GPU linker path and anything else that is needed to compile.

[walter-erquinigo]

nice!

[clayborg]

We should thing about a few things here: do we want to make a generic framework for GPU building, or have each vendor (AMD, NVidia, etc) to have to add their own Makefile settings. I would vote we make a generic way to specify a GPU compiler and linker and enable GPU tests for one GPU at a time. See inline comments where I talk about this. The idea is if you are testing for AMD would would specify the GPU compiler in dotest_args.py like:

--gpu-type=amd --gpu-cc=/path/to/rocm/hpicc

And for NVidia:

--gpu-type=nvidia --gpu-cc=/path/to/nvidia/compiler/cc

Then the test suite will run one set of GPU tests. And all GPU makefiles would look similar, but each use their own compiler. Any GPU tests that are specific to the GPU type would be run, but none of the others.

[clayborg]

If HIP a valid term for all GPU compilers? Or do we need a separate setting for AMD GPU, NVidia and MTIA?

Rename HIP_SOURCES to GPU_SOURCES?

[clayborg]

We shoulld make this function name more readable. What does Rocm stand for? So should this be something like getGPUCompilerArgs(self) if this is not specific to AMD. If this is specific to AMD this should be getAMDCompilerArgs(self)

[dmpots]

ROCm is the AMD platform for GPU computing. Previously I was passing the root rocm dir, but now I am passing the hipcc compiler path directly . Switched the name to getHipccArgs to match the getNvccArgs used in the nvidia plugin.

[clayborg]

Is HPICC valid for all GPUs, or just AMD?

[clayborg]

Is this AMD specific? If so, please name this better to specify this like amd_gpu_cc_path. If this is a generic compiler for any GPU then gpu_cc_path. Not sure if we ever expect the test suite to run for more than one GPU? If not, then we should do something like gpu_cc_path. Else we might need a dictionary of gpu name or target triple to the compiler path.

[clayborg]

This should be more GPU agnostic. We probably don't want to have each GPU maker having to define they own settings like this.

    group.add_argument(
        "--gpu-cc-path",
        metavar="path",
        dest="gpu_cc_path",
        default=None,
        help=textwrap.dedent(
            """Specify the path to a GPU compiler compiler."""
        ),
    )

[clayborg]

If we go with agnostic GPU compiler settings, we might need to enable a GPU type. That way if we have many tests that are AMD specific and some that are NVidia specific, we can enable one GPU to be tested and then the GPU tests would only run if they have this GPU type.

group.add_argument(
        "--gpu-type",
        type="string",
        dest="gpu_type",
        default=None,
        help=textwrap.dedent(
            """Specify the the GPU directory to test. Valid values should match a GPU test directory name in llvm-project/lldb/test/gpu folder."""
        ),
    )

That way anyone can add a directory like:

llvm-project/lldb/test/gpu/amd
llvm-project/lldb/test/gpu/nvidia
llvm-project/lldb/test/gpu/mtia

And then we can test one of these directories. If the --gpu-type option is not set, not GPU tests will be run. If it is set, we would check for a directory in llvm-project/lldb/test/gpu and if it matches this setting, we will run the tests for that GPU in addition.

[clayborg]

s/HIP_SOURCES/GPU_SOURCES/ ?

[clayborg]

Should we have a GPUCC defined for the compiler and GPULD for the linker? This way any GPU client can work with this build system?

override CC := $(GPUCC)
override CXX := $(GPULD)

[clayborg]

Do we need a separate HIPFLAGS and should it be named GPU_CFLAGS? If we only allow something to compile as a GPU binary, then we can just use CFLAGS?

[clayborg]

/HIP_SOURCES/GPU_SOURCES/?

[clayborg]

Are all of these flags AMD specific? If they are then we need a way to set the default compiler flags for the specific GPU type that we are using, and this would be best done somewhere that checks the GPU type being tested and sets these options as needed for AMD, NVidia, etc

	$(GPUCC) $(GPUFLAGS) -MT $@ -MD -MP -MF $*.d -c -o $@ $<

[walter-erquinigo]

for the nvcc compiler, which is not clang based, we need a custom version of this line, something like

$(NVCC) $(NVCCFLAGS) -Xcompiler "-MT $@ -MD -MP -MF $*.d" -c -o $@ $<

[walter-erquinigo]

@clayborg , I think we shouldn't go the route of making a generic set of flags in the Makefile for all GPUs. A big reason is that there are multiple compilers that can be used for producing binaries for the same GPU. For example, at some point I'll want tests that will compile both triton and cuda source files, and maybe mojo.
This means that it's better if the Makefile is agnostic of GPUs and simply operates on the notion of compilers and source languages. Then, test files become very generic, to the point at which you could have single python test compile both triton and cuda files without any restrictions.

[dmpots]

We should thing about a few things here: do we want to make a generic framework for GPU building, or have each vendor (AMD, NVidia, etc) to have to add their own Makefile settings. I would vote we make a generic way to specify a GPU compiler and linker and enable GPU tests for one GPU at a time.

@clayborg I agree with @walter-erquinigo here that we should not try to make a generic set of flags that work for all GPUs. What we have added to the Makefile here (and the version Walter added) is support for a new language and the toolchain to compile that language.

In this PR we are enabling tests to be built from hip sources. It is actually possible to target a CPU with hipcc so it is technically not even restricted to running only on a GPU. It can also be targeted to run on Nvidia GPUs so its possible that we could have some Nvidia tests that use hipcc and some that use nvcc.

Just like we have separate variables and settings for C (CC) and C++ (CXX) I think it makes sense to have separate variables for hip (HIPCC) and cuda (NVCC). These are different languages and so having different make rules to support them seems to fit well with the existing framework.