Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[LLVM][runtimes] Prepopulate LLVM_BUILTIN_TARGETS with runtimes values #95554

Merged
merged 1 commit into from
Jul 12, 2024

Conversation

jhuber6
Copy link
Contributor

@jhuber6 jhuber6 commented Jun 14, 2024

Summary:
We create the builtins separately because these are required to set up
before others are built. It's configured with a default value if not
specified, but this doesn't respect the runtimes from other targets.
This patch changes the behavior to prepopulate the builtins list with
all the targets that have compiler-rt enabled if not overridden by the
user.

@jhuber6
Copy link
Contributor Author

jhuber6 commented Jun 18, 2024

Also, @petrhosek, what's the specific reason we need to build these in a separate CMake invocation? Doesn't the list of runtimes have an implicit order which we build them? I think we could add a dependency for the builtins target within a single CMake invocation if needed. I'm wondering because adding these as separate targets creates a significant overhead to rebuild an extra CMake project.

@llvm-beanz
Copy link
Collaborator

@jhuber6 the reason I initially implemented builtins and runtimes separately is because the runtime configuration completing successfully and accurately depends on the builtin libraries being available during configuration. We could probably make some educated guess hard-coded assumptions about what the toolchain supports, but you can quickly get off the rails with more exotic target support.

@jhuber6
Copy link
Contributor Author

jhuber6 commented Jun 19, 2024

@jhuber6 the reason I initially implemented builtins and runtimes separately is because the runtime configuration completing successfully and accurately depends on the builtin libraries being available during configuration. We could probably make some educated guess hard-coded assumptions about what the toolchain supports, but you can quickly get off the rails with more exotic target support.

I see, so it's related to the CMake configure step when it tries to identify features? I can see that being an issue for really specific checks where the compiler defaults to compiler-rt for its runtime instead of some GNU libs. Still unfortunate that it adds such a large overhead to build times.

@llvm-beanz
Copy link
Collaborator

I see, so it's related to the CMake configure step when it tries to identify features? I can see that being an issue for really specific checks where the compiler defaults to compiler-rt for its runtime instead of some GNU libs. Still unfortunate that it adds such a large overhead to build times.

To make the point, who has GNU libs at all? I don't in 3 of the 4 configurations I run LLVM's build for.

Yes it is unfortunate, but there are more build configurations than we can reasonably hard code predictions for.

@jhuber6
Copy link
Contributor Author

jhuber6 commented Jun 20, 2024

I see, so it's related to the CMake configure step when it tries to identify features? I can see that being an issue for really specific checks where the compiler defaults to compiler-rt for its runtime instead of some GNU libs. Still unfortunate that it adds such a large overhead to build times.

To make the point, who has GNU libs at all? I don't in 3 of the 4 configurations I run LLVM's build for.

Yes it is unfortunate, but there are more build configurations than we can reasonably hard code predictions for.

Alright, good to know the motivation behind it. In any case, this patch should fix it to where users will have the builtins built without needing to manually specify an extra variable.

@jhuber6
Copy link
Contributor Author

jhuber6 commented Jun 24, 2024

Ping, this is required for #95304.

@compnerd
Copy link
Member

I think that @petrhosek and @llvm-beanz definitely should have a look over this - they have been working on the runtimes build, and would have a better idea of how they would like to see it evolve.

@llvm-beanz
Copy link
Collaborator

I think this is probably fine. I think @petrhosek's team is probably the most impacted here, and it has been a few years since I was too involved in this area. From looking at the Fuchsia in-tree CMake cache scripts it looks like they explicitly set both builtin and runtime configuration values so this shouldn't break anything.

@petrhosek any thoughts?

Comment on lines +148 to +151
if("compiler-rt" IN_LIST LLVM_ENABLE_RUNTIMES)
list(APPEND builtin_targets "default")
endif()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that applying DRY here is worth it. I'd rather just duplicate the builtin_default_target invocation and remove the string handling for "default".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need the following case to work,

   -DLLVM_ENABLE_RUNTIMES=compiler-rt \
   -DRUNTIMES_nvptx64-nvidia-cuda_LLVM_ENABLE_RUNTIMES=compiler-rt               \
   -DRUNTIMES_amdgcn-amd-amdhsa_LLVM_ENABLE_RUNTIMES=compiler-rt                 \
   -DLLVM_RUNTIME_TARGETS="default;amdgcn-amd-amdhsa;nvptx64-nvidia-cuda"

Honestly I think we should get rid of the distinction between the "default" target, but that's a much larger refactoring I think @petrhosek wanted to do.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "default" target is also important for how the compiler-rt parts of this are built on Darwin, which is a mess.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure that applying DRY here is worth it. I'd rather just duplicate the builtin_default_target invocation and remove the string handling for "default".

I have a WIP change that removes the "default" altogether since that part is a relic, but it needs some more testing to make sure I don't break any of existing configurations.

@jhuber6
Copy link
Contributor Author

jhuber6 commented Jul 1, 2024

ping

@jhuber6
Copy link
Contributor Author

jhuber6 commented Jul 8, 2024

@petrhosek

Comment on lines 20 to 25
set(all_runtimes ${runtimes})
foreach(name ${LLVM_RUNTIME_TARGETS})
list(APPEND all_runtimes RUNTIMES_${name}_LLVM_ENABLE_RUNTIMES)
endforeach()
foreach(entry ${all_runtimes})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like an unrelated change that could land separately.

Summary:
We create the builtins separately because these are required to set up
before others are built. It's configured with a default value if not
specified, but this doesn't respect the runtimes from other targets.
This patch changes the behavior to prepopulate the builtins list with
all the targets that have `compiler-rt` enabled if not overridden by the
user.
@jhuber6 jhuber6 merged commit da1f491 into llvm:main Jul 12, 2024
5 of 6 checks passed
aaryanshukla pushed a commit to aaryanshukla/llvm-project that referenced this pull request Jul 14, 2024
…ues (llvm#95554)

Summary:
We create the builtins separately because these are required to set up
before others are built. It's configured with a default value if not
specified, but this doesn't respect the runtimes from other targets.
This patch changes the behavior to prepopulate the builtins list with
all the targets that have `compiler-rt` enabled if not overridden by the
user.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants