[compiler-rt] Realtime Sanitizer: Introduce RADSan backend #92460

cjappl · 2024-05-16T21:34:46Z

This is PR 1/?? (3? 4?) introducing the main guts of the realtime sanitizer. For more information, please see the discourse thread. We have also put together a reviewer support document to show what our intention is.

This review introduces the sanitizer backend. This includes:

CMake build files (largely adapted from asan).
Main Radsan architecture (the external API, thread local context, stack).
Interceptors.
Many unit tests.
Minimal changes in clang, enough to get the unit tests compiling.

Please see the reviewer support document for what our next steps are. We are moving in lockstep with this PR #84983 for the codegen coming up next.

github-actions · 2024-05-16T21:36:09Z

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Discourse for more information.

github-actions · 2024-05-16T21:37:51Z

✅ With the latest revision this PR passed the Python code formatter.

github-actions · 2024-05-16T21:37:51Z

✅ With the latest revision this PR passed the C/C++ code formatter.

cjappl · 2024-05-16T21:46:01Z

clang/lib/Driver/ToolChains/CommonArgs.cpp

@@ -1382,6 +1382,10 @@ collectSanitizerRuntimes(const ToolChain &TC, const ArgList &Args,
      StaticRuntimes.push_back("asan_cxx");
  }

+  if (!SanArgs.needsSharedRt() && SanArgs.needsRadsanRt()) {
+    StaticRuntimes.push_back("radsan");
+  }


One question we have here: What is the SharedRuntimes vs StaticRuntimes? Why would we need to support both? This is one change we "eyeballed" based on the other sanitizers.

I have some notes at https://maskray.me/blog/2023-01-08-all-about-sanitizer-interceptors#static-runtime

Nit: we omit braces in this single-line statement case https://llvm.org/docs/CodingStandards.html#don-t-use-braces-on-simple-single-statement-bodies-of-if-else-loop-statements

Ok, if I can summarize the important bits in my own words:

Different platforms have different defaults they use for the sanitizers, some (Darwin, for example) use shared, others (clang on Linux) use static.

A user can specify and override the default with the flags -[shared|static]-libsan.

To support the most possible configurations, do you think we should add back in the ability to link the shared library in this file? In Darwin.cpp we already show the ability to link the shared runtime, so it seemingly would be as simple as adding in a:

if (SanArgs.needsRadsanRt()) SharedRuntimes.push_back("radsan");

To the block above. Any advice?

cjappl · 2024-05-16T21:46:56Z

compiler-rt/cmake/Modules/AllSupportedArchDefs.cmake

@@ -32,6 +32,9 @@ set(ALL_ASAN_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64} ${RISCV64}
    ${LOONGARCH64})
 set(ALL_ASAN_ABI_SUPPORTED_ARCH ${X86_64} ${ARM64} ${ARM64_32})
 set(ALL_DFSAN_SUPPORTED_ARCH ${X86_64} ${MIPS64} ${ARM64} ${LOONGARCH64})
+set(ALL_RADSAN_SUPPORTED_ARCH ${X86} ${X86_64} ${ARM32} ${ARM64} ${RISCV64}
+    ${MIPS32} ${MIPS64} ${PPC64} ${S390X} ${SPARC} ${SPARCV9} ${HEXAGON}
+    ${LOONGARCH64})


This was copied from ASAN, is this acceptable? We don't have all architectures to test, although we think that our interceptors are simple enough to work on any machine (no platform specific assembly in our code, for example)

cjappl · 2024-05-16T21:48:44Z

Pinging possibly interested parties for review.

@yln @vitalybuka @Sirraide @AaronBallman @dougsonos @davidtrevelyan

Please feel free to add more, we don't know who all may be interested. Thanks in advance

MaskRay · 2024-05-20T02:35:37Z

compiler-rt/test/radsan/lit.cfg.py

@@ -0,0 +1,71 @@
+# -*- Python -*-


Delete

Some older lit.cfg.py files might use this because previously they were named lit.cfg

MaskRay · 2024-05-20T02:37:34Z

compiler-rt/test/radsan/lit.cfg.py

+config.name = "RADSAN" + config.name_suffix
+
+# Setup source root.
+config.test_source_root = os.path.dirname(__file__)


msan, lsan, gwp_asan places lit tests directly under the directory, while some use TestCases/. Avoiding TestCases is preferred. Is the lit set up to avoid TestCases?

Yes, the lit tests may be placed directly in compiler-rt/test/radsan/ and they run as intended. Just checked on our branch that has them implemented.

MaskRay · 2024-05-20T02:40:14Z

compiler-rt/lib/radsan/radsan_stack.cpp

+} // namespace __sanitizer
+
+namespace {
+void SetGlobalStackTraceFormat() {


https://llvm.org/docs/CodingStandards.html#anonymous-namespaces

MaskRay · 2024-05-20T02:47:48Z

compiler-rt/lib/radsan/radsan_context.h

+
+#pragma once
+
+namespace radsan {


__radsan to avoid conflicts with user programs

MaskRay · 2024-05-20T02:53:17Z

compiler-rt/lib/radsan/radsan_context.cpp

+
+namespace detail {
+
+static pthread_key_t Key;


Avoid VariableName in new compiler-rt code.

Use the recommended variable_name like asan/hwasan/lsan. Some new compiler-rt subprojects used Variable as cargo culting llvm/ and clang/, which is a pity (you can also read https://llvm.org/docs/Proposals/VariableNames.html for more history)

Great thanks!! Before I go through and edit these, snake_case is preferred for variables in all contexts?

Really appreciate the review. Pushing up most of your requested changes now.

Yes, variable_name and FunctionName.. Sorry if this change may take a while.
asan/hwasan/lsan are good examples to follow.

No apology necessary! Happy to do this right the first time.

Does this also apply to lambdas?

auto Func = [&vec]() { vec.push_back(0.4f); };

Should this become:

auto func = [&vec]() { vec.push_back(0.4f); };

Or remain "function case"?

I pushed the change for all non-lambda variable names, happy to change the lambdas too based on your feedback.

MaskRay · 2024-05-20T02:55:23Z

compiler-rt/lib/radsan/radsan_stack.cpp

+} // namespace
+
+namespace radsan {
+void PrintStackTrace() {


https://llvm.org/docs/CodingStandards.html#use-namespace-qualifiers-to-implement-previously-declared-functions

void radsan::PrintStackTrace

MaskRay · 2024-05-20T02:55:47Z

compiler-rt/lib/radsan/radsan_interceptors.cpp

+}
+} // namespace radsan
+
+/*


Avoid /* in C++ files

MaskRay · 2024-05-20T03:03:53Z

compiler-rt/lib/radsan/radsan.h

+  violations are detected. Calls to this method are injected at the code
+  generation stage when RADSan is enabled.
+*/
+SANITIZER_INTERFACE_ATTRIBUTE void radsan_realtime_enter();


Use __* names to avoid conflicts with user code. Use //.

If there are user APIs, define them at include/sanitizer/*_interface.h

MaskRay · 2024-05-21T00:25:14Z

https://discourse.llvm.org/t/rfc-nolock-and-noalloc-attributes/76837/75 contains a discussion about the subproject name.
Do you have opinions? @llvm/pr-subscribers-sanitizer @compnerd @petrhosek @isanbard

cjappl · 2024-05-23T22:33:26Z

Weekly ping of reviewers

@yln @vitalybuka @Sirraide @AaronBallman @zygoloid @compnerd @petrhosek @isanbard

Looking for weighing in on any of the code as it stands now, or the naming question posed by MaskRay here, so far there seems to be support for rtsan over radsan :)

cjappl · 2024-05-30T13:58:28Z

Ping

@yln @vitalybuka @Sirraide @AaronBallman @zygoloid @compnerd @petrhosek @isanbard @MaskRay

dougsonos · 2024-05-30T15:37:46Z

Looking for weighing in on any of the code as it stands now, or the naming question posed by MaskRay here, so far there seems to be support for rtsan over radsan :)

+1 for rtsan

* Pass on CREAT mode propertly in open interceptors * Add unit tests * Remove two unneccessary bits * Fix fcntl

* Fix issue where fcntl was not using last arg correctly * Comment update

cjappl commented May 16, 2024

View reviewed changes

cjappl closed this May 16, 2024

cjappl reopened this May 16, 2024

cjappl changed the title ~~Realtime Sanitizer: Introduce RADSan backend~~ [compiler-rt] Realtime Sanitizer: Introduce RADSan backend May 16, 2024

AaronBallman requested review from zygoloid and MaskRay May 17, 2024 13:06

MaskRay reviewed May 20, 2024

View reviewed changes

cjappl force-pushed the RadsanBackend branch from 1697dca to 288836a Compare May 21, 2024 20:50

cjappl and others added 14 commits May 30, 2024 09:48

First try

c46c324

Fixing some clang tidy foo

e781641

One round of renaming

2e5f8ac

Another round of style cleanups

7fc5236

more auto removal, more case fixing

3232a4b

Missing end of namespace comment

2dd39a3

Fix one naming issue

3ae58fa

No more unistd

9d1624d

Radsan context moved to impl

a03a228

Promise lit tests later

4c721db

Move radsan_test and add helper comment to top of file

6ac0c66

Only link radsan statically

1be7de3

Fix comments in radsan.h

de92a9c

Replace inlined exit with selected action stub

688422d

cjappl and others added 14 commits May 30, 2024 09:48

Codeowners

ee7c1c4

Fix python formatting errors

e4d2609

Fix cpp style issues

37f60b7

PR: Delete python header comment

1cfac7d

PR: Change main namespace name to __radsan

8b7da4b

PR: Fix stack trace namespace

4d6008c

PR: Fix incorrect C++ style comments

0038d3f

PR: Prefix external API with dunderscores

73cfdc7

PR: Fix variable naming cases for variables, leave lambdas for now

53a1828

PR: Fix style nit

0e5ea54

PR: Fix small style fallout

21836c0

Update comment and TU-local function style

8a1a7a8

PR: Remove too early inclusion of sanitizer common lit tests

64df9c5

Test failure: fix initialization of pthread_cond_t

167349a

cjappl force-pushed the RadsanBackend branch from 288836a to 167349a Compare May 30, 2024 17:01

cjappl and others added 5 commits May 30, 2024 12:02

Test failure: Try again with initialization of pthread_cond_t

0b60190

Fix code formatting on unit test

615b65f

Ensure all pthread_cond_t are initialized in unit tests

e2a4baf

Fix issue where CREAT flag was not properly passing mode (#23)

6fa7e25

* Pass on CREAT mode propertly in open interceptors * Add unit tests * Remove two unneccessary bits * Fix fcntl

Fix issue where fcntl was not using last arg correctly (#24)

66c56c1

* Fix issue where fcntl was not using last arg correctly * Comment update

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[compiler-rt] Realtime Sanitizer: Introduce RADSan backend #92460

[compiler-rt] Realtime Sanitizer: Introduce RADSan backend #92460

cjappl commented May 16, 2024 •

edited

github-actions bot commented May 16, 2024

github-actions bot commented May 16, 2024 •

edited

github-actions bot commented May 16, 2024 •

edited

cjappl May 16, 2024

MaskRay May 20, 2024

cjappl May 20, 2024

cjappl May 16, 2024

cjappl commented May 16, 2024

MaskRay May 20, 2024

MaskRay May 20, 2024

cjappl May 20, 2024

MaskRay May 20, 2024

MaskRay May 20, 2024

MaskRay May 20, 2024

cjappl May 20, 2024

MaskRay May 20, 2024

cjappl May 20, 2024

cjappl May 20, 2024

MaskRay May 20, 2024

MaskRay May 20, 2024

MaskRay May 20, 2024

MaskRay commented May 21, 2024

cjappl commented May 23, 2024

cjappl commented May 30, 2024

dougsonos commented May 30, 2024

[compiler-rt] Realtime Sanitizer: Introduce RADSan backend #92460

Are you sure you want to change the base?

[compiler-rt] Realtime Sanitizer: Introduce RADSan backend #92460

Conversation

cjappl commented May 16, 2024 • edited

github-actions bot commented May 16, 2024

github-actions bot commented May 16, 2024 • edited

github-actions bot commented May 16, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cjappl commented May 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MaskRay commented May 21, 2024

cjappl commented May 23, 2024

cjappl commented May 30, 2024

dougsonos commented May 30, 2024

cjappl commented May 16, 2024 •

edited

github-actions bot commented May 16, 2024 •

edited

github-actions bot commented May 16, 2024 •

edited