Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
93 commits
Select commit Hold shift + click to select a range
2cf8c65
WIP
jbachorik May 14, 2024
c9de9da
Work in progress to try out sanitizers
r1viollet May 15, 2024
febacfb
Add suppression options
r1viollet May 15, 2024
d26011e
Avoid division by 0
r1viollet May 15, 2024
7435234
Tsan fix - Linear allocator
r1viollet May 22, 2024
ea6aa49
stackFrame - minor asan fix
r1viollet May 22, 2024
336384d
Tsan spinlock fix
r1viollet May 23, 2024
1ed9cd5
Minor tsan fixes for the CPU smoke test
r1viollet May 23, 2024
826f860
Allow new build/test configurations
jbachorik May 14, 2024
f6906b9
Add build comments
jbachorik May 14, 2024
b63c5de
Minor adjustements in sanitizer flags
r1viollet May 24, 2024
a44f570
Rework the build configurations
jbachorik May 25, 2024
8fa4dd6
Change asan location command
jbachorik May 25, 2024
778675a
Remove debug output
jbachorik May 25, 2024
845c889
Fix the linux build
jbachorik May 25, 2024
8e1fa05
Add a tsan configuration
r1viollet May 27, 2024
6d03be8
Minor tsan fix - Ensure we use atomic primitives on the epoch value.
r1viollet May 27, 2024
9559071
TSan fix - Call Trace Storage
r1viollet May 27, 2024
77ccc3f
TSan: WallClock running flag
r1viollet May 27, 2024
384b379
- Add suppressions for java issues
r1viollet May 27, 2024
2b90274
Move gtest to gradle
jbachorik May 27, 2024
62f0ca3
Merge branch 'jb/asan' of github.com:DataDog/java-profiler into jb/asan
jbachorik May 27, 2024
61d9dec
Remove access to non-existent file in CI
jbachorik May 27, 2024
9f1da2e
Debug gtests on CI
jbachorik May 27, 2024
5b3f463
TMPDIR fallback
jbachorik May 27, 2024
2266b46
TMPDIR fallback (1)
jbachorik May 27, 2024
6e9ce04
Turn off CI script debug
jbachorik May 27, 2024
dbd3fdf
Fix codecheck
jbachorik May 27, 2024
1a24bfa
TSan fix - Silence alignment issue from checkInterruptedSyscall
r1viollet May 27, 2024
368859f
Merge branch 'jb/asan' of github.com:DataDog/java-profiler into jb/asan
r1viollet May 27, 2024
45b713f
Gtest task per configuration
jbachorik May 27, 2024
6118292
Remove -lz
jbachorik May 27, 2024
9056385
More gtest build compatibilty fixes
jbachorik May 27, 2024
1c3f4d6
Merge branch 'jb/asan' of github.com:DataDog/java-profiler into jb/asan
r1viollet May 28, 2024
f9b3e7b
Minor fix on folder used to run the gtest build
r1viollet May 28, 2024
8f5769a
Remove minor leak in unit test
r1viollet May 28, 2024
ea61263
Fix CI build typo
jbachorik May 28, 2024
5a19586
Allow excluding tests for sanitized configs
jbachorik May 28, 2024
bc66484
Add comment about Buffer var-size-array
jbachorik May 28, 2024
756b903
Asan minor fixes
r1viollet May 28, 2024
fe6e0d9
Merge branch 'jb/asan' of github.com:DataDog/java-profiler into jb/asan
r1viollet May 28, 2024
a786605
Use the cmake defs from the gtest suite
jbachorik May 28, 2024
0605196
Do not delegate ddprof-test:test to ddprof-test:testDebug
jbachorik May 28, 2024
aa1d0f6
Allow gradle up-to-date checks for custom tasks
jbachorik May 28, 2024
0f140e8
Remove cmake files
jbachorik May 28, 2024
f688aa9
Merge remote-tracking branch 'origin/jb/asan' into jb/asan
jbachorik May 28, 2024
b5af49d
Use common sanitizer suppression files
jbachorik May 28, 2024
68f8649
Fix test env setup
jbachorik May 28, 2024
26d56b8
fix for com.datadoghq.profiler.wallclock.ContextWallClockTest
r1viollet May 28, 2024
525234d
Merge branch 'jb/asan' of github.com:DataDog/java-profiler into jb/asan
r1viollet May 28, 2024
df24fcf
Merge branch 'refs/heads/main' into jb/asan
jbachorik May 28, 2024
715b74e
Try migrating the CI Run workflow to shared tests
jbachorik May 28, 2024
cba18d5
Track failures in nightly
jbachorik May 28, 2024
1109744
WIP
jbachorik May 28, 2024
f7949f9
Improved CI tests
jbachorik May 28, 2024
966f0aa
Migrate gtests to gradle
jbachorik May 29, 2024
9944d64
Merge branch 'jb/asan_3' into jb/asan
jbachorik May 29, 2024
be360f9
Make gtest tasks more flexible
jbachorik May 29, 2024
c175bf1
Fix the gtest failure for release
jbachorik May 29, 2024
6feecfb
Update readme
jbachorik May 29, 2024
4273b88
Enable nightly run
jbachorik May 29, 2024
83beece
Try to avoid read-only systemIncludes error
jbachorik May 29, 2024
62642c5
Merge branch 'jb/asan' of github.com:DataDog/java-profiler into jb/asan
jbachorik May 29, 2024
e346229
Fix arm64 linux build
jbachorik May 29, 2024
029951e
Aarch64 clang workaround
jbachorik May 29, 2024
16a8696
Merge remote-tracking branch 'origin/jb/asan' into jb/asan
jbachorik May 29, 2024
ac782b6
Fix the release jar assembly
jbachorik May 30, 2024
d89b649
Fix assembly on aarch64
jbachorik May 30, 2024
1032728
Explicit sanitizers linker paths
jbachorik May 30, 2024
40d7220
Try more systematic system includes
jbachorik May 30, 2024
51db011
Skip strip symmbols if 'skip-native' is specified
jbachorik May 30, 2024
66ab531
Upgrade Gradle to support JDK 22
jbachorik May 30, 2024
54e3447
Update CI definitions
jbachorik May 30, 2024
eb5d9c3
Fix vmstruct stackwalker asan issue
jbachorik May 30, 2024
d0ea979
Do not try to resolve heap usage function on JDK 11
jbachorik May 30, 2024
2f5c875
Do not attempt to memcpy from a NULL source
jbachorik May 30, 2024
e9afed6
Remove commented out code
jbachorik May 30, 2024
7974902
Fix the test job exclusion for musl
jbachorik May 31, 2024
ac78bc0
Do not attempt to read JavaThread state from non-JavaThread threads (…
jbachorik May 31, 2024
9653303
Remove asan exclusion for VMThread::state()
jbachorik May 31, 2024
fee54cc
Add sanity check for elfsymbol value offset (#104)
jbachorik Jun 3, 2024
9e6c379
Add a binary for unresolved functions
r1viollet Jun 4, 2024
38159dc
Remove debug output
jbachorik Jun 4, 2024
fb5015d
Tsan fixes - Adjust atomic operations
r1viollet Jun 4, 2024
b6b31ae
Merge branch 'jb/asan' of github.com:DataDog/java-profiler into jb/asan
r1viollet Jun 4, 2024
98032a2
TSan fixes - Change the enabled flags to atomics
r1viollet Jun 4, 2024
706b40b
Plug the test reources build into gradle scripts (#105)
jbachorik Jun 4, 2024
c9cc5b0
Try setting up images with gtest
jbachorik Jun 4, 2024
3a4f7e8
Adjust memory orders for enabled flag
r1viollet Jun 4, 2024
b4e02f9
Merge branch 'jb/asan' of github.com:DataDog/java-profiler into jb/asan
r1viollet Jun 4, 2024
33c8c13
Valgrind analysis - silence an UMR
r1viollet Jun 13, 2024
be6fbbd
epoch minor atomic adjustement
r1viollet Jun 18, 2024
7192381
Merge branch 'main' of github.com:DataDog/java-profiler into jb/asan
r1viollet Jun 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
304 changes: 5 additions & 299 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,311 +37,17 @@ jobs:
echo "skip=true" >> "$GITHUB_OUTPUT"
fi
fi

test-linux-glibc:
test-matrix:
needs: check-for-pr
if: needs.check-for-pr.outputs.skip != 'true'
strategy:
matrix:
java_version: [8u362+9, 11.0.18+10, 17.0.6+10, 21.0.1+12]
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- uses: actions/checkout@v3
- name: Prepare build JDK
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: "11"
- name: Prepare JDK ${{ matrix.java_version }}
run: |
wget -nv https://download.bell-sw.com/java/${{ matrix.java_version }}/bellsoft-jdk${{ matrix.java_version }}-linux-amd64.tar.gz -O jdk.tar.gz
tar xzf *.tar.gz
find . -type d -name 'jdk*' -maxdepth 1| xargs -I {} mv {} test_jdk
- name: Test
run: |
export TEST_COMMIT=${{ github.sha }}
export TEST_CONFIGURATION=glibc/${{ matrix.java_version }}
export LIBC=glibc
export JAVA_TEST_HOME=$(pwd)/test_jdk
export BUILD_TYPE=SanitizedDebug
chmod a+x ddprof-lib/test.sh
./gradlew :ddprof-test:test --info
- name: Upload logs
uses: actions/upload-artifact@v3
if: always()
with:
name: reports-linux-glibc-${{ matrix.java_version }}.zip
path: |
ddprof-test/hs_err_*
ddprof-test/build/reports/tests
ddprof-lib/src/test/build/Testing/Temporary/LastTest.log
ddprof-lib/build/tmp/compileReleaseLinuxCpp/output.txt
- uses: actions/upload-artifact@v3
if: success()
with:
name: x64-glibc
path: build/

test-ubuntu-jdk:
needs: check-for-pr
if: needs.check-for-pr.outputs.skip != 'true'
strategy:
matrix:
java_version: [11, 17, 21]
runs-on: ubuntu-22.04
timeout-minutes: 180
steps:
- uses: actions/checkout@v3
- name: Prepare build JDK
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: "11"
- name: Prepare JDK ${{ matrix.java_version }}
run: |
sudo apt-get update
sudo apt-get install openjdk-${{ matrix.java_version }}-jdk
uname -r
- name: Test
run: |
export TEST_COMMIT=${{ github.sha }}
export TEST_CONFIGURATION=glibc/${{ matrix.java_version }}
export LIBC=glibc
export JAVA_TEST_HOME=/usr/lib/jvm/java-${{ matrix.java_version }}-openjdk-amd64
export JAVA_HOME=$JAVA_HOME
export PATH=$JAVA_HOME/bin:$PATH
export BUILD_TYPE=SanitizedDebug
./gradlew :ddprof-test:test --info
- name: Upload logs
uses: actions/upload-artifact@v3
if: always()
with:
name: reports-linux-ubuntu-jdk-${{ matrix.java_version }}.zip
path: |
ddprof-test/hs_err_*
ddprof-test/build/reports/tests
ddprof-lib/src/test/build/Testing/Temporary/LastTest.log
ddprof-lib/build/tmp/compileReleaseLinuxCpp/output.txt
- uses: actions/upload-artifact@v3
if: success()
with:
name: x64-ubuntu-jdk
path: build/

test-linux-glibc-j9:
needs: check-for-pr
if: needs.check-for-pr.outputs.skip != 'true'
strategy:
matrix:
java_version: [8, 11, 17]
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- uses: actions/checkout@v3
- name: Prepare test JDK
uses: actions/setup-java@v3
with:
distribution: 'adopt-openj9'
java-version: "${{ matrix.java_version }}"
- name: Store JAVA_TEST_HOME
run: JAVA_PATH=$(which java) && echo "JAVA_TEST_HOME=${JAVA_PATH/\/bin\/java/\/}" >> $GITHUB_ENV
- name: Prepare build JDK
uses: actions/setup-java@v3
with:
distribution: 'adopt-openj9'
java-version: "11"
- name: Test
run: |
export TEST_COMMIT=${{ github.sha }}
export TEST_CONFIGURATION=glibc/${{ matrix.java_version }}
export LIBC=glibc
export BUILD_TYPE=SanitizedDebug
chmod a+x gradlew
chmod a+x ddprof-lib/test.sh
./gradlew :ddprof-test:test --info
- name: Upload logs
uses: actions/upload-artifact@v3
if: always()
with:
name: reports-linux-glibc-j9-${{ matrix.java_version }}.zip
path: |
ddprof-test/javacore*
ddprof-test/build/reports/tests
ddprof-lib/src/test/build/Testing/Temporary/LastTest.log
ddprof-lib/build/tmp/compileReleaseLinuxCpp/output.txt
- uses: actions/upload-artifact@v3
if: success()
with:
name: x64-glibc-j9
path: build/

test-linux-glibc-oracle8:
needs: check-for-pr
if: needs.check-for-pr.outputs.skip != 'true'
runs-on: ubuntu-22.04
timeout-minutes: 180
steps:
- uses: actions/checkout@v3
- name: Prepare build JDK
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: "11"
- name: Prepare JDK ${{ matrix.java_version }}
run: |
sudo apt-get -y update && sudo apt-get -y install curl
set -eux;
sudo mkdir -p /usr/lib/jvm/oracle8;
# https://gist.github.com/wavezhang/ba8425f24a968ec9b2a8619d7c2d86a6?permalink_comment_id=4444663#gistcomment-4444663
# jdk1.8.0_361
curl -L --fail "https://javadl.oracle.com/webapps/download/AutoDL?BundleId=247926_0ae14417abb444ebb02b9815e2103550" | sudo tar -xvzf - -C /usr/lib/jvm/oracle8 --strip-components 1
uname -r
- name: Test
run: |
export TEST_COMMIT=${{ github.sha }}
export TEST_CONFIGURATION=glibc/8
export LIBC=glibc
export JAVA_TEST_HOME=/usr/lib/jvm/oracle8
export JAVA_HOME=$JAVA_HOME
export PATH=$JAVA_HOME/bin:$PATH
export BUILD_TYPE=SanitizedDebug
./gradlew :ddprof-test:test --info
- name: Upload logs
uses: actions/upload-artifact@v3
if: always()
with:
name: reports-linux-oracle-jdk-8.zip
path: |
ddprof-test/hs_err_*
ddprof-test/build/reports/tests
ddprof-lib/src/test/build/Testing/Temporary/LastTest.log
ddprof-lib/build/tmp/compileReleaseLinuxCpp/output.txt
- uses: actions/upload-artifact@v3
if: success()
with:
name: x64-oracle-jdk8
path: build/

test-linux-musl:
needs: check-for-pr
if: needs.check-for-pr.outputs.skip != 'true'
strategy:
matrix:
java_version: [8u362+9, 11.0.18+10, 17.0.6+10, 21.0.1+12]
runs-on: ubuntu-latest
timeout-minutes: 10
container:
image: "bellsoft/liberica-openjdk-alpine-musl:11.0.17"
options: --cpus 2
steps:
- uses: actions/checkout@v3
- name: Setup system
run: apk update && apk add curl moreutils wget hexdump linux-headers bash make g++ clang git cppcheck jq cmake >/dev/null
- name: Prepare build JDK
run: |
wget -nv https://download.bell-sw.com/java/11.0.18+10/bellsoft-jdk11.0.18+10-linux-x64-musl.tar.gz -O jdk.tar.gz
tar xzf *.tar.gz
find . -type d -name 'jdk*' -maxdepth 1 | xargs -I {} mv {} build_jdk
- name: Prepare JDK ${{ matrix.java_version }}
run: |
wget -nv https://download.bell-sw.com/java/${{ matrix.java_version }}/bellsoft-jdk${{ matrix.java_version }}-linux-x64-musl.tar.gz -O jdk.tar.gz
tar xzf *.tar.gz
find . -type d -name 'jdk*' -maxdepth 1 | xargs -I {} mv {} test_jdk
- name: Test
run: |
set -x
export JAVA_HOME=$(pwd)/build_jdk
export PATH=$JAVA_HOME/bin:$PATH
export TEST_COMMIT=${{ github.sha }}
export TEST_CONFIGURATION=musl/${{ matrix.java_version }}
export LIBC=musl
export JAVA_TEST_HOME=$(pwd)/test_jdk
chmod a+x ddprof-lib/test.sh
./gradlew :ddprof-test:test --info
- name: Upload logs
uses: actions/upload-artifact@v3
if: always()
with:
name: reports-linux-musl-${{ matrix.java_version }}.zip
path: |
ddprof-test/hs_err_*
ddprof-test/build/reports/tests
ddprof-lib/src/test/build/Testing/Temporary/LastTest.log
ddprof-lib/build/tmp/compileReleaseLinuxCpp/output.txt
- uses: actions/upload-artifact@v3
if: success()
with:
name: x64-musl
path: build/

test-linux-glibc-zing:
needs: check-for-pr
if: needs.check-for-pr.outputs.skip != 'true'
strategy:
matrix:
java_version: [ 8, 11, 17, 21 ]
runs-on: ubuntu-latest
timeout-minutes: 180
steps:
- uses: actions/checkout@v3
- name: Prepare build JDK
uses: actions/setup-java@v3
with:
distribution: 'temurin'
java-version: "11"
- name: Prepare JDK ${{ matrix.java_version }}
run: |
sudo apt-get -y update && sudo apt-get -y install curl g++-9 gcc-9
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 100 --slave /usr/bin/g++ g++ /usr/bin/g++-9
sudo update-alternatives --set gcc /usr/bin/gcc-9
set -eux
sudo mkdir -p /usr/lib/jvm/zing
if [ "${{ matrix.java_version }}" = "8" ]; then
# jdk1.8.0_372
curl -L --fail "https://cdn.azul.com/zing-zvm/ZVM23.05.0.0/zing23.05.0.0-2-jdk8.0.372-linux_x64.tar.gz" | sudo tar -xvzf - -C /usr/lib/jvm/zing --strip-components 1
elif [ "${{ matrix.java_version }}" = "11" ]; then
# jdk 11.0.19
curl -L --fail "https://cdn.azul.com/zing-zvm/ZVM23.05.0.0/zing23.05.0.0-2-jdk11.0.19-linux_x64.tar.gz" | sudo tar -xvzf - -C /usr/lib/jvm/zing --strip-components 1
elif [ "${{ matrix.java_version }}" = "17" ]; then
# jdk 17.0.7
curl -L --fail "https://cdn.azul.com/zing-zvm/ZVM23.05.0.0/zing23.05.0.0-2-jdk17.0.7-linux_x64.tar.gz" | sudo tar -xvzf - -C /usr/lib/jvm/zing --strip-components 1
elif [ "${{ matrix.java_version }}" = "21" ]; then
# jdk 21.0.2
curl -L --fail "https://cdn.azul.com/zing-zvm/ZVM23.10.0.0/zing23.10.0.0-3-jdk21.0.1-linux_x64.tar.gz" | sudo tar -xvzf - -C /usr/lib/jvm/zing --strip-components 1
fi
# rename the bundled libstdc++.so to avoid conflicts with the system one
sudo mv /usr/lib/jvm/zing/etc/libc++/libstdc++.so.6 /usr/lib/jvm/zing/etc/libc++/libstdc++.so.6.bak
- name: Test
run: |
export TEST_COMMIT=${{ github.sha }}
export TEST_CONFIGURATION=glibc/${{ matrix.java_version }}
export LIBC=glibc
export JAVA_TEST_HOME=/usr/lib/jvm/zing
export JAVA_HOME=$JAVA_HOME
export PATH=$JAVA_HOME/bin:$PATH
export BUILD_TYPE=SanitizedDebug
./gradlew :ddprof-test:test --info
- name: Upload logs
uses: actions/upload-artifact@v3
if: always()
with:
name: reports-linux-zing-jdk-${{ matrix.java_version }}.zip
path: |
ddprof-test/hs_err_*
ddprof-test/build/reports/tests/test
ddprof-lib/src/test/build/Testing/Temporary/LastTest.log
ddprof-lib/build/tmp/compileReleaseLinuxCpp/output.txt
- uses: actions/upload-artifact@v3
if: success()
with:
name: x64-glibc-zing-${{ matrix.java_version }}
path: build/
uses: ./.github/workflows/test_workflow.yml
with:
configuration: '["debug"]'

gh-release:
if: startsWith(github.event.ref, 'refs/heads/release/')
runs-on: ubuntu-latest
needs: [test-linux-musl, test-linux-glibc, test-linux-glibc-j9]
needs: [test-matrix]
steps:
- name: Create Github Release
uses: ./.github/workflows/gh_release.yml@gh-release
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/codecheck.yml
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,6 @@ jobs:
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main
- run: ./gradlew -x test assembleJar
- run: ./gradlew -x test assembleReleaseJar
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
6 changes: 3 additions & 3 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
scenarios=$(cat ./artifacts/failures.txt | tr '\n' ',')
echo "Failed scenarios: $scenarios"

curl -X POST "${{ secrets.SLACK_WEBHOOK }}" \
-H 'Content-Type: application/json' \
-d "{'scenarios': '${scenarios}', 'failed_run_url': '${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}'}"
# curl -X POST "${{ secrets.SLACK_WEBHOOK }}" \
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to leave notifications ?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Once we are ready to merge we should uncomment this block

# -H 'Content-Type: application/json' \
# -d "{'scenarios': '${scenarios}', 'failed_run_url': '${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}'}"

Loading