Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[receiver/jmx] Add metric gatherer to artifacts #3262

Merged
merged 143 commits into from
Sep 7, 2023
Merged
Show file tree
Hide file tree
Changes from 126 commits
Commits
Show all changes
143 commits
Select commit Hold shift + click to select a range
3ef91f4
[receiver/jmx] Add Docker support for JMX receiver
crobert-1 Apr 28, 2023
8da94a7
Add back accidental deletion
crobert-1 Jun 14, 2023
f8206b0
Updated based on Jeff's comments
crobert-1 Jun 30, 2023
91ab283
Fix import order
crobert-1 Jul 11, 2023
31b6ea0
Update cmd/otelcol/Dockerfile.windows
crobert-1 Jul 11, 2023
b80f572
Update cmd/otelcol/Dockerfile.windows
crobert-1 Jul 11, 2023
32fc072
Download from github instead of maven
crobert-1 Jul 11, 2023
bdbe71f
Add missing space
crobert-1 Jul 11, 2023
ce0cdeb
Fix version numbering
crobert-1 Jul 11, 2023
a3d183c
Update version style for new remote repo
crobert-1 Jul 11, 2023
952e423
Add v back for versioning download
crobert-1 Jul 11, 2023
a0d0cf5
Update cmd/otelcol/Dockerfile.windows
crobert-1 Jul 12, 2023
7946c56
Update cmd/otelcol/Dockerfile.windows
crobert-1 Jul 12, 2023
5617145
Update file names to match downloaded files
crobert-1 Jul 12, 2023
96b2257
Fix sprintf formatting
crobert-1 Jul 12, 2023
7c2187a
Ensure required directories exist
crobert-1 Jul 13, 2023
8736eec
Remove unecessary steps
crobert-1 Jul 13, 2023
81041c7
Make container delay longer, curl into specific filename
crobert-1 Jul 14, 2023
13f5ce4
Rename JAR files to match default JMX path
crobert-1 Jul 14, 2023
495563a
Fix variable syntax in config file
crobert-1 Jul 14, 2023
c3ebf5a
Fix jmx integration test
crobert-1 Jul 17, 2023
8c48cdc
Increase timeout to see if integration test will pass
crobert-1 Jul 17, 2023
99cdd6d
Update cmd/otelcol/Dockerfile.windows
crobert-1 Jul 18, 2023
f597d5b
Update cmd/otelcol/Dockerfile.windows
crobert-1 Jul 18, 2023
b64a0ac
Fix integration test
crobert-1 Jul 18, 2023
a625462
Remove integration test
crobert-1 Jul 18, 2023
f9293f0
Add check for jmx jar path in windows MSI
crobert-1 Jul 18, 2023
9db557d
Update path setting in windows dockerfile
crobert-1 Jul 18, 2023
dce7d2d
Fix path setting variable in windows dockerfile
crobert-1 Jul 18, 2023
d406c77
Try another variation of dockerfile windows path
crobert-1 Jul 18, 2023
9618705
Another try at the path syntax
crobert-1 Jul 18, 2023
594f034
Remove extra path arguments
crobert-1 Jul 18, 2023
fd9b894
Use servercore syntax for setting path
crobert-1 Jul 18, 2023
3e5d5d7
Add missing terminator
crobert-1 Jul 18, 2023
96f2b34
Match servercore
crobert-1 Jul 18, 2023
6a13123
Move around values
crobert-1 Jul 18, 2023
ed661c4
Try another format for path
crobert-1 Jul 18, 2023
c19dd6a
Add integration tests back in
crobert-1 Jul 18, 2023
c8545a3
Ran make fmt
crobert-1 Jul 18, 2023
4e37978
Get path a different way
crobert-1 Jul 18, 2023
a9b978f
Try debugging failure
crobert-1 Jul 18, 2023
1b6dc57
Change tmp dir's name
crobert-1 Jul 18, 2023
d43470a
Change temp dir's permission
crobert-1 Jul 18, 2023
7d0be25
Try base case without any temp dir
crobert-1 Jul 19, 2023
b5b33f7
Manually create tmp dir and mount it
crobert-1 Jul 19, 2023
3157e7c
Remove extra will fail call
crobert-1 Jul 19, 2023
7da95a7
Regular mount, made tmpdir with all permissions
crobert-1 Jul 19, 2023
848d616
Create tmp dir in scratch docker image
crobert-1 Jul 19, 2023
76986aa
Update Java version and re-use with different format
crobert-1 Jul 21, 2023
8be0ba9
Use powershell for string replacement
crobert-1 Jul 21, 2023
aa6cd51
Fix powershell command formatting
crobert-1 Jul 21, 2023
c968146
Fix formatting again
crobert-1 Jul 21, 2023
fbabb06
Try another powershell formatting
crobert-1 Jul 21, 2023
84a2a05
Update variable reference to env reference
crobert-1 Jul 21, 2023
7f5fedb
Fix syntax
crobert-1 Jul 21, 2023
bbc0a7e
Remove command option
crobert-1 Jul 21, 2023
05bf1cd
Try moving variable outside parenthesis
crobert-1 Jul 21, 2023
007edc1
Use explicit setenv method of powershell
crobert-1 Jul 21, 2023
adc42c2
Name directory generic name
crobert-1 Jul 21, 2023
1d06994
Try to see if tmp dir is reason tests are failing
crobert-1 Jul 24, 2023
11c684a
Mount local tmpdir for test only
crobert-1 Jul 24, 2023
9400dff
Manually set local tmp if not set
crobert-1 Jul 24, 2023
67b73bf
Skip integration test when running on arm
crobert-1 Jul 27, 2023
39afea0
Fix whitespace
crobert-1 Jul 27, 2023
3d93e59
Change method to find arch running test
crobert-1 Jul 28, 2023
f5767d9
Try env variable for go arch
crobert-1 Jul 28, 2023
781f2cc
Add test data for ARM architecture
crobert-1 Jul 28, 2023
1c03302
Add comment about arm test check
crobert-1 Jul 28, 2023
946d192
Update .github/workflows/win-package-test.yml
crobert-1 Jul 28, 2023
11b9c0f
Update cmd/otelcol/Dockerfile
crobert-1 Jul 28, 2023
5c0f1bc
Changes suggested by Jeff
crobert-1 Jul 28, 2023
c712094
Add component ref for JMX gatherer
crobert-1 Jul 28, 2023
3a67f96
Add changelog entry
crobert-1 Jul 28, 2023
e1793c3
Add PR link
crobert-1 Jul 28, 2023
2a0afe7
add tmpdir to images
rmfitzpatrick Jul 31, 2023
4fb2284
Bump boto3 in /internal/buildscripts/packaging/release (#3425)
dependabot[bot] Jul 31, 2023
5801c64
Bump github.com/aws/aws-sdk-go from 1.44.280 to 1.44.312 in /internal…
dependabot[bot] Jul 31, 2023
a4b7b9c
Bump github.com/snowflakedb/gosnowflake from 1.6.22 to 1.6.23 in /int…
dependabot[bot] Jul 31, 2023
ac3791e
Bump github.com/docker/docker from 24.0.4+incompatible to 24.0.5+inco…
dependabot[bot] Jul 31, 2023
29e9bcb
Bump github.com/docker/docker in /tests (#3432)
dependabot[bot] Jul 31, 2023
34b1a99
Bump org.apache.commons:commons-lang3 (#3434)
dependabot[bot] Jul 31, 2023
1f06e68
Bump github.com/prometheus/prometheus from 0.45.0 to 0.46.0 (#3433)
dependabot[bot] Jul 31, 2023
473a422
Upgrade go to 1.20.6 (#3437)
jeffreyc-splunk Jul 31, 2023
77490ea
Bump github.com/google/cadvisor from 0.47.1 to 0.47.3 in /internal/si…
dependabot[bot] Jul 31, 2023
cd694e4
Bump github.com/prometheus/prometheus (#3435)
dependabot[bot] Jul 31, 2023
e071bb1
Update OpenTelemetry Dependencies to latest (#3436)
srv-gh-o11y-gdi Aug 1, 2023
40403ee
Cherry-pick a fix for resource detection processor (#3438)
dmitryax Aug 1, 2023
73aab12
Revert "Revert "OTL-2195: Disable fluentd by default in installer scr…
jeffreyc-splunk Aug 1, 2023
12cb13a
OTL-2195: Disable fluentd by default for choco package (#3377)
jeffreyc-splunk Aug 1, 2023
331ac2d
set TMPDIR in docker image
rmfitzpatrick Aug 1, 2023
4856c43
remove unnecesary fixtures and mount
rmfitzpatrick Aug 2, 2023
bb6bd07
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 2, 2023
68e31cc
update gatherer version
rmfitzpatrick Aug 2, 2023
f55ec94
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 7, 2023
a7b3916
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 8, 2023
c8750ef
use standard tmp dir in docker image
rmfitzpatrick Aug 8, 2023
7b0f1af
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 9, 2023
7982107
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 10, 2023
e244b92
fix telegraf-exec test tmp race
rmfitzpatrick Aug 10, 2023
8bf0801
fix windows jdk path
rmfitzpatrick Aug 14, 2023
345e3ea
use jre instead of jdk and disable java otlp log exporter
rmfitzpatrick Aug 14, 2023
3cdbc22
Bump pygithub in /internal/buildscripts/packaging/release (#3490)
dependabot[bot] Aug 14, 2023
0873fee
Bump github.com/openzipkin/zipkin-go from 0.4.1 to 0.4.2 in /pkg/rece…
dependabot[bot] Aug 14, 2023
c66623a
Bump boto3 in /internal/buildscripts/packaging/release (#3491)
dependabot[bot] Aug 14, 2023
491e4aa
Bump golang.org/x/tools from 0.11.1 to 0.12.0 in /internal/tools (#3494)
dependabot[bot] Aug 14, 2023
eb77e33
Bump golang in /examples/prometheus-federation/prom-counter (#3493)
dependabot[bot] Aug 14, 2023
3e92203
Bump github.com/golangci/golangci-lint in /internal/tools (#3496)
dependabot[bot] Aug 14, 2023
1f88e5a
Workaround for windows installer script signing (#3505)
jeffreyc-splunk Aug 14, 2023
ae5757d
Bump github.com/antonmedv/expr from 1.12.6 to 1.13.0 in /internal/sig…
dependabot[bot] Aug 14, 2023
188bc51
Bump golang.org/x/net from 0.12.0 to 0.14.0 in /internal/signalfx-age…
dependabot[bot] Aug 14, 2023
4c9d8d4
Add a github issue template to send folks to support and splunk ideas…
atoulme Aug 14, 2023
02a9f68
Bump golang in /examples/splunk-hec-traces/tracing (#3495)
dependabot[bot] Aug 14, 2023
ede0c20
Bump golang in /examples/splunk-hec/logging (#3497)
dependabot[bot] Aug 14, 2023
80ddb19
Bump github.com/hashicorp/consul/api in /internal/signalfx-agent (#3500)
dependabot[bot] Aug 14, 2023
d0679d0
Bump github.com/hashicorp/golang-lru from 0.6.0 to 1.0.2 in /internal…
dependabot[bot] Aug 14, 2023
30ca3cd
Bump github.com/beevik/ntp from 1.2.0 to 1.3.0 in /internal/signalfx-…
dependabot[bot] Aug 14, 2023
1a815d3
corect changelog
rmfitzpatrick Aug 15, 2023
ced2890
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 15, 2023
afc2d3e
update default msi gatherer version
rmfitzpatrick Aug 15, 2023
6cc2cf1
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 16, 2023
d4c84b0
remove arm-specific telegraf-proctstat test metrics
rmfitzpatrick Aug 16, 2023
9d9bd54
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 21, 2023
1cace05
corect cl entry location
rmfitzpatrick Aug 21, 2023
6837806
Merge branch 'main' into jmx_integration
crobert-1 Aug 23, 2023
a782056
update jmx gatherer to 1.29.0
rmfitzpatrick Aug 25, 2023
4901d5c
Merge branch 'main' of github.com:signalfx/splunk-otel-collector into…
rmfitzpatrick Aug 25, 2023
7bcfe6a
Update .gitlab/update-openjdk.sh
rmfitzpatrick Aug 28, 2023
9334a87
Update cmd/otelcol/Dockerfile
rmfitzpatrick Aug 28, 2023
5f40455
move autoconfigure env vars to int test
rmfitzpatrick Aug 28, 2023
a0ccf85
Changes requested in PR
crobert-1 Aug 30, 2023
1a6d47b
Add missing parenthesis
crobert-1 Aug 30, 2023
b2ae714
Fix string comparison
crobert-1 Aug 31, 2023
b7d7667
Fix missing paren
crobert-1 Aug 31, 2023
fd3a7e9
Add test to make sure jar is installed properly
crobert-1 Sep 1, 2023
0f87e4c
Test paths for jmx jar
crobert-1 Sep 1, 2023
ed3b209
Update install path for jmx jar
crobert-1 Sep 1, 2023
073a072
Pass in version for jmx download method
crobert-1 Sep 1, 2023
b7b3513
Move install path of jmx jar
crobert-1 Sep 1, 2023
6db2787
Update tests/receivers/jmxreceiver/jmxreceiver_test.go
crobert-1 Sep 5, 2023
7379f97
Update internal/buildscripts/packaging/msi/msi-builder/Dockerfile
crobert-1 Sep 5, 2023
297a7db
Update internal/buildscripts/packaging/msi/msi-builder/build.sh
crobert-1 Sep 5, 2023
893474c
Merge branch 'main' into jmx_integration
crobert-1 Sep 7, 2023
5cb0177
Merge branch 'main' into jmx_integration
atoulme Sep 7, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/win-package-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -195,8 +195,9 @@ jobs:
Start-Sleep -s 30
& ${{ github.workspace }}\.github\workflows\scripts\win-test-services.ps1 -mode "${{ matrix.MODE }}" -access_token "${{ env.token }}" -realm "${{ env.realm }}" -memory "${{ env.memory }}" -with_fluentd "${{ matrix.WITH_FLUENTD }}"
& ${{ github.workspace }}\.github\workflows\scripts\win-test-support-bundle.ps1 -mode "${{ matrix.MODE }}" -with_fluentd "${{ matrix.WITH_FLUENTD }}"
Test-Path -Path "$env:ProgramFiles\Splunk\OpenTelemetry Collector\agent-bundle\python\python.exe"
Test-Path -Path "$env:ProgramFiles\Splunk\OpenTelemetry Collector\agent-bundle\collectd-python"
Resolve-Path -Path "$env:ProgramFiles\Splunk\OpenTelemetry Collector\agent-bundle\python\python.exe"
Resolve-Path -Path "$env:ProgramFiles\Splunk\OpenTelemetry Collector\agent-bundle\collectd-python"
Resolve-Path -Path "$env:SYSTEMDRIVE\opt\opentelemetry-java-contrib-jmx-metrics.jar"

- name: splunk-otel-collector logs
if: ${{ always() }}
Expand Down Expand Up @@ -351,9 +352,9 @@ jobs:
Copy-Item .\bin\translatesfx_windows_amd64.exe .\cmd\otelcol\translatesfx.exe
Copy-Item .\dist\agent-bundle_windows_amd64.zip .\cmd\otelcol\agent-bundle_windows_amd64.zip
if ("${{ matrix.OS }}" -eq "windows-2019") {
docker build -t otelcol-windows --build-arg BASE_IMAGE=mcr.microsoft.com/windows/servercore:1809 -f .\cmd\otelcol\Dockerfile.windows .\cmd\otelcol\
docker build -t otelcol-windows --build-arg BASE_IMAGE=mcr.microsoft.com/windows/servercore:1809 --build-arg JMX_METRIC_GATHERER_RELEASE=$(Get-Content internal\buildscripts\packaging\jmx-metric-gatherer-release.txt) -f .\cmd\otelcol\Dockerfile.windows .\cmd\otelcol\
} else {
docker build -t otelcol-windows --build-arg BASE_IMAGE=mcr.microsoft.com/windows/servercore:ltsc2022 -f .\cmd\otelcol\Dockerfile.windows .\cmd\otelcol\
docker build -t otelcol-windows --build-arg BASE_IMAGE=mcr.microsoft.com/windows/servercore:ltsc2022 --build-arg JMX_METRIC_GATHERER_RELEASE=$(Get-Content internal\buildscripts\packaging\jmx-metric-gatherer-release.txt) -f .\cmd\otelcol\Dockerfile.windows .\cmd\otelcol\
}
Remove-Item .\cmd\otelcol\otelcol.exe
Remove-Item .\cmd\otelcol\translatesfx.exe
Expand Down
6 changes: 4 additions & 2 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -656,8 +656,9 @@ build-push-windows-image:
$IMAGE_NAME = "quay.io/signalfx/splunk-otel-collector-windows-dev"
$IMAGE_TAG = $env:CI_COMMIT_SHA
}
$JMX_METRIC_GATHERER_RELEASE = $(Get-Content internal\buildscripts\packaging\jmx-metric-gatherer-release.txt)
echo "Building ${IMAGE_NAME}:${IMAGE_TAG}"
docker build -t ${IMAGE_NAME}:${IMAGE_TAG} --build-arg BASE_IMAGE=mcr.microsoft.com/windows/servercore:1809 -f .\cmd\otelcol\Dockerfile.windows .\cmd\otelcol\
docker build -t ${IMAGE_NAME}:${IMAGE_TAG} --build-arg BASE_IMAGE=mcr.microsoft.com/windows/servercore:1809 --build-arg JMX_METRIC_GATHERER_RELEASE=${JMX_METRIC_GATHERER_RELEASE} -f .\cmd\otelcol\Dockerfile.windows .\cmd\otelcol\
echo "Pushing ${IMAGE_NAME}:${IMAGE_TAG}"
docker push ${IMAGE_NAME}:${IMAGE_TAG}
if ($env:CI_COMMIT_BRANCH -eq "main" -or $env:CI_COMMIT_TAG -match '^v\d+\.\d+\.\d+$') {
Expand Down Expand Up @@ -698,8 +699,9 @@ build-push-windows2022-image:
$IMAGE_NAME = "quay.io/signalfx/splunk-otel-collector-windows-dev"
$IMAGE_TAG = "${env:CI_COMMIT_SHA}-2022"
}
$JMX_METRIC_GATHERER_RELEASE = $(Get-Content internal\buildscripts\packaging\jmx-metric-gatherer-release.txt)
echo "Building ${IMAGE_NAME}:${IMAGE_TAG}"
docker build -t ${IMAGE_NAME}:${IMAGE_TAG} --build-arg BASE_IMAGE=mcr.microsoft.com/windows/servercore:ltsc2022 -f .\cmd\otelcol\Dockerfile.windows .\cmd\otelcol\
docker build -t ${IMAGE_NAME}:${IMAGE_TAG} --build-arg BASE_IMAGE=mcr.microsoft.com/windows/servercore:ltsc2022 --build-arg JMX_METRIC_GATHERER_RELEASE=${JMX_METRIC_GATHERER_RELEASE} -f .\cmd\otelcol\Dockerfile.windows .\cmd\otelcol\
echo "Pushing ${IMAGE_NAME}:${IMAGE_TAG}"
docker push ${IMAGE_NAME}:${IMAGE_TAG}
if ($env:CI_COMMIT_BRANCH -eq "main" -or $env:CI_COMMIT_TAG -match '^v\d+\.\d+\.\d+$') {
Expand Down
1 change: 1 addition & 0 deletions .gitlab/update-openjdk.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ create_collector_pr() {
if [[ -n "$version" ]]; then
echo ">>> Updating openjdk version to $version ..."
sed -i "s|^ARG JDK_VERSION=.*|ARG JDK_VERSION=${version}|" internal/signalfx-agent/bundle/Dockerfile
sed -i "s|^ARG JDK_VERSION=.*|ARG JDK_VERSION=${version}|" cmd/otelcol/Dockerfile.windows
rmfitzpatrick marked this conversation as resolved.
Show resolved Hide resolved
else
echo "ERROR: Failed to get version from tag name '${tag}'!" >&2
exit 1
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

### 💡 Enhancements 💡

- (Splunk) `jmxreceiver`: Bundle latest [JMX Metric Gatherer](https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/jmx-metrics) in installer packages and images for Windows and Linux ([#3262](https://github.com/signalfx/splunk-otel-collector/pull/3262))
- (Splunk) `receiver/smartagent`: Use `Leases` instead of `ConfigMapLeases` for leader-election in k8s. ([#3521](https://github.com/signalfx/splunk-otel-collector/pull/3521))

## v0.83.0
Expand Down
7 changes: 4 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ BUILD_X2=-X $(BUILD_INFO_IMPORT_PATH_CORE).Version=$(VERSION)
BUILD_INFO=-ldflags "${BUILD_X1} ${BUILD_X2}"
BUILD_INFO_TESTS=-ldflags "-X $(BUILD_INFO_IMPORT_PATH_TESTS).Version=$(VERSION)"

JMX_METRIC_GATHERER_RELEASE=$(shell cat internal/buildscripts/packaging/jmx-metric-gatherer-release.txt)
SKIP_COMPILE=false
ARCH=amd64
BUNDLE_SUPPORTED_ARCHS := amd64 arm64
Expand Down Expand Up @@ -167,7 +168,7 @@ endif
ifneq ($(filter $(ARCH), $(BUNDLE_SUPPORTED_ARCHS)),)
cp ./dist/agent-bundle_linux_$(ARCH).tar.gz ./cmd/otelcol/dist/agent-bundle.tar.gz
endif
docker buildx build --platform linux/$(ARCH) -o type=image,name=otelcol:$(ARCH),push=false --build-arg ARCH=$(ARCH) --build-arg DOCKER_REPO=$(DOCKER_REPO) ./cmd/otelcol/
docker buildx build --platform linux/$(ARCH) -o type=image,name=otelcol:$(ARCH),push=false --build-arg ARCH=$(ARCH) --build-arg DOCKER_REPO=$(DOCKER_REPO) --build-arg JMX_METRIC_GATHERER_RELEASE=$(JMX_METRIC_GATHERER_RELEASE) ./cmd/otelcol/
docker tag otelcol:$(ARCH) otelcol:latest
rm -rf ./cmd/otelcol/dist

Expand Down Expand Up @@ -221,15 +222,15 @@ ifneq ($(SKIP_BUNDLE), true)
endif
endif
docker build -t otelcol-fpm internal/buildscripts/packaging/fpm
docker run --rm -v $(CURDIR):/repo -e PACKAGE=$* -e VERSION=$(VERSION) -e ARCH=$(ARCH) otelcol-fpm
docker run --rm -v $(CURDIR):/repo -e PACKAGE=$* -e VERSION=$(VERSION) -e ARCH=$(ARCH) -e JMX_METRIC_GATHERER_RELEASE=$(JMX_METRIC_GATHERER_RELEASE) otelcol-fpm

.PHONY: msi
msi:
ifneq ($(SKIP_COMPILE), true)
$(MAKE) binaries-windows_amd64
endif
test -f ./dist/agent-bundle_windows_amd64.zip || (echo "./dist/agent-bundle_windows_amd64.zip not found! Either download a pre-built bundle to ./dist/, or run './internal/signalfx-agent/bundle/scripts/windows/make.ps1 bundle' on a windows host and copy it to ./dist/." && exit 1)
./internal/buildscripts/packaging/msi/build.sh "$(VERSION)" "$(DOCKER_REPO)"
./internal/buildscripts/packaging/msi/build.sh "$(VERSION)" "$(DOCKER_REPO)" "$(JMX_METRIC_GATHERER_RELEASE)"

.PHONY: update-examples
update-examples:
Expand Down
18 changes: 17 additions & 1 deletion cmd/otelcol/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,9 @@ RUN chmod 755 /translatesfx
RUN chmod 755 /migratecheckpoint
RUN echo "splunk-otel-collector:x:999:999::/:" > /etc_passwd
# create base dirs since we cannot chown in scratch image except via COPY
RUN mkdir -p /otel/collector /splunk-otel-collector
RUN mkdir -p /otel/collector /splunk-otel-collector /tmp/tmp
# used by jmx receiver to pass tmp properties. Nested to avoid inheriting existing tmp files.
RUN chmod 755 /tmp/tmp

FROM ${DOCKER_REPO}/alpine:3.17.0 AS agent-bundle
ARG ARCH="amd64"
Expand All @@ -35,6 +37,12 @@ RUN if [[ "$ARCH" = "amd64" || "$ARCH" = "arm64" ]]; then \
mkdir -p /usr/lib/splunk-otel-collector/agent-bundle; \
fi

FROM alpine:3.17.0 AS jmx
rmfitzpatrick marked this conversation as resolved.
Show resolved Hide resolved
ARG JMX_METRIC_GATHERER_RELEASE
RUN JMX_METRICS_JAR=opentelemetry-jmx-metrics.jar && \
crobert-1 marked this conversation as resolved.
Show resolved Hide resolved
URL=https://github.com/open-telemetry/opentelemetry-java-contrib/releases/download/v${JMX_METRIC_GATHERER_RELEASE}/${JMX_METRICS_JAR} && \
cd /tmp && wget "$URL";

FROM ${DOCKER_REPO}/debian:11.5 as journalctl
RUN apt update
RUN apt install -y systemd
Expand All @@ -45,14 +53,22 @@ FROM scratch
ENV SPLUNK_BUNDLE_DIR=/usr/lib/splunk-otel-collector/agent-bundle
ENV SPLUNK_COLLECTD_DIR=${SPLUNK_BUNDLE_DIR}/run/collectd
ENV PATH=${PATH}:${SPLUNK_BUNDLE_DIR}/bin
ENV PATH=${PATH}:${SPLUNK_BUNDLE_DIR}/jre/bin
crobert-1 marked this conversation as resolved.
Show resolved Hide resolved
# avoid otel-java/autoconfigure default env vars for jmx receiver
ENV OTEL_TRACES_EXPORTER=none
ENV OTEL_LOGS_EXPORTER=none
rmfitzpatrick marked this conversation as resolved.
Show resolved Hide resolved

COPY --from=certs /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt
COPY --from=otelcol /etc_passwd /etc/passwd
COPY --from=otelcol --chown=999 /otelcol /
COPY --from=otelcol --chown=999 /translatesfx /
COPY --from=otelcol --chown=999 /migratecheckpoint /
COPY --from=otelcol --chown=999 /otel /etc/otel
COPY --from=otelcol --chown=999 /otel/collector /etc/otel/collector
COPY --from=otelcol --chown=999 /tmp/tmp /tmp
COPY --from=agent-bundle --chown=999 /usr/lib/splunk-otel-collector /usr/lib/splunk-otel-collector
COPY --from=jmx --chown=999 /tmp/opentelemetry-jmx-metrics.jar /opt/opentelemetry-java-contrib-jmx-metrics.jar


COPY --from=journalctl --chown=999 /bin/journalctl /bin/journalctl
COPY --from=journalctl --chown=999 /opt/journalctl /
Expand Down
19 changes: 19 additions & 0 deletions cmd/otelcol/Dockerfile.windows
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,25 @@ RUN Add-Type -AssemblyName System.IO.Compression.FileSystem ;\
[System.IO.Compression.ZipFile]::ExtractToDirectory('agent-bundle_windows_amd64.zip', '.')
RUN Remove-Item agent-bundle_windows_amd64.zip -force

# Download JMX Metric Gatherer
ARG JMX_METRIC_GATHERER_RELEASE
RUN New-Item -Path "C:\\" -Name "opt" -ItemType "directory"
RUN Invoke-WebRequest -Uri "https://github.com/open-telemetry/opentelemetry-java-contrib/releases/download/v${env:JMX_METRIC_GATHERER_RELEASE}/opentelemetry-jmx-metrics.jar" -Outfile "c:\opt\opentelemetry-java-contrib-jmx-metrics.jar"

ARG JDK_VERSION="11.0.20_8"
ARG OPENJDK_BASE_URL="https://github.com/adoptium/temurin11-binaries/releases/download"
ENV JAVA_HOME="C:\java\jre"
ENV JAVA_PATH="$JAVA_HOME\bin"
RUN Set-Variable -Name "ENCODED_VER" -Value "\"$env:JDK_VERSION\".replace(\"_\", \"%2B\")"; \
Invoke-WebRequest -Uri "${env:OPENJDK_BASE_URL}/jdk-${ENCODED_VER}/OpenJDK11U-jre_x64_windows_hotspot_${env:JDK_VERSION}.zip" -OutFile "openjdk_jre.zip" ; \
Expand-Archive "openjdk_jre.zip" -DestinationPath "C:\java" ;
RUN Move-Item -Path C:\java\jdk-* -Destination ${Env:JAVA_HOME}
RUN Remove-Item "openjdk_jre.zip" -force
RUN setx /M PATH $(${Env:PATH} + \";${Env:JAVA_PATH}\")
# update otel-java/autoconfigure defaults to prevent undesired telemetry
RUN setx /M OTEL_TRACES_EXPORTER "none"
RUN setx /M OTEL_LOGS_EXPORTER "none"
crobert-1 marked this conversation as resolved.
Show resolved Hide resolved

# Setting environment variables
ENV SPLUNK_BUNDLE_DIR="C:\Program Files\Splunk\OpenTelemetry Collector\agent-bundle"
ENV SPLUNK_CONFIG="C:\ProgramData\Splunk\OpenTelemetry Collector\gateway_config.yaml"
Expand Down
4 changes: 2 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ require (
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/fluentforwardreceiver v0.83.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver v0.83.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jaegerreceiver v0.83.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.83.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.83.1-0.20230824192551-2c559459cf6c
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/journaldreceiver v0.83.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.83.0
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8seventsreceiver v0.83.0
Expand Down Expand Up @@ -306,7 +306,7 @@ require (
github.com/google/go-cmp v0.5.9
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/uuid v1.3.0 // indirect
github.com/google/uuid v1.3.1 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.2.5 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/gophercloud/gophercloud v1.5.0 // indirect
Expand Down
8 changes: 4 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -633,8 +633,8 @@ github.com/google/renameio v0.1.0/go.mod h1:KWCgfxg9yswjAJkECMjeO8J8rahYeXnNhOm4
github.com/google/s2a-go v0.1.4 h1:1kZ/sQM3srePvKs3tXAvQzo66XfcReoqFpIpIccE7Oc=
github.com/google/s2a-go v0.1.4/go.mod h1:Ej+mSEMGRnqRzjc7VtF+jdBwYG5fuJfiZ8ELkjEwM0A=
github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.0 h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
github.com/google/uuid v1.3.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.2.5 h1:UR4rDjcgpgEnqpIEvkiqTYKBCKLNmlge2eVjoZfySzM=
github.com/googleapis/enterprise-certificate-proxy v0.2.5/go.mod h1:RxW0N9901Cko1VOCW3SXCpWP+mlIEkk2tP7jnHy9a3w=
github.com/googleapis/gax-go/v2 v2.0.4/go.mod h1:0Wqv26UfaUD9n4G6kQubkQ+KchISgw+vpHVxEJEs9eg=
Expand Down Expand Up @@ -1246,8 +1246,8 @@ github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsre
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver v0.83.0/go.mod h1:BtoPj2YDHhWy8LRxlWWjK+vOu3MJ6r8pRW56eJ7fk+g=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jaegerreceiver v0.83.0 h1:wu0ZdOTDqjl0Ug6J7rAO43GIe7nW/hvPXZ/Kcs7Hmis=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jaegerreceiver v0.83.0/go.mod h1:qhWK3ADYGvYaS54XbaH6yK8/iPGQLtimdtIcXevTLkw=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.83.0 h1:5ix9FlAdeMWdGcWgoXoDswZj6UknsVi4aner2igA/+s=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.83.0/go.mod h1:rZVBP7d3WfsXWv47HsG0zF97WBkcHseJpnIsNPq/cIA=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.83.1-0.20230824192551-2c559459cf6c h1:DBhqI71lEBTLbejLNshua+5C3zDE8lMbIGhFXMH+qAM=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/jmxreceiver v0.83.1-0.20230824192551-2c559459cf6c/go.mod h1:MSmgdhxc1dbbMQq6/cW/YPPfxd/92L03feKoqigR1nA=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/journaldreceiver v0.83.0 h1:e2P2HmEgHf81QIVllRitiueDxxVj3C7Ph++hxXBodmY=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/journaldreceiver v0.83.0/go.mod h1:Ota92x/vC7FBTvP6l6gCe5Ftq1YTP0yg/mp2nrNFyv0=
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/k8sclusterreceiver v0.83.0 h1:NSW1bxQ1J48dnT5MZvRqZMWsZi/cYCQUekp4nB8SJ4I=
Expand Down
2 changes: 1 addition & 1 deletion internal/buildscripts/packaging/fpm/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,4 +14,4 @@ RUN /install-deps.sh

RUN git config --global --add safe.directory /repo

CMD ./internal/buildscripts/packaging/fpm/$PACKAGE/build.sh "$VERSION" "$ARCH" "$OUTPUT_DIR"
CMD ./internal/buildscripts/packaging/fpm/$PACKAGE/build.sh "$VERSION" "$ARCH" "$OUTPUT_DIR" "$JMX_METRIC_GATHERER_RELEASE"
18 changes: 18 additions & 0 deletions internal/buildscripts/packaging/fpm/common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ SERVICE_INSTALL_PATH="/lib/systemd/system/$SERVICE_NAME.service"

FLUENTD_CONFIG_INSTALL_DIR="/etc/otel/collector/fluentd"

JMX_METRIC_GATHERER_RELEASE_PATH="${FPM_DIR}/../jmx-metric-gatherer-release.txt"
BUNDLE_BASE_DIR="/usr/lib/splunk-otel-collector"
AGENT_BUNDLE_INSTALL_DIR="$BUNDLE_BASE_DIR/agent-bundle"

Expand All @@ -67,6 +68,17 @@ create_user_group() {
sudo useradd --system --user-group --no-create-home --shell /sbin/nologin $SERVICE_USER
}

download_jmx_metric_gatherer() {
local version="$1"
local buildroot="$2"

JMX_METRIC_GATHERER_RELEASE_DL_URL="https://github.com/open-telemetry/opentelemetry-java-contrib/releases/download/v$version/opentelemetry-jmx-metrics.jar"
mkdir -p "$buildroot/opt"

echo "Downloading ${JMX_METRIC_GATHERER_RELEASE_DL_URL}"
curl -sL "$JMX_METRIC_GATHERER_RELEASE_DL_URL" -o "$buildroot/opt/opentelemetry-java-contrib-jmx-metrics.jar"
crobert-1 marked this conversation as resolved.
Show resolved Hide resolved
}

setup_files_and_permissions() {
local otelcol="$1"
local translatesfx="$2"
Expand Down Expand Up @@ -104,4 +116,10 @@ setup_files_and_permissions() {
sudo chown -R $SERVICE_USER:$SERVICE_GROUP "$buildroot/$BUNDLE_BASE_DIR"
sudo chmod -R 755 "$buildroot/$BUNDLE_BASE_DIR"
fi

JMX_INSTALL_PATH="$buildroot/opt/opentelemetry-java-contrib-jmx-metrics.jar"
if [[ -e "$JMX_INSTALL_PATH" ]]; then
sudo chown root:root "$JMX_INSTALL_PATH"
sudo chmod 755 "$JMX_INSTALL_PATH"
fi
}
7 changes: 7 additions & 0 deletions internal/buildscripts/packaging/fpm/deb/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,18 +22,25 @@ SCRIPT_DIR="$( cd "$( dirname ${BASH_SOURCE[0]} )" && pwd )"
VERSION="${1:-}"
ARCH="${2:-amd64}"
OUTPUT_DIR="${3:-$REPO_DIR/dist}"
JMX_METRIC_GATHERER_RELEASE="${4:-}"

if [[ -z "$VERSION" ]]; then
VERSION="$( get_version )"
fi
VERSION="${VERSION#v}"

if [[ -z "$JMX_METRIC_GATHERER_RELEASE" ]]; then
JMX_METRIC_GATHERER_RELEASE="$(cat $JMX_METRIC_GATHERER_RELEASE_PATH)"
fi

otelcol_path="$REPO_DIR/bin/otelcol_linux_${ARCH}"
translatesfx_path="$REPO_DIR/bin/translatesfx_linux_${ARCH}"
agent_bundle_path="$REPO_DIR/dist/agent-bundle_linux_${ARCH}.tar.gz"

buildroot="$(mktemp -d)"

download_jmx_metric_gatherer "$JMX_METRIC_GATHERER_RELEASE" "$buildroot"

setup_files_and_permissions "$otelcol_path" "$translatesfx_path" "$buildroot" "$agent_bundle_path"

mkdir -p "$OUTPUT_DIR"
Expand Down
7 changes: 7 additions & 0 deletions internal/buildscripts/packaging/fpm/rpm/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ SCRIPT_DIR="$( cd "$( dirname ${BASH_SOURCE[0]} )" && pwd )"
VERSION="${1:-}"
ARCH="${2:-amd64}"
OUTPUT_DIR="${3:-$REPO_DIR/dist}"
JMX_METRIC_GATHERER_RELEASE="${4:-}"

if [[ -z "$VERSION" ]]; then
VERSION="$( get_version )"
Expand All @@ -31,6 +32,10 @@ fi
VERSION="${VERSION/'-'/'~'}"
VERSION="${VERSION#v}"

if [[ -z "$JMX_METRIC_GATHERER_RELEASE" ]]; then
JMX_METRIC_GATHERER_RELEASE="$(cat $JMX_METRIC_GATHERER_RELEASE_PATH)"
fi

otelcol_path="$REPO_DIR/bin/otelcol_linux_${ARCH}"
translatesfx_path="$REPO_DIR/bin/translatesfx_linux_${ARCH}"
agent_bundle_path="$REPO_DIR/dist/agent-bundle_linux_${ARCH}.tar.gz"
Expand All @@ -43,6 +48,8 @@ elif [[ "$ARCH" = "amd64" ]]; then
ARCH="x86_64"
fi

download_jmx_metric_gatherer "$JMX_METRIC_GATHERER_RELEASE" "$buildroot"

setup_files_and_permissions "$otelcol_path" "$translatesfx_path" "$buildroot" "$agent_bundle_path"

mkdir -p "$OUTPUT_DIR"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.29.0
7 changes: 7 additions & 0 deletions internal/buildscripts/packaging/msi/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@ set -euxo pipefail

SCRIPT_DIR="$( cd "$( dirname ${BASH_SOURCE[0]} )" && pwd )"
REPO_DIR="$( cd "$SCRIPT_DIR/../../../../" && pwd )"
JMX_METRIC_GATHERER_RELEASE_PATH="${SCRIPT_DIR}/../jmx-metric-gatherer-release.txt"

VERSION="${1:-}"
DOCKER_REPO="${2:-docker.io}"
JMX_METRIC_GATHERER_RELEASE="${3:-}"

get_version() {
commit_tag="$( git -C "$REPO_DIR" describe --abbrev=0 --tags --exact-match --match 'v[0-9]*' 2>/dev/null || true )"
Expand All @@ -36,6 +38,10 @@ get_version() {
fi
}

if [ -z "$JMX_METRIC_GATHERER_RELEASE" ]; then
JMX_METRIC_GATHERER_RELEASE=$(cat "$JMX_METRIC_GATHERER_RELEASE_PATH")
fi

if [ -z "$VERSION" ]; then
VERSION="$( get_version )"
fi
Expand All @@ -44,6 +50,7 @@ docker build -t msi-builder --build-arg DOCKER_REPO="$DOCKER_REPO" -f "${SCRIPT_
docker rm -fv msi-builder 2>/dev/null || true
docker run -d --name msi-builder msi-builder sleep inf
docker exec \
-e JMX_METRIC_GATHERER_RELEASE="${JMX_METRIC_GATHERER_RELEASE}" \
-e OUTPUT_DIR=/project/dist \
-e VERSION="${VERSION#v}" \
msi-builder /docker-entrypoint.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ RUN cp /project/internal/buildscripts/packaging/msi/msi-builder/docker-entrypoin
chmod a+x /docker-entrypoint.sh

ENV OUTPUT_DIR=/project/dist
ENV JMX_METRIC_GATHERER_RELEASE=1.28.0
crobert-1 marked this conversation as resolved.
Show resolved Hide resolved
ENV VERSION=""

WORKDIR /work
Expand Down
Loading
Loading