Skip to content

Commit

Permalink
1.6.4
Browse files Browse the repository at this point in the history
* Update jars
* Add support for passing query params from proxy to agent
  • Loading branch information
pambrose committed Apr 30, 2020
1 parent 69fb4e5 commit 29b870d
Show file tree
Hide file tree
Showing 67 changed files with 852 additions and 740 deletions.
9 changes: 8 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
VERSION=1.6.3
VERSION=1.6.4

default: compile

Expand All @@ -23,11 +23,15 @@ distro: clean compile jars

docker-build:
docker build -f ./etc/docker/proxy.df -t pambrose/prometheus-proxy:${VERSION} .
docker build -f ./etc/docker/proxy.df -t pambrose/prometheus-proxy:latest .
docker build -f ./etc/docker/agent.df -t pambrose/prometheus-agent:${VERSION} .
docker build -f ./etc/docker/agent.df -t pambrose/prometheus-agent:latest .

docker-push:
docker push pambrose/prometheus-proxy:${VERSION}
docker push pambrose/prometheus-proxy:latest
docker push pambrose/prometheus-agent:${VERSION}
docker push pambrose/prometheus-agent:latest

all: distro docker-build docker-push

Expand All @@ -49,5 +53,8 @@ tree:
versioncheck:
./gradlew dependencyUpdates

refresh:
./gradlew --refresh-dependencies

depends:
./gradlew dependencies
28 changes: 14 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@
[![Kotlin](https://img.shields.io/badge/%20language-Kotlin-red.svg)](https://kotlinlang.org/)

[Prometheus](https://prometheus.io) is an excellent systems monitoring and alerting toolkit, which uses a pull model for
collecting metrics. The pull model is problematic when a Prometheus server and its metrics endpoints are separated by a
firewall. [Prometheus Proxy](https://github.com/pambrose/prometheus-proxy) enables Prometheus to reach metrics endpoints
collecting metrics. The pull model is problematic when a firewall separates a Prometheus server and its metrics endpoints.
[Prometheus Proxy](https://github.com/pambrose/prometheus-proxy) enables Prometheus to reach metrics endpoints
running behind a firewall and preserves the pull model.

The `prometheus-proxy` runtime comprises 2 services:
Expand Down Expand Up @@ -45,7 +45,7 @@ java -jar prometheus-agent.jar -Dagent.proxy.hostname=mymachine.local --config h

If prometheus-proxy were running on a machine named *mymachine.local* and the
`agent.pathConfigs` value in the [myapps.conf](https://raw.githubusercontent.com/pambrose/prometheus-proxy/master/examples/myapps.conf)
config file were defined as:
config file had the contents:

```hocon
agent {
Expand Down Expand Up @@ -97,8 +97,8 @@ scrape_configs:

The docker images are available via:
```bash
docker pull pambrose/prometheus-proxy:1.6.3
docker pull pambrose/prometheus-agent:1.6.3
docker pull pambrose/prometheus-proxy:1.6.4
docker pull pambrose/prometheus-agent:1.6.4
```

Start a proxy container with:
Expand All @@ -107,15 +107,15 @@ Start a proxy container with:
docker run --rm -p 8082:8082 -p 8092:8092 -p 50051:50051 -p 8080:8080 \
--env ADMIN_ENABLED=true \
--env METRICS_ENABLED=true \
pambrose/prometheus-proxy:1.6.3
pambrose/prometheus-proxy:1.6.4
```

Start an agent container with:

```bash
docker run --rm -p 8083:8083 -p 8093:8093 \
--env AGENT_CONFIG='https://raw.githubusercontent.com/pambrose/prometheus-proxy/master/examples/simple.conf' \
pambrose/prometheus-agent:1.6.3
pambrose/prometheus-agent:1.6.4
```

Using the config file [simple.conf](https://raw.githubusercontent.com/pambrose/prometheus-proxy/master/examples/simple.conf),
Expand All @@ -131,7 +131,7 @@ is in your current directory, run an agent container with:
docker run --rm -p 8083:8083 -p 8093:8093 \
--mount type=bind,source="$(pwd)"/prom-agent.conf,target=/app/prom-agent.conf \
--env AGENT_CONFIG=prom-agent.conf \
pambrose/prometheus-agent:1.6.3
pambrose/prometheus-agent:1.6.4
```

**Note:** The `WORKDIR` of the proxy and agent images is `/app`, so make sure
Expand All @@ -141,7 +141,7 @@ to use `/app` as the base directory in the target for `--mount` options.

The proxy and agent use the [Typesafe Config](https://github.com/typesafehub/config) library for configuration.
Highlights include:
* supports files in three formats: Java properties, JSON, and a human-friendly JSON superset ([HOCON](https://github.com/typesafehub/config#using-hocon-the-json-superset))
* support for files in three formats: Java properties, JSON, and a human-friendly JSON superset ([HOCON](https://github.com/typesafehub/config#using-hocon-the-json-superset))
* config files can be files or urls
* config values can come from CLI options, environment vars, Java system properties, and/or config files.
* config files can reference environment variables
Expand Down Expand Up @@ -196,7 +196,7 @@ Misc notes:
* JSON config files must have a *.json* suffix
* Java Properties config files must have a *.properties* or *.prop* suffix
* HOCON config files must have a *.conf* suffix
* Option values are evaluated in the order: CLI, enviroment vars, and finally config file vals
* Option values are evaluated in the order: CLI, environment vars, and finally config file vals
* Property values can be set as a java -D arg to or as a proxy or agent jar -D arg.

### Admin Servlets
Expand All @@ -223,8 +223,8 @@ Agents connect to a proxy using [gRPC](https://grpc.io). gRPC supports TLS with
necessary certificate and key file paths can be specified via CLI args, environment variables and configuration file settings.

The gRPC docs describe [how to setup TLS](https://github.com/grpc/grpc-java/tree/master/examples/example-tls).
The certificates and keys necessary to test TLS support are included in the
[repo](https://github.com/pambrose/prometheus-proxy/tree/master/testing/certs).
The [repo](https://github.com/pambrose/prometheus-proxy/tree/master/testing/certs) includes the
certificates and keys necessary to test TLS support.

These settings are required to run TLS without mutual authentication:
* certChainFilePath and privateKeyFilePath on the proxy
Expand All @@ -250,15 +250,15 @@ docker run --rm -p 8082:8082 -p 8092:8092 -p 50440:50440 -p 8080:8080 \
--env PROXY_CONFIG=tls-no-mutual-auth.conf \
--env ADMIN_ENABLED=true \
--env METRICS_ENABLED=true \
pambrose/prometheus-proxy:1.6.3
pambrose/prometheus-proxy:1.6.4

docker run --rm -p 8083:8083 -p 8093:8093 \
--mount type=bind,source="$(pwd)"/testing/certs,target=/app/testing/certs \
--mount type=bind,source="$(pwd)"/examples/tls-no-mutual-auth.conf,target=/app/tls-no-mutual-auth.conf \
--env AGENT_CONFIG=tls-no-mutual-auth.conf \
--env PROXY_HOSTNAME=mymachine.lan:50440 \
--name docker-agent \
pambrose/prometheus-agent:1.6.3
pambrose/prometheus-agent:1.6.4
```

**Note:** The `WORKDIR` of the proxy and agent images is `/app`, so make sure
Expand Down
2 changes: 1 addition & 1 deletion bin/docker-agent.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
docker run --rm -p 8083:8083 -p 8093:8093 \
--env AGENT_CONFIG='https://raw.githubusercontent.com/pambrose/prometheus-proxy/master/examples/simple.conf' \
--env PROXY_HOSTNAME=mymachine.lan \
pambrose/prometheus-agent:1.6.3
pambrose/prometheus-agent:1.6.4
2 changes: 1 addition & 1 deletion bin/docker-proxy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@

docker run --rm -p 8082:8082 -p 8092:8092 -p 50051:50051 -p 8080:8080 \
--env PROXY_CONFIG='https://raw.githubusercontent.com/pambrose/prometheus-proxy/master/examples/simple.conf' \
pambrose/prometheus-proxy:1.6.3
pambrose/prometheus-proxy:1.6.4
108 changes: 45 additions & 63 deletions build.gradle
Original file line number Diff line number Diff line change
@@ -1,40 +1,21 @@
plugins {
id 'idea'
id 'java'
id 'org.jetbrains.kotlin.jvm' version '1.3.61'
id 'org.jetbrains.kotlin.plugin.serialization' version '1.3.61'
id 'com.google.protobuf' version '0.8.10'
id "com.github.ben-manes.versions" version '0.27.0'
id 'org.jetbrains.kotlin.jvm' version '1.3.71'
id 'org.jetbrains.kotlin.plugin.serialization' version '1.3.71'
id 'com.google.protobuf' version '0.8.12'
id "com.github.ben-manes.versions" version '0.28.0'
id 'com.github.johnrengelman.shadow' version '5.2.0'
id 'jacoco'
id 'com.github.kt3k.coveralls' version '2.8.4'
id 'com.github.kt3k.coveralls' version '2.10.1'
}

group = 'io.prometheus'
version = '1.6.3'
version = '1.6.4'

sourceCompatibility = 1.8
targetCompatibility = 1.8

def annotationVersion = '1.3.2'
def coroutinesVersion = '1.3.3'
def dropwizardVersion = '4.1.2'
def grpcVersion = '1.26.0'
def jcommanderVersion = '1.78'
def jettyVersion = '9.4.22.v20191022'
def junitVersion = '5.5.2'
def kluentVersion = '1.58'
def ktorVersion = '1.2.6'
def logbackVersion = '1.2.3'
def loggingVersion = '1.7.8'
def prometheusVersion = '0.8.0'
def protocVersion = '3.11.2'
def serializationVersion = '0.13.0'
def slf4jVersion = '1.7.28'
def typesafeVersion = '1.4.0'
def utilsVersion = '1.1.14'
def zipkinVersion = '5.9.1'

repositories {
maven { url = "https://kotlin.bintray.com/kotlinx" }
maven { url = 'https://maven-central.storage-download.googleapis.com/repos/central/data/' }
Expand All @@ -46,58 +27,58 @@ dependencies {
implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk8"
implementation "org.jetbrains.kotlin:kotlin-reflect"

implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:${serializationVersion}"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core:${coroutinesVersion}"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-jdk8:${coroutinesVersion}"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-slf4j:${coroutinesVersion}"
implementation "org.jetbrains.kotlinx:kotlinx-serialization-runtime:${serialization_version}"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-core:${coroutines_version}"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-jdk8:${coroutines_version}"
implementation "org.jetbrains.kotlinx:kotlinx-coroutines-slf4j:${coroutines_version}"

implementation "io.grpc:grpc-all:${grpcVersion}"
implementation "io.grpc:grpc-all:${grpc_version}"

implementation "com.github.pambrose.common-utils:core-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:corex-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:dropwizard-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:guava-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:grpc-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:jetty-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:ktor-client-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:prometheus-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:service-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:zipkin-utils:${utilsVersion}"
implementation "com.github.pambrose.common-utils:core-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:corex-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:dropwizard-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:guava-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:grpc-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:jetty-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:ktor-client-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:prometheus-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:service-utils:${utils_version}"
implementation "com.github.pambrose.common-utils:zipkin-utils:${utils_version}"

implementation "org.eclipse.jetty:jetty-servlet:${jettyVersion}"
implementation "org.eclipse.jetty:jetty-servlet:${jetty_version}"

implementation "javax.annotation:javax.annotation-api:${annotationVersion}"
implementation "com.beust:jcommander:${jcommanderVersion}"
implementation "com.typesafe:config:${typesafeVersion}"
implementation "javax.annotation:javax.annotation-api:${annotation_version}"
implementation "com.beust:jcommander:${jcommander_version}"
implementation "com.typesafe:config:${typesafe_version}"

implementation "io.prometheus:simpleclient:${prometheusVersion}"
implementation "io.prometheus:simpleclient:${prometheus_version}"

implementation "io.ktor:ktor-server-core:${ktorVersion}"
implementation "io.ktor:ktor-server-cio:${ktorVersion}"
implementation "io.ktor:ktor-client-core:${ktorVersion}"
implementation "io.ktor:ktor-client-cio:${ktorVersion}"
implementation "io.ktor:ktor-server-core:${ktor_version}"
implementation "io.ktor:ktor-server-cio:${ktor_version}"
implementation "io.ktor:ktor-client-core:${ktor_version}"
implementation "io.ktor:ktor-client-cio:${ktor_version}"

implementation "io.dropwizard.metrics:metrics-healthchecks:${dropwizardVersion}"
implementation "io.dropwizard.metrics:metrics-healthchecks:${dropwizard_version}"

implementation "io.zipkin.brave:brave-instrumentation-grpc:${zipkinVersion}"
implementation "io.zipkin.brave:brave-instrumentation-grpc:${zipkin_version}"

implementation "ch.qos.logback:logback-classic:${logbackVersion}"
implementation "org.slf4j:jul-to-slf4j:${slf4jVersion}"
implementation "io.github.microutils:kotlin-logging:${loggingVersion}"
implementation "ch.qos.logback:logback-classic:${logback_version}"
implementation "org.slf4j:jul-to-slf4j:${slf4j_version}"
implementation "io.github.microutils:kotlin-logging:${logging_version}"

testImplementation "org.amshove.kluent:kluent:${kluentVersion}"
testImplementation "org.junit.jupiter:junit-jupiter-api:$junitVersion"
testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:$junitVersion"
testImplementation "org.amshove.kluent:kluent:${kluent_version}"
testImplementation "org.junit.jupiter:junit-jupiter-api:$junit_version"
testRuntimeOnly "org.junit.jupiter:junit-jupiter-engine:$junit_version"
}

compileKotlin.dependsOn ':generateProto'

protobuf {
protoc {
artifact = "com.google.protobuf:protoc:${protocVersion}"
artifact = "com.google.protobuf:protoc:${protoc_version}"
}
plugins {
grpc { artifact = "io.grpc:protoc-gen-grpc-java:${grpcVersion}" }
grpc { artifact = "io.grpc:protoc-gen-grpc-java:${grpc_version}" }
}
generateProtoTasks {
all()*.plugins {
Expand Down Expand Up @@ -136,10 +117,10 @@ jacocoTestReport {
afterEvaluate {
getClassDirectories().setFrom(files(classDirectories.files.collect {
fileTree(dir: it,
excludes: [
'**/ConfigVals*',
'io/prometheus/grpc/**'
])
excludes: [
'**/ConfigVals*',
'io/prometheus/grpc/**'
])
}))
}
}
Expand Down Expand Up @@ -176,6 +157,7 @@ compileTestKotlin {
}
}

// Required for multiple uberjar targets
shadowJar {
mergeServiceFiles()
}
Expand Down
2 changes: 1 addition & 1 deletion docs/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
9) Build distro with: `make distro`

10) Create release on github (https://github.com/pambrose/prometheus-proxy/releases) and
upload the *build/distro/prometheus-proxy.jar* and *build/distro/prometheus-agent.jar* files.
upload the *build/libs/prometheus-proxy.jar* and *build/libs/prometheus-agent.jar* files.

11) Build and push docker images with: `make docker-build docker-push`

Expand Down
2 changes: 1 addition & 1 deletion etc/compose/proxy.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
prometheus-proxy:
autoredeploy: true
image: 'pambrose/prometheus-proxy:1.6.3'
image: 'pambrose/prometheus-proxy:1.6.4'
ports:
- '8080:8080'
- '8082:8082'
Expand Down
18 changes: 18 additions & 0 deletions gradle.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
annotation_version=1.3.2
coroutines_version=1.3.4
dropwizard_version=4.1.5
grpc_version=1.28.1
jcommander_version=1.78
jetty_version=9.4.22.v20191022
junit_version=5.6.1
kluent_version=1.61
ktor_version=1.3.2
logback_version=1.2.3
logging_version=1.7.9
prometheus_version=0.8.1
protoc_version=3.11.4
serialization_version=0.20.0-1.3.70-eap-274-2
slf4j_version=1.7.28
typesafe_version=1.4.0
utils_version=b107c71
zipkin_version=5.11.2
Binary file modified gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
4 changes: 2 additions & 2 deletions gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#Fri Nov 22 10:04:40 PST 2019
distributionUrl=https\://services.gradle.org/distributions/gradle-6.0.1-all.zip
#Wed Mar 25 12:27:26 PDT 2020
distributionUrl=https\://services.gradle.org/distributions/gradle-6.3-all.zip
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
zipStorePath=wrapper/dists
Expand Down
3 changes: 3 additions & 0 deletions gradlew.bat
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ if "%DIRNAME%" == "" set DIRNAME=.
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%

@rem Resolve any "." and ".." in APP_HOME to make it shorter.
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi

@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"

Expand Down
6 changes: 3 additions & 3 deletions grafana/prometheus-proxy.json
Original file line number Diff line number Diff line change
Expand Up @@ -981,7 +981,7 @@
},
"targets": [
{
"expr": "proxy_cummulative_agent_backlog_size",
"expr": "proxy_cumulative_agent_backlog_size",
"intervalFactor": 2,
"legendFormat": "",
"metric": "proxy_connected_agents",
Expand Down Expand Up @@ -1145,7 +1145,7 @@
},
"targets": [
{
"expr": "proxy_cummulative_agent_backlog_size",
"expr": "proxy_cumulative_agent_backlog_size",
"intervalFactor": 2,
"legendFormat": "",
"metric": "proxy_connected_agents",
Expand All @@ -1154,7 +1154,7 @@
}
],
"thresholds": "",
"title": "Cummulative Agent Backlog Size",
"title": "Cumulative Agent Backlog Size",
"type": "singlestat",
"valueFontSize": "80%",
"valueMaps": [
Expand Down
Loading

0 comments on commit 29b870d

Please sign in to comment.