From 67eac5dd25c86e33b155335fba36027c5dc98022 Mon Sep 17 00:00:00 2001 From: Milind Srivastava Date: Mon, 27 Apr 2026 10:16:24 -0400 Subject: [PATCH 1/2] feat: Updated quickstart to use precompute engine and v0.3.0; updated readme --- README.md | 23 +- asap-quickstart/README.md | 10 +- asap-quickstart/config/prometheus.yml | 4 +- .../docker-compose-precompute.dev.yml | 12 - .../docker-compose-precompute.local.yml | 12 - asap-quickstart/docker-compose-precompute.yml | 322 ------------------ asap-quickstart/docker-compose.dev.yml | 4 - asap-quickstart/docker-compose.yml | 158 +-------- 8 files changed, 29 insertions(+), 516 deletions(-) delete mode 100644 asap-quickstart/docker-compose-precompute.dev.yml delete mode 100644 asap-quickstart/docker-compose-precompute.local.yml delete mode 100644 asap-quickstart/docker-compose-precompute.yml diff --git a/README.md b/README.md index fd8f465b..c436f499 100644 --- a/README.md +++ b/README.md @@ -8,11 +8,11 @@ - **Configurable query accuracy** - **Ease-of-use with your existing tech stack** -![ASAPQuery v0.1.0 intercepts queries between Grafana and Prometheus, and accelerates them. Currently, it ingests data using Prometheus' remote_write interface](assets/img/asapquery_intro_figure.jpg) +![ASAPQuery intercepts queries between Grafana and Prometheus, and accelerates them. It ingests data using Prometheus' remote_write interface](assets/img/asapquery_intro_figure.jpg) -ASAPQuery v0.1.0 sits between Prometheus and Grafana. It intercepts queries from Grafana and answers them using streaming sketches, instead of scanning large volumes of raw data in Prometheus. -Currently, it ingests data using Prometheus' remote_write interface. -Future versions of ASAPQuery will accelerate queries against other observability systems (e.g. VictoriaMetrics) and time-series databases (e.g. Clickhouse, Elastic). +ASAPQuery sits between Prometheus and Grafana. It intercepts queries from Grafana and answers them using streaming sketches, instead of scanning large volumes of raw data in Prometheus. +It ingests data using Prometheus' remote_write interface. +ASAPQuery also targets other observability systems (e.g. VictoriaMetrics) and time-series databases (e.g. Clickhouse, Elastic). ## Quick Start @@ -47,29 +47,24 @@ ASAPQuery uses **streaming sketches** to: ## Architecture -ASAPQuery has four main components: the **asap-planner-rs** generates sketch configurations from your query workload, **asap-summary-ingest** deploys streaming pipelines in **Arroyo** that continuously build sketches from live Prometheus metrics, and **asap-query-engine** intercepts PromQL queries and serves them from those pre-computed sketches. +ASAPQuery has two main components: **asap-planner-rs** analyzes your PromQL query workload and generates sketch configurations, and **asap-query-engine** intercepts PromQL queries and serves them from pre-computed sketches. The query engine includes a built-in **precompute engine** that continuously builds sketches directly from live metrics via Prometheus `remote_write` — no external streaming infrastructure required. ### Components -- **[asap-planner-rs](asap-planner-rs/)** - Analyzes a PromQL query workload and auto-generates sketch configurations for asap-summary-ingest and asap-query-engine -- **[asap-summary-ingest](asap-summary-ingest/)** - Deploys Arroyo streaming pipelines that continuously compute and publish sketches from live metrics -- **[arroyo](https://github.com/ProjectASAP/arroyo)** - Fork of the [Arroyo](https://github.com/ArroyoSystems/arroyo) stream processing engine that runs the sketch-building SQL pipelines -- **[asap-query-engine](asap-query-engine/)** - Intercepts incoming PromQL queries and serves them from pre-computed sketches, falling back to Prometheus for unsupported queries +- **[asap-planner-rs](asap-planner-rs/)** - Analyzes a PromQL query workload and auto-generates sketch configurations for asap-query-engine +- **[asap-query-engine](asap-query-engine/)** - Intercepts incoming PromQL queries and serves them from pre-computed sketches, falling back to Prometheus for unsupported queries; includes a built-in precompute engine that continuously builds sketches from live metrics via Prometheus `remote_write` ### Repository Structure ``` ├── asap-quickstart/ # Self-contained demo (start here!) ├── asap-planner-rs/ # Auto-configuration service -├── asap-summary-ingest/ # Arroyo pipeline deployer -└── asap-query-engine/ # Query serving engine -# Note: Arroyo fork lives at https://github.com/ProjectASAP/arroyo +└── asap-query-engine/ # Query serving and sketch precomputation engine ``` ## Coming soon -1. Drop-in ASAPQuery artifact that works with your existing pre-configured Prometheus-Grafana stack -2. Drop-in ASAPQuery artifact that accelerates Clickhouse queries +1. Drop-in ASAPQuery artifact that accelerates Clickhouse queries ## Current state diff --git a/asap-quickstart/README.md b/asap-quickstart/README.md index e0142419..796d0551 100644 --- a/asap-quickstart/README.md +++ b/asap-quickstart/README.md @@ -2,7 +2,7 @@ ## What is ASAPQuery? -ASAPQuery is a drop-in accelerator that reduces query latency by 100x. The current version of ASAPQuery (v0.1.0) sits between Prometheus and Grafana and accelerates repeating PromQL queries. +ASAPQuery is a drop-in accelerator that reduces query latency by 100x. It sits between Prometheus and Grafana and accelerates repeating PromQL queries. ASAPQuery is compatible with the Prometheus query API and integrates seamlessly with existing Grafana dashboards. ## What This Demo Shows @@ -13,9 +13,7 @@ This quickstart simulates a typical monitoring deployment with components you mi - Grafana - send queries to Prometheus and visualizing query results Then it adds ASAPQuery's components on top: -- **Query Engine** - Prometheus-compatible API with sketch-based acceleration -- **[Arroyo](https://github.com/ProjectASAP/arroyo) + asap-summary-ingest** - Streaming engine with pipelines configured for building sketches -- **Kafka** - Message broker for streaming data from Arroyo to the Query Engine +- **Query Engine** - Prometheus-compatible API with sketch-based acceleration and built-in precompute engine - **asap-planner-rs** - Automatically configures sketches from PromQL queries Once you run the quickstart, you will see a pre-configured Grafana dashboard that compares Prometheues and ASAPQuery side-by-side. You will see **visually indistinguishable** results from Prometheus and ASAPQuery, with ASAPQuery being 100x faster @@ -23,7 +21,7 @@ Once you run the quickstart, you will see a pre-configured Grafana dashboard tha ## Prerequisites - **Docker & Docker Compose** v2.0+ -- **Ports available**: 3000 (Grafana), 5115 (Arroyo), 8088 (ASAPQuery), 9090 (Prometheus) +- **Ports available**: 3000 (Grafana), 8088 (ASAPQuery), 9090 (Prometheus) ## Quick Start @@ -117,4 +115,4 @@ docker compose restart ### Running ASAPQuery with your own Grafana and Prometheus setup -**Coming Soon**: A drop-in ASAPQuery artifact that works with your existing already-configured Grafana and Prometheus deployments. +See [asap-dropin/](../asap-dropin/README.md) for a single-container setup that integrates with an existing Prometheus and Grafana deployment. diff --git a/asap-quickstart/config/prometheus.yml b/asap-quickstart/config/prometheus.yml index 6ec5ffc2..c336930d 100644 --- a/asap-quickstart/config/prometheus.yml +++ b/asap-quickstart/config/prometheus.yml @@ -4,9 +4,9 @@ global: scrape_interval: 1s evaluation_interval: 1s -# Remote write configuration to send metrics to Arroyo for sketch building +# Remote write configuration to send metrics to ASAPQuery for sketch building remote_write: - - url: http://arroyo:9091/receive + - url: http://queryengine:9091/receive queue_config: batch_send_deadline: 1s # Drop samples older than 5 minutes before enqueuing — prevents WAL replay diff --git a/asap-quickstart/docker-compose-precompute.dev.yml b/asap-quickstart/docker-compose-precompute.dev.yml deleted file mode 100644 index 9786ea9e..00000000 --- a/asap-quickstart/docker-compose-precompute.dev.yml +++ /dev/null @@ -1,12 +0,0 @@ -name: asapquery-quickstart-precompute - -# Development override for precompute variant: builds query engine from local source. -# -# Usage: -# docker compose -f docker-compose-precompute.yml -f docker-compose-precompute.dev.yml up -d --build - -services: - queryengine: - build: - context: .. - dockerfile: asap-query-engine/Dockerfile diff --git a/asap-quickstart/docker-compose-precompute.local.yml b/asap-quickstart/docker-compose-precompute.local.yml deleted file mode 100644 index c2a2320a..00000000 --- a/asap-quickstart/docker-compose-precompute.local.yml +++ /dev/null @@ -1,12 +0,0 @@ -name: asapquery-quickstart-precompute - -# Override: builds query engine from pre-built local binary (fast). -# Usage: -# cargo build --release -p query_engine_rust # build locally first -# sudo docker compose -f docker-compose-precompute.yml -f docker-compose-precompute.local.yml up -d --build - -services: - queryengine: - build: - context: .. - dockerfile: asap-quickstart/Dockerfile.queryengine-local diff --git a/asap-quickstart/docker-compose-precompute.yml b/asap-quickstart/docker-compose-precompute.yml deleted file mode 100644 index e0353ada..00000000 --- a/asap-quickstart/docker-compose-precompute.yml +++ /dev/null @@ -1,322 +0,0 @@ -name: asapquery-quickstart-precompute - -# Docker Compose file for pattern-based fake exporters demo -# Uses the hand-crafted precompute engine instead of Arroyo for streaming aggregation -# This eliminates the need for Kafka, Arroyo, and asap-summary-ingest - -networks: - asap-network: - driver: bridge - ipam: - driver: default - config: - - subnet: 172.25.0.0/16 - -volumes: - prometheus-data: - grafana-data: - asap-planner-output: - -services: - ############################################################################# - # INFRASTRUCTURE SERVICES - ############################################################################# - - prometheus: - image: prom/prometheus:v3.9.1 - container_name: asap-prometheus - hostname: prometheus - networks: - - asap-network - ports: - - "9090:9090" - volumes: - - ./config/prometheus-precompute.yml:/etc/prometheus/prometheus.yml:ro - - prometheus-data:/prometheus - command: - - "--config.file=/etc/prometheus/prometheus.yml" - - "--storage.tsdb.path=/prometheus" - - "--web.console.libraries=/usr/share/prometheus/console_libraries" - - "--web.console.templates=/usr/share/prometheus/consoles" - - "--web.enable-lifecycle" - healthcheck: - test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:9090/-/healthy || exit 1"] - interval: 10s - timeout: 5s - retries: 5 - depends_on: - queryengine: - condition: service_healthy - restart: no - - grafana: - image: grafana/grafana-enterprise:12.3.3 - container_name: asap-grafana - hostname: grafana - networks: - - asap-network - ports: - - "3000:3000" - environment: - - GF_SECURITY_ADMIN_PASSWORD=admin - - GF_SECURITY_ADMIN_USER=admin - - GF_USERS_ALLOW_SIGN_UP=false - - GF_SERVER_ROOT_URL=http://localhost:3000 - - GF_SECURITY_ALLOW_EMBEDDING=true - volumes: - - grafana-data:/var/lib/grafana - - ./config/grafana/provisioning:/etc/grafana/provisioning:ro - healthcheck: - test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:3000/api/health || exit 1"] - interval: 10s - timeout: 5s - retries: 5 - restart: no - - ############################################################################# - # INIT CONTAINERS - ############################################################################# - - asap-planner-rs: - image: ghcr.io/projectasap/asap-planner-rs:v0.2.0 - container_name: asap-planner-rs - hostname: asap-planner-rs - networks: - - asap-network - command: - - "--input_config=/config/controller-config.yaml" - - "--output_dir=/asap-planner-output" - - "--prometheus_scrape_interval=1" - - "--streaming_engine=arroyo" - - "--range-duration=300" - - "--step=10" - volumes: - - ./config/controller-config.yaml:/config/controller-config.yaml:ro - - asap-planner-output:/asap-planner-output - restart: "no" - - ############################################################################# - # CORE SERVICES - ############################################################################# - - queryengine: - image: ghcr.io/projectasap/asap-query-engine:v0.2.0-precompute - container_name: asap-queryengine - hostname: queryengine - networks: - - asap-network - ports: - - "8088:8088" - expose: - - "9091" - environment: - - RUST_LOG=INFO - - RUST_BACKTRACE=1 - volumes: - - asap-planner-output:/asap-planner-output:ro - - ./output/queryengine:/app/outputs - command: - - "--config=/asap-planner-output/inference_config.yaml" - - "--streaming-config=/asap-planner-output/streaming_config.yaml" - - "--prometheus-server=http://prometheus:9090" - - "--prometheus-scrape-interval=1" - - "--streaming-engine=precompute" - - "--prometheus-remote-write-port=9091" - - "--delete-existing-db" - - "--log-level=INFO" - - "--output-dir=/app/outputs" - - "--query-language=PROMQL" - - "--lock-strategy=per-key" - - "--forward-unsupported-queries" - healthcheck: - test: ["CMD-SHELL", "bash -c 'echo > /dev/tcp/localhost/8088' 2>/dev/null || exit 1"] - interval: 10s - timeout: 5s - retries: 10 - start_period: 15s - depends_on: - asap-planner-rs: - condition: service_completed_successfully - restart: no - - ############################################################################# - # PATTERN-BASED FAKE EXPORTERS - # Each exporter generates one pattern type with the 'pattern' label - # All series within an exporter follow the same pattern shape with variation - ############################################################################# - - # Constant values - baseline for comparison - fake-exporter-constant: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-constant - hostname: fake-exporter-constant - networks: - - asap-network - expose: - - "50000" - command: - - "--port=50000" - - "--valuescale=1000" - - "--dataset=constant" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Linear increasing - tests trend preservation - fake-exporter-linear-up: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-linear-up - hostname: fake-exporter-linear-up - networks: - - asap-network - expose: - - "50001" - command: - - "--port=50001" - - "--valuescale=1000" - - "--dataset=linear-up" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Linear decreasing - tests trend preservation - fake-exporter-linear-down: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-linear-down - hostname: fake-exporter-linear-down - networks: - - asap-network - expose: - - "50002" - command: - - "--port=50002" - - "--valuescale=1000" - - "--dataset=linear-down" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Sine wave - tests periodicity preservation - fake-exporter-sine: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-sine - hostname: fake-exporter-sine - networks: - - asap-network - expose: - - "50003" - command: - - "--port=50003" - - "--valuescale=1000" - - "--dataset=sine" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Sine with noise - tests signal extraction / smoothing - fake-exporter-sine-noise: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-sine-noise - hostname: fake-exporter-sine-noise - networks: - - asap-network - expose: - - "50004" - command: - - "--port=50004" - - "--valuescale=1000" - - "--dataset=sine-noise" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Step function - tests edge preservation - fake-exporter-step: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-step - hostname: fake-exporter-step - networks: - - asap-network - expose: - - "50005" - command: - - "--port=50005" - - "--valuescale=1000" - - "--dataset=step" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Spiky pattern - tests sudden spikes/drops - fake-exporter-spiky: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-spiky - hostname: fake-exporter-spiky - networks: - - asap-network - expose: - - "50006" - command: - - "--port=50006" - - "--valuescale=1000" - - "--dataset=spiky" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no - - # Exponential growth - tests non-linear patterns - fake-exporter-exp-up: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 - container_name: asap-fake-exporter-exp-up - hostname: fake-exporter-exp-up - networks: - - asap-network - expose: - - "50007" - command: - - "--port=50007" - - "--valuescale=1000" - - "--dataset=exp-up" - - "--num-labels=3" - - "--num-values-per-label=30,30,30" - - "--metric-type=gauge" - - "--metric-name=sensor_reading" - - "--label-names=region,service,host" - - "--label-value-prefixes=region,svc,host" - - "--add-pattern-label" - restart: no diff --git a/asap-quickstart/docker-compose.dev.yml b/asap-quickstart/docker-compose.dev.yml index 96b5fc0f..10531c8d 100644 --- a/asap-quickstart/docker-compose.dev.yml +++ b/asap-quickstart/docker-compose.dev.yml @@ -14,10 +14,6 @@ services: context: .. dockerfile: asap-planner-rs/Dockerfile - asap-summary-ingest: - build: - context: ../asap-summary-ingest - queryengine: build: context: .. diff --git a/asap-quickstart/docker-compose.yml b/asap-quickstart/docker-compose.yml index 9622d723..fd83f944 100644 --- a/asap-quickstart/docker-compose.yml +++ b/asap-quickstart/docker-compose.yml @@ -13,7 +13,6 @@ networks: - subnet: 172.25.0.0/16 volumes: - kafka-data: prometheus-data: grafana-data: asap-planner-output: @@ -23,97 +22,6 @@ services: # INFRASTRUCTURE SERVICES ############################################################################# - kafka: - image: apache/kafka:3.7.0 - container_name: asap-kafka - hostname: kafka - networks: - - asap-network - environment: - KAFKA_NODE_ID: 1 - KAFKA_PROCESS_ROLES: broker,controller - KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:9093 - KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092 - KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:9093 - KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT - KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 - KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1 - KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1 - KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 - KAFKA_MESSAGE_MAX_BYTES: 20971520 - KAFKA_REPLICA_FETCH_MAX_BYTES: 20971520 - KAFKA_LOG_RETENTION_HOURS: 1 - KAFKA_LOG_DIRS: /tmp/kraft-combined-logs - CLUSTER_ID: MkU3OEVBNTcwNTJENDM2Qk - volumes: - - kafka-data:/tmp/kraft-combined-logs - user: "0:0" - entrypoint: /bin/bash - command: - - -c - - | - chown -R appuser:appuser /tmp/kraft-combined-logs - chmod -R 755 /tmp/kraft-combined-logs - exec su appuser -c "/etc/kafka/docker/run" - healthcheck: - test: ["CMD-SHELL", "/opt/kafka/bin/kafka-broker-api-versions.sh --bootstrap-server localhost:9092 || exit 1"] - interval: 10s - timeout: 5s - retries: 5 - start_period: 30s - restart: no - - kafka-init: - image: apache/kafka:3.7.0 - container_name: asap-kafka-init - networks: - - asap-network - depends_on: - kafka: - condition: service_healthy - entrypoint: /bin/bash - command: - - -c - - | - echo "Creating Kafka topics..." - /opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka:9092 \ - --create --if-not-exists --topic flink_input \ - --partitions 1 --replication-factor 1 \ - --config max.message.bytes=20971520 - - /opt/kafka/bin/kafka-topics.sh --bootstrap-server kafka:9092 \ - --create --if-not-exists --topic flink_output \ - --partitions 1 --replication-factor 1 \ - --config max.message.bytes=20971520 - - echo "Kafka topics created successfully" - restart: "no" - - arroyo: - image: ghcr.io/projectasap/asap-arroyo:v0.1.0 - container_name: asap-arroyo - hostname: arroyo - networks: - - asap-network - ports: - - "5115:5115" - volumes: - - ./config/arroyo-config.yaml:/config.yaml:ro - command: ["--config", "/config.yaml", "cluster"] - environment: - - ARROYO__API__RUN_HTTP_PORT=5115 - - KAFKA_BOOTSTRAP_SERVERS=kafka:9092 - depends_on: - kafka: - condition: service_healthy - healthcheck: - test: ["CMD-SHELL", "curl -f http://localhost:5115/api/v1/pipelines || exit 1"] - interval: 10s - timeout: 5s - retries: 5 - restart: no - prometheus: image: prom/prometheus:v3.9.1 container_name: asap-prometheus @@ -137,9 +45,6 @@ services: timeout: 5s retries: 10 start_period: 3m - depends_on: - asap-summary-ingest: - condition: service_completed_successfully restart: no grafana: @@ -171,7 +76,7 @@ services: ############################################################################# asap-planner-rs: - image: ghcr.io/projectasap/asap-planner-rs:v0.2.0 + image: ghcr.io/projectasap/asap-planner-rs:v0.3.0 container_name: asap-planner-rs hostname: asap-planner-rs networks: @@ -180,7 +85,7 @@ services: - "--input_config=/config/controller-config.yaml" - "--output_dir=/asap-planner-output" - "--prometheus_scrape_interval=1" - - "--streaming_engine=arroyo" + - "--streaming_engine=precompute" - "--range-duration=300" - "--step=10" - "-v" @@ -189,47 +94,19 @@ services: - asap-planner-output:/asap-planner-output restart: "no" - asap-summary-ingest: - image: ghcr.io/projectasap/asap-summary-ingest:v0.2.0 - container_name: asap-summary-ingest - hostname: asap-summary-ingest - networks: - - asap-network - command: - - "--config_file_path=/asap-planner-output/streaming_config.yaml" - - "--source_type=prometheus_remote_write" - - "--prometheus_base_port=9091" - - "--prometheus_path=/receive" - - "--prometheus_bind_ip=0.0.0.0" - - "--parallelism=1" - - "--output_kafka_topic=flink_output" - - "--output_format=json" - - "--pipeline_name=asap-demo" - - "--output_dir=/asap-summary-ingest-output" - - "--arroyo_url=http://arroyo:5115/api/v1" - - "--bootstrap_servers=kafka:9092" - volumes: - - asap-planner-output:/asap-planner-output:ro - - ./output/asap-summary-ingest:/asap-summary-ingest-output - depends_on: - asap-planner-rs: - condition: service_completed_successfully - arroyo: - condition: service_healthy - restart: "no" - ############################################################################# # CORE SERVICES ############################################################################# queryengine: - image: ghcr.io/projectasap/asap-query-engine:v0.2.0 + image: ghcr.io/projectasap/asap-query-engine:v0.3.0 container_name: asap-queryengine hostname: queryengine networks: - asap-network ports: - "8088:8088" + - "9091:9091" environment: - RUST_LOG=INFO - RUST_BACKTRACE=1 @@ -237,27 +114,20 @@ services: - asap-planner-output:/asap-planner-output:ro - ./output/queryengine:/app/outputs command: - - "--kafka-topic=flink_output" - - "--kafka-broker=kafka:9092" - - "--input-format=json" - "--config=/asap-planner-output/inference_config.yaml" - "--streaming-config=/asap-planner-output/streaming_config.yaml" - "--prometheus-server=http://prometheus:9090" - "--prometheus-scrape-interval=1" - - "--streaming-engine=arroyo" + - "--streaming-engine=precompute" + - "--prometheus-remote-write-port=9091" - "--delete-existing-db" - "--log-level=INFO" - "--output-dir=/app/outputs" - "--query-language=PROMQL" - "--lock-strategy=per-key" - - "--decompress-json" - "--forward-unsupported-queries" depends_on: - asap-summary-ingest: - condition: service_completed_successfully - kafka: - condition: service_healthy - kafka-init: + asap-planner-rs: condition: service_completed_successfully prometheus: condition: service_healthy @@ -271,7 +141,7 @@ services: # Constant values - baseline for comparison fake-exporter-constant: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-constant hostname: fake-exporter-constant networks: @@ -293,7 +163,7 @@ services: # Linear increasing - tests trend preservation fake-exporter-linear-up: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-linear-up hostname: fake-exporter-linear-up networks: @@ -315,7 +185,7 @@ services: # Linear decreasing - tests trend preservation fake-exporter-linear-down: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-linear-down hostname: fake-exporter-linear-down networks: @@ -337,7 +207,7 @@ services: # Sine wave - tests periodicity preservation fake-exporter-sine: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-sine hostname: fake-exporter-sine networks: @@ -359,7 +229,7 @@ services: # Sine with noise - tests signal extraction / smoothing fake-exporter-sine-noise: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-sine-noise hostname: fake-exporter-sine-noise networks: @@ -381,7 +251,7 @@ services: # Step function - tests edge preservation fake-exporter-step: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-step hostname: fake-exporter-step networks: @@ -403,7 +273,7 @@ services: # Exponential growth - tests non-linear patterns fake-exporter-exp-up: - image: ghcr.io/projectasap/asap-fake-exporter:v0.2.0 + image: ghcr.io/projectasap/asap-fake-exporter:v0.3.0 container_name: asap-fake-exporter-exp-up hostname: fake-exporter-exp-up networks: From 81fef4c778b35ed2686b9e7935a3b4efccd99297 Mon Sep 17 00:00:00 2001 From: Milind Srivastava Date: Mon, 27 Apr 2026 10:22:44 -0400 Subject: [PATCH 2/2] updated readmes --- README.md | 11 +++++++++++ asap-quickstart/README.md | 2 +- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index c436f499..e0470f58 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,16 @@ Open http://localhost:3000 and see ASAPQuery vs Prometheus side-by-side! Full quickstart instructions at [**Quickstart Guide**](asap-quickstart/README.md) +**Already have Prometheus and Grafana running?** Use the drop-in instead: + +```bash +cd asap-dropin +# Edit .env to point at your Prometheus, then: +docker compose up -d +``` + +Full drop-in instructions at [**Drop-in Guide**](asap-dropin/README.md) + ## Why ASAPQuery? ### The Problem @@ -58,6 +68,7 @@ ASAPQuery has two main components: **asap-planner-rs** analyzes your PromQL quer ``` ├── asap-quickstart/ # Self-contained demo (start here!) +├── asap-dropin/ # Drop-in for existing Prometheus-Grafana stacks ├── asap-planner-rs/ # Auto-configuration service └── asap-query-engine/ # Query serving and sketch precomputation engine ``` diff --git a/asap-quickstart/README.md b/asap-quickstart/README.md index 796d0551..645f706d 100644 --- a/asap-quickstart/README.md +++ b/asap-quickstart/README.md @@ -21,7 +21,7 @@ Once you run the quickstart, you will see a pre-configured Grafana dashboard tha ## Prerequisites - **Docker & Docker Compose** v2.0+ -- **Ports available**: 3000 (Grafana), 8088 (ASAPQuery), 9090 (Prometheus) +- **Ports available**: 3000 (Grafana), 8088 (ASAPQuery query), 9091 (ASAPQuery ingest), 9090 (Prometheus) ## Quick Start