Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 20 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@
- **Configurable query accuracy**
- **Ease-of-use with your existing tech stack**

![ASAPQuery v0.1.0 intercepts queries between Grafana and Prometheus, and accelerates them. Currently, it ingests data using Prometheus' remote_write interface](assets/img/asapquery_intro_figure.jpg)
![ASAPQuery intercepts queries between Grafana and Prometheus, and accelerates them. It ingests data using Prometheus' remote_write interface](assets/img/asapquery_intro_figure.jpg)

ASAPQuery v0.1.0 sits between Prometheus and Grafana. It intercepts queries from Grafana and answers them using streaming sketches, instead of scanning large volumes of raw data in Prometheus.
Currently, it ingests data using Prometheus' remote_write interface.
Future versions of ASAPQuery will accelerate queries against other observability systems (e.g. VictoriaMetrics) and time-series databases (e.g. Clickhouse, Elastic).
ASAPQuery sits between Prometheus and Grafana. It intercepts queries from Grafana and answers them using streaming sketches, instead of scanning large volumes of raw data in Prometheus.
It ingests data using Prometheus' remote_write interface.
ASAPQuery also targets other observability systems (e.g. VictoriaMetrics) and time-series databases (e.g. Clickhouse, Elastic).

## Quick Start

Expand All @@ -27,6 +27,16 @@ Open http://localhost:3000 and see ASAPQuery vs Prometheus side-by-side!

Full quickstart instructions at [**Quickstart Guide**](asap-quickstart/README.md)

**Already have Prometheus and Grafana running?** Use the drop-in instead:

```bash
cd asap-dropin
# Edit .env to point at your Prometheus, then:
docker compose up -d
```

Full drop-in instructions at [**Drop-in Guide**](asap-dropin/README.md)

## Why ASAPQuery?

### The Problem
Expand All @@ -47,29 +57,25 @@ ASAPQuery uses **streaming sketches** to:

## Architecture

ASAPQuery has four main components: the **asap-planner-rs** generates sketch configurations from your query workload, **asap-summary-ingest** deploys streaming pipelines in **Arroyo** that continuously build sketches from live Prometheus metrics, and **asap-query-engine** intercepts PromQL queries and serves them from those pre-computed sketches.
ASAPQuery has two main components: **asap-planner-rs** analyzes your PromQL query workload and generates sketch configurations, and **asap-query-engine** intercepts PromQL queries and serves them from pre-computed sketches. The query engine includes a built-in **precompute engine** that continuously builds sketches directly from live metrics via Prometheus `remote_write` — no external streaming infrastructure required.

### Components

- **[asap-planner-rs](asap-planner-rs/)** - Analyzes a PromQL query workload and auto-generates sketch configurations for asap-summary-ingest and asap-query-engine
- **[asap-summary-ingest](asap-summary-ingest/)** - Deploys Arroyo streaming pipelines that continuously compute and publish sketches from live metrics
- **[arroyo](https://github.com/ProjectASAP/arroyo)** - Fork of the [Arroyo](https://github.com/ArroyoSystems/arroyo) stream processing engine that runs the sketch-building SQL pipelines
- **[asap-query-engine](asap-query-engine/)** - Intercepts incoming PromQL queries and serves them from pre-computed sketches, falling back to Prometheus for unsupported queries
- **[asap-planner-rs](asap-planner-rs/)** - Analyzes a PromQL query workload and auto-generates sketch configurations for asap-query-engine
- **[asap-query-engine](asap-query-engine/)** - Intercepts incoming PromQL queries and serves them from pre-computed sketches, falling back to Prometheus for unsupported queries; includes a built-in precompute engine that continuously builds sketches from live metrics via Prometheus `remote_write`

### Repository Structure

```
├── asap-quickstart/ # Self-contained demo (start here!)
├── asap-dropin/ # Drop-in for existing Prometheus-Grafana stacks
├── asap-planner-rs/ # Auto-configuration service
├── asap-summary-ingest/ # Arroyo pipeline deployer
└── asap-query-engine/ # Query serving engine
# Note: Arroyo fork lives at https://github.com/ProjectASAP/arroyo
└── asap-query-engine/ # Query serving and sketch precomputation engine
```

## Coming soon

1. Drop-in ASAPQuery artifact that works with your existing pre-configured Prometheus-Grafana stack
2. Drop-in ASAPQuery artifact that accelerates Clickhouse queries
1. Drop-in ASAPQuery artifact that accelerates Clickhouse queries

## Current state

Expand Down
10 changes: 4 additions & 6 deletions asap-quickstart/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## What is ASAPQuery?

ASAPQuery is a drop-in accelerator that reduces query latency by 100x. The current version of ASAPQuery (v0.1.0) sits between Prometheus and Grafana and accelerates repeating PromQL queries.
ASAPQuery is a drop-in accelerator that reduces query latency by 100x. It sits between Prometheus and Grafana and accelerates repeating PromQL queries.
ASAPQuery is compatible with the Prometheus query API and integrates seamlessly with existing Grafana dashboards.

## What This Demo Shows
Expand All @@ -13,17 +13,15 @@ This quickstart simulates a typical monitoring deployment with components you mi
- Grafana - send queries to Prometheus and visualizing query results

Then it adds ASAPQuery's components on top:
- **Query Engine** - Prometheus-compatible API with sketch-based acceleration
- **[Arroyo](https://github.com/ProjectASAP/arroyo) + asap-summary-ingest** - Streaming engine with pipelines configured for building sketches
- **Kafka** - Message broker for streaming data from Arroyo to the Query Engine
- **Query Engine** - Prometheus-compatible API with sketch-based acceleration and built-in precompute engine
- **asap-planner-rs** - Automatically configures sketches from PromQL queries

Once you run the quickstart, you will see a pre-configured Grafana dashboard that compares Prometheues and ASAPQuery side-by-side. You will see **visually indistinguishable** results from Prometheus and ASAPQuery, with ASAPQuery being 100x faster

## Prerequisites

- **Docker & Docker Compose** v2.0+
- **Ports available**: 3000 (Grafana), 5115 (Arroyo), 8088 (ASAPQuery), 9090 (Prometheus)
- **Ports available**: 3000 (Grafana), 8088 (ASAPQuery query), 9091 (ASAPQuery ingest), 9090 (Prometheus)

## Quick Start

Expand Down Expand Up @@ -117,4 +115,4 @@ docker compose restart

### Running ASAPQuery with your own Grafana and Prometheus setup

**Coming Soon**: A drop-in ASAPQuery artifact that works with your existing already-configured Grafana and Prometheus deployments.
See [asap-dropin/](../asap-dropin/README.md) for a single-container setup that integrates with an existing Prometheus and Grafana deployment.
4 changes: 2 additions & 2 deletions asap-quickstart/config/prometheus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ global:
scrape_interval: 1s
evaluation_interval: 1s

# Remote write configuration to send metrics to Arroyo for sketch building
# Remote write configuration to send metrics to ASAPQuery for sketch building
remote_write:
- url: http://arroyo:9091/receive
- url: http://queryengine:9091/receive
queue_config:
batch_send_deadline: 1s
# Drop samples older than 5 minutes before enqueuing — prevents WAL replay
Expand Down
12 changes: 0 additions & 12 deletions asap-quickstart/docker-compose-precompute.dev.yml

This file was deleted.

12 changes: 0 additions & 12 deletions asap-quickstart/docker-compose-precompute.local.yml

This file was deleted.

Loading
Loading