Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ src/
apps.rs Apps command (fetch, DB write)
consumers.rs Consumers command (paginated fetch, DB write)
request_logs.rs Request logs command (Arrow IPC or NDJSON streaming)
request_details.rs Request details command (single request fetch, DB write)
sql.rs SQL command (query DuckDB, output NDJSON)
utils.rs Shared helpers (open DuckDB connection, check HTTP response)
npm/
Expand All @@ -33,14 +34,15 @@ npm/

## CLI Subcommands

| Subcommand | Data source |
| -------------- | -------------------------------------------- |
| `auth` | — |
| `whoami` | `GET /v1/team` |
| `apps` | `GET /v1/apps` |
| `consumers` | `GET /v1/apps/{app_id}/consumers` |
| `request-logs` | `POST /v1/apps/{app_id}/request-logs/stream` |
| `sql` | Local DuckDB |
| Subcommand | Data source |
| ----------------- | --------------------------------------------------- |
| `auth` | — |
| `whoami` | `GET /v1/team` |
| `apps` | `GET /v1/apps` |
| `consumers` | `GET /v1/apps/{app_id}/consumers` |
| `request-logs` | `POST /v1/apps/{app_id}/request-logs/stream` |
| `request-details` | `GET /v1/apps/{app_id}/request-logs/{request_uuid}` |
| `sql` | Local DuckDB |

## Authentication with API

Expand Down
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,14 +72,15 @@ You can also set the API key via the `APITALLY_API_KEY` environment variable or

## Commands

| Command | Description |
| -------------- | ----------------------------------------------- |
| `auth` | Configure API key |
| `whoami` | Check authentication and show team info |
| `apps` | List all apps in your team |
| `consumers` | List consumers for an app |
| `request-logs` | Fetch request log data for an app |
| `sql` | Run SQL queries against a local DuckDB database |
| Command | Description |
| ----------------- | ----------------------------------------------- |
| `auth` | Configure API key |
| `whoami` | Check authentication and show team info |
| `apps` | List all apps in your team |
| `consumers` | List consumers for an app |
| `request-logs` | Fetch request log data for an app |
| `request-details` | Fetch details for a specific request |
| `sql` | Run SQL queries against a local DuckDB database |

Commands that fetch data from the API output NDJSON to stdout by default. They accept a `--db` flag to write data to a local DuckDB database instead, which can then be queried with the `sql` command. The database defaults to `~/.apitally/data.duckdb` if no other path is specified.

Expand Down
18 changes: 5 additions & 13 deletions skills/apitally-cli/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ All commands are run via `npx @apitally/cli <command>`. For full details, see [r
- `apps [--db [<path>]]` -- list apps (get app IDs)
- `consumers <app-id> [--requests-since <dt>] [--db [<path>]]` -- list consumers for an app
- `request-logs <app-id> --since <dt> [--until <dt>] [--fields <json>] [--filters <json>] [--limit <n>] [--db [<path>]]` -- fetch request logs (max 1,000,000 rows at once)
- `request-details <app-id> <request-uuid> [--db [<path>]]` -- fetch full details for a single request (headers, body, logs, spans)
- `sql "<query>" [--db <path>]` -- run SQL against local DuckDB

## Investigation Patterns
Expand Down Expand Up @@ -126,19 +127,10 @@ ORDER BY p95_ms DESC

### Inspect a specific request

To inspect headers and payloads, first re-fetch with additional fields:
Use `request-details` to fetch full details (headers, body, exception, application logs, spans) for a single request:

```
npx @apitally/cli request-logs <app-id> --since "2026-03-23T00:00:00Z" \
--fields '["timestamp","request_uuid","method","url","status_code","response_time_ms","request_headers","request_body_json","response_body_json","exception_type","exception_message","exception_stacktrace"]' \
--filters '[{"field":"request_uuid","op":"eq","value":"<uuid>"}]' \
--db
```

Then query:

```sql
SELECT * FROM request_logs WHERE request_uuid = '<uuid>'
npx @apitally/cli request-details <app-id> <request-uuid>
```

### Trace a consumer's activity
Expand Down Expand Up @@ -189,11 +181,11 @@ ORDER BY count DESC

### Query headers

Headers are stored as arrays of `STRUCT("1" VARCHAR, "2" VARCHAR)` (name-value tuples). Use DuckDB list comprehensions:
Headers are stored as arrays of `STRUCT(name VARCHAR, value VARCHAR)`. Use DuckDB list comprehensions:

```sql
SELECT timestamp, method, path,
[s."2" FOR s IN request_headers IF lower(s."1") = 'content-type'][1] as content_type
[s.value FOR s IN request_headers IF lower(s.name) = 'content-type'][1] as content_type
FROM request_logs
WHERE request_headers IS NOT NULL
LIMIT 20
Expand Down
21 changes: 18 additions & 3 deletions skills/apitally-cli/references/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,6 @@ npx @apitally/cli apps [--db [<path>]]

List all apps in the team. Use this to get app IDs for other commands. Outputs NDJSON to stdout by default.

**Arguments:**

- `--db`: Write to `apps` and `app_envs` tables in DuckDB instead of outputting NDJSON to stdout

Example NDJSON output (without `--db`):
Expand Down Expand Up @@ -158,6 +156,23 @@ Example NDJSON output (without `--db`):
{"timestamp":"2026-01-01T00:16:00.000Z","request_uuid":"c6d32f8a-0bc1-43c1-b6c5-7d04363dc97c","env":"prod","method":"GET","path":"/test/2","url":"https://api.example.com/test/2","consumer_id":1,"request_size_bytes":0,"status_code":500,"response_time_ms":68,"response_size_bytes":66,"client_ip":"198.51.100.22","client_country_iso_code":"US"}
```

## `request-details`

```
npx @apitally/cli request-details <app-id> <request-uuid> [--db [<path>]]
```

Get full details for a specific request identified by its UUID, including headers, request/response body, exception info, application logs, and spans. Outputs a JSON object to stdout by default.

- `--db`: Write to `request_logs`, `application_logs`, and `spans` tables in DuckDB instead of outputting JSON to stdout

Example JSON output (without `--db`):

<!-- prettier-ignore -->
```json
{"timestamp":"2026-01-01T00:15:00.000Z","request_uuid":"2fbc1df6-3124-4ed1-a376-7d2c64e4d5cf","env":"prod","method":"GET","path":"/test/1","url":"https://api.example.com/test/1","consumer":"bob@example.com","request_headers":[["content-type","application/json"]],"request_size_bytes":0,"request_body_json":null,"status_code":200,"response_time_ms":122,"response_headers":[["x-request-id","abc"]],"response_size_bytes":66,"response_body_json":"{\"ok\":true}","client_ip":"203.0.113.10","client_country_iso_code":"DE","trace_id":"0000000000000000aaaaaaaaaaaaaaaa","exception":null,"logs":[{"timestamp":"2026-01-01T00:15:00.100Z","message":"handling request","level":"INFO","logger":"app","file":"main.py","line":42}],"spans":[{"span_id":"00000000000000aa","parent_span_id":null,"name":"GET /test/1","kind":"SERVER","start_time_ns":1735689600000000000,"end_time_ns":1735689600050000000,"duration_ns":50000000,"status":"OK","attributes":{"http.method":"GET"}}]}
```

## `sql`

```
Expand All @@ -170,7 +185,7 @@ Run a SQL query against a local DuckDB database. The query can be passed as an a

- `--db`: Path to DuckDB database

Available tables: `apps`, `app_envs`, `consumers`, `request_logs`. See [tables.md](tables.md) for schemas.
Available tables: `apps`, `app_envs`, `consumers`, `request_logs`, `application_logs`, `spans`. See [tables.md](tables.md) for schemas.

DuckDB uses a [PostgreSQL-compatible SQL dialect](https://duckdb.org/docs/stable/sql/dialect/overview).

Expand Down
53 changes: 46 additions & 7 deletions skills/apitally-cli/references/tables.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# DuckDB Table Schemas

Tables are created automatically when using the `--db` flag with `apps`, `consumers`, or `request-logs` commands. DuckDB uses a [PostgreSQL-compatible SQL dialect](https://duckdb.org/docs/stable/sql/dialect/overview).
Tables are created automatically when using the `--db` flag with `apps`, `consumers`, `request-logs`, or `request-details` commands. DuckDB uses a [PostgreSQL-compatible SQL dialect](https://duckdb.org/docs/stable/sql/dialect/overview).

## apps

Expand Down Expand Up @@ -56,12 +56,12 @@ CREATE TABLE request_logs (
path VARCHAR,
url VARCHAR NOT NULL,
consumer_id INTEGER,
request_headers STRUCT("1" VARCHAR, "2" VARCHAR)[],
request_headers STRUCT(name VARCHAR, value VARCHAR)[],
request_size_bytes BIGINT,
request_body_json JSON,
status_code INTEGER,
response_time_ms INTEGER,
response_headers STRUCT("1" VARCHAR, "2" VARCHAR)[],
response_headers STRUCT(name VARCHAR, value VARCHAR)[],
response_size_bytes BIGINT,
response_body_json JSON,
client_ip VARCHAR,
Expand All @@ -77,25 +77,64 @@ CREATE TABLE request_logs (

Columns are only populated if the corresponding field was included in the `--fields` flag during fetch.

## application_logs

```sql
CREATE TABLE application_logs (
app_id INTEGER NOT NULL,
request_uuid VARCHAR NOT NULL,
timestamp TIMESTAMPTZ NOT NULL,
message VARCHAR NOT NULL,
level VARCHAR,
logger VARCHAR,
file VARCHAR,
line INTEGER
);
```

Populated by the `request-details` command when using `--db`.

## spans

```sql
CREATE TABLE spans (
app_id INTEGER NOT NULL,
request_uuid VARCHAR NOT NULL,
span_id VARCHAR NOT NULL,
parent_span_id VARCHAR,
name VARCHAR NOT NULL,
kind VARCHAR NOT NULL,
start_time_ns BIGINT NOT NULL,
end_time_ns BIGINT NOT NULL,
duration_ns BIGINT NOT NULL,
status VARCHAR NOT NULL,
attributes JSON
);
```

Populated by the `request-details` command when using `--db`.

## Relationships

- `request_logs.consumer_id` references `consumers.consumer_id` (join on both `app_id` and `consumer_id`)
- `request_logs.app_id` references `apps.app_id`
- `app_envs.app_id` references `apps.app_id`
- `request_logs.env` matches `app_envs.name` (string, not a foreign key to `app_env_id`)
- `application_logs.request_uuid` references `request_logs.request_uuid` (join on both `app_id` and `request_uuid`)
- `spans.request_uuid` references `request_logs.request_uuid` (join on both `app_id` and `request_uuid`)

## Special Types

### Headers (`STRUCT("1" VARCHAR, "2" VARCHAR)[]`)
### Headers (`STRUCT(name VARCHAR, value VARCHAR)[]`)

Headers are arrays of structs where `"1"` is the header name and `"2"` is the header value. Use DuckDB list comprehensions:
Headers are arrays of structs with `name` and `value` fields. Use DuckDB list comprehensions:

```sql
-- Extract a specific header value
[s."2" FOR s IN request_headers IF lower(s."1") = 'content-type'][1]
[s.value FOR s IN request_headers IF lower(s.name) = 'content-type'][1]

-- Check if a header exists
len([s FOR s IN request_headers IF lower(s."1") = 'authorization']) > 0
len([s FOR s IN request_headers IF lower(s.name) = 'authorization']) > 0
```

### JSON body fields (`JSON`)
Expand Down
12 changes: 10 additions & 2 deletions src/apps.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,16 @@ fn ensure_apps_tables(conn: &duckdb::Connection) -> Result<()> {
}

fn write_apps_to_db(conn: &duckdb::Connection, apps: &[AppItem]) -> Result<()> {
let mut app_stmt = conn.prepare("INSERT OR REPLACE INTO apps VALUES (?, ?, ?, ?, ?)")?;
let mut env_stmt = conn.prepare("INSERT OR REPLACE INTO app_envs VALUES (?, ?, ?, ?, ?)")?;
let mut app_stmt = conn.prepare(
"INSERT OR REPLACE INTO apps (
app_id, name, framework, client_id, created_at
) VALUES (?, ?, ?, ?, ?)",
)?;
let mut env_stmt = conn.prepare(
"INSERT OR REPLACE INTO app_envs (
app_id, app_env_id, name, created_at, last_sync_at
) VALUES (?, ?, ?, ?, ?)",
)?;
for app in apps {
app_stmt.execute(duckdb::params![
app.id,
Expand Down
7 changes: 6 additions & 1 deletion src/consumers.rs
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,12 @@ fn write_consumers_to_db(
app_id: i64,
consumers: &[ConsumerItem],
) -> Result<()> {
let mut stmt = conn.prepare("INSERT OR REPLACE INTO consumers VALUES (?, ?, ?, ?, ?, ?, ?)")?;
let mut stmt = conn.prepare(
"INSERT OR REPLACE INTO consumers (
app_id, consumer_id, identifier, name, \"group\",
created_at, last_request_at
) VALUES (?, ?, ?, ?, ?, ?, ?)",
)?;
for consumer in consumers {
let group_name = consumer.group.as_ref().map(|g| g.name.as_str());
stmt.execute(duckdb::params![
Expand Down
62 changes: 60 additions & 2 deletions src/main.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
mod apps;
mod auth;
mod consumers;
mod request_details;
mod request_logs;
mod sql;
mod utils;
Expand Down Expand Up @@ -153,9 +154,33 @@ enum Command {
db: Option<Option<PathBuf>>,
},

/// Get details for a specific request
///
/// Outputs a JSON object with full request details including headers, body,
/// application logs, and spans.
/// With --db, upserts the request into the `request_logs` table and inserts
/// rows into the `application_logs` and `spans` tables.
RequestDetails {
#[command(flatten)]
api: ApiArgs,

/// App ID
app_id: i64,

/// Request UUID
request_uuid: String,

/// Store results in DuckDB instead of outputting JSON
///
/// Defaults to ~/.apitally/data.duckdb if no path is given.
#[arg(long, num_args = 0..=1)]
db: Option<Option<PathBuf>>,
},

/// Run a SQL query against local DuckDB
///
/// Available tables: apps, app_envs, consumers, request_logs.
/// Available tables: apps, app_envs, consumers, request_logs,
/// application_logs, spans.
Sql {
/// SQL query to execute (reads from stdin if omitted)
query: Option<String>,
Expand Down Expand Up @@ -262,6 +287,22 @@ fn run(cli: Cli) -> Result<()> {
std::io::stdout().lock(),
)
}
Command::RequestDetails {
api,
app_id,
request_uuid,
db,
} => {
let db = utils::resolve_db(db)?;
request_details::run(
app_id,
&request_uuid,
db.as_deref(),
api.api_key.as_deref(),
api.api_base_url.as_deref(),
std::io::stdout().lock(),
)
}
Command::Sql { query, db } => {
let db = db.map_or_else(utils::default_db_path, Ok)?;
let query = match query {
Expand Down Expand Up @@ -298,6 +339,8 @@ mod tests {
assert!(Cli::try_parse_from(["apitally"]).is_err()); // missing command
assert!(Cli::try_parse_from(["apitally", "consumers"]).is_err()); // missing app_id
assert!(Cli::try_parse_from(["apitally", "request-logs", "42"]).is_err()); // missing --since
assert!(Cli::try_parse_from(["apitally", "request-details", "42"]).is_err()); // missing request_uuid
assert!(Cli::try_parse_from(["apitally", "sql", "SELECT 1", "--db"]).is_err()); // missing db path

// Valid subcommands should parse correctly
assert!(matches!(
Expand Down Expand Up @@ -325,7 +368,22 @@ mod tests {
Command::RequestLogs { app_id: 42, .. }
));
assert!(matches!(
Cli::try_parse_from(["apitally", "sql", "--db", "test.db"])
Cli::try_parse_from([
"apitally",
"request-details",
"42",
"f328bb2a-93e1-4c4a-a263-47be6a1bcb15"
])
.unwrap()
.command,
Command::RequestDetails {
app_id: 42,
ref request_uuid,
..
} if request_uuid == "f328bb2a-93e1-4c4a-a263-47be6a1bcb15"
));
assert!(matches!(
Cli::try_parse_from(["apitally", "sql", "--db", "test.duckdb"])
.unwrap()
.command,
Command::Sql { query: None, .. }
Expand Down
Loading
Loading