Skip to content

Server discovery runs one lsof per clickhouse PID — ~29s command stall on machines with many clickhouse-local/chDB processes #329

Description

@sdairs

Found during 0.4.0 RC local-command testing.

Observed

discover_clickhouse_processes() (crates/clickhousectl/src/local/discovery.rs) does pgrep -x clickhouse, then for each PID shells out to lsof -a -d cwd -Fn -p <pid> to resolve its cwd. On macOS each lsof call costs ~0.27s, so total discovery time is linear in the number of processes named clickhouse on the machine — including ones the CLI didn't start.

On a machine with 106 such processes (chDB / clickhouse-local children spawned by an AI-agent app), every command that touches discovery stalled ~29s:

$ time chctl local server list        # zero project servers
{"servers": [], "total_servers": 0, "total_running_servers": 0}
1.65s user 9.11s system 37% cpu 28.847 total

Affected: server list, server start, server stop/stop-all, and named-server client lookup. client --host/--port is unaffected (skips discovery). Not telemetry-related (reproduces with DO_NOT_TRACK=1).

After killing the stray processes, the same commands take 0.06–0.36s — confirming the per-PID lsof loop is the entire cost.

This setup isn't exotic: chDB and clickhouse-local are exactly what coding agents spawn, so agent-heavy machines are the ones that hit it.

Fix candidates

  1. Batch the lsof call: lsof accepts a comma-separated PID list (lsof -a -d cwd -Fn -p pid1,pid2,...), and -Fn output includes p<pid> markers to attribute each cwd. One ~0.3s call regardless of N.
  2. Or drop the subprocess entirely on macOS and use libproc (proc_pidinfo/PROC_PIDVNODEPATHINFO) for the cwd, mirroring the existing /proc/<pid>/cwd fast path on Linux.

Option 1 is the minimal change; option 2 also removes the dependency on lsof being present/fast.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions