Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,11 +119,12 @@ hyp query sql "select count(*) from ai_gateway_messages" --remote
```

**Privacy review before anything ships.** Nothing is forwarded immediately.
The first sync (which includes backfilled history) waits until at least
11:59pm local time on the day you enroll, and the login prints the exact
deadline. Before then, open Claude or Codex and run the `hypaware-privacy`
skill to review what will ship, mark directories ignore / local-only / sync,
and purge anything sensitive.
The first sync (which includes backfilled history) waits until 11:59pm local
time on the day you enroll, and the login prints the exact deadline. Before
then, open Claude or Codex and run the `hypaware-privacy` skill to review what
will ship, mark directories ignore / local-only / sync, and purge anything
sensitive. Done reviewing, or want to see data on the server sooner? `hyp sync`
shows what would leave and sends it once you confirm.

Useful login flags: `--no-forward` signs in for remote queries only (no
enrollment), `--no-browser` prints the sign-in URL instead of opening one,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ hyp status --json # daemon running? enrolled (a central sink present)?
hyp query status # cache state and last refresh
```

Then run the enumeration query (Step 3) **twice, a short interval apart** (say ~30-60s). If the per-directory `rows` counts are still climbing, backfill is still landing: **warn the user and offer to wait** until counts stabilize before proposing any markings. Surveying mid-backfill risks marking against an incomplete picture. There is no deadline pressure here - the first-sync hold gives hours.
Then run the enumeration query (Step 3) **twice, a short interval apart** (say ~30-60s). If the per-directory `rows` counts are still climbing, backfill is still landing: **warn the user and offer to wait** until counts stabilize before proposing any markings. Surveying mid-backfill risks marking against an incomplete picture. There is no deadline pressure here - the first-sync hold gives hours. Note the user can also end that window early at any time with `hyp sync` (it prints what would leave and asks first), so if they say they are in a hurry, finishing the review is what unblocks them, not waiting.

## Step 3 - Survey the captured directories, then sample content (R4 applies)

Expand Down Expand Up @@ -137,6 +137,6 @@ hyp policy set <dir> ignore && hyp purge <dir>

## After the review

- Nothing you did contacts the server. At the deadline, the hold expires and export begins: `ignore`d data was never recorded (or was purged), `local-only` rows are withheld at the export seam, and everything else - the `sync` directories and anything left at the default - ships, backfill included.
- Nothing you did contacts the server. At the deadline - or sooner, if the user runs `hyp sync` and confirms the prompt - the hold expires and export begins: `ignore`d data was never recorded (or was purged), `local-only` rows are withheld at the export seam, and everything else - the `sync` directories and anything left at the default - ships, backfill included.
- Check the pending deadline any time with `hyp status` (it shows the first-sync deadline while the hold is live).
- Re-running this skill later is safe and idempotent; already-decided directories drop out of the survey.
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ hyp status --json # daemon running? enrolled (a central sink present)?
hyp query status # cache state and last refresh
```

Then run the enumeration query (Step 3) **twice, a short interval apart** (say ~30-60s). If the per-directory `rows` counts are still climbing, backfill is still landing: **warn the user and offer to wait** until counts stabilize before proposing any markings. Surveying mid-backfill risks marking against an incomplete picture. There is no deadline pressure here - the first-sync hold gives hours.
Then run the enumeration query (Step 3) **twice, a short interval apart** (say ~30-60s). If the per-directory `rows` counts are still climbing, backfill is still landing: **warn the user and offer to wait** until counts stabilize before proposing any markings. Surveying mid-backfill risks marking against an incomplete picture. There is no deadline pressure here - the first-sync hold gives hours. Note the user can also end that window early at any time with `hyp sync` (it prints what would leave and asks first), so if they say they are in a hurry, finishing the review is what unblocks them, not waiting.

## Step 3 - Survey the captured directories, then sample content (R4 applies)

Expand Down Expand Up @@ -165,6 +165,6 @@ hyp policy set <dir> ignore && hyp purge <dir>

## After the review

- Nothing you did contacts the server. At the deadline, the hold expires and export begins: `ignore`d data was never recorded (or was purged), `local-only` rows are withheld at the export seam, and everything else - the `sync` directories and anything left at the default - ships, backfill included.
- Nothing you did contacts the server. At the deadline - or sooner, if the user runs `hyp sync` and confirms the prompt - the hold expires and export begins: `ignore`d data was never recorded (or was purged), `local-only` rows are withheld at the export seam, and everything else - the `sync` directories and anything left at the default - ships, backfill included.
- Check the pending deadline any time with `hyp status` (it shows the first-sync deadline while the hold is live).
- Re-running this skill later is safe and idempotent; already-decided directories drop out of the survey.
4 changes: 2 additions & 2 deletions hypaware-core/plugins-workspace/format-parquet/src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ const DEFAULT_MAX_CLUSTER_KEYS = 16
const DEFAULT_MAX_ROWS_PER_GROUP = 50_000
// Hard ceiling on how many estimated row bytes accumulate in one in-memory
// group before it is written out as a row group and freed. This is the knob
// that bounds peak heap during a sink force: the encoder never holds more than
// that bounds peak heap during a forced sync: the encoder never holds more than
// ~one group (plus its columnar copy) at once, instead of materializing the
// whole partition. Independent of blob size, so a fat-`tools` partition cannot
// push a group into the gigabytes.
Expand Down Expand Up @@ -183,7 +183,7 @@ async function encodePartition(partition, ctx, settings) {
// Derive a stable schema from the declared column types (not from the
// data) so we can write row groups incrementally: never holding more
// than one cluster group of rows (plus its columnar copy) in memory.
// This is what stops `hyp sink force` on a large partition from OOMing
// This is what stops `hyp sync` on a large partition from OOMing
// while materializing the whole partition at once.
const schema = schemaFromColumnData({ columnData: rowsToColumnSources(columns, []) })
const writer = new ByteWriter()
Expand Down
16 changes: 9 additions & 7 deletions hypaware-core/smoke/flows/local_parquet_export.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ import { dispatch } from '../../../src/core/cli/dispatch.js'
* ENOTDIR. The driver then routes the failed batch
* into `<state>/sinks/broken/outbox/<batchId>.json`.
*
* The forced tick is driven through `hyp sink force` so the CLI
* The forced tick is driven through `hyp sync` so the CLI
* surface is exercised, not just the in-process driver. Dispatch is
* handed the daemon's runtime via `opts.kernel` because Phase 7 does
* not yet wire config-driven sink instantiation. The smoke owns the
Expand All @@ -40,7 +40,7 @@ import { dispatch } from '../../../src/core/cli/dispatch.js'
* - The good sink produces a Parquet file decodable by
* `parquetReadObjects` containing the captured log row.
* - The broken sink's failure lands in the outbox.
* - `hyp sink force good` exits 0 and the report mentions the
* - `hyp sync --yes` exits 0 and the report mentions the
* instance as `exported`.
* - Daemon self-telemetry includes `source.start` (otlp),
* `sink.tick`, `sink.export_batch` (status=ok for `good`, !=ok
Expand Down Expand Up @@ -144,11 +144,13 @@ export async function run({ harness, expect }) {
(v) => v !== undefined && v.kind === 'blob',
)

// ----- Drive the forced tick through the CLI (`hyp sink force`) -----
// ----- Drive the forced tick through the CLI (`hyp sync`) -----
// `--yes` because the smoke has no TTY and `hyp sync` always confirms
// before exporting (LLP 0101 #no-release).
const forceStdout = makeBuf()
const forceStderr = makeBuf()
const forceCode = await dispatch(
['sink', 'force'],
['sync', '--yes'],
{
stdout: forceStdout,
stderr: forceStderr,
Expand All @@ -157,18 +159,18 @@ export async function run({ harness, expect }) {
},
)
expect.that(
`dispatch: hyp sink force exited 0 (stderr=${forceStderr.text()})`,
`dispatch: hyp sync exited 0 (stderr=${forceStderr.text()})`,
forceCode,
(v) => v === 0,
)
const forceOut = forceStdout.text()
expect.that(
'stdout: hyp sink force reported the good instance as exported',
'stdout: hyp sync reported the good instance as exported',
forceOut,
(v) => typeof v === 'string' && /good: exported/.test(v),
)
expect.that(
'stdout: hyp sink force reported the broken instance as partial (failed mkdir)',
'stdout: hyp sync reported the broken instance as partial (failed mkdir)',
forceOut,
(v) => typeof v === 'string' && /broken: (partial|failed)/.test(v),
)
Expand Down
21 changes: 14 additions & 7 deletions llp/0100-enrollment-privacy-review.spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,10 +101,12 @@ Its job, in order:
## Requirements {#requirements}

- **R1.** An attended enrolling login MUST print the first-sync deadline as an
absolute local time and the skill invocation hint, and MUST state that the
first sync includes backfilled history. A non-TTY enrolling login gets the
same hold and the same message on stderr; nothing prompts
([LLP 0063 D3](./0063-login-auto-provision-forward-sink.decision.md#d3)
absolute local time **with its time zone**, the skill invocation hint, and
the verb that ends the window early (`hyp sync`), and MUST state that the
first sync includes backfilled history. A deadline the reader cannot place
on a clock, or cannot act on, is a countdown rather than a choice. A non-TTY
enrolling login gets the same hold and the same message on stderr; nothing
prompts ([LLP 0063 D3](./0063-login-auto-provision-forward-sink.decision.md#d3)
stands).
- **R1a.** The enrolling login's destination surfaces - the forwarding line and
the privacy block - MUST name the server by its configured target name and
Expand All @@ -120,9 +122,14 @@ Its job, in order:
paths (`already connected to <origin>`, the replaced-identity note) still
print origins, where the origin is the fact the user must act on. Revisit if
the server root becomes a browsable landing page.
- **R2.** No export tick may run before the deadline
([LLP 0101](./0101-first-sync-review-window.decision.md)); the hold MUST be
written before `enrollCentralSink` so no daemon tick can precede it.
- **R2.** *(amended 2026-07-27, [LLP 0101 #no-release](./0101-first-sync-review-window.decision.md#no-release))*
No export tick may run before the deadline **except one the user releases
through an interactive, unscoped `hyp sync` whose plan named every
destination the release unblocks**; the hold MUST be written before
`enrollCentralSink` so no daemon tick can precede it, and the daemon MUST
NOT clear the marker itself. An instance-scoped run and `--yes` MUST refuse
to release, since the hold is driver-wide and their consent is not.
Originally: "No export tick may run before the deadline."
- **R3.** The skill MUST opt its own session out of capture
([LLP 0066](./0066-session-opt-out.spec.md)) as its first action and verify
success; on failure it MUST say so and continue only with explicit user
Expand Down
66 changes: 57 additions & 9 deletions llp/0101-first-sync-review-window.decision.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@

> On an attended enrolling login, the sink driver exports nothing until a
> **printed, absolute deadline**: the next local 11:59pm, rolled to the
> following day when that is less than four hours away. No early release, no
> extension. Supersedes the pick-pending marker of
> following day when that is less than four hours away. The deadline is the
> latest the first sync can happen, not the earliest: a confirmed `hyp sync`
> ends the window early (amended 2026-07-27, see [#no-release](#no-release)).
> No extension. Supersedes the pick-pending marker of
> [LLP 0093](./0093-pick-pending-export-hold.decision.md), generalizing its
> enforcement point from a 10-minute picker guard to an hours-long review
> window.
Expand Down Expand Up @@ -60,12 +62,52 @@ deadline; `createSinkDriver.tick` exports nothing while `now < deadline`.**
fleet rollout by hours would be the surprise in the other direction.
Re-logins hold nothing: the daemon is already forwarding, so there is no
"first" sync to defer.
- **No early release** {#no-release}: rejected a release verb ("sync now")
and release-on-review-completion. The hold simply runs to its deadline,
even for a user who finished the review in ten minutes. This keeps the
printed message unconditionally true, needs no completion signal from the
skill, and costs only latency on a machine that was not forwarding
yesterday either.
- **Release only by confirmed, attended request** {#no-release}
*(amended 2026-07-27; the original text is kept below it)*: `hyp sync`
prints what would leave, warns that the window is open and that sent
history cannot be un-sent, and on an explicit `y` clears the marker and
exports. Release-on-review-completion stays rejected: it would need a
completion signal from the skill, and a skill that finished is not the
same as a user who decided. The daemon never clears the marker, so an
unattended machine still waits out the full window.

Two shapes of `hyp sync` cannot release, because the consent they carry is
narrower than what the release does:

- **Instance-scoped** (`hyp sync <instance>`) refuses while a hold is live.
The hold is driver-wide (`#hold`), so a plan built from one named handle
omits every destination the release would unblock; confirming it would
forward the others unseen. Releasing is all-or-nothing because the hold
is.
- **`--yes`** refuses while a hold is live. "Attended" is the operative
word: a provisioning script is not a person, and its destination list
scrolls past in a log nobody reads. `--yes` still works for ordinary
syncs; what it must not buy is somebody's review window.

A release that cannot be completed (an unlink failure) is an error, not a
quiet no-op: the marker survives, the driver holds the tick, and the
command must say so rather than exit 0 having sent nothing.

What the amendment concedes is the third clause of the original: the cost
is *not* only latency. On an attended onboarding the hold blocks the
demonstration that the product works at all, and the person running it has
no way to say "I have seen enough". Worse, the window is sized for a review
(`hypaware-privacy`) too slow to run in the meeting the hold was scoped to
(`#which` holds the attended lane and lets unattended `hyp join` forward
immediately, which is backwards for exactly this case). Nothing in
[LLP 0069](./0069-local-only-dir-selection.spec.md) R6 is given up: R6
forbids a *silent* first forward, and an explicit confirmation naming the
destination is the consent it exists to obtain.

The first clause survives with a wording change: the printed message says
"no later than <deadline>", so it stays unconditionally true.

> *Original (2026-07-13):* **No early release**: rejected a release verb
> ("sync now") and release-on-review-completion. The hold simply runs to
> its deadline, even for a user who finished the review in ten minutes.
> This keeps the printed message unconditionally true, needs no completion
> signal from the skill, and costs only latency on a machine that was not
> forwarding yesterday either.
- **No extension**: bounded always. A hold that can be pushed out is a kill
switch with extra steps ([LLP 0093 #bounded](./0093-pick-pending-export-hold.decision.md#bounded)).

Expand All @@ -77,7 +119,13 @@ deadline; `createSinkDriver.tick` exports nothing while `now < deadline`.**
- Live capture during the window lands in the cache and ships at the deadline
unless marked; watermarks ([LLP 0040](./0040-incremental-sink-reads.design.md))
make the first post-deadline tick a plain catch-up.
- `hyp status` shows the pending deadline (LLP 0100 R9).
- `hyp status` shows the pending deadline (LLP 0100 R9), and names `hyp sync`
beside it: a countdown the reader cannot act on is where the original
decision's cost was hiding.
- The three surfaces that print the deadline (the login message, the wizard's
privacy narration, `hyp status`) all name `hyp sync`, and the formatted
deadline carries its time zone - an absolute time is only memorable if the
reader knows which clock it is on.
- LLP 0093's pick-pending semantics retire with the picker
([LLP 0102](./0102-skill-replaces-enrollment-picker.decision.md)); its
driver-side hold machinery is reused, resized, and renamed.
55 changes: 55 additions & 0 deletions src/core/cli/confirm.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
// @ts-check

import process from 'node:process'
import readline from 'node:readline/promises'

import { isTty } from './stdio.js'

/**
* @import { CommandRunContext } from '../../../hypaware-plugin-kernel-types.js'
*/

/**
* Ask a y/N question on the interactive terminal. The question goes to
* stderr, not stdout, so a command's machine-readable output stays clean
* for a caller that pipes it.
*
* Anything other than `y`/`yes` is a no: the default has to be the safe
* one for a verb nobody can undo.
*
* @param {CommandRunContext} ctx
* @param {string} question rendered verbatim, including its `[y/N]` suffix
* @returns {Promise<boolean>}
*/
export async function askYesNo(ctx, question) {
const rl = readline.createInterface({
input: /** @type {NodeJS.ReadableStream} */ (ctx.stdin ?? process.stdin),
output: /** @type {NodeJS.WritableStream} */ (/** @type {unknown} */ (ctx.stderr)),
})
try {
const answer = await rl.question(question)
return /^y(es)?$/i.test(answer.trim())
} finally {
rl.close()
}
}

/**
* The confirmation posture `hyp purge` established and every irreversible
* verb has followed since (LLP 0104, LLP 0111#delete-confirm): prompt on an
* interactive TTY, require an explicit `--yes` anywhere else. A command that
* cannot ask must not assume.
*
* Returning a discriminant rather than a boolean keeps the refusal text with
* the command: "refusing to purge" and "refusing to sync" name different
* flags in their hints, and a shared string would have to be vague about
* both.
*
* @param {{ ctx: CommandRunContext, yes: boolean, question: string }} opts
* @returns {Promise<'confirmed' | 'declined' | 'no-tty'>}
*/
export async function requireConfirmation({ ctx, yes, question }) {
if (yes) return 'confirmed'
if (!isTty(ctx.stdin)) return 'no-tty'
return (await askYesNo(ctx, question)) ? 'confirmed' : 'declined'
}
Loading
Loading