Skip to content

Release v0.1.9 — Google Sheets, managed .venv, Quack, notify & schedule#59

Merged
db-tycoon-stephen merged 11 commits into
mainfrom
v0.1.9
Jun 11, 2026
Merged

Release v0.1.9 — Google Sheets, managed .venv, Quack, notify & schedule#59
db-tycoon-stephen merged 11 commits into
mainfrom
v0.1.9

Conversation

@db-tycoon-stephen

Copy link
Copy Markdown
Contributor

v0.1.9

Headline: a first-class Google Sheets source, plus tycoon managing a project-local .venv so Python setup stops blocking new users. Folded in alongside: the live multi-client warehouse (Quack), pipeline notifications, and scheduled runs.

Shipped

Deferred

Verification

Local preflight all green: pytest (721 passed, 2 skipped), ruff check src tests, mkdocs build --strict, uv build (wheel + sdist), tycoon --version0.1.9. Version coherent across pyproject.tomlsrc/tycoon/__init__.py ↔ CHANGELOG (2026-06-07) ↔ docs/releases/v0.1.9.md.

Full narrative: docs/releases/v0.1.9.md · CHANGELOG.md.

🤖 Generated with Claude Code

db-tycoon-stephen and others added 10 commits May 30, 2026 19:17
- docs/proposals/v0.1.9-scope.md: cycle goal + ordered scope (Google
  Sheets #52 headline; managed .venv onboarding #57; then Quack #42 +
  carried #31/#46/#48), the #57 Option-A design, and the release-wrap
  checklist.
- CHANGELOG [0.1.9] - Unreleased stub + docs/releases/v0.1.9.md skeleton.
- mkdocs: v0.1.9 in Releases nav; scope doc suppressed from --strict.

No code/version change — branch is the v0.1.8 main tail per the
branch-per-release convention; version bumps at release time.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Python-interpreter check as doctor's first check, verifying the
running interpreter is in tycoon's supported range (>=3.12,<3.14, mirroring
requires-python). tycoon runs dbt out of the same interpreter it lives in,
so a too-new Python (notably 3.14, which has no dbt wheels) previously
failed far from its cause at `data transform run` — exactly how #55 stayed
invisible. The error hints point at the managed-.venv direction
(`uv venv --python 3.13`). Environment-level, runs without a tycoon.yml.

First sub-piece of the managed project-local .venv onboarding work (#57).
Includes TestPythonVersionCheck (5 cases) + docs/changelog.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Register a Google spreadsheet as a first-class source via
`tycoon data sources add google_sheets`: tabs/ranges land in DuckDB with
the header row as typed columns. Auth is a service-account JSON key
(headless/cron-friendly), defaulting to ${GOOGLE_APPLICATION_CREDENTIALS};
a blank key falls back to dlt's own resolution (OAuth / ADC). Config is
spreadsheet_url_or_id (required) + optional comma-separated range_names
(blank = all sheets). Works under --no-prompt.

Per project convention the dlt google_sheets verified source isn't bundled
— it's pulled on demand via `dlt init`. First cut is full-refresh replace
per run (Sheets has no native incremental key); the existing post-ingest
auto-scaffold generates the staging dbt model.

- catalog: CatalogEntry (creds + spreadsheet/range config fields)
- source_manager: _run.py shim mapping config → google_spreadsheet kwargs,
  + _DLT_INIT_NAME entry
- tests: catalog shape, shim/dlt-init mapping, no-prompt registration, and
  shim translation (range subset, all-sheets, service-account JSON load,
  dlt fallback, max-records) — Sheets client mocked, no network
- docs: Google Sheets recipe in the sources guide
- CHANGELOG + v0.1.9 release notes

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Second piece of the managed-environment work (after the doctor interpreter
check). tycoon runs dbt out of the same interpreter it lives in, so a
mismatched Python is the most common first-mile failure; `tycoon setup`
removes it by owning the environment.

- `tycoon setup` builds a project-local .venv beside tycoon.yml on a
  supported interpreter via `uv venv --python` (uv auto-downloads a
  python-build-standalone CPython if needed — zero manual installs), pins it
  with .python-version, and installs tycoon's dbt/dlt/duckdb stack into it.
  Flags: --python (default 3.13), --from (install spec, `-e .` for dev),
  --no-install, --force, --no-prompt. uv-absent → points at the standalone
  installer rather than running it.
- `tycoon doctor --fix` runs the same .venv-build flow when the interpreter
  check fails; the check now returns a bool so --fix can key off it.
- New `tycoon.venv` module holds the subprocess-isolated core (find_uv,
  parse/validate version, create_venv) so it's fully tested without ever
  creating a real env.
- Centralize the supported-Python range in constants.py (MIN/MAX_PYTHON,
  DEFAULT_SETUP_PYTHON, python_range_str); doctor imports it.
- Tests: 24 cases (version helpers, create_venv happy/failure/force/no-install,
  setup command, doctor --fix). Docs: new commands/setup.md + doctor --fix
  section + mkdocs nav. CHANGELOG + v0.1.9 notes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…art (#42)

Make use of DuckDB's new Quack RPC protocol (core_nightly) to turn the
single-writer warehouse file into a multi-client local server — folded into
existing commands, no new user-facing surface.

Probe first: confirmed quack INSTALLs/LOADs on the pinned duckdb 1.5.2 and a
full serve->attach->read/write round-trip works (see test_quack real round-trip).

- New `tycoon.quack` module: token lifecycle (persisted once in the gitignored
  .tycoon/secrets.yml), serve_command (mirrors the duckdb_ui service), a client
  `connect()` that ATTACHes + USEs the remote so unqualified SQL resolves, and a
  core_nightly extension-availability probe.
- `tycoon start` folds in a `quack` service that serves the warehouse on :9494
  when the extension is available; preflight ensures the token and skips quack
  silently otherwise. No new command or flag.
- `tycoon data query` (warehouse only) attaches over Quack when the server is up
  — reading the LIVE warehouse instead of hitting the file lock — and falls back
  to direct file access otherwise. Labelled `warehouse (Quack)`.
- PORTS[quack]=9494.
- Tests (16): token lifecycle, serve cmd, probes, service def gating, start
  preflight folding, query routing, + a real serve->attach round-trip (skipped
  if the extension can't load). Docs: tycoon start Quack section + query note.
  CHANGELOG + v0.1.9 notes.

Deferred follow-ups (out of scope per design): auto-pause/coordinate standalone
dbt writes against a served warehouse; Nao quack-aware config.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tify (#46)

Close the simplest production-grade gap: an unattended `tycoon data run-all`
printed to the terminal and exited, so nobody knew a scheduled run
succeeded/failed. Add a small, local-first webhook notification surface.

- New `tycoon.notify` module: payload construction (Slack coloured-attachment
  shape when the URL is a hooks.slack.com incoming webhook, generic JSON
  envelope otherwise) + best-effort `send()` (never raises) over httpx. URL
  from $TYCOON_NOTIFY_WEBHOOK_URL so the secret stays out of tycoon.yml.
- `tycoon notify <success|error|info> <message> [-f key=val ...] [--label]`
  standalone command.
- `tycoon data run-all --notify` emits success on completion and error (with
  failing stage + message tail) on ingest/dbt failure; best-effort so a
  notification problem never fails the run.
- Optional `notify:` config block (severities to emit, source label) — the
  webhook URL is never stored there.
- Tests (19): payload shapes, url resolution, send success/failure/not-
  configured, the command (severities, missing webhook, field parsing), and
  run-all --notify wiring. Docs: commands/notify.md + run-all note. CHANGELOG
  + v0.1.9 notes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Answer "how do I run this every morning?" with one command instead of
hand-writing cron/launchd/systemd. `tycoon schedule` wraps the platform-native
scheduler — local-first, no daemon.

- New `tycoon.schedule` module: spec validation, launchd plist + systemd
  timer/service rendering, and add/list/remove/status primitives dispatched by
  platform. Scheduler calls (launchctl/systemctl) go through subprocess at
  module scope; everything else is file rendering against $HOME.
- `tycoon schedule add <name> --command "data run-all" --at HH:MM
  [--cadence daily|hourly|weekly] [--weekday N] [--notify] [--force]`, plus
  list / remove / status (tails the run log).
  - macOS: ~/Library/LaunchAgents/com.databasetycoon.<name>.plist + launchctl.
  - Linux: ~/.config/systemd/user/tycoon-<name>.{timer,service} + systemctl --user.
  - Windows: clear Task Scheduler pointer (unsupported v1).
  - Logs append to ~/.local/share/tycoon/schedule/<name>/run.log; runs execute
    from the project root so tycoon.yml resolves.
- `tycoon doctor` row: count of installed schedules.
- Tests (38): validation, plist/timer rendering (daily/hourly/weekly), launchd
  + systemd add/list/remove round-trips (platform + subprocess patched, real
  file writes to a temp HOME), unsupported-platform error, and the command
  surface. Docs: commands/schedule.md + run-all/notify cross-links.
  CHANGELOG + v0.1.9 notes.

Note: an executable docs/recipes doctest is intentionally omitted — it would
install a real system job in CI. The command doc carries the examples instead.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#31 (layer-granular backup) is framed as making an existing backup.skip_raw
mechanism layer-aware, but no cloud-bucket backup track ever shipped (it
depended on the deferred DuckLake path) — there's nothing to make layer-aware
yet. Document the deferral in the release notes, scope proposal decisions, and
CHANGELOG header, and reflect that #52/#57/#42/#46/#48 landed this cycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Version coherence for the publish.yml preflight gate:
- pyproject.toml + src/tycoon/__init__.py → 0.1.9
- CHANGELOG.md `## [0.1.9] - 2026-06-07` (was Unreleased)
- docs/releases/v0.1.9.md `_Released: 2026-06-07_` + past-tense intro
- mkdocs nav `v0.1.9` (dropped "in flight")

Local preflight all green: pytest (721 passed), ruff, mkdocs --strict,
uv build (wheel + sdist), tycoon --version → 0.1.9.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several major features for v0.1.9, including a first-class Google Sheets data source, a managed project-local virtual environment setup (tycoon setup and tycoon doctor --fix), a live multi-client warehouse via DuckDB Quack, platform-native scheduled runs (tycoon schedule), and pipeline-completion notifications (tycoon notify). The review feedback is highly constructive, highlighting opportunities to improve virtual environment recreation robustness, handle redundant command prefixes in schedules, optimize DuckDB Quack loading for offline environments, prevent unhandled exceptions in webhook requests, ensure UTF-8 encoding compatibility on Windows, and validate command-line notification fields.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread src/tycoon/venv.py
Comment on lines +137 to +146
target = venv_path(project_root)
if target.exists() and not force:
return VenvResult(
ok=False,
venv_path=target,
message=(
f"{target} already exists. Re-run with --force to recreate it "
"(this removes the existing environment)."
),
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When force=True is passed, the existing .venv directory is not actually removed before running uv venv. This can leave stale packages or files in the environment, violating the clean slate promise. Deleting the directory using shutil.rmtree ensures a clean recreation.

    target = venv_path(project_root)
    if target.exists():
        if not force:
            return VenvResult(
                ok=False,
                venv_path=target,
                message=(
                    f"{target} already exists. Re-run with --force to recreate it "
                    "(this removes the existing environment)."
                ),
            )
        import shutil
        try:
            shutil.rmtree(target)
        except OSError as exc:
            return VenvResult(
                ok=False,
                venv_path=target,
                message=f"Failed to remove existing environment at {target}: {exc}",
            )

Comment on lines +61 to +63
args = shlex.split(command)
if notify and "--notify" not in args:
args.append("--notify")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If a user accidentally prefixes the scheduled command with tycoon (e.g., --command "tycoon data run-all"), the scheduler will attempt to run tycoon tycoon data run-all, which will fail. Stripping the leading tycoon argument prevents this common configuration mistake.

    args = shlex.split(command)
    if args and args[0] == "tycoon":
        args = args[1:]
    if notify and "--notify" not in args:
        args.append("--notify")

Comment thread src/tycoon/quack.py
Comment on lines +133 to +141
result = subprocess.run(
["duckdb", "-c", _LOAD_QUACK],
capture_output=True,
text=True,
timeout=60,
)
except (subprocess.TimeoutExpired, OSError):
return False
return result.returncode == 0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Running INSTALL quack FROM core_nightly; requires internet access to check the remote repository, even if the extension is already installed locally. Attempting to LOAD quack; first avoids unnecessary network calls and supports offline usage when the extension is already cached.

    try:
        # Try loading first (offline-friendly)
        result = subprocess.run(
            ["duckdb", "-c", "LOAD quack;"],
            capture_output=True,
            text=True,
            timeout=10,
        )
        if result.returncode == 0:
            return True
    except (subprocess.TimeoutExpired, OSError):
        pass

    try:
        result = subprocess.run(
            ["duckdb", "-c", _LOAD_QUACK],
            capture_output=True,
            text=True,
            timeout=60,
        )
    except (subprocess.TimeoutExpired, OSError):
        return False
    return result.returncode == 0

Comment thread src/tycoon/notify.py
Comment on lines +112 to +116
try:
response = httpx.post(resolved, json=payload, timeout=timeout)
except httpx.HTTPError:
return False
return response.is_success

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The send function is documented as never raising exceptions. However, httpx.post can raise non-HTTPError exceptions (such as ValueError or InvalidURL if the resolved URL is malformed). Catching Exception ensures that notification failures never crash the calling CLI process.

Suggested change
try:
response = httpx.post(resolved, json=payload, timeout=timeout)
except httpx.HTTPError:
return False
return response.is_success
try:
response = httpx.post(resolved, json=payload, timeout=timeout)
except Exception:
return False
return response.is_success

if creds_path:
key_file = Path(creds_path).expanduser()
if key_file.is_file():
kwargs["credentials"] = json.loads(key_file.read_text())

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Specifying encoding="utf-8" when reading the service account JSON file prevents potential encoding errors on platforms where the default system encoding is not UTF-8 (such as Windows).

Suggested change
kwargs["credentials"] = json.loads(key_file.read_text())
kwargs["credentials"] = json.loads(key_file.read_text(encoding="utf-8"))

Comment on lines +18 to +19
key, value = pair.split("=", 1)
out[key.strip()] = value.strip()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If the user passes an empty key (e.g., --field =value), the current implementation will accept it with an empty string key. Validating that the key is not empty prevents malformed payloads.

        key, value = pair.split("=", 1)
        key = key.strip()
        if not key:
            error(f"Invalid --field '{pair}' — key cannot be empty.")
            raise typer.Exit(2)
        out[key] = value.strip()

…dentifier injection (#60, #61)

#60: api_key/api_secret are now pydantic.SecretStr (masked in repr) with
field docs recommending ${FIVETRAN_API_SECRET} env-var indirection.
save_project preserves the hand-authored ingestion_metadata block verbatim
so a sources-add/register write no longer round-trips the expanded secret
back into tycoon.yml. Scaffolded .gitignore now excludes .env,
.dlt/secrets.toml, **/profiles.yml; tycoon.yml stays committable as the
shareable stack template.

#61: schema/table/column names from config or DB introspection are quoted
via a shared quote_identifier helper before SQL interpolation (rill Parquet
export + data schema row counts). The COPY destination-path literal is
escaped too. Closes the arbitrary-file-write vector via a crafted schema:.

Tests: round-trip secret preservation (test_project) + quote-bearing
identifier handling (test_utils). Full suite green except pre-existing
dagster-not-installed import failures in test_orchestration.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@db-tycoon-stephen
db-tycoon-stephen merged commit 2d1ccfc into main Jun 11, 2026
7 checks passed
db-tycoon-stephen added a commit that referenced this pull request Jun 11, 2026
fix(v0.1.9): address PR #59 review feedback (robustness polish)
@db-tycoon-stephen
db-tycoon-stephen deleted the v0.1.9 branch June 12, 2026 10:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant