💡 Unified Python Client SDK and Standalone CLI Library

[don-petry]

Context & Objective

To unify all access paths to BroodMinder data under a polished developer and user experience, we will evolve the existing standalone scripts into a published, installable Python Library and Unified CLI (broodminder).

Instead of running individual loose scripts (scripts/discover.py, scripts/extract_all.py, scripts/flatten.py), users and automated tools will have access to a coherent CLI and a clean Python SDK with typed data models, authentication helpers, and output formatters.

Key Goals & Capabilities

1. Clean Python SDK API:

   • Transport-clean, reusable client (BroodMinderClient) with typed models (dataclasses / Pydantic).
   • Domain objects: Apiary, Hive, Device, Reading, Note, ApiarySummary, HiveStatus.
   • Programmatic convenience methods:

     from bm import BroodMinderClient
     
     client = BroodMinderClient()  # Loads BROODMINDER_API_KEY from env/.env
     apiaries = client.list_apiaries()
     hives = client.list_hives(apiary_id="123")
     readings = client.fetch_readings(device_id="456", start_epoch=..., end_epoch=...)
     df = readings.to_dataframe()  # Optional pandas/polars conversion

2. Unified CLI Tool (broodminder):

   # Quick status overview
   broodminder status
   
   # List topology
   broodminder apiaries
   broodminder hives --apiary 123
   
   # Bulk export (resumable, rate-limit aware)
   broodminder export --start 2025-01-01 --out ./data --format csv,ndjson
   
   # Incremental delta sync
   broodminder sync --delta
   
   # Offline test sandbox
   broodminder --sandbox status

3. Packaging & PyPI Distribution:

   • pyproject.toml defining entry point broodminder = "bm.cli:main" and broodminder-mcp = "bm.mcp_server:main".
   • Standard optional dependencies:
     • pip install "broodminder-data[mcp]" (FastMCP / MCP support)
     • pip install "broodminder-data[dataframe]" (Pandas / Polars / PyArrow support)
     • pip install "broodminder-data[all]"

4. Offline Sandbox Mode:

   • Provide built-in mock/sandbox fixtures so users can explore the CLI and test automation pipelines without registering for an API key or when offline.

Acceptance Criteria

• Pip-installable package broodminder-data publishing the broodminder CLI entry point.
• Typed models representing core BroodMinder entities.
• Full backward compatibility for existing extraction scripts.
• Comprehensive unit and offline tests matching TDD and quality gate requirements.

[github-actions[bot]]

Research Enhancement — Strategic Analysis & Implementation Path

Prior Art & Competitive Signal

No beekeeping platform currently offers a pip-installable Python SDK. This is a genuine first-mover opportunity in precision apiculture tooling:

Project	Type	Language	API Access	PyPI Package?
BroodMinder (this project)	Cloud REST API	Python client	X-Api-Key header, alpha, 1000 calls/day	Not yet — broodminder-data name appears available on PyPI
BMBase (wcheswick/BMBase)	Local BLE reader	Python	Raspberry Pi Bluetooth scanning	No
broodminder-diy (dstrickler/broodminder-diy)	BLE scanner	Python	Direct BLE advertising	No
BEEP (beepnl/BEEP)	Full platform	PHP (Laravel) + Vue.js	LoRaWAN → REST API	No (PHP backend)
Hiveeyes (hiveeyes.org)	DIY toolkit	Arduino + Python	MQTT + InfluxDB	No unified SDK
BeeOpen (MDPI 2024)	Data sharing ecosystem	Research framework	Proposed open data standard	No

The closest analogues from adjacent IoT domains — Adafruit IO Python (REST + MQTT, feed-based pagination), ThingSpeak (minimal HTTP wrapper) — validate the pattern but set a low bar. The openai-python SDK and stripe-python set the standard for modern Python SDK design with Pydantic models, structured error hierarchies, and sync/async parity.

Key Design Decisions — Options Tables

Domain Model Layer

Approach	Pros	Cons	Recommendation
Pydantic v2 models	Runtime validation on API responses, automatic JSON serialization, .model_dump() / .model_dump_json(), JSON Schema generation for OpenAPI alignment	~2-3× overhead vs raw dataclasses; adds pydantic as a core dependency (~4 MB)	Recommended for API response models (Apiary, Hive, Reading, Note, DeviceHealth)
dataclasses (current)	Zero-dependency, ~6.5× faster instantiation, already used for BroodMinderClient	No built-in validation, manual serialization, weaker IDE support for nested types	Keep for internal config (client settings, window parameters)
dataclasses + cattrs	Structured serialization without Pydantic weight, composable validators	Smaller ecosystem, less familiar to most developers	Viable middle ground if Pydantic dependency is rejected

Note: Pydantic v2 is Rust-backed and dramatically faster than v1 — the performance gap vs dataclasses has narrowed. It also provides TypeAdapter for validating raw collections (e.g., list[Reading]) without wrapper models.

CLI Framework

Framework	Pros	Cons	Fit for this project
Typer	Type-hint-driven, auto-generates --help, built on Click, pairs naturally with Pydantic	Adds dependency; younger ecosystem	Best fit — aligns with typed-model philosophy
Click	Mature (38.7% market share), extensive plugin ecosystem, decorator-based	More boilerplate than Typer for simple commands	Good fallback if Typer is too opinionated
argparse (current in scripts)	Zero-dependency, stdlib	Verbose, no rich output, poor subcommand ergonomics	Already outgrown by the project's ambitions

Async Strategy

Approach	Pros	Cons
Sync-only (current)	Simple, sufficient for CLI + MCP stdio transport	Limits notebook/server integration
Dual sync + async clients (BroodMinderClient / AsyncBroodMinderClient)	Matches openai-python pattern, httpx already supports both, future-proofs for streaming MCP	Doubles the surface area to maintain
Sync with async shim	One implementation, asyncio.to_thread() wrapper	Not truly async (blocks a thread)

Recommendation: Start sync-only (Phase 1), add AsyncBroodMinderClient when MCP SSE transport or notebook async patterns demand it (Phase 3). The existing httpx transport makes the async upgrade straightforward.

Sandbox Mode Design

The BroodMinder API's 1000 calls/day cap and alpha status make an offline sandbox essential — not just nice-to-have. Recommended approach (combining Stripe test-mode and client-side mock router patterns):

1. Activation: BroodMinderClient(api_key="sandbox") or BROODMINDER_API_KEY=sandbox auto-routes to bundled fixtures. No network I/O.
2. Fixture data: Bundle bm/_fixtures/ with realistic seasonal scenarios:
   • apiary_demo → 3 hives × 4 devices, spanning spring buildup through winter cluster
   • Magic IDs: hive_healthy (normal curves), hive_swarming (weight-drop event), hive_offline (stale sensor data)
3. Request history: client.sandbox_history records all calls for test assertions
4. Deterministic: Same inputs always produce same outputs (seeded, not randomized)

This unlocks: workshops without API keys, CI without secrets, documentation with runnable examples, and MCP server development without live data.

Risks & Unknowns

Risk	Severity	Mitigation
API schema instability (alpha API, no versioning)	High	Pydantic's model_config = ConfigDict(extra="ignore") absorbs additive changes gracefully; contract tests (test_contract.py) catch breaking changes
Rate limit blocks dev iteration	Medium	Sandbox mode eliminates API calls during development; VCR-style response recording for integration tests
PyPI name squatting	Low	broodminder-data appears available today — register the name early with a minimal 0.1.0 placeholder if needed
Breaking existing script users	Medium	Phase 1 keeps scripts/ working as-is; CLI wraps same logic. Deprecation notices after CLI reaches parity, removal no earlier than 1.0
Pydantic as core dependency	Low	Keep it in dependencies only if models are core to the SDK; otherwise make it optional with a [typed] extra and fall back to dicts
Semver commitment before API stabilizes	Medium	Stay on 0.x (no semver stability guarantee) until BroodMinder's API exits alpha

Phased Validation Path

Phase 0 — Foundation (1-2 PRs)
Extract domain models from the implicit schemas in discover.py / extract_all.py / flatten.py. Define Apiary, Hive, Device, Reading, Note as Pydantic models in bm/models.py. Validate against live contract test fixtures. No behavioral changes.

Phase 1 — CLI + Entry Points (2-3 PRs)
Add bm/cli.py with Typer, wrapping existing script logic as subcommands (broodminder apiaries, broodminder export, broodminder flatten). Register [project.scripts] entry points in pyproject.toml. Keep scripts/ working via thin wrappers that delegate to the new CLI internals.

Phase 2 — Sandbox + Packaging (2 PRs)
Implement client-side sandbox mode with bundled fixtures. Add [cli] and [dataframe] optional dependency groups. Test pip install broodminder-data && broodminder --sandbox status end-to-end. Publish 0.1.0 to PyPI.

Phase 3 — Polish + Async (2-3 PRs)
Add AsyncBroodMinderClient. Add .to_dataframe() convenience on Reading collections (when [dataframe] extra is installed). Wire MCP server to use typed models instead of raw dicts. Add broodminder sync --delta subcommand for the delta sync engine.

Success Metrics

Metric	Target	Measurement
Time-to-first-query for new users	< 5 minutes from pip install to first API response	README walkthrough test
Sandbox test coverage	100% of SDK methods exercisable without API key	Offline test suite passes with api_key="sandbox"
Script parity	CLI reproduces all scripts/*.py functionality	Feature matrix checklist
PyPI installability	pip install broodminder-data && broodminder --help works	CI publish + smoke test
Type coverage	All public API methods return typed models, not Any	mypy strict mode on bm/
Downstream adoption signal	MCP server config simplifies to python -m bm.mcp_server (already in README)	Config validation test