Local integration tests with real Meticulous machine API

[hessius]

Overview

We need a comprehensive local integration test suite that validates MeticAI functionality against a real Meticulous machine, real API connections, and real telemetry data. This complements the existing mock test suite which runs in CI.

Background

Currently, our test suite uses mocks for all machine interactions, which is necessary for CI/CD but doesn't verify actual hardware integration. Since we have access to a real Meticulous machine on the local network, we should leverage this for more comprehensive testing.

Requirements

1. Test Infrastructure

• Create a separate test runner/config for local integration tests (e.g., pytest --integration or separate test files like test_integration_*.py)
• Tests should be opt-in and not run by default in CI
• Support for environment-based configuration (machine IP, credentials)
• Clear documentation on how to set up and run integration tests

2. Connection Tests

• Verify WebSocket connection to meticulous machine
• Test Socket.IO → MQTT bridge connectivity
• Validate MQTT subscription to sensor topics
• Test connection recovery after network interruption

3. API Tests (Real Data)

• Test profile CRUD against real machine
• Verify profile schema validation with actual machine responses
• Test last-shot retrieval from machine history
• Validate shot history parsing with real telemetry data

4. Telemetry Tests

• Verify real-time weight data reception
• Test flow rate calculation accuracy
• Validate temperature readings
• Test pressure sensor data parsing

5. Command Tests

• Test tare command and verify scale response
• Test preheat command
• Test profile load command
• Test brightness/sound settings

6. Pour-Over Mode Tests

• Verify scale weight polling in real-time
• Test auto-start detection with actual flow
• Validate flow rate calculations with real pours

Implementation Notes

• Integration tests should be clearly separated from unit tests
• Use @pytest.mark.integration or similar marker
• Add safety guards to prevent tests from affecting production machine state
• Consider test data cleanup/reset procedures
• May need test fixtures that wait for machine readiness

Local Test Execution

# Set environment variables
export METICULOUS_IP=192.168.x.x
export TEST_INTEGRATION=true

# Run integration tests only
cd apps/server && pytest test_integration*.py -v

# Or with marker
cd apps/server && pytest -m integration -v

CI Considerations

• Integration tests should be excluded from GitHub Actions by default
• Consider future option for scheduled integration test runs on self-hosted runner
• Mock tests remain the primary CI validation

Acceptance Criteria

• Integration test suite created with 80%+ coverage of machine interactions
• Clear documentation for local setup
• All integration tests pass with real Meticulous machine
• Integration tests isolated from CI runs
• Test results provide actionable feedback on failures

[hessius]

v2.1.0-beta.7 Milestone Audit

Status: ✅ Implemented — Infrastructure Complete

Criterion	Status	Evidence
Integration test suite created	✅	test_integration_machine.py (445 lines), 8 test classes
Clear documentation for local setup	✅	INTEGRATION_TESTING.md with full setup guide
Integration tests isolated from CI	✅	conftest_integration.py auto-skips unless TEST_INTEGRATION=true
Actionable feedback on failures	✅	IntegrationTestHelpers class with descriptive assertions

Test classes implemented:

• TestMachineConnection — HTTP/WebSocket connectivity
• TestMQTTConnection — MQTT broker connectivity
• TestProfileAPI — Profile CRUD against real machine
• TestLastShotAPI — Shot retrieval from machine history
• TestTelemetry — Real-time weight/temp/pressure data
• TestMachineCommands — Tare, preheat, profile load, brightness/sound
• TestPourOverMode — Scale weight polling, flow rate, auto-start detection
• TestIntegrationSmoke — Full workflow validation

Run with: TEST_INTEGRATION=true METICULOUS_IP=192.168.x.x pytest test_integration_machine.py -v

Note: Actual 80%+ coverage metric requires running against real hardware. The test infrastructure covers all categories from the requirements (connection, API, telemetry, commands, pour-over). Acceptance of "all tests pass" is pending real-machine validation.

[hessius]

✅ Integration tests validated against real Meticulous machine in v2.1.0-beta.7 (cda8359):

• 17 passed, 4 skipped (MQTT broker not running locally, history endpoint N/A, preheat skipped for safety)
• Fixed 8 test failures: machine has no /api/v1/machine/state REST endpoint — tests now use /api/v1/settings and Socket.IO status events
• Telemetry polling latency: ~110ms average
• All acceptance criteria met.

[hessius]

Shipped in v2.1.0 — see release notes