Refactor models to use Pydantic by eman · Pull Request #37 · eman/nwp500-python

eman · 2025-11-19T04:04:05Z

Refactor models to use Pydantic

Date: 2025-11-20

Summary

Replaces all legacy dataclass-based models (DeviceInfo, Location, Device, FirmwareInfo, DeviceStatus, DeviceFeature, EnergyUsage*) with Pydantic BaseModel implementations (NavienBaseModel) providing automatic validation, camelCase alias mapping, and declarative field conversions.

Breaking Changes

Dataclass models removed; all models now inherit from NavienBaseModel (Pydantic).
Field metadata conversion system (meta/apply_field_conversions) removed.
Attribute names standardized to snake_case (e.g. operationMode -> operation_mode).
Manual conversion logic (+20 offset, /10 scaling, decicelsius->F) replaced by BeforeValidator Annotated types (Add20, Div10, DeciCelsiusToF).
Boolean device fields now use DeviceBool validator instead of manual value == 2 checks.
Enum parsing handled directly; defaults set via Field declarations (operation_mode defaults to STANDBY).
API typo handled via Field alias (heLowerOnTDiffempSetting).

Migration Guide

Remove apply_field_conversions and meta usage.
Replace dataclass imports with existing paths (no path changes) — models are now Pydantic.
Use Model.model_validate(raw_payload) instead of from_dict + manual conversions.
Update attribute access to snake_case (see CHANGELOG for common mappings).
Stop adding +20 or dividing by 10; values are already converted.
Remove boolean normalization (value == 2); attributes are bool.
Eliminate manual camelCase->snake_case dict key renaming.
Use model.model_dump() for logging converted data.
Simplify enum handling — remove try/except around enum constructors.
Energy usage models remain accessible; properties unchanged.

Benefits

~400 lines of imperative conversion logic removed.
Single-pass parsing improves performance and reduces memory copies.
Automatic camelCase alias mapping reduces maintenance.
Clear, declarative conversion definitions via Annotated types.
Rich validation errors aid debugging malformed MQTT/API payloads.
Easier future extension (add field + validator).

Testing & Quality

Linting: Passed (ruff check & format).
Type Checking: mypy passed (no errors).
Tests: 123/123 passed.

Follow-Up

Update any external integrations relying on old dataclass patterns.
Consider adding additional custom validators for edge-case payloads if discovered.

Reference

See CHANGELOG v6.0.3 entry for full details and examples.

Replaces dataclasses with Pydantic models for better validation and type safety. Updates auth, api_client, and mqtt_client to use new models.

Fixes whitespace, line length, and import sorting issues reported by ruff.

Auto-formatted code to pass CI checks.

Copilot

Pull Request Overview

This PR refactors the data models to use Pydantic instead of dataclasses, improving data validation, type safety, and maintainability. The changes replace manual field conversion logic with Pydantic's validator system and leverage automatic camelCase/snake_case aliasing.

Key Changes

Replaced dataclasses with Pydantic BaseModel across models.py and auth.py
Introduced custom validators for device-specific field conversions (boolean encoding, temperature offsets, etc.)
Migrated from from_dict() factory methods to Pydantic's model_validate() for consistency

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`src/nwp500/models.py`	Core refactor: replaced dataclasses with Pydantic models, converted field metadata to validators, simplified data conversion logic
`src/nwp500/auth.py`	Refactored authentication models to Pydantic, replaced `__post_init__` with `model_post_init`, added validator for empty alias handling
`src/nwp500/api_client.py`	Updated model instantiation from `from_dict()` to `model_validate()`
`src/nwp500/mqtt_client.py`	Updated docstring examples to use snake_case field names
`src/nwp500/mqtt_subscriptions.py`	Updated field references and docstrings to use snake_case naming
`src/nwp500/__init__.py`	Removed deprecated `EnergyUsageData` and `MonthlyEnergyData` exports, added duplicate exports
`setup.cfg`	Added `pydantic>=2.0.0` dependency

Copilot · 2025-11-19T04:16:45Z

src/nwp500/models.py

+    he_upper_off_diff_temp_setting: Div10
+    he_lower_on_diff_temp_setting: Div10 = Field(
+        alias="heLowerOnTDiffempSetting"
+    )  # Handle typo


The inline comment "# Handle typo" is unclear. Consider expanding it to: "# Handle API typo: heLowerOnTDiffempSetting -> heLowerOnDiffTempSetting" to better document that this is a workaround for an API-level typo.

Suggested change

) # Handle typo

) # Handle API typo: heLowerOnTDiffempSetting -> heLowerOnDiffTempSetting

Copilot · 2025-11-19T04:16:45Z

src/nwp500/models.py

+        # Handle the typo field explicitly if needed, though alias handles it
+        if "heLowerOnTDiffempSetting" in data:
+            # Pydantic alias will handle this, but if we want to be safe
+            pass


The from_dict method contains dead code (lines 316-318). Since Pydantic's alias mechanism handles the typo field automatically, remove this unnecessary conditional check and simplify to just the docstring and return statement.

Suggested change

# Handle the typo field explicitly if needed, though alias handles it

if "heLowerOnTDiffempSetting" in data:

# Pydantic alias will handle this, but if we want to be safe

pass

Copilot · 2025-11-19T04:16:46Z

src/nwp500/models.py

+    heat_pump_usage: int
+    heat_element_usage: int


The EnergyUsageTotal model has breaking changes - field names have changed from heUsage/hpUsage to heat_element_usage/heat_pump_usage without aliases. Consider adding Field aliases to maintain backward compatibility: heat_pump_usage: int = Field(alias="hpUsage") and heat_element_usage: int = Field(alias="heUsage").

Suggested change

heat_pump_usage: int

heat_element_usage: int

heat_pump_usage: int = Field(alias="hpUsage")

heat_element_usage: int = Field(alias="heUsage")

Copilot · 2025-11-19T04:16:46Z

src/nwp500/models.py

+    day: int
+    total_usage: int
+    heat_pump_usage: int
+    heat_element_usage: int
+    heat_pump_time: int
+    heat_element_time: int


The EnergyUsageDay model appears to be a new structure replacing EnergyUsageData. If this is meant to parse the same API response data, field aliases may be needed to map from the API's camelCase format (e.g., hpUsage, heUsage, hpTime, heTime) to these snake_case names.

Suggested change

day: int

total_usage: int

heat_pump_usage: int

heat_element_usage: int

heat_pump_time: int

heat_element_time: int

day: int = Field(alias="day")

total_usage: int = Field(alias="totalUsage")

heat_pump_usage: int = Field(alias="hpUsage")

heat_element_usage: int = Field(alias="heUsage")

heat_pump_time: int = Field(alias="hpTime")

heat_element_time: int = Field(alias="heTime")

src/nwp500/models.py

src/nwp500/__init__.py

src/nwp500/auth.py

Clarifies that the alias handles an API-level typo.

Removes unnecessary conditional check for typo field as Pydantic alias handles it automatically.

Adds explicit aliases for heat_pump_usage and heat_element_usage to match API keys (hpUsage, heUsage).

- Auth: Add idToken to handle_empty_aliases validator for consistency - Models: Restructure energy usage models to match actual API response - Add MonthlyEnergyData class for monthly grouping - Change EnergyUsageResponse from flat daily list to usage with monthly grouping - Add time fields to EnergyUsageTotal (heat_pump_time, heat_element_time) - Make total_usage computed properties where appropriate - Add get_month_data() method to EnergyUsageResponse - Models: Fix temp_formula_type to accept Union[int, str] for API compatibility - Exports: Remove duplicate MqttRequest/MqttCommand, add new model exports - Docs: Add datetime serialization backward compatibility notes - Examples: Fix 100+ camelCase attribute references to snake_case across all example files - Examples: Fix encoding function imports (decode_week_bitfield, decode_price) All changes verified with actual API responses from Navien device.

- Remove trailing whitespace from blank lines - Apply ruff formatting to modified files

- Fix camelCase attribute usage in CLI commands and monitoring - Fix return type annotation in boolean validator - Update TOUInfo.model_validate signature to match base class - Fix Optional[ClientSession] assignment in NavienAPIClient

- Extract shared topic matching utility to mqtt_utils - Refactor MqttSubscriptionManager to use shared utility - Clean up MqttClient imports and remove dead code - Make save/restore arguments optional in token_restoration_example.py

- Refine configuration.rst to reduce duplication - Move Protocol Reference to Advanced section in index.rst - Add warnings to all protocol docs clarifying internal nature

- Fix deprecated typing.Tuple usage in scripts/bump_version.py - Fix line length issues in scripts/ - Apply ruff formatting to all files

…lained.rst

- Updated Energy model fields (heUsage -> heat_element_usage, etc.) - Updated MQTT model fields (clientID -> client_id, etc.) - Updated common fields (deviceType -> device_type, etc.)

…ings

…bers

…to text

eman added 3 commits November 18, 2025 20:03

Refactor models to use Pydantic

84300ef

Replaces dataclasses with Pydantic models for better validation and type safety. Updates auth, api_client, and mqtt_client to use new models.

Fix linting errors

8ff7163

Fixes whitespace, line length, and import sorting issues reported by ruff.

Apply ruff formatting

c87ebea

Auto-formatted code to pass CI checks.

eman requested a review from Copilot November 19, 2025 04:15

Copilot AI reviewed Nov 19, 2025

View reviewed changes

eman added 25 commits November 18, 2025 20:18

Update comment for API typo workaround

45d2c84

Clarifies that the alias handles an API-level typo.

Remove dead code in from_dict

feaac72

Removes unnecessary conditional check for typo field as Pydantic alias handles it automatically.

Fix EnergyUsageTotal aliases

7a37281

Adds explicit aliases for heat_pump_usage and heat_element_usage to match API keys (hpUsage, heUsage).

fix: Apply ruff linting and formatting fixes

e2771a4

- Remove trailing whitespace from blank lines - Apply ruff formatting to modified files

docs: improve structure and add protocol disclaimers

e6b2caf

- Refine configuration.rst to reduce duplication - Move Protocol Reference to Advanced section in index.rst - Add warnings to all protocol docs clarifying internal nature

chore: fix linting and formatting issues

6c53ff0

- Fix deprecated typing.Tuple usage in scripts/bump_version.py - Fix line length issues in scripts/ - Apply ruff formatting to all files

docs: update Python code examples to use snake_case attributes

a019f23

docs: fix reStructuredText heading hierarchy in advanced_features_exp…

108732f

…lained.rst

fix: remove duplicate @classmethod decorator in TOUInfo.model_validate

52350e0

docs: add advanced_features_explained to User Guides toctree

b16b6b6

docs: update all remaining Python examples to use snake_case attributes

d2114b8

docs: convert field name listings in models.rst to snake_case

a4b5281

docs: convert remaining field names in models.rst to snake_case

9f000dc

- Updated Energy model fields (heUsage -> heat_element_usage, etc.) - Updated MQTT model fields (clientID -> client_id, etc.) - Updated common fields (deviceType -> device_type, etc.)

docs: convert code example attributes in models.rst to snake_case

0a60276

docs: normalize snake_case field names in mqtt_client examples

5c62b1f

docs: add :no-index: to package automodule to suppress duplicate warn…

f86519b

…ings

docs: convert remaining camelCase to snake_case in mqtt examples

9e3d471

docs: fix remaining dhw_temperatureSetting and exclude duplicated mem…

022bede

…bers

docs: finalize snake_case field and switch problematic JSON examples …

d25fc6a

…to text

docs: finalize snake_case and switch json placeholders to text

d3b4f9d

docs: update REST API and MQTT client docs (2025-11-20)

8c6eec2

docs: update changelog with v6.0.3 model migration breaking changes

4dbb2a8

clean-up readme

fb73625

eman merged commit 566806f into main Nov 20, 2025
10 checks passed

eman deleted the code-review-refactor branch November 20, 2025 02:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor models to use Pydantic#37

Refactor models to use Pydantic#37
eman merged 29 commits intomainfrom
code-review-refactor

eman commented Nov 19, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Nov 19, 2025

Uh oh!

Copilot AI Nov 19, 2025

Uh oh!

Copilot AI Nov 19, 2025

Uh oh!

Copilot AI Nov 19, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	) # Handle typo
	) # Handle API typo: heLowerOnTDiffempSetting -> heLowerOnDiffTempSetting

Conversation

eman commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Breaking Changes

Migration Guide

Benefits

Testing & Quality

Follow-Up

Reference

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Key Changes

Reviewed Changes

Uh oh!

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

eman commented Nov 19, 2025 •

edited

Loading