Skip to content

Conversation

@ltucker
Copy link
Contributor

@ltucker ltucker commented Apr 3, 2025

  • Adjusts cycle breaking to support primary_ip4/primary_ip6 on devices using topo sort for long cycles
  • Add logical matchers to allow resolving exising objects not technically constrainted by uniqueness for ip addresses, prefixes
  • Additional tests, cleanup

etc

This pull request includes several changes to the netbox_diode_plugin to improve validation, matching, transformation, and testing functionalities. The most important changes are grouped by their themes below:

Validation Improvements:

  • Added _validate_relations method to check and validate relation fields and included its call in the validate method to handle relational validation errors. (netbox_diode_plugin/api/common.py, [1] [2]

Matching Enhancements:

  • Added new matching criteria for ipam.ipaddress and ipam.prefix to handle different conditions based on the presence of VRF. (netbox_diode_plugin/api/matcher.py, netbox_diode_plugin/api/matcher.pyL33-R71)

Transformation Enhancements:

  • Introduced topological sorting for entities to ensure proper order during transformation and deduplication processes. (netbox_diode_plugin/api/transformer.py, [1] [2] [3]
  • Replaced merge_data with _merge_nodes to handle merging of nodes more effectively during deduplication. (netbox_diode_plugin/api/transformer.py, netbox_diode_plugin/api/transformer.pyL197-R281)
  • Improved handling of post-create steps by merging entities when possible and ensuring proper reference updates. (netbox_diode_plugin/api/transformer.py, [1] [2]

Testing Enhancements:

@github-actions
Copy link

github-actions bot commented Apr 3, 2025

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines Covered Coverage Threshold Status
1830 1675 92% 0% 🟢

New Files

No new covered files...

Modified Files

File Coverage Status
netbox_diode_plugin/api/common.py 92% 🟢
netbox_diode_plugin/api/matcher.py 84% 🟢
netbox_diode_plugin/api/transformer.py 86% 🟢
netbox_diode_plugin/tests/test_api_diff_and_apply.py 100% 🟢
TOTAL 90% 🟢

updated for commit: badb842 by action🐍

Copy link
Member

@mfiedorowicz mfiedorowicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👏

@ltucker ltucker merged commit bf99b50 into update_data_model Apr 3, 2025
3 checks passed
@ltucker ltucker deleted the logical_matchers branch April 3, 2025 17:03
ltucker added a commit that referenced this pull request Apr 7, 2025
* fix: scope support on apply change set (#64)

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* wip diff api

* set default values and missing slugs

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* tidy up setting defaults

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* remove unused imports

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* fix constructor of object type

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* set slugs (if not present) after resolving existing instances

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* emit ref_id instead of variable object_id field for new objects

* improve entity field mapping coverage

* fill in primary value mapping, use primary value for slug

* use canonical field ordering in change dicts

* first pass at certain common circular refs

* remove ref id to itself

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* tidy up

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* add applier

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* fix resolve ref before lookup, use field name directly, not field attr

* don't query with unresolved references

* fix _build_expressions_queryset

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* resolve lint issues

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* exclude fields with GenericRelation type

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* fix sorting dict

we may get ints strings etc

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* rework applier logic

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* applier with content type fields

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* fix content type related existing value

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* exclude foreign key fields with many to one rel

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* fix: support for post create updates eg (primary mac address) (#68)

* fix: support for post create updates eg (primary mac address)

* use serializers, fix relevent tests

* linting

* filter fields in the prior state

* add some basic smoke tests for diff / diff+apply

* Change set validation (#69)

* add change set validation

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* remove redundant serializers

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* remove redundant serializers tests

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* tidy up

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* adjust tests, lighter validation of refs on diff, handle generic

---------

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>
Co-authored-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>

* fix: expand support for cycle breaking, add additional logical matchers (#70)

* fix: fix error fingerprinting tags (#71)

* fix: all noops -> no changes, show noops as only prior state (#72)

---------

Signed-off-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>
Co-authored-by: Michal Fiedorowicz <mfiedorowicz@netboxlabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants