Skip to content

Conversation

@dgarros
Copy link
Contributor

@dgarros dgarros commented Mar 26, 2025

This PR refactor the internal store to be easier to use and to cover more use cases
The main changes is the introduction of a new internal_id on all nodes to simplify how the nodes are organized within the store event before have been created in Infrahub.
Also with this change the store is now branch-aware and by default objects from 2 different branches will be isolated from each other.
And last but not least, it's now easier to search node based on their HFID both with the flat version or the structured one.

@codecov
Copy link

codecov bot commented Mar 26, 2025

Codecov Report

Attention: Patch coverage is 74.21053% with 49 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
infrahub_sdk/store.py 75.51% 15 Missing and 21 partials ⚠️
infrahub_sdk/node.py 66.66% 6 Missing and 2 partials ⚠️
infrahub_sdk/protocols_base.py 40.00% 0 Missing and 3 partials ⚠️
infrahub_sdk/client.py 83.33% 1 Missing ⚠️
infrahub_sdk/generator.py 0.00% 1 Missing ⚠️
@@             Coverage Diff             @@
##           develop     #323      +/-   ##
===========================================
+ Coverage    71.32%   71.46%   +0.13%     
===========================================
  Files           87       87              
  Lines         7871     7990     +119     
  Branches      1517     1544      +27     
===========================================
+ Hits          5614     5710      +96     
- Misses        1865     1875      +10     
- Partials       392      405      +13     
Flag Coverage Δ
integration-tests 22.76% <24.73%> (-0.04%) ⬇️
python-3.10 45.66% <52.10%> (+0.21%) ⬆️
python-3.11 45.66% <52.10%> (+0.18%) ⬆️
python-3.12 45.66% <52.10%> (+0.18%) ⬆️
python-3.13 45.66% <52.10%> (+0.18%) ⬆️
python-3.9 44.60% <52.10%> (+0.22%) ⬆️
python-filler-3.12 24.60% <23.15%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
infrahub_sdk/exceptions.py 82.29% <100.00%> (+0.37%) ⬆️
infrahub_sdk/utils.py 80.25% <100.00%> (+0.34%) ⬆️
infrahub_sdk/client.py 67.78% <83.33%> (ø)
infrahub_sdk/generator.py 35.44% <0.00%> (ø)
infrahub_sdk/protocols_base.py 77.77% <40.00%> (-2.03%) ⬇️
infrahub_sdk/node.py 77.27% <66.66%> (-0.21%) ⬇️
infrahub_sdk/store.py 76.43% <75.51%> (+9.36%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@BeArchiTek
Copy link
Contributor

Tested as replacement of #321, all good for me

Copy link
Contributor

@ogenstad ogenstad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall to me, but there seems to be some issues that need to be resolved as highlighted by the infrastructure_edge script. Also I don't think that we should add that extra package.

return None

if kind and found_invalid:
raise NodeInvalidError(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error is raised when running infrahub_edge.py would be good with a pipeline in the Infrahub repo for this PR.

we need to save them in order to reuse them later to associate them with another node for example.
"""

def __init__(self, default_branch: str = "main") -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it was like this before but it might be unexpected for users that have configured a different default branch when initializing the SDK and then just use these store objects where we default to "main".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the default branch for now but it will introduce a "breaking change" so not 100% sure if we should go with it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No with that in mind I'd suggest to keep it as is and instead create an issue so that we cycle back and refactor this in a controlled manner.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the default branch back along with a deprecation warning

@dgarros dgarros force-pushed the dga-20250326-refactor-store branch 2 times, most recently from 1541e27 to 4c7c2a7 Compare March 30, 2025 15:08
@dgarros dgarros marked this pull request as ready for review March 31, 2025 07:43
@ogenstad
Copy link
Contributor

Based on the changes done in infrastructure_edge.py are we introducing any kind of breaking changes what would impact other scripts out there? Also does this need a news fragment or will that come in the follow up work?

@dgarros
Copy link
Contributor Author

dgarros commented Mar 31, 2025

Based on the changes done in infrastructure_edge.py are we introducing any kind of breaking changes what would impact other scripts out there? Also does this need a news fragment or will that come in the follow up work?

I will add a fragment

If we add back the default branch I think it should work out of the box for existing users but I asked the CS team to do a quick check just to be sure

@dgarros dgarros force-pushed the dga-20250326-refactor-store branch from 4c7c2a7 to daa6fdf Compare April 1, 2025 06:37
@dgarros dgarros requested a review from ogenstad April 1, 2025 06:40
@dgarros dgarros merged commit 057ab0f into develop Apr 1, 2025
17 checks passed
@dgarros dgarros deleted the dga-20250326-refactor-store branch April 1, 2025 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants