Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: omegaconf inventory by @MatteoVoges #1173

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open

Conversation

ademariag
Copy link
Contributor

@ademariag ademariag commented Apr 24, 2024

Proposed Changes

  • Implements OmegaConf based Inventory by @MatteoVoges (@kapicorp/nexenio-team)
  • Speeds up inventory loading and compilation (2x faster)

Instructions

Activate the new inventory

 global:
   compose-target-name: true
   inventory-backend: omegaconf

or use the command line flag --inventory-backend=omegaconf

Migrate your inventory to the new setup

Migrate your inventory
 ./kapitan compile --migrate

Docs and Tests

  • Tests added
  • Updated documentation

@ademariag ademariag marked this pull request as ready for review April 24, 2024 07:59
@ademariag ademariag changed the title Test/omegaconf feat: omegaconf inventory by @MatteoVoges Apr 24, 2024
ademariag added a commit that referenced this pull request May 1, 2024
Incremental changes from OmegaConf work #1173
@ademariag ademariag force-pushed the test/omegaconf branch 4 times, most recently from 249d61b to 69d5883 Compare May 13, 2024 23:43
kapitan/inventory/inventory.py Show resolved Hide resolved
kapitan/inventory/inventory.py Show resolved Hide resolved
kapitan/inventory/inventory.py Show resolved Hide resolved
@@ -100,10 +105,10 @@ def get_parameters(self, target_names: str | list[str], ignore_class_not_found:
target = self.get_target(target_names, ignore_class_not_found)
return target.parameters

return {name: target.parameters for name, target in self.get_targets(target_names)}
return {name: {"parameters": Dict(target.parameters)} for name, target in self.get_targets(target_names)}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return {name: {"parameters": Dict(target.parameters)} for name, target in self.get_targets(target_names)}
return {name: {"parameters": dict(target.parameters)} for name, target in self.get_targets(target_names)}


logger = logging.getLogger(__name__)


def compile_targets(
inventory_path, search_paths, output_path, parallel, targets, labels, ref_controller, **kwargs
inventory_path, search_paths, output_path, parallelism, desidered_targets, labels, ref_controller, **kwargs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is desidered_targets a typo of desired_targets?

kapitan/cached.py Show resolved Hide resolved
kapitan/cached.py Show resolved Hide resolved
kapitan/cached.py Show resolved Hide resolved
@@ -38,6 +38,7 @@ coverage = "^7.5.0"
docker = "^7.0.0"
pytest-md = "^0.2.0"
pytest-emoji = "^0.2.0"
regex = "^2024.5.10"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be a non-dev dependency, since the omegaconf migration implementation depends on the library

shutil.rmtree(compile_path)
shutil.copytree(temp_compile_path, compile_path)
logger.debug("Copied %s into %s", temp_compile_path, compile_path)
logger.info(f"Compiled {len(targets)} targets in (%.2fs)", time.time() - compile_start)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just noticed that this log message looks odd, you probably want either

Suggested change
logger.info(f"Compiled {len(targets)} targets in (%.2fs)", time.time() - compile_start)
logger.info(f"Compiled {len(targets)} targets (%.2fs)", time.time() - compile_start)

or

Suggested change
logger.info(f"Compiled {len(targets)} targets in (%.2fs)", time.time() - compile_start)
logger.info(f"Compiled {len(targets)} targets in %.2fs", time.time() - compile_start)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants