feat: Central Hierarchy + Environment design populated by Actions

[fblanchetNaN]

[#9 automatically closed after rebase + push --force, cannot be reopen]

The design is based on two central classes:

• Hierarchywith two attributes _configurations and _templates

  incipyt/incipyt/system.py

  Lines 127 to 143 in dc3fdde

  	class Hierarchy:
  	"""Represents all configuration and template files to commit for the new project.
  	An instance internally stores mappables between path objects and template
  	files or dictionaries modeling configuration files.
  	Functions :meth:`get_configuration` and :meth:`register_template` add
  	respectively configuration dictionary and template to the instance.
  	When the hierarchy is finally ready, functions :meth:`mkdir` :meth:`commit`
  	can be used to write folder and files in a new folder after substituting
  	variables in path and files using an :class:`incipyt.system.Environment`.
  	"""
  	def __init__(self):
  	self._configurations = {}
  	self._templates = {}

  They are dictionaries with instances of classes in incipyt/_internal/dumpers -- path-like classes with a specific method to write them, e.g. for Toml

  incipyt/incipyt/_internal/dumpers.py

  Lines 71 to 74 in dc3fdde

  	class Toml(BaseDumper):
  	def dump_in(self, config):
  	with self.substitute_path().open("w+") as file:
  	toml.dump(config, file)

  -- as keys. Each association key-value corresponds to a file path and its content to be written. For _templates, values are just Jinja template. _configurations is dedicated to all configuration files based on key-value syntax (toml, cfg, ini, yaml, json, etc), values of this dictionary are then nested dictionary describing this configuration file.
• Envrionment is basically a wrapper around a dictionary initiated with environment variables,

  incipyt/incipyt/system.py

  Lines 13 to 65 in dc3fdde

  	class Environment:
  	"""Manage environment variables using for substitutions in patterns.
  	Functions :meth:`pull`and :meth:`push` can be used to add or request a
  	specific envirnoment variable.
  	Function :meth:`requests` ask the user if a value is missing.
  	An instance is also a visitor for :class:`incipyt.system.Hierarchy`
  	elements involving pattern with environment variables.
  	"""
  	def __init__(self):
  	self._variables = os.environ.copy()
  	if "PYTHON_CMD" not in self._variables:
  	self._variables["PYTHON_CMD"] = "python"
  	def pull(self, key):
  	"""Try to pull the actual value for `key`.
  	If `key` already exists, just return the associated value, if not,
  	first asks for it -- see :func:`incipyt.system.Environment.requests`
  	-- then returns it.
  	:param key: Environment key asked.
  	:type key: str
  	:return: The actual value for `key`.
  	:rtype: str
  	"""
  	if key not in self._variables:
  	logger.info(f"Missing environement variable {key}, request it.")
  	self._variables[key] = self.requests(key)
  	return self._variables[key]
  	def push(self, key, value, update=False):
  	"""Try to push a `key` = `value` associaton.
  	:param key: Key of the association to push.
  	:type key: str
  	:param value: Value of the association to push.
  	:type value: str
  	:param update: Allow existing keys.
  	:type update: bool
  	:raises RuntimeError: Raise if `key` already exists in the actual environment.
  	"""
  	if key in self._variables and not update:
  	raise RuntimeError(
  	f"Environment variable {key} already exists, use update."
  	)
  	logger.info(f"Push environement variable {key}={value}.")
  	self._variables[key] = value

  its purpose is to deal with all variables, like {NAME} or {AUTHOR} that should be asked to the user. To do so,

  incipyt/incipyt/system.py

  Lines 67 to 74 in dc3fdde

  	def requests(self, key):
  	"""Request to the user the value to associate to `key`.
  	:param key: Key to request to the user.
  	:type key: str
  	:raises NotImplementedError: TO-DO.
  	"""
  	raise NotImplementedError(f"Request [for {key}] is not implemented.")

  have to be implemented and ask the user for a specific variable. Then it behaves as a visitor for values of Hierarchy._configuration

  incipyt/incipyt/system.py

  Lines 97 to 108 in dc3fdde

  	for key, value in template.items():
  	logger.debug(f"Visit {key} to process environment variables.")
  	if callable(value):
  	template[key] = value(self)
  	elif isinstance(value, collections.abc.MutableMapping):
  	self.visit(value)
  	elif isinstance(value, collections.abc.MutableSequence):
  	for index, element in enumerate(value):
  	if callable(element):
  	value[index] = element(self)
  	if isinstance(element, collections.abc.MutableMapping):
  	self.visit(element)

  actually values can contain some callable instead of str, if so those callable are call and the value substituted. Typically, it can be utils.Requires("{NAME}") with

  incipyt/incipyt/_internal/utils.py

  Lines 12 to 28 in dc3fdde

  	class Requires:
  	def __init__(self, template, sanitizer=None):
  	self._sanitizer = sanitizer
  	self._template = template
  	def __call__(self, environment):
  	return self._template.format(
  	**{
  	key: (
  	self._sanitizer(key, environment.pull(key))
  	if self._sanitizer
  	else environment.pull(key)
  	)
  	for _, key, _, _ in Formatter().parse(self._template)
  	if key is not None
  	}
  	)

  then it asks Environment for {NAME} : environment.pull(key).

There are similar substitution mechanisms for _template and also for using subprocess.run

incipyt/incipyt/system.py

Lines 110 to 124 in dc3fdde

	def run(self, command):
	"""Run a command after substitution using the environment.
	:param command: List of the command elements.
	:type command: List
	"""
	completed_process = subprocess.run(
	[c(self) if callable(c) else c for c in command],
	capture_output=True,
	check=True,
	)
	logger.info(
	f"""{' '.join(completed_process.args)}
	{completed_process.stdout.decode()}"""
	)

The main workflow is then to initialize a Hierarchy and let tool specific classes populate _configurations and _template

incipyt/incipyt/system.py

Lines 242 to 245 in dc3fdde

	hierarchy = Hierarchy()
	for action in actions:
	logger.info(f"Add {action} to hierarchy.")
	action.add_to(hierarchy)

---

An instance of Hierarchy stores all information about configurations files and templated files to write for the new project.

Then each tool is modeled by Action instances, they populate a hierarchy instance and define pre and post methods to run before and after the creation of the project files

incipyt/incipyt/system.py

Lines 250 to 259 in dc3fdde

	for action in actions:
	logger.info(f"Running pre-action for {action}.")
	action.pre(workon_path, environment)
	logger.info(f"Commit hierarchy on {workon_path}.")
	hierarchy.commit(workon_path, environment)
	for action in actions:
	logger.info(f"Running post-action for {action}.")
	action.post(workon_path, environment)

e.g. Git Action pre method runs git init, while post method runs git add --all and git commit.

Definitions of configurations and templates can be done with environment variables, like {NAME}. All those variables are stored in an Environment instance, it is initialized with environment variables system and can be modified by any Action using pre or post methods. Then values are used when needed and asked to user if missing.

When an Action is created, it can also register itself a hook, e.g. Git Action register a VCSIgnore hook that defines how a pattern is ignored by git. Then any Action can used those hooks, e.g. Venv Action use VCSIgnore to make VCS system aware of ignoring .env folder

incipyt/incipyt/actions/git.py

Lines 6 to 24 in dc3fdde

	class Git:
	"""Action to add Git to :class:`incipyt.system.Hierarchy`."""
	def __init__(self):
	hooks.VCSIgnore.register(self._hook)
	def add_to(self, hierarchy):
	"""Add git configuration to `hierarchy`, do nothing.
	:param hierarchy: The actual hierarchy to update with git configuration.
	:type hierarchy: :class:`incipyt.system.Hierarchy`
	"""
	def _hook(self, hierarchy, value):
	gitignore = hierarchy.get_configuration(Requirement.make(".gitignore"))
	if None not in gitignore:
	gitignore[None] = []
	gitignore[None].append(value)

[LeMinaw]

About last push

• append_unique needs to be moved as a class member
• docs need update
• still unsure about how TemplateDict internals work, maybe iteration should be preferred for __getitem__

Maybe we should ditch

template_dict["first", "second", "third"] = something

in favor of

template_dict["first"] = {"second": {"third": something}}

and check for overrides in __getitem__

[fblanchetNaN]

• still unsure about how TemplateDict internals work, maybe iteration should be preferred for __getitem__

I don't really like all those staticmethod.

Maybe we should ditch

template_dict["first", "second", "third"] = something

I don't think so, this syntax seems more clear and simple than the other.

[LeMinaw]

I don't really like all those staticmethod.

Me neither, but I'm unsure about how to get rid of them. Maybe an iterative approach will suit this case better than the current, recursive one.

Maybe we should ditch

template_dict["first", "second", "third"] = something

I don't think so, this syntax seems more clear and simple than the other.

It is, but I see two issues with it:

• It shawows all sequence-like (and not string-like) dict keys (though this can be with circumvented with template_dict.data['hello', 'tuple', 'key'])
• template_dict["1"] = {"2": ...}} will bypass all the TemplateDict collision logic for nested keys

[Julien00859]

The last few commits are great, I've only a few nitpicking comments huhu. Let's look forward to merge this branch!

[codecov-commenter]

Codecov Report

Merging #15 (86fb76f) into main (a7aaeba) will increase coverage by 66.28%.
The diff coverage is 66.28%.

@@            Coverage Diff            @@
##           main      #15       +/-   ##
=========================================
+ Coverage      0   66.28%   +66.28%     
=========================================
  Files         0       13       +13     
  Lines         0      519      +519     
=========================================
+ Hits          0      344      +344     
- Misses        0      175      +175     

Impacted Files	Coverage Δ
incipyt/__main__.py	0.00% <0.00%> (ø)
incipyt/_internal/sanitizers.py	0.00% <0.00%> (ø)
incipyt/signals.py	0.00% <0.00%> (ø)
incipyt/tools/__init__.py	0.00% <0.00%> (ø)
incipyt/tools/base.py	0.00% <0.00%> (ø)
incipyt/tools/git.py	0.00% <0.00%> (ø)
incipyt/tools/setuptools.py	0.00% <0.00%> (ø)
incipyt/tools/venv.py	0.00% <0.00%> (ø)
incipyt/commands.py	68.96% <68.96%> (ø)
incipyt/_internal/templates.py	88.75% <88.75%> (ø)
... and 3 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7aaeba...86fb76f. Read the comment docs.

[Julien00859]

Plus qu'à nettoyer un peu les commits :)