Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
Changelog
=========

Unreleased
----------

* Add closed layers to layer contract.

3.9 (2025-05-05)
----------------

Expand Down
17 changes: 15 additions & 2 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -304,7 +304,8 @@ Higher level analysis
passing ``independent=False`` when instantiating the :class:`.Layer`. For convenience, if a layer consists
only of one module name then a string may be passed in place of the :class:`.Layer` object. Additionally, if
the layer consists of multiple *independent* modules, that can be passed as a set of strings instead of a
:class:`.Layer` object.
:class:`.Layer` object. A closed layer may be created by passing ``closed=True`` to prevent higher layers
from importing directly from layers below the closed layer (see `Closed layers`_ section below).
*Any modules specified that don't exist in the graph will be silently ignored.*
:param set[str] containers: The parent modules of the layers, as absolute names that you could
import, such as ``mypackage.foo``. (Optional.)
Expand Down Expand Up @@ -409,6 +410,18 @@ Higher level analysis
),
)

Closed layers
^^^^^^^^^^^^^

A closed layer may be created by passing ``closed=True``. Closed layers provide an additional
constraint in your architecture that prevents higher layers from "reaching through" to access
lower layers directly. Imports from higher to lower layers cannot bypass closed layers - the
closed layer must be included in the import chain.

This is particularly useful for enforcing architectural boundaries where you want to hide
implementation details of lower layers and ensure that higher layers only interact with
the public interface provided by the closed layer.

Return value
^^^^^^^^^^^^

Expand Down Expand Up @@ -575,4 +588,4 @@ Module expressions
- ``mypackage.foo*``: is not a valid expression. (The wildcard must replace a whole module name.)

.. _namespace packages: https://docs.python.org/3/glossary.html#term-namespace-package
.. _namespace portion: https://docs.python.org/3/glossary.html#term-portion
.. _namespace portion: https://docs.python.org/3/glossary.html#term-portion
132 changes: 127 additions & 5 deletions rust/src/graph/higher_order_queries.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ pub struct Level {

#[getset(get_copy = "pub")]
independent: bool,

#[getset(get_copy = "pub")]
closed: bool,
}

#[derive(Debug, Clone, PartialEq, Eq, new, Getters)]
Expand Down Expand Up @@ -57,7 +60,7 @@ impl Graph {
.flat_map(|m| m.conv::<FxHashSet<_>>().with_descendants(self))
.collect::<FxHashSet<_>>();

self.generate_module_permutations(levels)
self.generate_illegal_import_permutations_for_layers(levels)
.into_par_iter()
.try_fold(
Vec::new,
Expand All @@ -81,15 +84,20 @@ impl Graph {
)
}

fn generate_module_permutations(&self, levels: &[Level]) -> Vec<(ModuleToken, ModuleToken)> {
let mut permutations = vec![];
/// Returns a set of tuples (importer, imported) describing the illegal
/// import permutations for the given layers.
fn generate_illegal_import_permutations_for_layers(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely better - might be worth adding a comment to explain what the tuple corresponds to (e.g. is the first one the source and the second one the module that should not be imported by it)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment in the docstring now.

&self,
levels: &[Level],
) -> FxHashSet<(ModuleToken, ModuleToken)> {
let mut permutations = FxHashSet::default();

for (index, level) in levels.iter().enumerate() {
for module in &level.layers {
// Should not be imported by lower layers.
for lower_level in &levels[index + 1..] {
for lower_module in &lower_level.layers {
permutations.push((*lower_module, *module));
permutations.insert((*lower_module, *module));
}
}

Expand All @@ -99,8 +107,19 @@ impl Graph {
if sibling_module == module {
continue;
}
permutations.push((*module, *sibling_module));
permutations.insert((*module, *sibling_module));
}
}

// Should not be imported by higher layers if there is a closed layer inbetween.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@seddonym I'm unsure if this is the definition of closed layers that we want. Is this behaviour consistent with your understanding of the term "closed layer"?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test_cannot_import_through_closed_mid makes the implemented behaviour clear.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think so. The question of whether to disallow indirect imports is interesting though, I wonder if that is something we would need to make optional. Let me have a think about it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coming back to it, I agree with how you've done it here. Can you see any reason why we should allow indirect imports?

Copy link
Collaborator Author

@Peter554 Peter554 Jul 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you.

Can you see any reason why we should allow indirect imports?

I think allowing indirect imports would undermine the point of the contract. I think indirect imports should be disallowed. If we find that for some reason we need that in the future, then we could always add that option then (but by default I think indirect imports should be disallowed).

let mut closed = false;
for higher_level in levels[..index].iter().rev() {
if closed {
for higher_module in &higher_level.layers {
permutations.insert((*higher_module, *module));
}
}
closed |= higher_level.closed;
}
}
}
Expand Down Expand Up @@ -208,3 +227,106 @@ impl Graph {
)
}
}

#[cfg(test)]
mod tests {
use super::*;
use crate::graph::Graph;
use rustc_hash::FxHashSet;

#[test]
fn test_generate_module_permutations_simple_layers() {
let mut graph = Graph::default();

let top_module = graph.get_or_add_module("app.top").token;
let middle_module = graph.get_or_add_module("app.middle").token;
let bottom_module = graph.get_or_add_module("app.bottom").token;

let mut top_layer = FxHashSet::default();
top_layer.insert(top_module);

let mut middle_layer = FxHashSet::default();
middle_layer.insert(middle_module);

let mut bottom_layer = FxHashSet::default();
bottom_layer.insert(bottom_module);

let top_level = Level::new(top_layer, false, false);
let middle_level = Level::new(middle_layer, false, false);
let bottom_level = Level::new(bottom_layer, false, false);

let levels = vec![top_level, middle_level, bottom_level];

let permutations = graph.generate_illegal_import_permutations_for_layers(&levels);

assert_eq!(
permutations,
FxHashSet::from_iter([
(bottom_module, middle_module),
(bottom_module, top_module),
(middle_module, top_module),
])
);
}

#[test]
fn test_generate_module_permutations_independent_layer() {
let mut graph = Graph::default();

let module_a = graph.get_or_add_module("app.independent.a").token;
let module_b = graph.get_or_add_module("app.independent.b").token;

let mut independent_layer = FxHashSet::default();
independent_layer.insert(module_a);
independent_layer.insert(module_b);

let independent_level = Level::new(independent_layer, true, false);

let levels = vec![independent_level];

let permutations = graph.generate_illegal_import_permutations_for_layers(&levels);

assert_eq!(
permutations,
FxHashSet::from_iter([(module_a, module_b), (module_b, module_a),])
);
}

#[test]
fn test_generate_module_permutations_closed_layer() {
let mut graph = Graph::default();

// Create three layers with the middle one closed
let top_module = graph.get_or_add_module("app.top").token;
let middle_module = graph.get_or_add_module("app.middle").token;
let bottom_module = graph.get_or_add_module("app.bottom").token;

let mut top_layer = FxHashSet::default();
top_layer.insert(top_module);

let mut middle_layer = FxHashSet::default();
middle_layer.insert(middle_module);

let mut bottom_layer = FxHashSet::default();
bottom_layer.insert(bottom_module);

let top_level = Level::new(top_layer, false, false);
let middle_level = Level::new(middle_layer, false, true); // Closed layer
let bottom_level = Level::new(bottom_layer, false, false);

let levels = vec![top_level, middle_level, bottom_level];

let permutations = graph.generate_illegal_import_permutations_for_layers(&levels);

assert_eq!(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I think this would be clearer as a single assertion on the result.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

permutations,
FxHashSet::from_iter([
(bottom_module, middle_module),
(bottom_module, top_module),
(middle_module, top_module),
// Top should not import Bottom due to closed middle layer
(top_module, bottom_module),
])
);
}
}
11 changes: 9 additions & 2 deletions rust/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -543,7 +543,7 @@ impl GraphWrapper {
.unwrap()
.into_iter()
.map(|name| match container.clone() {
Some(container) => format!("{}.{}", container, name),
Some(container) => format!("{container}.{name}"),
None => name,
})
.filter_map(|name| match self.get_visible_module_by_name(&name) {
Expand All @@ -561,7 +561,14 @@ impl GraphWrapper {
.extract::<bool>()
.unwrap();

levels.push(Level::new(layers, independent));
let closed = level_dict
.get_item("closed")
.unwrap()
.unwrap()
.extract::<bool>()
.unwrap();

levels.push(Level::new(layers, independent, closed));
}
levels_by_container.push(levels);
}
Expand Down
1 change: 1 addition & 0 deletions rust/tests/large.rs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ fn test_large_graph_deep_layers() {
.token()
.conv::<FxHashSet<_>>(),
true,
false,
)
})
.collect();
Expand Down
10 changes: 7 additions & 3 deletions src/grimp/adaptors/graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,11 @@ def find_illegal_dependencies_for_layers(
try:
result = self._rustgraph.find_illegal_dependencies_for_layers(
layers=tuple(
{"layers": layer.module_tails, "independent": layer.independent}
{
"layers": layer.module_tails,
"independent": layer.independent,
"closed": layer.closed,
}
for layer in layers
),
containers=set(containers) if containers else set(),
Expand Down Expand Up @@ -201,9 +205,9 @@ def _parse_layers(layers: Sequence[Layer | str | set[str]]) -> tuple[Layer, ...]
if isinstance(layer, Layer):
out_layers.append(layer)
elif isinstance(layer, str):
out_layers.append(Layer(layer, independent=True))
out_layers.append(Layer(layer))
else:
out_layers.append(Layer(*tuple(layer), independent=True))
out_layers.append(Layer(*tuple(layer)))
return tuple(out_layers)


Expand Down
5 changes: 5 additions & 0 deletions src/grimp/application/ports/graph.py
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,11 @@ def find_illegal_dependencies_for_layers(
compatibility it is also possible to pass a simple `set[str]` to describe a layer. In this
case the sibling modules within the layer will be considered independent.

By default layers are open. `Layer.closed` can be set to True to create a closed layer.
Imports from higher to lower layers cannot bypass closed layers - the closed layer must be
included in the import chain. For example, given the layers high -> mid (closed) -> low then
all import chains from high -> low must go via mid.

Arguments:

- layers: A sequence, each element of which consists either of a `Layer`, the name
Expand Down
7 changes: 5 additions & 2 deletions src/grimp/domain/valueobjects.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,12 +69,15 @@ class Layer:

module_tails: Set[str]
independent: bool
closed: bool

# A custom `__init__` is needed since `module_tails` is a variadic argument.
def __init__(self, *module_tails: str, independent: bool = True) -> None:
def __init__(self, *module_tails: str, independent: bool = True, closed: bool = False) -> None:
# `object.__setattr__` is needed since the dataclass is frozen.
object.__setattr__(self, "module_tails", set(module_tails))
object.__setattr__(self, "independent", independent)
object.__setattr__(self, "closed", closed)

def __str__(self) -> str:
return f"{self.module_tails}, independent={self.independent}"
module_tails = sorted(self.module_tails)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sorted needed to make the test stable.

return f"{module_tails}, independent={self.independent}, closed={self.closed}"
Loading
Loading