[Discussion] Put files into buckets of responsibilities/seams #145

kenzan100 · 2021-10-13T18:30:46Z

What are you trying to accomplish?

Trying to find a seam within the Packwerk module.

We have some rough exploration of how it works (see below diagram). This PR will acts as a self-annotated PR to invite more discussions on what module structure Packwerk should enjoy.

Responsibilities and its dependencies

Data/object transformation journey

What approach did you choose and why?

A couple of motivations to do this;

We're interested in what kind of contracts (interface) Packwerk can and should expose, so that 3rd party integration becomes possible. (checkers?)
We're also interested in documenting the high-level overview of "how it works", much like explained in https://matklad.github.io/2021/02/06/ARCHITECTURE.md.html

What should reviewers focus on?

Currently, the files completely ignore the proposed namespace. No need to commit editing each file until we're comfortable finalizing the proposed change.

Does a file belong to an appropriate directory?
Naming of a directory (namespace) sounds right to you?
What should be the pattern to bridge these seams? (how to interact/orchestrate between them)

Type of Change

Discussion only. The merge-able artifact should be created as a separate PR.

kenzan100 · 2021-10-13T19:18:44Z

lib/packwerk/run_context.rb

+      # 1. file path to node
+      # It needs to return ancestors relative to node
+      node, ancestors = file_processor.call(file)
+
+      # Inside NodeProcessor - @reference_extractor.reference_from_node(node, ancestors: ancestors, file_path: @filename)
+      # 2. node to constant
+      # 3. constant to reference
+      @constant_name_inspectors.each do |inspector|
+        constant_name = inspector.constant_name_from_node(node, ancestors: ancestors)
+        break if constant_name
+      end
+
+      reference_from_constant(constant_name, node: node, ancestors: ancestors, file_path: file_path) if constant_name
+
+      # Inside NodeProcessor
+      # 4. reference to an offence
+      @checkers.each_with_object([]) do |checker, violations|
+        next unless checker.invalid_reference?(reference)
+        offense = Packwerk::ReferenceOffense.new(
+          location: Node.location(node),
+          reference: reference,
+          violation_type: checker.violation_type
+        )
+        violations << offense
+      end


If we will divide code's responsibility by their explicit "steps", something needs to have the responsibility of orchestration.

RunContext is just one convenient place atm, but what's more important is to find out if this way of separation of responsibility is what we want to go for, and whether or not the interface between each step is clean and extensible.

Does this direction make sense to you guys? @exterm @dougedey-shopify

Can we eliminate RunContext and let ParseRun orchestrate?

yup, certainly! I'll first focus on isolating the responsibilities of data transformation within the RunContext, then will follow up with the PR to eliminate the redundancy between RunContext and ParseRun.

kenzan100 · 2021-10-13T19:46:09Z

lib/packwerk/checking_references/checkers/README.md

+# Packwerk::Checkers
+
+- Checker implements `invalid_reference?(reference)`, which takes a `Packwerk::Reference` and returns Boolean.
+
+### Packwerk::Reference
+
+- It contains the information about source package and destination package.
+
+
+## Example: DependencyChecker
+
+- Dependency means any outgoing dependency from the source package.
+- If source package is enforcing dependency, it will check if declared dependency includes the destination package.
+
+
+## Example: PrivacyChecker
+
+- Privacy means any incoming dependency towards the destination package.
+- If destination package is enforcing privacy, it till check if source package references non-public constants.


Checker is an obvious candidate of what we can accept as 3rd party integration.

Does current implementation leak too much of Reference and subsequent objects?
For example, relative_path in Reference doesn't seem to be used at all, but I might be wrong.

I think relative_path is used in, or through, the offense

exterm · 2021-10-13T21:14:33Z

lib/packwerk/checking_references/checkers/README.md

@@ -0,0 +1,19 @@
+# Packwerk::Checkers
+
+- Checker implements `invalid_reference?(reference)`, which takes a `Packwerk::Reference` and returns Boolean.


Nitpick: This is accurate, but it duplicates the interface definition in Packwerk::Checker, so I think in the final docs we should just point out that the interface exists instead of putting the details here

exterm · 2021-10-13T23:19:23Z

lib/packwerk/configurations/README.md

@@ -0,0 +1,3 @@
+### Configurations
+
+Includes validators which are outside of primary `check` runs.


I would probably go with the comment in https://github.com/Shopify/packwerk/pull/144/files#diff-5ca9dcd54fd0b9eeef94e00ba6adc7822dc83e5dd2bfb9e8d96fd3c356de8b8aR10-R11

exterm · 2021-10-13T23:21:31Z

lib/packwerk/extracting_references/README.md

@@ -0,0 +1,4 @@
+# Extracting references
+
+- Given a `AST::Node`, this module extracts `Packwerk::Reference`.


If you put it like this, it sounds like everything here works in the context of a single node. However, ParsedConstantDefinitions works on a whole syntax tree.

exterm · 2021-10-13T23:23:27Z

lib/packwerk/file_processor.rb

@@ -31,6 +33,7 @@ def call(file_path)
        node_processor = @node_processor_factory.for(filename: file_path, node: node)
        node_visitor = Packwerk::NodeVisitor.new(node_processor: node_processor)

+        # node to offence
        node_visitor.visit(node, ancestors: [], result: result)


Yeah, the content of this conditional doesn't really seem to fit here. I'd rather just hand off to something that processes the AST instead of instantiating processors and visitors here.

exterm · 2021-10-13T23:25:18Z

lib/packwerk/run_context.rb

@@ -52,7 +54,32 @@ def initialize(

    sig { params(file: String).returns(T::Array[T.nilable(::Packwerk::Offense)]) }
    def process_file(file:)
-      file_processor.call(file)
+      # 1. file path to node


it's more like file path to AST, but that contains nodes, so... you're not wrong

exterm · 2021-10-13T23:26:39Z

lib/packwerk/run_context.rb

+        break if constant_name
+      end
+
+      reference_from_constant(constant_name, node: node, ancestors: ancestors, file_path: file_path) if constant_name


yeah maybe at this point we'd have a list of references?

exterm · 2021-10-13T23:27:21Z

I think in general this makes sense but we can talk more tomorrow. I think there'll be more interesting questions coming up when you start putting it together.

Put files into buckets of responsibilities/seams

3a8b828

kenzan100 requested review from dougedey-shopify and exterm October 13, 2021 18:31

kenzan100 added 3 commits October 13, 2021 14:44

Add temp README for each folder

6fcd8a5

Add a bit more comments

e46b253

Write down the data transformation step at to clarify its steps

896797e

kenzan100 commented Oct 13, 2021

View reviewed changes

Run a draft documentation of Checker interface

ac57f56

kenzan100 commented Oct 13, 2021

View reviewed changes

exterm reviewed Oct 13, 2021

View reviewed changes

More flattened extraction

009df05

kenzan100 mentioned this pull request Oct 14, 2021

Refactor - simplify orchestration layer #147

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Discussion] Put files into buckets of responsibilities/seams #145

[Discussion] Put files into buckets of responsibilities/seams #145

kenzan100 commented Oct 13, 2021

kenzan100 Oct 13, 2021

exterm Oct 13, 2021

kenzan100 Oct 14, 2021

kenzan100 Oct 13, 2021

exterm Oct 13, 2021

exterm Oct 13, 2021

exterm Oct 13, 2021

exterm Oct 13, 2021

exterm Oct 13, 2021 •

edited

Loading

exterm Oct 13, 2021

exterm Oct 13, 2021

exterm commented Oct 13, 2021

		@@ -0,0 +1,19 @@
		# Packwerk::Checkers

		- Checker implements `invalid_reference?(reference)`, which takes a `Packwerk::Reference` and returns Boolean.

		@@ -0,0 +1,3 @@
		### Configurations

		Includes validators which are outside of primary `check` runs.

		@@ -0,0 +1,4 @@
		# Extracting references

		- Given a `AST::Node`, this module extracts `Packwerk::Reference`.

[Discussion] Put files into buckets of responsibilities/seams #145

Are you sure you want to change the base?

[Discussion] Put files into buckets of responsibilities/seams #145

Conversation

kenzan100 commented Oct 13, 2021

What are you trying to accomplish?

Responsibilities and its dependencies

Data/object transformation journey

What approach did you choose and why?

What should reviewers focus on?

Type of Change

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

exterm Oct 13, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

exterm commented Oct 13, 2021

exterm Oct 13, 2021 •

edited

Loading