Quick Start Questions

[KyleKing]

I started asking a question, then mostly answered it. This is the condensed version number ~5. Essentially I just ran into a bunch of issues understanding how archan works and how to apply it to my project. I think my below notes might be helpful to others and could be optionally integrated to the README

Would you consider reordering the example commands? I think creating the CSV with dependenpy might be the most common use case and would be helpful if listed first. I wasn't sure where the CSV files were coming from until I dug further

# Generate CSV DSM data for archan (dependenpy must be installed)
dependenpy archan --format=csv archan_dsm.csv

# No configuration, read CSV data from file
archan --no-config --input archan_dsm.csv
# or read CSV data from STDIN
dependenpy archan --format=csv | archan --no-config

# Load configuration file and run archan (see Configuration section to know how archan finds the config file)
archan
# or specify the configuration file to load
archan --config my_config.yml

# Output the list of available plugins in the current environment
archan --list-plugins

I created a modified version of the archan config, but I had to comment out the names to prevent skipping the steps:

analysis:
  archan_pylint.PylintProvider:
    # name: Pylint
    arguments:
      pylint_args:
        - dash_charts
    checkers:
      archan.CodeClean:
        allow_failure: true
        arguments:
          threshold: 10
  dependenpy.InternalDependencies:
    # name: Software Architecture
    arguments:
      packages:
        - dash_charts
    checkers:
      - archan.CompleteMediation
      - archan.EconomyOfMechanism:
          allow_failure: true
          arguments:
            simplicity_factor: 2
      - archan.LayeredArchitecture
      - archan.LeastCommonMechanism:
          allow_failure: true
          arguments:
            independence_factor: 5
  Open Design:
    Source Code: true

Installed the dependencies and ran archan

> poetry add dependenpy git+https://github.com/pawamoy/archan-pylint.git git+https://github.com/pawamoy/archan.git -D

Updating dependencies
Resolving dependencies... (25.4s)

Writing lock file

Package operations: 8 installs, 0 updates, 0 removals

  • Installing dependenpy (3.3.0)
  • Installing lazy-object-proxy (1.4.3)
  • Installing wrapt (1.12.1)
  • Installing astroid (2.4.2)
  • Installing pylint (2.6.0)
  • Installing archan-pylint (0.1.0 ef4a609)
  • Installing tap.py (3.0)
  • Installing archan (0.3.0 9443064)

❯ poetry run archan -c .archan.yaml
TAP version 13
# TAP results for Pylint Provider: Issues per Module
ok 1 Code Clean (ALLOWED FAILURE)
# TAP results for Internal Dependencies
ok 2 Complete Mediation
ok 3 Economy of Mechanism
ok 4 Layered Architecture
ok 5 Least Common Mechanism (ALLOWED FAILURE)
# TAP results for Open Design
ok 6 Source Code

On a side-note, can you publish a new wheel to PyPi? The current one has a typo bug for .ingo and some API differences (notably analyzers vs. analysis)

[KyleKing]

I finished a course on SW Architecture and the professor briefly mentioned DSMs and the proprietary Lattix tooling, but didn't go into depth. I've read a few whitepapers/blogs and now want to try using DSMs with my personal and later work projects

I found your response in pawamoy/dependenpy#47 (comment) which answers some of my followup questions

I have some followup questions that sort of all revolve around "What are the next steps from here?" Besides django-meerkat, are there other examples of archan/dependenpy in use? It looks like archan is intended to be an architecture or security test tool with binary PASS/FAIL results. How do you work with the raw quantitative data from the DSM? (This might be a better issue for dependenpy) What do you use for visualization?

[KyleKing]

In summary, I think there are some ways that the documentation could be improved for new users:

• Can I reorder the example snippets like I've shown above?
• How should a name/provider/checker be used in the archan.yaml file?
• Can I add archan-pylint to the README plugin section with dependenpy (with a disclaimer that it is not yet released)?
  • (I'll probably also add a pypi search if future plugins are published: https://pypi.org/search/?q=archan-)
• Can you push a new wheel for archan to PyPi?

I would be happy to submit a PR with documentation updates. Let me know if there is anything I should know before starting edits in a fork

[pawamoy]

Hello @KyleKing! Thank you for your patience (it's been two weeks since you opened this issue)! And thank you for your interest in archan/dependenpy/django-meerkat!

I'm sorry you had troubles getting started with archan, and I'm glad you were able to make it work.

I am absolutely favorable to Pull Requests to improve the documentation! This would be really, really appreciated 🙂 If you send one, and after it's merged, I can push a new version on PyPI of course.

About archan-pylint: if you're interested in this plugin, I can give you ownership or add you as a collaborator 🙂

To conclude, I'm open to any suggestion or improvement you'd like to submit!

[KyleKing]

Thanks! I'll try to open a PR when I have some time this weekend. I'm wrapping up some changes to migrate to copier and starting to use mkdocstrings and your name keeps popping up 😄 . Thanks for contributing to and building such useful tools!

[KyleKing]

I started making a few minor documentation changes (master...KyleKing:fix/44), but ran into an error:

❯ archan --no-config --input pytest_dsm.csv
 error  archan.config:60: Error while inflating "archan.CSVInput" analysis group. The group will not be added to the list. Exception: <class 'tuple'> type is not supported for a plugin list, use list or dict.
❯ dependenpy pytest --format=csv | archan --no-config
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/kyleking/Developer/Pull_Requests/archan/src/archan/cli.py", line 158, in main
    config = Config.default_config(file_path)
  File "/Users/kyleking/Developer/Pull_Requests/archan/src/archan/config.py", line 141, in default_config
    return Config(
  File "/Users/kyleking/Developer/Pull_Requests/archan/src/archan/config.py", line 46, in __init__
    self.config_dict = deepcopy(config_dict)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 230, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/usr/local/Caskroom/miniconda/base/lib/python3.8/copy.py", line 161, in deepcopy
    rv = reductor(4)
TypeError: cannot pickle '_io.TextIOWrapper' object

pytest_dsm.csv

module,pytest.__init__,pytest.__main__,pytest.collect
pytest.__init__,0,0,1
pytest.__main__,1,0,0
pytest.collect,1,0,0

[pawamoy]

The default config contains sys.stdin as a file path. It seems deepcopy is choking on it. We could probably remove sys.stdin from the call chain and replace it by None, and assign sys.stdin to the variable in CSVInput.get_data() instead.

[KyleKing]

I have had this on my list in awhile and I finally had some time to catch up on open source contributions today. I think I fixed the issue with sys.stdin and deepcopy (7c42f28)

I had trouble running the tests locally because invoke wasn't using my conda configuration, but figured I could also let the PR run the tasks. I'll revisit and make sure everything is passing