Code Quality Tool Lib

[gauthamnair]

This is a library supporting the behavior proposed in #17729 (comment)

The usage example is a bit awkward but there is a lot of variation and discussion about the steps to make the use-case smooth. That work is separated out to a subsequent PR. The core is here.

Goal

Allows a user to define custom linters or formatters by:

1. defining an adhoc-tool compatible target
2. wrapping it in a new code_quality_tool target
3. referring to this target from a thin in-repo plugin.

Example Use-case

in pants.toml:

[GLOBAL]
pythonpath = ["%(buildroot)s/pants-plugins"]
backend_packages.add = [

    # needed for python_requirement
    "pants.backend.python",

    # code_quality_tool's target and rules are defined in the adhoc backend
    "pants.backend.experimental.adhoc",

    # thin in-repo plugin activates a particular target as a linter
    "flake8_linter_plugin",
]

in BUILD:

# a python requirement is an adhoc_tool-compatible runnable
python_requirement( 
    name="flake8",
    requirements=["flake8==5.0.4"]
)

# new target type describes how to use a runnable as a code quality tool
code_quality_tool(
    name="flake8_tool",
    runnable=":flake8",
    execution_dependencies=[":flake8_conf"],
    file_glob_include=["**/*.py"],
    file_glob_exclude=["dont_lint_me/**"],
    args=["--indent-size=2"],
)

and an in-repo plugin pants-plugins/flake8_linter_plugin/register.py

from pants.backend.adhoc.code_quality_tool import CodeQualityToolRuleBuilder

def rules():
    cfg = CodeQualityToolRuleBuilder(
            goal="lint", target="//:flake8_tool", name="Flake8", scope="flake8_tool"
    )
    return cfg.rules()

The lib supports:

• fmt, lint and fix
• any runnable supported by adhoc_tool
• passing additional runnables that need to be available on the PATH
• passing configuration via exec dependencies
• passing args
• file-based lint/fmt/fix only (as opposed to target based - therefore no target-level skip_ fields)
• skip and only working similarly to regular code quality tools.

See the tests for demos.

Full demo repo

See this branch:
https://github.com/gauthamnair/example-pants-byo-tool/tree/code-quality-tool-lib-demo

Follow-ups

• Add the ability to access this functionality without having to write even a thin in-repo plugin but in pants.toml directly. (as discussed in Allow using a shell command/adhoc tool as part of any goal (check, lint, fmt, fix, deploy, ...) #17729 (comment))
• DRY/factor out common bit of machinery with adhoc_tool

[gauthamnair]

@huonw @kaos I think I've added the missing ingredients.

• package has been shuffled around and is simplified and contained in adhoc
• fix works now
• Repaired some docs issue (turns out the target had a broken help - how do we usually test for that?)
• I confirmed that the runtime graph looks fine for this thing despite the dynamic generation.

[lilatomic]

Really neat!
I left a few comments. I think we're good to go without them. They're mostly possibilities for improvement.
There's some duplication between the _build_*_rules. I think that's fine. I think it's better to have 3 easy-to-follow Transaction Scripts than a pile of small "reusable" components (like constructing the FileSpecMatcher).

---

One thing which we should think about is rule deduplication. Some tools support multiple operations. For example, https://radon.readthedocs.io/en/latest/ has 4 different metrics. If you set up 4 different CodeQualityTools to achieve this, I think each one will generate a separate CodeQualityToolInstance. I'm not sure how much of a problem that will be, since I think that the runnable target will deduplicate the hard work of installing the tool which is where most of the gains would be. Deduplicating isn't that hard, Metalint and other places in Pants do this with a simple cache decorator.

[gauthamnair]

Yes, agreed, there is some factoring of commonality that needs to be done, which I can take a crack at after these prs are in.

[kaos]

lgtm. Thanks for iterating on this! 🚀

[huonw]

I like it, a few things to tweak, but getting very close!

[gauthamnair]

@huonw I think the issues you brought up are addressed now.

[benjyw]

Checking in on the status of this. @huonw looks like it needs a last pass over the response to your code review comments?

[huonw]

Nice!