docs/source/test_a_lint_rule.ipynb

{
 "cells": [
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "================\n",
    "Test a Lint Rule\n",
    "================\n",
    "\n",
    "This guide assumes you've already started implementing your lint rule. If not, please read :doc:`build_a_lint_rule`.\n",
    "\n",
    "You've written your rule, but now you need a test plan for it.\n",
    "Every lint rule should include some test cases.\n",
    "\n",
    "Writing Unit Tests\n",
    "==================\n",
    "While manually running your lint rule is useful, a unit test makes it easier to validate that your rule isn't failing to report certain violations, prevents future regressions, and helps your reviewers understand your rule.\n",
    "An ideal lint rule written should warn on every violation (avoiding false-negatives), and never warn where there's no violation (false-positives). For this reason, we require that every rule includes both ``VALID`` and ``INVALID`` test cases.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "nbsphinx": "hidden"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.chdir(\"../../\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fixit import (\n",
    "    CstLintRule,\n",
    "    InvalidTestCase as Invalid,\n",
    "    ValidTestCase as Valid,\n",
    ")\n",
    "\n",
    "\n",
    "class NoInheritFromObjectRule(CstLintRule):\n",
    "    MESSAGE = \"Inheriting from object is a no-op. 'class Foo:' is just fine =)\"\n",
    "    VALID = []\n",
    "    INVALID = []"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Valid test cases are defined using :class:`~fixit.ValidTestCase` (imported as a shorter name ``Valid``), and invalid test cases are defined using :class:`~fixit.InvalidTestCase` (imported as a shorter name ``Invalid``).\n",
    "You should add test cases until all potential edge-cases are covered.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import libcst as cst\n",
    "import libcst.matchers as m\n",
    "\n",
    "class NoInheritFromObjectRule(CstLintRule):\n",
    "    MESSAGE = \"Inheriting from object is a no-op. 'class Foo:' is just fine =)\"\n",
    "\n",
    "    VALID = [Valid(\"class A(something):\\n    pass\"), Valid(\"class A:\\n    pass\")]\n",
    "    INVALID = []"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Adding Autofix Test Case\n",
    "========================\n",
    "Testing an auto-fixer is as easy as adding the expected replacement code to your `Invalid` test case.\n",
    "Function strings can be multi-line, similar to docstrings, the leading whitespace is trimmed.\n",
    "The position of the lint suggestion is the starting position of the reported CSTNode.\n",
    "(In our example, it's the start of ``ClassDef`` node which is the position of ``c`` in ``class`` )\n",
    "It can be specified by providing ``line`` and ``column`` numbers in an ``Invalid`` test case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class NoInheritFromObjectRule(CstLintRule):\n",
    "    \"\"\"\n",
    "    In Python 3, a class is inherited from ``object`` by default.\n",
    "    Explicitly inheriting from ``object`` is redundant, so removing it keeps the code simpler.\n",
    "    \"\"\"\n",
    "    MESSAGE = \"Inheriting from object is a no-op. 'class Foo:' is just fine =)\"\n",
    "    VALID = [\n",
    "        Valid(\"class A(something):    pass\"),\n",
    "        Valid(\n",
    "            \"\"\"\n",
    "            class A:\n",
    "                pass\"\"\"\n",
    "        ),\n",
    "    ]\n",
    "    INVALID = [\n",
    "        Invalid(\n",
    "            \"\"\"\n",
    "            class B(object):\n",
    "                pass\"\"\",\n",
    "            line=1,\n",
    "            column=1,\n",
    "            expected_replacement=\"\"\"\n",
    "            class B:\n",
    "                pass\"\"\",\n",
    "        ),\n",
    "        Invalid(\n",
    "            \"\"\"\n",
    "            class B(object, A):\n",
    "                pass\"\"\",\n",
    "            line=1,\n",
    "            column=1,\n",
    "            expected_replacement=\"\"\"\n",
    "            class B(A):\n",
    "                pass\"\"\",\n",
    "        ),\n",
    "    ]\n",
    "\n",
    "    def visit_ClassDef(self, node: cst.ClassDef) -> None:\n",
    "        new_bases = tuple(\n",
    "            base for base in node.bases if not m.matches(base.value, m.Name(\"object\"))\n",
    "        )\n",
    "\n",
    "        if tuple(node.bases) != new_bases:\n",
    "            # reconstruct classdef, removing parens if bases and keywords are empty\n",
    "            new_classdef = node.with_changes(\n",
    "                bases=new_bases,\n",
    "                lpar=cst.MaybeSentinel.DEFAULT,\n",
    "                rpar=cst.MaybeSentinel.DEFAULT,\n",
    "            )\n",
    "\n",
    "            # report warning and autofix\n",
    "            self.report(node, replacement=new_classdef)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Run Tests\n",
    "=========\n",
    "\n",
    "Fixit provides :func:`~fixit.add_lint_rule_tests_to_module` to automatically generate `unittest <https://docs.python.org/3/library/unittest.html>`_ test cases in a module.\n",
    "If you're contributing a new lint rule to Fixit, you can add the rule in ``fixit/rules/`` of fixit repository.\n",
    "The :func:`~fixit.add_lint_rule_tests_to_module` is already configured in ``fixit/tests/__init__.py``.\n",
    "Run the added tests by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "! python -m unittest fixit.tests.NoInheritFromObjectRule"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "If you're developing a custom lint rule in your codebase, you can configure :func:`~fixit.add_lint_rule_tests_to_module` in your test module \n",
    "by passing `globals()` as the `module_attrs` argument and providing a collection of the rules you would like to test as the `rules` argument.\n",
    "E.g. ``add_lint_rule_tests_to_module(globals(), {my_package.Rule1, my_package.Rule2})``\n",
    "Then run your test module by::\n",
    "\n",
    "    python -m unittest your_test_module"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from fixit import add_lint_rule_tests_to_module\n",
    "\n",
    "add_lint_rule_tests_to_module(globals(), rules=[NoInheritFromObjectRule])\n",
    "import unittest\n",
    "\n",
    "unittest.main(argv=[\"first-arg-is-ignored\"], exit=False)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Run Your Rule in a Codebase\n",
    "===========================\n",
    "\n",
    "The easiest way to see the effects of your lint rule is to run it against the entire codebase. This is pretty easy::\n",
    "    \n",
    "    python -m fixit.cli.run_rules path --help  # see all the supported flags\n",
    "    python -m fixit.cli.run_rules path --rules RewriteToLiteralRule\n",
    "    \n",
    "This runs the linter in parallel across all the Python files in the path.\n",
    "It ignores ``# noqa``, ``# lint-fixme``, and ``# lint-ignore`` comments by default, but that can be overridden with the ``--use-ignore-comments`` flag (see ``--help`` for more details).\n",
    "\n",
    "Check out a few of the reported violations to see that generated reports are accurate.\n",
    "If possible, you should consider cleaning up these existing violations before shipping your lint rule.\n",
    "Otherwise, consider that fixing these preexisting violations in future changes may slow other developers down."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "raw_mimetype": "text/restructuredtext"
   },
   "source": [
    "Reference\n",
    "=========\n",
    "\n",
    ".. autoclass:: fixit.ValidTestCase\n",
    ".. autoclass:: fixit.InvalidTestCase\n",
    ".. autofunction:: fixit.add_lint_rule_tests_to_module"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}