Support: requirements.txt file (#75)

* Support: requirements.txt file

* docs: update

* unused function remove & docs update

* check info fix

* fix: constant, type error

* libcst update

* requirements.txt remove

* Path.read_text & readlines

* _any_unimport variable name more meaningful

* workflow update

.github/workflows/test.yml

@@ -20,7 +20,6 @@ jobs:
         run: |
           python -m pip install --upgrade pip
           python -m pip install -r requirements-dev.txt
-          python -m pip install -r requirements.txt
 
       - name: Test with pytest
         run: |

docs/CHANGELOG.md

@@ -4,6 +4,13 @@ All notable changes to this project will be documented in this file.
 
 ## [0.2.8] - ././2020
 
+- [💪 Support: requirements.txt file by @hakancelik96](https://github.com/hakancelik96/unimport/pull/75)
+
+  - Now, You can automatically delete unused modules from the requirements.txt file (
+    `unimport --requirements --remove`), see the difference (
+    `unimport --requirements --diff`), delete it by requesting permission (
+    `unimport --requirements --permission`), or just check ( `unimport --requirements`).
+
 - [🔥 Fix: match error between import name and name by @hakancelik96](https://github.com/hakancelik96/unimport/pull/74)
 
 - [💪 Support for type hints (#58) by @hakancelik96 & string typing @isidentical](https://github.com/hakancelik96/unimport/pull/71)

docs/README.md

@@ -289,14 +289,20 @@ import os
 __all__ = ["os"] # this import is used and umimport can understand
 ```
 
+## Requirements.txt
+
+You can automatically delete unused modules from the requirements.txt file
+(`unimport --requirements --remove`), see the difference (
+`unimport --requirements --diff`), delete it by requesting permission (
+`unimport --requirements --permission`), or just check ( `unimport --requirements`).
+
 ## Command line options
 
 You can list many options by running unimport --help
 
 ```
-usage: unimport [-h] [-c PATH] [--include include] [--exclude exclude]
-                [--include-star-import] [--show-error] [-d] [-r | -p]
-                [--check] [-v]
+usage: unimport [-h] [-c PATH] [--include include] [--exclude exclude] [--include-star-import] [--show-error]
+                [-d] [-r | -p] [--requirements] [--check] [-v]
                 [sources [sources ...]]
 
 A linter, formatter for finding and removing unused import statements.
@@ -312,12 +318,11 @@ optional arguments:
   --exclude exclude     file exclude pattern.
   --include-star-import
                         Include star imports during scanning and refactor.
-  --show-error          Show or don't show errors captured during static
-                        analysis.
-  -d, --diff            Prints a diff of all the changes unimport would make
-                        to a file.
+  --show-error          Show or don't show errors captured during static analysis.
+  -d, --diff            Prints a diff of all the changes unimport would make to a file.
   -r, --remove          remove unused imports automatically.
   -p, --permission      Refactor permission after see diff.
+  --requirements        Include requirements.txt file, You can use it with all other arguments
   --check               Prints which file the unused imports are in.
   -v, --version         Prints version of unimport
 
@@ -368,16 +373,16 @@ repos:
     rev: v0.2.8
     hooks:
       - id: unimport
-        args: [-r, --include-star-import]
+        args: [--remove, --requirements, --include-star-import]
 ```
 
 ## Our badge
 
-[![style](https://img.shields.io/badge/unimport-v0.2.8-red)](https://github.com/hakancelik96/unimport)
+[![style](https://img.shields.io/badge/unimport-ED1C24)](https://github.com/hakancelik96/unimport)
 
 **Please insert this badge into your project**
 
-`[![style](https://img.shields.io/badge/unimport-v0.2.8-red)](https://github.com/hakancelik96/unimport)`
+`[![style](https://img.shields.io/badge/unimport-ED1C24)](https://github.com/hakancelik96/unimport)`
 
 ## CONTRIBUTING
 
@@ -391,13 +396,6 @@ All notable changes to this project will be documented in this file.
 
 [CHANGELOG.md](/CHANGELOG.md)
 
-## Contact
-
-- [![Telegram](https://img.shields.io/badge/telegram-@hakancelik-brightgreen?logo=telegram)](https://t.me/hakancelik96)
-- [![Twitter](https://img.shields.io/twitter/follow/hakancelik96?style=social)](https://twitter.com/hakancelik96)
-- [![Github](https://img.shields.io/github/followers/hakancelik96?label=hakancelik96&style=social)](https://github.com/hakancelik96)
-- <hakancelik96@outlook.com>
-
 ## Who's using Unimport
 
 [![radity.com](https://raw.githubusercontent.com/hakancelik96/unimport/master/images/clients/radity.jpg)](https://radity.com/?ref=unimport)

requirements-dev.txt

@@ -3,3 +3,4 @@ pytest
 pytest-cov
 pre-commit==2.4.0
 wheel==0.34.2
+libcst==0.3.8

setup.py

@@ -30,7 +30,7 @@ def get_long_description():
     license_file="LICENSE",
     python_requires=">=3.6.0",
     packages=["unimport"],
-    install_requires=["libcst==0.3.5"],
+    install_requires=["libcst==0.3.8"],
     extras_require={"pyproject": ["toml"]},
     zip_safe=False,
     include_package_data=True,

unimport/__main__.py

@@ -1,13 +1,17 @@
 import argparse
+import difflib
 import re
 import sys
 from pathlib import Path
-from typing import List, Optional
+from typing import TYPE_CHECKING, List, Optional, Tuple
 
 from unimport import __description__, __version__
 from unimport.color import Color
 from unimport.session import Session
 
+if TYPE_CHECKING:
+    from unimport.models import TYPE_IMPORT
+
 parser = argparse.ArgumentParser(
     prog="unimport",
     description=__description__,
@@ -75,6 +79,11 @@
     action="store_true",
     help="Refactor permission after see diff.",
 )
+parser.add_argument(
+    "--requirements",
+    action="store_true",
+    help="Include requirements.txt file, You can use it with all other arguments",
+)
 parser.add_argument(
     "--check",
     action="store_true",
@@ -89,7 +98,7 @@
 )
 
 
-def color_diff(sequence: tuple) -> str:
+def color_diff(sequence: Tuple[str, ...]) -> str:
     contents = "\n".join(sequence)
     lines = contents.split("\n")
     for i, line in enumerate(lines):
@@ -106,50 +115,44 @@ def color_diff(sequence: tuple) -> str:
     return "\n".join(lines)
 
 
-def print_if_exists(sequence: tuple):
+def print_if_exists(sequence: Tuple[str, ...]) -> bool:
     if sequence:
         print(color_diff(sequence))
-        return True
-    return None
+    return bool(sequence)
 
 
-def output(name: str, path: Path, lineno: int, modules: str) -> str:
-    modules = modules or ""
-    return (
-        f"{Color(name).yellow} at "
-        f"{Color(str(path)).green}:{Color(str(lineno)).green}"
-        f" {modules}"
-    )
-
-
-def get_modules(imp: str, is_star: bool, modules: str):
+def get_as_import_from(
+    import_name: str, is_star: bool, modules: List[str]
+) -> Optional[str]:
+    _modules = ""
     if is_star:
         _modules = ", ".join(modules)
         if len(_modules) > 5:
-            modules = f"({_modules})"
-        elif len(_modules) == 0:
-            modules = ""
-        else:
-            modules = f"{_modules}"
-    else:
-        modules = ""
+            _modules = "(" + _modules + ")"
     if modules:
-        return f"from {imp} import {modules}"
+        return f"from {import_name} import {_modules}"
     return None
 
 
-def show(unused_import, py_path):
+def show(unused_import: "List[TYPE_IMPORT]", py_path: Path) -> None:
     for imp in unused_import:
-        modules = get_modules(imp["name"], imp["star"], imp["modules"])
+        import_from = get_as_import_from(
+            imp["name"], imp["star"], imp["modules"]
+        )
         if (imp["star"] and imp["module"]) or (not imp["star"]):
-            print(output(imp["name"], py_path, imp["lineno"], modules))
+            print(
+                f"{Color(imp['name']).yellow} at "
+                f"{Color(str(py_path)).green}:{Color(str(imp['lineno'])).green}"
+                f" {import_from or ''}"
+            )
 
 
 def main(argv: Optional[List[str]] = None) -> None:
     namespace = parser.parse_args(argv)
     namespace.check = namespace.check or not any(
-        [value for value in vars(namespace).values()][6:-1]
+        [namespace.diff, namespace.remove, namespace.permission]
     )
+    namespace.diff = namespace.diff or namespace.permission
     session = Session(
         config_file=namespace.config,
         include_star_import=namespace.include_star_import,
@@ -167,17 +170,25 @@ def main(argv: Optional[List[str]] = None) -> None:
         exclude_list.append(session.config.exclude)  # type: ignore
     include = re.compile("|".join(include_list)).pattern
     exclude = re.compile("|".join(exclude_list)).pattern
-    _any_unimport = False
+    is_unused_module = False
+    unused_modules = set()
     for source_path in namespace.sources:
         for py_path in session.list_paths(source_path, include, exclude):
+            session.scanner.run_visit(source=session.read(py_path)[0])
+            unused_imports = session.scanner.unused_imports
+            if not is_unused_module and unused_imports:
+                is_unused_module = True
+            unused_modules.update(
+                {
+                    imp["module"].__name__.split(".")[0]  # type: ignore
+                    for imp in unused_imports
+                    if imp["module"]
+                }
+            )
+            session.scanner.clear()
             if namespace.check:
-                session.scanner.run_visit(source=session.read(py_path)[0])
-                unused_imports = session.scanner.unused_imports
                 show(unused_imports, py_path)
-                if not (not _any_unimport and not unused_imports):
-                    _any_unimport = True
-                session.scanner.clear()
-            if namespace.diff or namespace.permission:
+            if namespace.diff:
                 exists_diff = print_if_exists(session.diff_file(py_path))
             if namespace.permission and exists_diff:
                 action = input(
@@ -192,12 +203,46 @@ def main(argv: Optional[List[str]] = None) -> None:
                 refactor_source = session.refactor_file(py_path, apply=True)
                 if refactor_source != source:
                     print(f"Refactoring '{Color(str(py_path)).green}'")
-    if not _any_unimport and namespace.check:
+    if not is_unused_module and namespace.check:
         print(
             Color(
                 "✨ Congratulations there is no unused import in your project. ✨"
             ).green
         )
+    if namespace.requirements and unused_modules:
+        requirements_path = Path("requirements.txt")
+        if not requirements_path.exists():
+            return
+        result = ""
+        source = requirements_path.read_text()
+        for index, requirement in enumerate(source.splitlines()):
+            if requirement.split("==")[0] not in unused_modules:
+                result += f"{requirement}\n"
+            else:
+                if namespace.check and requirement:
+                    print(
+                        f"{Color(requirement).cyan} at "
+                        f"{Color(str(requirements_path)).cyan}:{Color(str(index + 1)).cyan}"
+                    )
+        if namespace.diff:
+            exists_diff = print_if_exists(
+                tuple(
+                    difflib.unified_diff(
+                        source.splitlines(),
+                        result.splitlines(),
+                        fromfile=str(requirements_path),
+                    )
+                )
+            )
+        if namespace.permission and exists_diff:
+            action = input(
+                f"Apply suggested changes to '{Color(str(requirements_path)).cyan}' [Y/n] ? >"
+            ).lower()
+            if action == "y" or action == "":
+                namespace.remove = True
+        if namespace.remove:
+            requirements_path.write_text(result)
+            print(f"Refactoring '{Color(str(requirements_path)).cyan}'")
 
 
 if __name__ == "__main__":

unimport/scan.py

@@ -147,6 +147,8 @@ def visit_Constant(
         self, node: ast.Constant, id_: Optional[int] = None
     ) -> None:
         id_ = id_ or id(node)
+        if not isinstance(node.value, (str, bytes)):
+            return
         try:
             parent = first_occurrence(node, ast.FunctionDef)
         except AttributeError:
@@ -157,11 +159,7 @@ def visit_Constant(
                 type_parent in {ast.AnnAssign, ast.arg}
                 for type_parent in map(type, get_parents(node))
             )
-            if (
-                isinstance(node.value, str)
-                and (parent and id(parent.returns) == id_)
-                or is_annasign_and_arg
-            ):
+            if (parent and id(parent.returns) == id_) or is_annasign_and_arg:
                 with contextlib.suppress(SyntaxError):
                     self.visit(ast.parse(node.value, mode="eval"))