DevTool

A single CLI bundling the small tools that come up constantly while working on a codebase: removing comments, counting lines, finding leftover TODOs, catching credentials before they are committed, spotting duplicate and oversized files, tidying a download folder, plus a few text and hashing helpers.

Every tool runs both from an interactive menu and as a subcommand, so the same code path serves an exploratory session and a CI pipeline.

     _            _              _
  __| | _____   _| |_ ___   ___ | |
 / _` |/ _ \ \ / / __/ _ \ / _ \| |
| (_| |  __/\ V /| || (_) | (_) | |
 \__,_|\___| \_/  \__\___/ \___/|_|

---

Demo

$ devtool stats ./src

Project statistics ─────────────────────────────────────────────────────
C:\projects\devtool\src

Totals
─────────────────────────────────────────────────────────────────────────
Files                    47
Lines                 5,637
Code         4,273  (75.8%)
Comments        390  (6.9%)
Blank lines             974

Languages
─────────────────────────────────────────────────────────────────────────
language  files  lines   code  comments   share
Python       47  5,637  4,273       390  100.0%  ████████████████████████

✓ Analysis completed

Files analysed     47
Files skipped       6
Time elapsed    0.11s

$ devtool scan-secrets ./project

Potential secrets
─────────────────────────────────────────────────────────────────────────

.env
     1  DB_PASSWORD=************  assignment · medium
        DB_PASSWORD=************

src\api.py
     6  API_KEY=************  stripe-key · high
        API_KEY = "************"

✓ Scan completed
Values are never printed. Rotate anything real that shows up here.

---

Features

Tool	Command	What it does
Repository health	check	Runs every check below at once and returns a CI-friendly exit code
Git inspector	git	Branch, upstream, status and pending changes
Conflict scanner	conflicts	Unresolved merge conflict markers left in the tree
.gitignore checker	gitignore	What is exposed that should not be, with an opt-in --fix
Health score	score	One weighted number, with every point accounted for
Complexity	complexity	Cyclomatic complexity per function
Dependencies	deps	Manifests, cross-checked against what the code imports
Snapshots	snapshot	Metrics saved over time, and the diff between them
CI setup	ci init	Generates a pipeline or Git hook that runs devtool check
Comment cleaner	clean-comments	Removes comments from source files and nothing else
Code statistics	stats	Files, lines, code, comments and blanks per language
TODO scanner	todos	TODO / FIXME / XXX / HACK found inside comments only
Secret scanner	scan-secrets	Heuristic search for committed credentials, values always masked
Duplicate finder	duplicates	Byte-identical files, by size then partial hash then SHA-256
Large file finder	large-files	Biggest files in a tree, with a --min-size threshold
File organizer	organize	Sorts loose files into category folders, after confirmation
JSON tool	json	Validate with a located error, pretty-print, minify
Text statistics	text-stats	Characters, words, lines, paragraphs
Hash generator	hash	MD5 / SHA-1 / SHA-256 / SHA-512, plus --check
UUID generator	uuid	v1 / v3 / v4 / v5, with formatting options

Two properties hold across all of them:

• Nothing is ever deleted. The only command that moves data is organize, and it shows the complete plan and waits for a yes.
• Every write is atomic. Files go through a temporary file and a rename, preserving text encoding, line-ending style and the final newline.

---

Repository health

devtool check is the pre-commit sweep: it runs the security, code, file and Git checks in one pass and reports a single verdict.

$ devtool check .

╔════════════════════════════════════════════════════╗
║                   DEVTOOL CHECK                    ║
║                 Repository Health                  ║
╚════════════════════════════════════════════════════╝

 SECURITY
 --------------------------------------------------
 ⚠ 1 potential secret found
     .env:1
 ✓ No private keys found

 CODE
 --------------------------------------------------
 ✗ 1 file with unresolved conflict markers
     src/api.py:2
 ✓ Code structure OK
 ⚠ 1 TODO found
 ⚠ 1 FIXME found

 FILES
 --------------------------------------------------
 ✓ No duplicate files
 ✓ No files larger than 50.0 MB

 GIT
 --------------------------------------------------
 ⚠ 4 uncommitted changes
     M README.md
     D src/util.py
 ✗ 4 untracked files (2 look risky)
     .env
     debug.log
     suspicious: .env, debug.log
 ⚠ 3 files that probably should be ignored

 PROJECT
 --------------------------------------------------
Files                        4
Lines                       14
Languages                    2
Branch     feature/new-scanner

----------------------------------------------------

Result: 2 errors, 6 warnings
Finished in 0.26s  ·  exit code 2

The check reuses the existing scanners rather than reimplementing them, so a # inside a string is still not a comment and a masked secret is still never printed.

What it covers

Section	Checks
SECURITY	potential secrets, committed private keys
CODE	merge conflict markers, unparsable files, TODO / FIXME / XXX / HACK
FILES	duplicate files, files above the size threshold (default 50 MB)
GIT	uncommitted changes, untracked files, unpushed commits, .gitignore coverage
PROJECT	files, lines and languages, from the same collector as devtool stats

Outside a Git repository the Git checks are skipped rather than failed, so check is useful on any folder.

Exit codes

Every command returns a meaningful code, so DevTool can gate a hook or a pipeline:

Code	Meaning	Examples
0	no problems	everything passed
1	warnings	TODOs, large files, duplicates, untracked files, missing .gitignore, medium-confidence secrets
2	errors	unresolved conflict markers, high-confidence secret, committed private key, risky untracked file — or a usage error such as a missing path

--strict promotes warnings to exit code 2 for a zero-tolerance gate.

devtool check . --strict

JSON output

devtool check . --output json

{
  "status": "warning",
  "exit_code": 1,
  "files": 184,
  "lines": 24821,
  "languages": 6,
  "todo": 12,
  "fixme": 3,
  "secrets": 0,
  "conflicts": 0,
  "large_files": 2,
  "duplicate_groups": 0,
  "is_git_repository": true,
  "branch": "main",
  "untracked_files": 3,
  "gitignore_exposed": 1,
  "checks": [ { "category": "SECURITY", "status": "ok", "message": "..." } ]
}

git, conflicts and gitignore accept --output json too. When stdout is not a terminal the JSON is written with a plain print, so no console width or theme can wrap a long string and break the parser reading it.

Git tools individually

$ devtool git .

Branch
  feature/new-scanner
  origin/feature/new-scanner  2 ahead

Status
Modified   2
Added      1
Deleted    1
Untracked  4

Changes
M  README.md
M  src/app.py
D  src/util.py
A  tests/test_scanner.py  staged

$ devtool conflicts .

⚠ Merge conflict markers found

src/api.py
     2  <<<<<<< HEAD
     4  =======
     6  >>>>>>> feature/other

1 file contains unresolved conflict markers.

A lone ======= line is not treated as a conflict — that is also a Markdown heading underline and a common ASCII separator. A file is only reported when it holds a matching <<<<<<< / >>>>>>> pair.

$ devtool gitignore .

⚠ Files that may not belong in Git

file          why                                          state
__pycache__/  Python bytecode cache                        not ignored
.env          environment file, usually holds credentials  not ignored
debug.log     log output                                   not ignored

Inside a repository the "ignored or not" verdict comes from git check-ignore itself, so it matches exactly what Git would do — including rules inherited from parent directories and global excludes. --fix appends the missing entries after showing them and asking:

devtool gitignore . --fix

It only ever appends: existing lines are never rewritten or reordered, and files Git already tracks stay tracked (git rm --cached is the tool for that, and the command says so).

---

Analysis

Health score

$ devtool score .

   B   83.6 / 100   ███████████████████████·····

Breakdown
─────────────────────────────────────────────────────────────────────────
component        score                    weight  points
Security           100  ████████████████      25    25.0
Correctness        100  ████████████████      20    20.0
Maintainability     38  ██████··········      20     7.6
Hygiene            100  ████████████████      15    15.0
Git                 76  ████████████····      10     7.6
Not applicable here, weight redistributed: Dependencies

Where the points went
─────────────────────────────────────────────────────────────────────────
  Maintainability (-12.4)
      -21.9  11% of functions over complexity 10
      -25    10 function(s) over complexity 20
      -15    24 function(s) longer than 50 lines
  Git (-2.4)
      -20    11 untracked file(s)
      -4     4 uncommitted change(s)

A single number is only useful if you can see where it came from, so the score never appears alone: each component reports its sub-score, its weight, and the specific reasons that cost points. The weights live as constants in scanners/score.py — disagreeing with them is a one-line change, not an argument.

Components with nothing to measure are skipped and their weight redistributed, not scored as zero: a small script should not get an F for having no package.json.

--min turns it into a gate:

devtool score . --min 80

The score is computed entirely from reports the other scanners already produced — it does no scanning of its own.

Complexity

$ devtool complexity ./src --limit 5

Most complex
─────────────────────────────────────────────────────────────────────────
   function     location                          cx  lines
F  run_check    devtool/scanners/health.py:122    51    226
F  scan         devtool/languages/scanner.py:159  36    126
D  apply_spans  devtool/cleaners/comments.py:61   28     73

Distribution
─────────────────────────────────────────────────────────────────────────
grade          meaning          count
A  1-5         simple             271
B  6-10        well structured     71
C  11-20       complex             32
D  21-30       hard to test         6
F  31+         unmaintainable       4

Two levels of fidelity, and the report says which one it used:

• Python — a real McCabe count from the ast module, per function, with nested functions counted separately.
• everything else — branch keywords counted at file level, marked ~, after the comment scanner has blanked out comments and string literals. That is the reuse that makes it worth doing: "if you see this" inside a string costs nothing, which a regex over the raw file would get wrong.

Per-function boundaries are not guessed outside Python. Finding them reliably needs a parser per language, and a wrong boundary produces a confidently wrong number — worse than an honest file-level one.

Dependencies

devtool deps .

Parses pyproject.toml (including Poetry), requirements*.txt, package.json, Cargo.toml, go.mod, Gemfile and composer.json, then reports dependencies without a version constraint, packages declared twice with conflicting constraints, and manifests missing a lock file — only where a lock file is actually a convention, so a plain setuptools pyproject.toml is not nagged.

For Python it also cross-checks declarations against the real imports:

• imported but not declared — a fresh install would fail
• declared but never imported — possibly dead weight

Import names are mapped to distribution names (yaml → PyYAML, cv2 → opencv-python, …), the standard library is excluded, and anything resolvable inside the project counts as local.

This is offline. There is no vulnerability database and no registry lookup. It answers "is my manifest tidy and honest?", not "am I running anything vulnerable?" — different questions needing different tools.

---

Snapshots

Record the project's metrics now, and see what moved later.

devtool snapshot save . --label v1.2

$ devtool snapshot compare .

  2026-08-11 01:10:26  →  2026-08-11 01:10:27
  score 100.0 → 94.4  -5.6

Changes
─────────────────────────────────────────────────────────────────────────
metric              before     after  change
score                100.0  →   94.4    -5.6
lines                    3  →      6      +3
todo                     0  →      1      +1
complexity_average     1.0  →    2.0      +1

Each snapshot is one JSON file under .devtool/snapshots/ — diffable, greppable, and committable if you want the trend in version control.

Knowing which way is up per metric is a judgement, so it lives in DIRECTION in core/snapshots.py: fewer TODOs is progress (green), more lines is neither good nor bad (grey). compare exits 1 when any metric regressed.

devtool snapshot list .

devtool snapshot prune . --keep 10

---

CI integration

devtool ci init . --target github

Generates a config that leans on the exit codes DevTool already returns, so the pipeline logic stays in one place — devtool check decides, CI reports:

      - name: Repository health check
        # exit 0 = clean, 1 = warnings, 2 = errors.
        # `|| [ $? -eq 1 ]` lets warnings pass but fails the job on errors.
        run: devtool check . || [ $? -eq 1 ]

Targets: github, gitlab, pre-commit, pre-push, or all. The pre-commit hook blocks only on errors; pre-push uses --strict so warnings block too.

Every file is listed before anything is written, existing files are skipped unless --overwrite is passed, and hooks are written with LF endings and marked executable (a CRLF hook will not run under sh).

devtool ci show github

prints a template to stdout without touching the disk.

---

Supported languages

The comment cleaner, the statistics collector and the TODO scanner all share one language registry:

Python, JavaScript/JSX, TypeScript/TSX, Java, C, C++, C#, Go, Rust, PHP, Ruby, Kotlin, Swift, Scala, Dart, Shell/Bash, SQL, HTML, XML, CSS, SCSS/LESS, Lua, YAML, TOML, INI/Properties, PowerShell, Dockerfile, R.

devtool languages prints the full table with extensions.

---

Installation

Requires Python 3.9 or newer.

git clone https://github.com/yourname/devtool
cd devtool
python -m venv .venv
.venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -e .

Runtime dependencies are typer and rich; everything else is standard library. For the tests:

pip install -e ".[dev]"

Without installing, the package also runs directly:

python -m devtool

---

Usage

Interactive menu:

devtool

  CODE TOOLS
  ------------------------------------------------
  [1] Clean comments
  [2] Code statistics
  [3] TODO / FIXME scanner
  [4] Secret scanner

  FILE TOOLS
  ------------------------------------------------
  [5] Find duplicate files
  [6] Find large files
  [7] File organizer

  TEXT TOOLS
  ------------------------------------------------
  [8] JSON formatter
  [9] Text statistics

  UTILITIES
  ------------------------------------------------
  [10] Hash generator
  [11] UUID generator

  [H] Help
  [0] Exit

devtool --help lists the subcommands, devtool help prints the full documentation, devtool --version prints the version.

---

CLI examples

# analysis
devtool score .                                      # one number + breakdown
devtool score . --min 80                             # gate on it
devtool complexity . --threshold 15
devtool deps . --all
devtool snapshot save . --label v1.2
devtool snapshot compare .
devtool ci init . --target github --dry-run

# repository health, before committing or pushing
devtool check .
devtool check . --output json
devtool check . --strict                             # warnings become exit 2
devtool check . --large-file-size 10MB
devtool git .
devtool conflicts .
devtool gitignore .
devtool gitignore . --fix

# comment cleaning -- one of --dry-run / --in-place / --output is required
devtool clean-comments ./project --dry-run
devtool clean-comments ./project --output ./project-clean
devtool clean-comments ./project --in-place --backup
devtool clean-comments ./main.py --in-place --keep-docstrings
devtool clean-comments ./query.txt --lang sql --in-place

# analysis
devtool stats ./project
devtool todos ./project
devtool todos ./project --tag TODO --tag FIXME --limit 20
devtool scan-secrets ./project
devtool scan-secrets ./project --fail-on-find        # exit 1 in CI

# files
devtool duplicates ./project --min-size 1KB
devtool large-files ./project --min-size 10MB --limit 10
devtool organize ./Downloads --dry-run
devtool organize ./Downloads --yes

# text and utilities
devtool json ./data.json
devtool json ./data.json --minify --output ./data.min.json
devtool text-stats ./notes.txt
devtool hash ./file.zip --algorithm all
devtool hash ./file.zip --check 9f86d081884c7d65...
devtool uuid --count 5 --plain

# any tree-walking command
devtool stats ./project --ignore fixtures --ignore generated --include-hidden

---

Architecture

src/devtool/
├── cli.py              Typer app: one subcommand per tool
├── menu.py             interactive menu (calls the same run_* functions)
├── console.py          the only Rich Console; theme, tables, progress, prompts
├── config.py           Config (where to look) + per-command option dataclasses
├── help_text.py        the [H] Help / `devtool help` screen
│
├── commands/           argument handling and rendering, one module per tool
├── scanners/           stats, todos, secrets, duplicates, large_files,
│                       conflicts, gitignore, health (the check aggregator),
│                       complexity, dependencies, score
├── generators/         CI pipelines and Git hooks (plan / apply split)
├── cleaners/           comments engine + the Python tokenize/ast stripper
├── languages/          language registry, comment scanner, hooks, markup
├── formatters/         json_tools
├── organizers/         file organiser (plan / apply split)
├── utils/              hashing, textstats, ids
└── core/               walker, fileio, sizes, git, snapshots

The rule that keeps this from turning into a pile of scripts:

scanners/, cleaners/, utils/ never print and never call sys.exit. They take a path and options, and return a dataclass. commands/ turns that dataclass into Rich output and an exit code. That is what makes every feature unit-testable without capturing stdout, and it is why the menu and the CLI can share code instead of duplicating it — both call the same run_* function.

The other shared piece is core/walker.py. Every tool that walks a tree uses it, so "skip node_modules" is implemented once and obeyed everywhere.

scanners/health.py is the clearest example of the split paying off: it owns no analysis at all. It calls the six existing scanners plus the Git and .gitignore inspectors, and turns their dataclasses into a flat list of findings with one severity each. Adding a check to devtool check means appending one CheckItem, not writing another scanner.

Talking to Git

core/git.py shells out to git rather than parsing .git by hand: the on-disk format changes between versions, git is installed wherever a repository exists, and porcelain output is a documented interface. Three habits keep it safe to run against a live working tree:

• --no-optional-locks, so inspecting a repository never writes to the index and never fights a running IDE for the lock file;
• -z everywhere, so paths with spaces, quotes or non-ASCII characters are never mangled by Git's own quoting;
• every call is wrapped — a missing binary, a timeout or a corrupt repository surfaces as GitInfo.error, never as a traceback.

Nothing is staged, committed, pushed or reset by any command.

How comment removal works

This is the part most likely to damage a file, so it does not use one regex per language — or one regex at all. languages/scanner.py is a state machine driven by a declarative LanguageSpec (string rules, line/block comment rules, and hooks for the awkward cases). At each position it asks, in order: protected region → hook → block comment → line comment → string literal → advance one character.

String literals are consumed atomically, so a comment marker inside one is never even considered:

print("hello # world")   →   print("hello # world")
x = 10  # comment        →   x = 10
url = "https://x/#test"  →   url = "https://x/#test"

The scanner returns ranges to delete, never rewritten text, so there is no code path that could rename a variable or reorder a statement.

Ambiguity always resolves to "leave it alone": an unterminated comment or string, a heredoc with no terminator, or an unrecognised file type means the file is not modified. Three language-specific refinements matter in practice: JavaScript regex literals (/https?:\/\// is not a comment), Rust lifetimes ('a is not a string), and shell parameter expansion (${v#p} is not a comment).

Python goes further and uses the standard library: tokenize for comments, ast for docstrings, and then both versions are re-parsed and their syntax trees compared. Any difference beyond removed docstrings aborts that file. This engine was validated by cleaning the entire Python standard library — 4,721 files, 216,665 comments, zero failures other than files that were already invalid Python 2.

Adding a language is one entry in languages/registry.py; detection, the menu, the help screen and the walker all read from that registry.

---

Testing

pytest

453 tests, no network, no fixtures on disk — everything uses tmp_path.

tests/conftest.py         throw-away Git repository fixture
tests/test_comments.py    comment removal, per language and per edge case
tests/test_languages.py   detection by extension, filename and shebang
tests/test_scanners.py    TODO detection, secret detection, line classification
tests/test_files.py       duplicates, large files, organizer, walker
tests/test_utils.py       JSON, hashing, text stats, size parsing, UUID
tests/test_git.py         repository detection, porcelain parsing, status
tests/test_repo_checks.py conflicts, .gitignore, health aggregation, metrics
tests/test_analysis.py    complexity, dependency parsing, score weighting
tests/test_snapshots_ci.py snapshot persistence and diffs, CI generation
tests/test_cli.py         every subcommand end to end, including exit codes

The Git tests build a real repository with git init and commit into it. They are skipped, not failed, when git is not installed, so the suite still runs anywhere. Porcelain parsing is additionally covered by pure unit tests that need no git at all.

The comment cleaner carries the edge cases that must never regress:

print("# not a comment")            # the string survives
url = "https://example.com/#test"   # the URL survives
x = 10  # comment                   # → x = 10
"""multiline docstring"""           # removed, unless --keep-docstrings

with equivalents for JavaScript regex literals, C++ raw strings, C# verbatim strings, Go raw strings, Rust nested comments and lifetimes, shell $# and heredocs, Ruby =begin/%w[], SQL '' escaping, HTML attributes containing <!--, and mixed HTML/PHP files.

---

Configuration

These directories are skipped by every scanning command:

.git  .hg  .svn  .idea  .vscode  .vs  node_modules  bower_components  vendor
__pycache__  .mypy_cache  .pytest_cache  .ruff_cache  .tox  venv  .venv  env
.env  virtualenv  dist  build  out  target  .next  .nuxt  .gradle  bin  obj
coverage  .terraform  .eggs

Override per run with --ignore NAME (repeatable), or permanently with a .devtool.toml file or a [tool.devtool] section in pyproject.toml, found by searching upwards from the target path:

[tool.devtool]
ignore_dirs       = ["node_modules", "dist"]
extra_ignore_dirs = ["fixtures"]
include_hidden    = false
follow_symlinks   = false
max_bytes         = 5242880

Hidden files are skipped everywhere except scan-secrets, which includes them deliberately — .env is exactly where credentials hide.

Comments that a compiler, runtime or tool actually reads are preserved by clean-comments unless you pass --no-preserve: shebangs, # -*- coding:, # frozen_string_literal, //go:build, MySQL's executable /*! */, SQL optimizer hints, linter and type-checker pragmas, and HTML conditional comments. Removing any of those would change how the program builds or runs.

---

Limitations

Worth knowing before you point this at something important.

• scan-secrets is a heuristic, not a security product. It recognises common shapes and filters obvious placeholders. It will miss anything deliberately hidden and will occasionally flag something harmless. A clean run means "nothing obvious", never "proven safe". Use a dedicated scanner and pre-commit hooks for real coverage.
• JSX text outside strings. <p>see https://x</p> is handled (a // right after : does not start a comment), but a bare // in JSX text in another context could still be misread.
• Ruby has the most ambiguous syntax on the list. The constructs that occur in practice are handled; exotic % delimiters and heredocs written without a space after << may be skipped — skipped, not corrupted.
• PHP short tags. <?php and <?= are recognised; bare <? is not.
• Python docstrings are technically code (__doc__), not comments. Removing them is the default because that is usually what is wanted, but it is a deliberate choice — --keep-docstrings reverses it.
• Indentation- and position-sensitive languages (Makefile, Perl, Haskell) are intentionally unsupported. No support beats a corrupted file.
• organize is not transactional. If it fails halfway, the files already moved stay moved. It reports exactly which ones failed.
• The Git tools need git on PATH. They shell out to it. Without it, devtool git reports the problem and check skips the Git section instead of failing.
• gitignore --fix appends, it does not tidy. It will not deduplicate, reorder or remove stale entries, and it cannot untrack a file that is already committed.
• check walks the tree several times, once per scanner. On a very large repository the individual commands are faster than the aggregate.
• The score weights are opinions. They are defensible defaults, not measurements. Read scanners/score.py before treating the number as truth, and change the constants if your project weighs things differently.
• Complexity outside Python is an estimate, computed at file level from branch keywords. It is marked ~ in the output and should be read as a rough signal, not a metric.
• deps is offline. It never contacts a registry, so it cannot tell you about vulnerabilities or newer versions. The "declared but never imported" list is heuristic — a package used only through an entry point or plugin will show up there.
• Snapshots compare only the two most recent. Longer trends are on the roadmap.
• Test fixtures look like findings. A repository containing deliberate sample credentials or conflict markers in its tests will report them, and there is no allow-list yet. Point check at ./src, or pass --ignore tests, until the planned baseline file lands. (DevTool's own test suite is exactly this case.)

Always run --dry-run first, and prefer --output or --backup over --in-place on anything not committed to version control.

---

Roadmap

Done in 1.1: repository health check, Git inspector, conflict scanner, .gitignore checker, JSON output and exit codes.

Done in 1.2: health score, complexity analysis, dependency analyzer, project snapshots, CI/CD generation.

Next:

• baseline / allow-list file so known-safe secret matches stop being reported
• --json output on the remaining scanners (stats, todos, scan-secrets)
• git-aware mode: scan only tracked or only changed files
• trend charts across many snapshots, not just the last two
• per-function complexity outside Python, where a parser is available
• a proper entropy model for scan-secrets instead of the current threshold
• more languages: Elixir, Haskell, Zig, Nim

---

License

MIT. See LICENSE.