setuppy Dependency Module Installer

setup.py: Dependency Module Installer

The setup.py script serves as the modular dependency management engine for the Claude Sleuth toolkit. It allows for granular installation of Python libraries based on the specific requirements of an investigation phase, minimizing environment bloat while ensuring all necessary tools for tasks like geolocation, corporate research, or network analysis are available.

Dependency Architecture

The toolkit categorizes dependencies into 13 distinct modules. These modules are defined in a central MODULES dictionary within the script, which maps module identifiers to their human-readable descriptions and specific Python package lists. This structure mirrors the [project.optional-dependencies] defined in the project's configuration file.

Module Definitions

Module	Description	Key Packages
core	Essential HTTP and data utilities	requests, pandas, rich, beautifulsoup4
identity	Username and identity research	sherlock-project, maigret
social	Media archiving and capture	yt-dlp, playwright, instaloader
network	DNS and infrastructure intel	dnspython, ipwhois, whois
corporate	Registry and financial analysis	edgartools
sanctions	Screening and fuzzy matching	rapidfuzz, jellyfish, nameparser
geo	Mapping and spatial analysis	geopandas, folium, exifread, pysolar
nlp	Natural language processing	spacy, nltk, scikit-learn
graph	Network graph visualization	networkx, pyvis, plotly
archiving	Evidence preservation	waybackpy, warcio, trafilatura
documents	PDF/OCR processing	pdfplumber, pytesseract, markitdown
reporting	Document generation	jinja2, weasyprint, docxtpl
entity_resolution	Record linkage	recordlinkage

Implementation Detail

The installer utilizes subprocess to interface directly with the system's Python interpreter to ensure packages are installed in the correct environment.

Key Functions

• install(package: str): Executes the command pip install <package> --break-system-packages -q. It captures the return code, execution time, and any error messages.
• get_all_packages(module_names: list): A helper function that iterates through requested modules, flattens the package lists, and removes duplicates using a seen set to ensure idempotent installation.
• main(): Handles CLI argument parsing via argparse. It validates module names against the MODULES keys and coordinates the installation loop.

Logical Flow: CLI to Execution

The following diagram illustrates how the main entry point processes user input and interacts with the underlying install logic.

Logic Flow of setup.py

graph TD
    "START[main()]" --> "PARSE[argparse.ArgumentParser]"
    "PARSE" --> "CHECK_LIST{args.list?}"
    "CHECK_LIST" -- "Yes" --> "PRINT_LIST[Print MODULES descriptions]"
    "CHECK_LIST" -- "No" --> "CHECK_MODS{args.modules?}"
    "CHECK_MODS" -- "Yes" --> "VAL[Validate module names]"
    "CHECK_MODS" -- "No" --> "ALL[Select all MODULES]"
    "VAL" --> "GET_PKGS[get_all_packages()]"
    "ALL" --> "GET_PKGS"
    "GET_PKGS" --> "DRY{args.dry_run?}"
    "DRY" -- "Yes" --> "PRINT_DRY[Print packages & Exit]"
    "DRY" -- "No" --> "LOOP[For pkg in packages]"
    "LOOP" --> "CALL_INST[install(pkg)]"
    "CALL_INST" --> "SUB[subprocess.run pip install]"
    "SUB" --> "RES[Log status/time/notes]"
    "RES" --> "LOOP"
    "LOOP" -- "Done" --> "EXIT[Print Summary & Exit]"

Loading

Command Line Interface

The script is registered as a CLI entry point sleuth-setup via pyproject.toml pyproject.toml:128.

Flags and Usage

• --modules <names>: A comma-separated list (e.g., core,geo,network) to install specific subsets.
• --list: Displays all available modules, their descriptions, and the number of packages contained in each.
• --dry-run: Lists the packages that would be installed without executing the pip command.

Natural Language to Code Entity Mapping

This diagram maps the high-level investigation requirements to the specific code-level package groups and the CLI tool.

Requirement to Code Entity Mapping

graph LR
    subgraph "Natural Language Space"
        "R1[I need to map locations]"
        "R2[I need to find social media]"
        "R3[I need to analyze networks]"
    end

    subgraph "Code Entity Space (setup.py)"
        "CLI[sleuth-setup / scripts.setup:main]"
        "M_GEO[MODULES['geo']]"
        "M_ID[MODULES['identity']]"
        "M_GRAPH[MODULES['graph']]"
    end

    "R1" --> "CLI"
    "R2" --> "CLI"
    "R3" --> "CLI"
    "CLI" -- "--modules geo" --> "M_GEO"
    "CLI" -- "--modules identity" --> "M_ID"
    "CLI" -- "--modules graph" --> "M_GRAPH"

    "M_GEO" --> "P1[geopandas]"
    "M_GEO" --> "P2[folium]"
    "M_ID" --> "P3[sherlock-project]"
    "M_GRAPH" --> "P4[networkx]"

Loading

Integration with Task Runner

While setup.py can be run manually, it is designed to be invoked programmatically by the task_runner.py (accessible via sleuth-task). When an investigator moves to a new phase or task, the task runner can trigger setup.py to ensure the environment is prepared for the specific analytical scripts required by that phase.

---