Skip to content

Releases: SariYanouche/PolyglotMiner

Release list

PolyglotMiner v1.0.1

PolyglotMiner v1.0.1 Latest
Latest

Choose a tag to compare

View all tags
@SariYanouche SariYanouche released this 08 Sep 23:50
v1.0.1
fe6e391

PyPI
License: MIT
Python Version

PolyglotMiner v1.0.1 is now officially published on PyPI! This release brings major performance optimizations, directed migration harvesting, false-positive elimination heuristics, an interactive Web Analytics Studio, and automated CLI tools.

📦 Quick Installation

You can now install PolyglotMiner directly from PyPI on any machine:

pip install polyglotminer

Verify your installation:

polyglotminer doctor
polyglotminer --version

🚀 What's New in v1.0.1

1. Official PyPI Distribution

  • Published to the Python Package Index (polyglotminer).
  • Both polyglotminer and its short alias pgminer are automatically added to your system PATH.

2. Directed Migration Pair Mining (-p, --pairs)

  • Harvest and analyze only repositories exhibiting specific cross-language transitions: 
    polyglotminer campaign -p "C->C++,JS->TS" --workers 4 --limit 50 --open
  • Integrated automated language normalization and targeted discovery filtering.

3. High-Throughput Multiprocess Campaign Engine

  • Worker Process Recycling (maxtasksperchild=25): Guarantees 100% OS-level memory reclamation after analyzing large repositories.
  • Watchdog Protection: 75-second timeout per repository prevents pathological repositories from stalling the batch queue.
  • Persistent Resume State: SQLite WAL tracking ensures failed or timed-out repositories are recorded and skipped automatically on subsequent runs.
  • Mining throughputs exceeding 70+ repositories per minute in blobless Empirical Module mode.

4. Rigorous Heuristics & False-Positive Elimination

  • Header Pairing Exclusion: Strictly eliminates false C/C++/Objective-C pairings (.h vs .c/.cpp/.m).
  • Rename/Move Filter: Discards identical file renames and pure directory relocations.
  • DevOps Script Exclusion: Excludes shell scripts (.sh, .bat), Dockerfiles, and CI workflow configurations from primary migration candidate detection.

5. Interactive Web Analytics Studio

  • Launch with polyglotminer ui --port 8080.
  • Graph Navigator & Spotlight Views: Longitudinal volume dynamics, 100% dominance curves, and state-space turnover trajectories.
  • Academic Export Suite: 1-click generation of LaTeX booktabs tables, high-resolution figures (300 DPI PNG), and raw CSV datasets.

6. Developer & CLI Ergonomics

  • Interactive Terminal Wizard (TUI): Launch polyglotminer without arguments to access an interactive guided workflow.
  • System Doctor (polyglotminer doctor): Built-in diagnostics for Python version, Git CLI, authentication tokens, rate-limits, and SQLite database connectivity.
  • Zero-Config Shell Completion (polyglotminer completion --install): Instant tab autocompletion support for PowerShell, Bash, Zsh, and Fish.

🧪 Verification & Test Suite

  • 100% automated test suite passing (74 unit tests covering CLI smoke tests, multi-worker campaign pipelines, heuristic edge-cases, SQLite persistence, and token fallbacks): 
    python -m unittest discover tests

📖 Documentation & Links

Assets 2
Loading

PolyglotMiner v1.0.0

PolyglotMiner v1.0.0

Choose a tag to compare

View all tags
@SariYanouche SariYanouche released this 01 Sep 02:23
v1.0.0
3a1e324

PolyglotMiner v1.0.0

This is the initial release of PolyglotMiner, an empirical software engineering tool built to analyze codebase evolution, multi-language composition, and software migrations across Git repositories.

What is in this release

  • Codebase Profiling: Fast line counting across 150+ languages, separating pure source code, comments, and blank lines.
  • Longitudinal Evolution: Samples equidistant historical milestones (from 0% to 100%) to calculate growth curves, dominance shifts, and language crossover points.
  • Dual Analysis Modes: Fast empirical module mode using blobless Git clones, and physical lines of code mode (--full-loc) for exact line counting.
  • Migration Mining: Streams commit logs to extract co-changing files and detect 1-to-1 language replacements (such as Java to Kotlin or JavaScript to TypeScript).
  • Empirical Heuristics: Classifies codebase trajectories using statistical correlation, code contraction metrics, and file-level churn.
  • Authoritative SQLite Storage: Built-in SQLite database using WAL mode for zero-memory, fast query performance.
  • Local Web Studio: Interactive browser interface for trajectory visualization, search, and publication exports (LaTeX tables, 300 DPI PNG figures, and CSV).

Installation

git clone https://github.com/SariYanouche/PolyglotMiner.git
cd PolyglotMiner

# Create and activate a virtual environment
python -m venv .venv

# On Linux / macOS:
source .venv/bin/activate
# On Windows (PowerShell):
.\.venv\Scripts\Activate.ps1

# Install in editable mode
pip install -e .

Quick Start

# Profile a local directory or remote repository
polyglotminer scan .

# Analyze evolution across 10 milestone intervals
polyglotminer history carlphilipp/chicago-commutes --samples 10 --open

# Run automated discovery and batch evolution mining across 100 repositories
polyglotminer campaign -n 100 --workers 2

# Launch the interactive web studio
polyglotminer ui --port 8080

Full documentation, heuristic details, and CLI options are available in the repository README.

Loading