Skip to content

kactlabs/mod2pip

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

422 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

mod2pip - Generate requirements.txt file for any project based on imports

Tests PyPI version codecov License Python 3.9+

A superior alternative to pipreqs - Actively maintained with comprehensive features for modern Python development.

About This Project

mod2pip is an enhanced requirements.txt generator that addresses all major limitations of pipreqs:

  • ✅ Detects dynamic imports (__import__, importlib)
  • ✅ Supports conda environments
  • ✅ Handles Python 3.9-3.13 (including latest 3.13)
  • ✅ Graceful syntax error handling (works with legacy Python 2 code)
  • ✅ 1,785 standard library modules filtered
  • ✅ 1,156 import-to-package mappings
  • ✅ Environment file generation and validation
  • ✅ Active maintenance with regular updates

Installation

pip install mod2pip

Note: If you don't want support for jupyter notebooks, you can install mod2pip without the dependencies that give support to it:

pip install --no-deps mod2pip
pip install yarg==0.1.9 docopt==0.6.2

Testing

To run the comprehensive test suite:

# Install in development mode first
pip install -e .

# Run tests
python test_mod2pip_features.py

Usage

Usage:
    mod2pip [options] [<path>]

Arguments:
    <path>                The path to the directory containing the application files for which a requirements file
                          should be generated (defaults to the current working directory)

Options:
    --use-local           Use ONLY local package info instead of querying PyPI
    --pypi-server <url>   Use custom PyPi server
    --proxy <url>         Use Proxy, parameter will be passed to requests library. You can also just set the
                          environments parameter in your terminal:
                          $ export HTTP_PROXY="http://10.10.1.10:3128"
                          $ export HTTPS_PROXY="https://10.10.1.10:1080"
    --debug               Print debug information
    --ignore <dirs>...    Ignore extra directories, each separated by a comma
    --no-follow-links     Do not follow symbolic links in the project
    --encoding <charset>  Use encoding parameter for file open
    --savepath <file>     Save the list of requirements in the given file
    --print               Output the list of requirements in the standard output
    --force               Overwrite existing requirements.txt
    --append              Append to existing requirements.txt (use with --lib)
    --diff <file>         Compare modules in requirements.txt to project imports
    --clean <file>        Clean up requirements.txt by removing modules that are not imported in project
    --mode <scheme>       Enables dynamic versioning with <compat>, <gt> or <non-pin> schemes
                          <compat> | e.g. Flask~=1.1.2
                          <gt>     | e.g. Flask>=1.1.2
                          <no-pin> | e.g. Flask
    --scan-notebooks      Look for imports in jupyter notebook files.
    --lib <packages>...   Add specific libraries with their installed versions (comma-separated)
    --generate-env        Scan Python files for environment variables and generate .env and .env.sample files
    --validate-env        Validate .env file values against known patterns (API keys, tokens, URLs, etc.)

Examples

Scan Project for Imports

$ mod2pip /home/project/location
Successfully saved requirements file in /home/project/location/requirements.txt

Contents of requirements.txt:

wheel==0.23.0
Yarg==0.1.9
docopt==0.6.2

Add Specific Libraries (New in v0.9.0)

Add specific libraries with their installed versions without scanning the project:

# Overwrite requirements.txt with specific libraries
$ mod2pip --force --lib langchain,langchain-core,numpy
Successfully saved requirements file in requirements.txt
# Append libraries to existing requirements.txt (skips duplicates)
$ mod2pip --append --lib requests,flask,django
Skipped 1 package(s) already in requirements.txt
Successfully appended to requirements file requirements.txt
# Preview libraries without writing to file
$ mod2pip --print --lib pandas,numpy,scipy
pandas==2.0.3
numpy==1.24.3
scipy==1.11.1
# Use different version schemes
$ mod2pip --force --lib flask,django --mode compat
flask~=2.0.1
django~=4.2.0

$ mod2pip --force --lib flask,django --mode gt
flask>=2.0.1
django>=4.2.0

$ mod2pip --force --lib flask,django --mode no-pin
flask
django

Generate Environment Variable Files (New in v0.10.0)

Automatically scan your Python project for environment variables and generate .env and .env.sample files:

$ mod2pip --generate-env --force
INFO: Scanning for environment variables in /home/project
INFO: Found 15 environment variables
INFO:   API_KEY: found in 2 location(s)
INFO:   DATABASE_URL: found in 3 location(s)
INFO:   DEBUG: found in 1 location(s)
INFO: Successfully created /home/project/.env
INFO: Successfully created /home/project/.env.sample
INFO: Environment files generated successfully!

Smart Merging: If .env or .env.sample files already exist, mod2pip will intelligently merge new variables with existing ones:

  • Preserves existing variable values
  • Adds newly discovered variables
  • Avoids duplicates
  • Keeps existing variables even if not detected in code
  • Supports both .env.sample and .env.example formats
$ mod2pip --generate-env --force
INFO: Successfully merged 5 new variable(s) with 3 existing in /home/project/.env
INFO: Successfully merged 5 new variable(s) with 3 existing in /home/project/.env.sample

Example generated .env file:

# Environment Variables
# Generated by mod2pip --generate-env
# Merged with 3 existing variable(s), added 5 new variable(s)

# API authentication key
# Used in: config.py:12, app.py:5
API_KEY=

# Database connection URL
# Used in: config.py:8, models.py:3, database.py:10
DATABASE_URL=postgresql://localhost/mydb

# Debug mode flag (True/False)
# Used in: config.py:15
DEBUG=False

# Secret key for encryption/signing
# Used in: config.py:20
SECRET_KEY=dev-secret-key

For more examples, see the demo_project.py file.

Validate Environment Variable Values (New in v0.11.0)

Validate your .env file to ensure API keys, tokens, and other sensitive values match expected patterns:

$ mod2pip --validate-env
INFO: Validating /home/project/.env...
✓ All 15 environment variable(s) validated successfully!

If validation issues are found:

$ mod2pip --validate-env
INFO: Validating /home/project/.env...
ERROR: ✗ Found 2 validation issue(s):

  Variable: SLACK_BOT_TOKEN
  Current value: abc123...
  Expected: Slack Bot Token (starts with xoxb-)
  Example: xoxb-YOUR-BOT-TOKEN-HERE

  Variable: OPENAI_API_KEY
  Current value: test-key...
  Expected: OpenAI API Key (starts with sk-)
  Example: sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Supported Patterns:

  • AI/ML Services: OpenAI, Gemini, DeepSeek, Anthropic, Groq, Cohere, Hugging Face, Replicate, Stability AI, ElevenLabs, AssemblyAI, LangChain
  • Vector Databases: Pinecone, Weaviate
  • Communication: Slack, Discord, Telegram, Zoom
  • Cloud Platforms: AWS, Azure, Google Cloud Platform
  • Hosting/Deployment: Netlify, Vercel, Heroku, Railway, Render, Fly.io, DigitalOcean, Cloudflare
  • Databases: MongoDB, PostgreSQL, Redis, Supabase, PlanetScale, CockroachDB, Neon, Upstash, MongoDB Atlas
  • CMS/Content: Contentful, Sanity, Airtable, Notion
  • Version Control: GitHub, GitLab, Bitbucket
  • Payment Processing: Stripe, PayPal, Square
  • Email Services: SendGrid, Mailgun, Twilio
  • CRM/Support: HubSpot, Intercom, Zendesk, Freshdesk, Salesforce
  • Analytics: Amplitude, Mixpanel, Segment, LogRocket, Datadog
  • Error Tracking: Sentry, Bugsnag, Rollbar, New Relic
  • Search/Logging: Algolia, Elasticsearch
  • Maps: Mapbox, Google Maps
  • Real-time: Pusher
  • Project Management: Linear, Asana, Jira
  • Other: Firebase, JWT secrets, and more

Total: 100+ validation patterns covering the most popular services and APIs.

Patterns are defined in mod2pip/env_patterns.json and can be customized.

Why mod2pip over pipreqs?

mod2pip addresses common limitations found in pipreqs and is actively maintained with regular updates.

1. Dynamic Import Detection

Unlike pipreqs which only uses static analysis, mod2pip detects:

  • __import__() calls
  • importlib.import_module() usage
  • Imports inside functions (late imports)
  • Conditional imports in try/except blocks
  • Imports in exec/eval statements
# These are detected by mod2pip but missed by pipreqs
import importlib
pandas = importlib.import_module('pandas')

def process_data():
    import numpy as np  # Late import
    return np.array([1, 2, 3])

2. Conda Environment Support

mod2pip automatically detects packages installed via conda:

  • Reads conda-meta directory for package information
  • Maps conda package names to Python import names
  • Works seamlessly in mixed pip/conda environments

3. Transitive Dependencies (Optional)

Use --include-transitive to include indirect dependencies:

mod2pip --include-transitive --transitive-depth 2

4. Enhanced Local Package Detection

Supports:

  • Editable packages (pip install -e .)
  • Namespace packages
  • Custom installation layouts
  • Non-standard package structures

5. Additional Features

  • Selective library addition: --lib flag to add specific packages
  • Environment file generation: --generate-env to create .env files
  • Environment validation: --validate-env to check API keys and tokens
  • Smart merging: Preserves existing requirements when appending

6. Correct Package Names

mod2pip ensures accurate package naming:

  • Uses official PyPI package names (not import names)
  • Maintains correct capitalization (e.g., Flask not flask)
  • Comprehensive mapping file for special cases (e.g., PILPillow, cv2opencv-python)
  • Deduplicates packages to prevent version conflicts

7. Robust File Handling

  • Automatic directory creation: Creates parent directories if they don't exist
  • Symbolic link support: --no-follow-links flag to control symlink behavior
  • Nested project structures: Handles deeply nested folders and unusual naming
  • Graceful error handling: Continues processing even if individual files fail

8. Superior Module-to-Package Resolution

mod2pip has the most comprehensive mapping system:

  • 1,785 standard library modules: Filters out stdlib to prevent false positives (e.g., json, os, sys)
  • 1,156 import-to-package mappings: Handles special cases correctly:
    • PILPillow
    • cv2opencv-python
    • sklearnscikit-learn
    • MySQLdbMySQL-python
    • And 1,152+ more mappings
  • Local-first resolution: Checks installed packages before querying PyPI
  • Clear warnings: Alerts when remote resolution is used, prompting manual verification

9. Robust Syntax and Compatibility

  • Graceful syntax error handling: Falls back to regex parsing when AST parsing fails
  • Legacy Python 2 support: Can parse old codebases without crashing
  • Python 3.9-3.13 support: Fully tested across all modern Python versions
  • Continues on errors: Processes remaining files even if some fail

10. Active Maintenance

Unlike pipreqs which has many unresolved issues, mod2pip is actively maintained with:

  • Regular bug fixes and feature updates
  • Comprehensive test suite with CI/CD
  • Support for latest Python versions (3.9-3.13)
  • Responsive issue resolution
  • Modern development practices (flake8, type hints, documentation)

Comparison: mod2pip vs pipreqs vs pip freeze

Feature mod2pip pipreqs pip freeze
Dynamic imports ✅ Yes ❌ No ❌ No
Conda support ✅ Yes ❌ No ⚠️ Partial
Transitive dependencies ✅ Optional ❌ No ✅ Yes
Standard library filtering ✅ 1,785 modules ✅ Basic ❌ No
Package name mapping ✅ 1,156 mappings ⚠️ Limited ✅ Correct
Syntax error handling ✅ Graceful fallback ❌ Crashes N/A
Python 3.13 support ✅ Yes ❌ No ✅ Yes
Symbolic link control ✅ Yes ❌ No N/A
Auto directory creation ✅ Yes ❌ No N/A
Environment file generation ✅ Yes ❌ No ❌ No
Environment validation ✅ Yes (100+ patterns) ❌ No ❌ No
Selective library addition ✅ Yes (--lib) ❌ No ❌ No
Smart merging/appending ✅ Yes ❌ No ❌ No
Active maintenance ✅ Yes ⚠️ Limited ✅ Yes
Only project dependencies ✅ Yes ✅ Yes ❌ No

Why not pip freeze?

  • pip freeze only saves the packages that are installed with pip install in your environment.
  • pip freeze saves all packages in the environment including those that you don't use in your current project (if you don't have virtualenv).
  • Sometimes you just need to create requirements.txt for a new project without installing modules.
  • With --lib flag, you can selectively add specific packages with their versions.

Citing mod2pip

If you find mod2pip useful in your research and wish to cite it, please use the following BibTex entry:

@software{mod2pip2025,
   author = {Raja CSP Raman},
   title = {mod2pip: Generate requirements.txt file for any project based on imports},
   url = {https://github.com/kactlabs/mod2pip/},
   version = {0.11.0},
   year = {2025},
}

About

No description, website, or topics provided.

Resources

License

Contributing

Stars

Watchers

Forks

Packages

 
 
 

Contributors