Add support for analyzing npm packages directly

[eastlondoner]

Add support for analyzing npm packages directly

Feature Request

Add the ability to analyze npm packages directly using their package name similar to the existing GitHub repository support.

Motivation

Many developers use npm and know packages by name rather than the corresponding github repo. This would streamline the workflow for:

• Quick analysis of dependencies
• Evaluating potential packages before installation
• Understanding package internals for debugging or learning purposes

Proposed Implementation

1. Primary Method: Extract GitHub repository information from package.json

   repomix --npm-package express

   • Parse the package's metadata to find the GitHub repository URL
   • Use existing GitHub repository analysis functionality

2. Fallback Method: Direct package analysis from npm tarball

   repomix --npm-package express --use-tarball

   • Download and extract the package tarball
   • Analyze contents directly when GitHub repository isn't available

Technical Details

1. Use the npm registry API to fetch package metadata:

   https://registry.npmjs.org/[package-name]
   

2. Extract repository information from:

   • repository field in package.json
   • homepage field (if it's a GitHub URL)
   • bugs field (if it points to GitHub issues)

3. For tarball fallback:

   • Use the dist.tarball URL from the package metadata
   • Extract and analyze the contents locally

Example Usage

# Using package name
repomix --npm-package express

# Specific version
repomix --npm-package express@4.18.2

# Force tarball analysis
repomix --npm-package express --use-tarball

# With other existing options
repomix --npm-package express --compress --style markdown

Note: scoped packages like @org/package-name also exist and should work just fine.

Questions

1. What would the website UI be for using npm package names & the tarball option?

[yamadashy]

Hi, @eastlondoner !
Thank you for your proposal! (And thanks for moving this from DMs)

The parameter name --npm-package seems to indicate we're targeting the npm package itself, so intuitively it might make more sense to retrieve the tarball from npm by default. I'd like to think about this approach a bit more.

Also, what do you think about --remote-npm as the parameter name instead?
And if we make tarball the default method, perhaps we could use something like this when source is preferred:

repomix --remote-npm express --use-source

I'll consider the website UI implementation later.

[eastlondoner]

You're welcome, it's an honour to contribute.

I have no strong opinion about names. But I think defaulting to the source makes sense for a couple of reasons.

1. the tarball may contain only thefiles array from the package.json, often this is bundled and minified and not a lot of use to LLMs
2. repomix in general is expected to work on GitHub repos.

We could do

# this uses source and falls back to tarball automatically?
repomix --remote-npm express

# this also uses source
repomix --remote-npm express --use-source

# this uses tarball
repomix --remote-npm express --use-tarball

Having said all that, I am not that fussed about tarball so we could say that we only use GitHub source and if the npm repo doesn't have a GitHub source then we just log an error.

[yamadashy]

@eastlondoner
Apologies for the late follow-up — I just realized it's already been a month since this was opened!

Would you still be up for taking this on?

I've assigned it to you for now, but if your schedule’s tight, I’m totally happy to handle it myself. Just let me know!