Skip to content

eilam-ashbell/npm-digger

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

34 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

npm-digger

API wrapper and data extractor on NPM packages without any requirements.

Table of content

Installation

npm i npm-digger

Features

  • No dependencies
  • built-in types system
  • Built-in management of caching data so it won't make unnecessary requests every time you ask for data in a same class.
  • Parsing data that not expose on the official npm API documentations

Usage

npm-digger expose all npm's API data and some more extras from npm's website and internal methods.

The main classes you can use are:

  • Registry - basic information on the npm registry and DB
  • Package - information on a specific package
  • Version - information on a specific version of a package
  • Downloads - statistics data on downloads of a specific package
  • User - information on npm's users
  • Search - results of searching on npm's website
  • Provenance - new feature introduced on Apr 23 and gives you a way to link package to its source repo and specific build instructions used to publish it.

Registry

By init the Registry class, you get access to all registry data npm's API is expose in its documentations.

import { Registry } from 'npm-digger'

const registry = new Registry()

const dbName = await registry.dbName()

console.log(dbName)

// console: "registry"

Functionality

Function Description Type
data() JSON response as returns from the npm API RegistryModel
dbName() The DB name of the registry string
engine() The engine name of the registry DB string
docCount() The number of docs in the db number
deletedDocCount() The number of deleted docs from the db number
updateSeq() An opaque string that describes the state of the database number
purgeSeq() An opaque string that describes the purge state of the database number
compactRunning() 'True' if the database compaction routine is operating on this database boolean
diskSize() The size of disk, in bytes number
dataSize() The size of data, in bytes number
sizes.active() The size of live data inside the database, in bytes number
sizes.external() The uncompressed size of database contents in bytes number
sizes.file() The size of the database file on disk in bytes number
diskFormatVersion() The version of the physical format used for the data when it is stored on disk number
committedUpdateSeq() number
compactedSeq() number
uuid() string

Package

By init the Package class, you get access to all data npm's API is expose in its documentations in the package route https://registry.npmjs.org/:package. Also, there is some more information that comes from package page on npm website.

This class includes also the Downloads class (can be imported separately too)

import { Package } from 'npm-digger'

const package = new Package('express')

const description = await package.description()

console.log(description)

// console: "Fast, unopinionated, minimalist web framework"

Params

Name Description Type Required
packageName The name of the package you want to get data on. If package is scoped, should be included too string yes

Functionality

Function Description Type
name() Package name (with scope if exist) string
maintainers() npm username list of maintainers NpmPersonModel[]
distTags() object includes main tags of the package {key: name, value: tag}
lastPublish.maintainer() npm username of the maintainer who publish the last release string
lastPublish.time() date and time of last publish string
types.isBundled() 'True' if types are bundled in the package boolean
types.isExternal() 'true if types are in external package boolean
types.entries() object of entries for locate types for the package {key: name, value: entry}
dependents.count() total number of dependents packages number
dependents.listAll() list of all dependents packages with main details array
githubApiRoute() main route for github API of the package repo string
npmUrl() URL for package page on npm string
latestVersion.data() main data objects of the latest version of the package JSON
latestVersion.author() author data of the latest version of the package
latestVersion.description() description of the latest version of the package string
latestVersion.homepage() homepage of the latest version of the package string
latestVersion.repository() link to the repo of the latest version of the package string
latestVersion.keywords() keywords of the latest version of the package string[]
latestVersion.maintainers() maintainers of the latest version of the package
latestVersion.license() license of the latest version of the package string
latestVersion.version() version of the latest version of the package string
latestVersion.dependencies() dependencies of the latest version of the package {key: package, value: version}
latestVersion.devDependencies() dev dependencies of the latest version of the package {key: package, value: version}
isPrivate() 'true' if package is private boolean
isSecurityPlaceholder() 'true' if security placeholder was published for this package boolean
readme() readme file content string
author() package author data string
description() package description string
homepage() homepage of the package string
repository() package repository url string
keywords() package keywords string[]
license() license of the package string
currentVersion() current version of the package string
versions.data() JSON object with versions data {key: version, value: data about the version}
versions.deprecations() list of deprecated versions string[]
versions.releaseDates() JSON object with all releases times {key: release tag, value: date-time string}
id() package name, used as an ID in CouchDB string
rev() revision number of this version of the document in CouchDB string
createdTime() time string of package creation string
lastModified() time string of package last modify time string
stars.count() number of users who star this package number
stars.usersList() list of users who star this package string[]
contributors() list of contributors of this package array
bugsReport() object with available URLs for bug report {key: name, value: link}
downloads() downloads class Downloads

Version

By init the Version class, you get access to all version data npm's API is expose in its documentations. If no version number provided, the latest version will be used.

This class includes also the Provenance class (can be imported separately too)

import { Version } from 'npm-digger'

const version = new Version('express', '4.18.2')

const name = await version.name()

console.log(name)

// console: "express@4.18.2"

Params

Name Description Type Required
packageName package name string yes
packageVersion requested version number string no

Functionality

Function Description Type
data() JSON object consist all data of specified version or latest version of a package JSON
name() The name of package string
version() The version of the package string
description() The description of the package string
author() author information PersonModel
contributors() List of contributors of the package array
keywords() List of keywords of the package string[]
engines() the node engines required for this version to run, if specified {key: engine, value: version}
scripts() all scripts defined on the package jason file of the package {key: name, value: script}
id() Id of the package on npm (A.K.A packageName@PackageVersion) string
nodeVersion() the version of node used to publish this version string
npmVersion() the version of the npm client used to publish this string
dist() Dist object with information about this version distribution DistModel
tarball() url of tarball file string
directories() directories included by this version
main() the package's entry point (e.g., index.js or main.js) string
license() the SPDX identifier of the package's license string
repository() url for repository as given in package.json
bugs() url for bugs reporting string
homepage() url of the homepage of the package string
bin() a mapping of bin commands to set up for this version
dependencies() a mapping of other packages this version depends on to the required semver ranges
peerDependencies() a mapping of package names to the required semver ranges of peer dependencies
devDependencies() a mapping of package names to the required semver ranges of development dependencies
optionalDependencies() an object mapping package names to the required semver ranges of optional dependencies
resolutions()
gitHead() git head signature of the version string
maintainers() list of the npm usernames of the people with permission to write to that package
npmUser() the account that did the publishing for that version
hasShrinkwrap() true if this version is known to have a shrinkwrap that must be used to install it; false if this version is known not boolean

Downloads

By init the Downloads class, you get access to analytics data about the package downloads.

import { Downloads } from 'npm-digger'

const downloads = new Downloads('express')

const pointDownload = await downloads.pointDownload('last-week')

console.log(pointDownload)

// console: 
//      {
//          "downloads": 26951829,
//          "start": "2023-08-10",
//          "end": "2023-08-16",
//          "package": "express"
//      }

Params

Name Description Type Required
packageName package name string yes

Functionality

Function Description Type params
weeklyDownloads() List of the last 52 weeks from today with their download count JSON
pointDownload() JSON object consist the total downloads count to a specific date or period JSON period - one of Period type options or date string of yyy-mm-dd:yyy-mm-dd format
rangeDownload() JSON object consist list of downloads count for each day in a range of time JSON period - one of Period type options or date string of yyy-mm-dd:yyy-mm-dd format
perVersion() JSON object consist all downloads count of the last week for each version {key: version, value: downloads count}

User

By init the User class, you get access to all available data exist on npm user page

import { User } from 'npm-digger'

const user = new User('username')

const accountType = await user.accountType()

console.log(accountType)

// console: "user"

Params

Name Description Type Required
username name of required user data string yes

Functionality

Function Description Type
rawData() JSON object consist all data from npm user page JSON
accountType() The user type definition by npm string
name() The user name on npm string
description() The user description paragraph on npm string
avatars() An object with links all the existing avatar images sizes object
avatar.small() Link to small avatar image string
avatar.medium() Link to medium avatar image string
avatar.large() Link to large avatar image string
createdDate() Account creation date string
updatedDate() Last date of updating the account string
id() User id on npm number
info.isGithubAccountConnected() 'true' if the user associate github account to his npm profile boolean
info.isTwitterAccountConnected() 'true' if the user associate twitter account to his npm profile boolean
info.githubAccount() associated github username string
info.twitterAccount() associated twitter username string
info.fullName() The full name provided by the user string
info.email() The email provided by the user string
packages.totalNumber() Number of all packages owned by the user number
packages.listAll() Array of all the packages owned by the user with their main info array
orgs.totalNumber() Number of all organizations the user is associated with number
orgs.listAll() Array of all the organizations the user is associated with, with their main info array

Search

By init the Search class, you get access npm search engine for package searching as define in the npm documentations.

import { Search } from 'npm-digger'

const search = new Search()

const res = await search.searchPackage('express', 5)

Functionality

Function Description Type
searchPackage() JSON object consist search results on npm website search engine JSON

Params

Name Description Type Required
query full text search to apply string yes
size how many results should be returned number no
from offset to return results from number no
quality how much of an effect should quality have on search results number no
popularity how much of an effect should popularity have on search results number no
maintenance how much of an effect should maintenance have on search results number no

Provenance

Provenance

By init the Provenance class, you get access npm & github provenance data.

import { Provenance } from 'npm-digger'

const provenance = new Provenance()

const res = await provenance.searchPackage('express', 5)

Params

Name Description Type Required
packageName package name string yes
packageVersion requested version number string yes

Functionality

Function Description Type
data() JSON object consist all provenance data of the specified package version JSON
subjectAlternativeName() string
certificateIssuer() string
issuer() string
issuerDisplayName() string
buildTrigger() string
buildConfigUri() string
sourceRepositoryUri() string
sourceRepositoryDigest() string
sourceRepositoryRef() string
runInvocationUri() string
expiresAt() string
includedAt() string
resolvedSourceRepositoryCommitUri() string
transparencyLogUri() string
buildConfigDisplayName() string
resolvedBuildConfigUri() string
sourceCommitResponseCode() number
sourceCommitUnreachable() boolean
sourceCommitNotFound() boolean

Vulnerabilities

By init the Vulnerabilities class, you get access osv data about the package and can search for vulnerabilities. this class uses the osv API to fetch the data.

import { Vulnerabilities } from 'npm-digger'

const vulnerabilities = new Vulnerabilities()

const res = await vulnerabilities.number('express', '3.0.0')

console.log(res)
// 1

Params

Name Description Type Required
packageName package name string yes
packageVersion requested version number string yes

Functionality

Function Description Type
data() JSON object consist all provenance data of the specified package version JSON
number() the number of vulnerabilities for the package number
ids() all vulnerabilities ids for the specified package string[]
keyInfo() an object with key info about the vulnerabilities: id, summary, publish date, modified date, affected versions and severity object