add dependency rules

jrburke edited this page Jun 21, 2012 · 15 revisions

These are the rules volo add uses to know what to keep from a bundle it installs. volo can also handle single JS file installs, but that installation is fairly straightforward.

add currently does recursively add dependencies. It is possible, but just not coded yet.

The logic is targeted at working for GitHub repos that exist today, to show the usefulness of this kind of tool, but the hope is to also evangelize best-practice layouts. There is also an overrides section in the config that specifies what to do for some libraries that do not quite fit the model yet, but could easily work with the addition of a property to a package.json structure.

Target resolution to an URL

When this command is run:

volo add target [localName]

the installed bundle has a name of target.

A target can have a protocol scheme: foo:baz, and volo allows for different scheme resolvers. Right now it just supports http: 'https:, 'github:, 'local:' and 'symlink:'.

If there is no scheme in the target:

target is checked to see if it is a local path to a directory. If so, then the scheme of local: is assigned to it, and that directory is copied and treated as the expanded contents from an archive.

GitHub identifiers

If target is not a local directory, it gets the github: scheme, and the target is assumed to be a GitHub identifier, either query, user/repo or user/repo/tag. volo distinguishes between these values based on the number of front slashes.

  • If query, then the GitHub V2 search API is used, and a user/repo value is generated from the first search result.
  • If user/repo, then the GitHub API is used to find the latest version tag (or master if no version tags).

So, all the options are normalized to a user/repo/tag identifier.

A package.json is fetched for that user/repo/tag. If it exists, and if it contains a volo.url property, then that value is used for the location of the dependency.

volo.url can contain a {version} substring, which is replaced with the numeric value from the tag.

Here is an example package.json for jQuery, since it places its releases on its CDN:

{
    "volo": {
        "url": "http://code.jquery.com/jquery-{version}.js"
    }
}

Example repo "foo", that specifies a path to a zip file in the GitHub Download area for its built releases:

{
    "volo": {
        "url": "https://github.com/downloads/jrburke/foo/foo-{version}.zip"
    }
}

If the package.json does not exist for that user/repo/tag combination, or if it does not contain a "volo" property, then the volojs/repos repo is consulted to see if there is an "override" for that project. The volojs/repos repo is organized by user/repo, and if there is a package.json in that location, the "volo" information from that package.json override is used.

If there is still no "volo" information from a package.json, then the a zipball URL from GitHub for that user/repo/tag combination is generated using GitHub's API. The normally end up looking like:

https://github.com/[user]/[repo]/zipball/[tag]

The end result is an URL, to either a single JS file or an archive. Right now only .zip archives are supported, but it might be nice to support .tar.gz files. .zip files were easier to get working cross-platform, but otherwise there was no special reason to choose .zip files to start.

That is the github: scheme resolution of target.

Other schemes

The target can also be one of the following:

  • symlink:path/to/dir/target - creates a symlink to the path/to/dir/target directory.
  • http(s)://some.domain.com/path/to/target.zip - directly specifying an archive URL.
  • http(s)://some.domain.com/path/to/lib.js - an URL to a single JS file for install. Its installation is very straightforward. The rest of this document is concerned with archive URLs.

Detecting AMD projects

If the add command either specifies -amd or it happens from a directory that has a package.json with:

"amd": {
    "baseUrl": "path/to/baseUrl"
}

then it will do the if AMD actions mentioned below.

Finding the localName destination

Once the archive URL is known, the localName is determined. It can be passed in to the add command, but usually it is not, so the localName value is determined from the name of the archive -- the archive file name without the file extension.

Once the archive is fetched from the archive URL, add looks in the current working directory for the following directories, and will use them as the parent directory for localName, the directory where localName will be placed.

Listed in order of preference:

  • package.json's volo.baseUrl value
  • package.json's amd.baseUrl value
  • js/
  • scripts/
  • current working directory.

The directory chosen is called parentDir below.

Unpacking the archive

From symlink

Just creates the symlink at parentDir/target.

if AMD: If there is a main property in parenDir/target/package.json, an adapter module at parentDir/target.js. is created, that simply return the main module:

define(['./targetDir/mainValue'], function (main) {
    return main;
});

By doing this, there is no need to specify a configuration for the AMD loader, it just finds all the modules by convention.

From a .zip bundle

volo unpacks the .zip file in a temporary directory. It assumes the zip file will create one directory that has all of the zip file contents in it. It looks in the directory and applies the following rules in this order:

  • if: there is a package.json file or just one .js file at the top level and it has a /package.json */comment, and *if**: there is a "main" property, read it.
    • Parse the file for AMD, then CommonJS, dependencies
      • if: there are dependencies that are relative (start with "./"), then keep the whole directory.
      • else: just grab the "main" JS file and use that as the target.
  • else:
    • if: there is only one JS file in the directory, use that one file as the target.
    • else if: there was a .js file that had the /*package.json */ comment use that as the target
    • else if: there is a file that matches [localName].js use that as the target.
    • else: keep the whole directory as the target for the add.

Once the final target is chosen:

  • if: the target is a directory:
    • Remove a configurable list of directories and files from target. Currently it removes:
      • .gitignore
      • test
      • tests
      • doc
      • docs
      • example
      • examples
      • demo
      • demos
    • if AMD: create an adapter module as mentioned in the From symlink section.
  • else: copy the single target file to the parentDir.

Once the target is in its final destination, only if AMD, then JS files are converted to AMD:

  • target is scanned for CommonJS/Node module usage. If it is a CommonJS/Node module it is wrapped in define(). If __dirname or __filename are used, values for those variables are generated.
  • else: if not a CJS/Node module, wrap in define call. If depends and exports are passed on the command line, use those in the conversion. Those are the same options that can be passed to the amdify volo command.

Recording the add

After addition of the dependency, if the project has a package.json file, the package.json file is modified. A volo.dependencies config is updated to have an entry in it that specifies the target name and the ID for the dependency. The ID for GitHub will be the final user/repo/tag name used. Otherwise it is URL used for the single JS file/archive. Example package.json for something added from GitHub:

{
    "volo": {
        "dependencies": {
            "underscore": "github:documentcloud/underscore/1.2.3"
        }
    }
}