Skip to content

Commit

Permalink
feat: Initial commit
Browse files Browse the repository at this point in the history
  • Loading branch information
dhoulb committed Jan 13, 2019
0 parents commit 942ef94
Show file tree
Hide file tree
Showing 46 changed files with 9,609 additions and 0 deletions.
2 changes: 2 additions & 0 deletions .eslintignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/node_modules/
/coverage/
39 changes: 39 additions & 0 deletions .eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"parserOptions": {
"sourceType": "module",
"ecmaVersion": 2017,
"experimentalObjectRestSpread": true
},
"extends": [
"eslint:recommended",
"plugin:prettier/recommended"
],
"env": {
"es6": true,
"node": true,
"jest": true
},
"rules": {
"eqeqeq": 2,
"no-alert": 2,
"no-caller": 2,
"no-eval": 2,
"no-extend-native": 2,
"no-floating-decimal": 2,
"no-implicit-globals": 2,
"no-labels": 2,
"no-loop-func": 2,
"no-new-require": 2,
"no-path-concat": 2,
"no-useless-rename": 2,
"no-var": 2,
"no-shadow": 2,
"prefer-const": 2,
"prefer-spread": 2,
"strict": 2,
"valid-jsdoc": 2,
"global-require": 0,
"no-console": 0,
"no-unused-vars": 0
}
}
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/node_modules/
/coverage/
7 changes: 7 additions & 0 deletions .prettierrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"tabWidth": 4,
"useTabs": true,
"printWidth": 120,
"parser": "babylon",
"proseWrap": "never"
}
17 changes: 17 additions & 0 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
language: node_js

cache: yarn

jobs:
include:
- stage: test
node_js: "8"
- stage: test
node_js: "10"
- stage: release
node_js: "10"
script: skip # Skip running "yarn test" again
deploy:
provider: script
skip_cleanup: true
script: yarn publish
12 changes: 12 additions & 0 deletions LICENSE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
Copyright (C) 2017 by Dave Houlbrooke <dave@shax.com>
Permission to use, copy, modify, and/or distribute this software
for any purpose with or without fee is hereby granted.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL
WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR
CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM
LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT,
NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN
CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
116 changes: 116 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,116 @@
# multi-semantic-release: hacky semantic-release for monorepos

[![Travis CI](https://travis-ci.org/dhoulb/blork.svg?branch=master)](https://travis-ci.org/dhoulb/blork)
[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg?style=flat)](https://github.com/semantic-release/semantic-release)
[![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-yellow.svg)](https://conventionalcommits.org)
[![Prettier](https://img.shields.io/badge/code_style-prettier-ff69b4.svg)](https://github.com/prettier/prettier)
[![npm](https://img.shields.io/npm/dm/blork.svg)](https://www.npmjs.com/package/blork)

Proof of concept that wraps `[semantic-release](https://github.com/semantic-release/semantic-release)` to work with monorepos.

This package should work well, but may not be fundamentally stable enough for important production use as it's pretty dependent on how semantic-release works (so it may break or get out-of-date in future versions of semantic-release).

One of the best things about semantic-release is forgetting about version numbers. In a monorepo though there's still a lot of version number management required for local deps (packages in the same monorepo referenced in `dependencies` or `devDependencies` or `peerDependencies`). However in multi-semantic-release the version numbers of local deps are written into `package.json` at release time. This means there's no need to hard-code versions any more (we recommend just using `*` asterisk instead in your repo code).

## Installation

```sh
yarn add multi-semantic-release --dev
```

## Usage

```sh
multi-semantic-release
```

## Configuration

Configuration for releases is the same as [semantic-release configuration](https://github.com/semantic-release/semantic-release/blob/master/docs/usage/configuration.md), i.e. using a `release` key under `package.json` or in `.releaserc` file of any type e.g. `.yaml`, `.json`.

_But_ in multi-semantic-release this configuration can be done globally (in your top-level dir), or per-package (in that individual package's dir). If you set both then per-package settings will override global settings.

multi-semantic-release does not support any command line arguments (this wasn't possible without duplicating files from semantic-release, which I've tried to avoid).

## API

multi-semantic-release default exports a `multirelease()` method which takes the following arguments:

- `packages` An array containing string paths to `package.json` files
- `options` An object containing default semantic-release configuration options

`multirelease()` returns an array of objects describing the result of the multirelease (corresponding to the `packages` array that is passed in).

```js
const multirelease = require("multi-semantic-release");

multirelease([
`${__dirname}/packages/my-pkg-1/package.json`,
`${__dirname}/packages/my-pkg-2/package.json`,
]);
```

## Implementation notes (and other thoughts)

### Support for monorepos

Automatically finds packages as long as workspaces are configured as-per [Yarn workspaces](https://yarnpkg.com/lang/en/docs/workspaces/). _You don't need to use Yarn but the way they define monorepos seems intuitive, and is likely what NPM will copy when they add this functionality (as rumoured)._

I'm aware Lerna is the best-known tool right now, but in future it seems clear it will be replaced by functionality in Yarn and NPM directly. If you use Yarn workspaces today (January 2019), then publishing is the only remaining feature Lerna is _really_ required for (though it'd be lovely if Yarn added parallel script execution). Thus using multi-semantic-release means you can probably remove Lerna entirely from your project.

### Iteration vs coordination

Other packages that enable semantic-release for monorepos work by iterating into each package and running the `semantic-release` command. This is conceptually simple but unfortunately not viable because:

- If a package is published that depends on minor changes that have been made in a sibling package it could cause extremely subtle errors (the worst kind!) — if the project follows semver religiously this should never happen, but it's better to eliminate the _potential_ for errors
- Dependency version numbers need to reflect the _next_ release at time of publishing, so a package needs to know the state of _all other packages_ before it can publish correctly — this central state needs to be coordinated by something

### Local dependencies and version numbers

A key requirement is handling local dep version numbers elegantly. multi-semantic-release does the following:

- The next version number of all packages is established first
- If a release has not changed but has local deps that _have_ changed... do a `patch` bump on that package too
- Before packages are released (in semantic-release's prepare step), the correct current/next version number of _all_ local dependencies is written into the `package.json` file (overwriting any existing value)
- This ensures the package at the time of publishing will be atomically correct with all other packages in the monorepo.

The above means that, possibly, if someone upgrades dependencies and pulls down a package from NPM _during the multirelease_ (before all its deps have been published at their next versions), then their `npm install` will fail (it will work if they try again in a few minutes). On balance I thought it was more important to be atomically correct (this situation should be fairly rare assuming projects commit their lockfiles).

### Integration with semantic-release

This is the jankiest part of multi-semantic-release and most likely part to break relies. I expect this to cause maintenance issues down the line. In an ideal world semantic-release will bake-in support for monorepos (making this package unnecessary).

The way I ended up integrating is to create a custom "inline plugin" for semantic-release, and passing that in to `semanticRelease()` as the only plugin. This then calls any other configured plugins to retrieve and potentially modify the response.

The plugin starts all release at once, then pauses them (using Promises) at various points to allow other packages in the multirelease to catch up. This is mainly needed so the version number of all packages can be established _before_ any package is released. This allows us to do a `patch` bump on releases whose local deps have bumped, and to accurately write in the version of local deps in each `package.json`

The inline plugin does the following:

- **verifyConditions:** _not used_
- **analyzeCommits:**
- Replaces `context.commits` with a list of commits filtered to the folder only
- Calls `plugins.analyzeCommits()` to get the next release type (e.g. from @semantic-release/commit-analyzer)
- Waits for _all_ packages to catch up to this point.
- For packages that haven't bumped, checks if it has local deps (or deps of deps) that have bumped and returns `patch` if that's true
- **verifyRelease:** _not used_
- **generateNotes:**
- Calls `plugins.generateNotes()` to get the notes (e.g. from @semantic-release/release-notes-generator)
- Appends a section listing any local deps bumps (e.g. "my-pkg-2: upgraded to 1.2.1")
- **prepare:**
- Writes in the correct version for local deps in `dependencies`, `devDependencies`, `peerDependencies` in `package.json`
- Serialize the releases so they happen one-at-a-time (because semantic-release calls `git push` asyncronously, multiple releases at once fail because Git refs aren't locked — semantic-release should use `execa.sync()` so Git operations are atomic)
- **publish:** _not used_
- **success:** _not used_
- **fail:** _not used_

### Git tags

Releases always use a `tagFormat` of `my-pkg-1@1.0.1` for Git tags, and always overrides any `gitTag` set in semantic-release configuration.

I can personally see the potential for this option in coordinating a semantic-release (e.g. so two packages with the same tag always bump and release simultaneously). Unfortunately with the points of integration available in semantic-release, it was effectively impossible when releasing to stop a second package creating a duplicate tag (causing an error).

To make the `tagFormat` option work as intended the following would need to happen:

- semantic-release needs to check if a given tag already exists at a given commit, and not create it / push it if that's true
- Release notes for multiple package releases need to be merged BUT the Github release only done once (by having the notes merged at the semantic-release level but only published once, or having the Github plugin merge them)
- Make it clear in documentation that the default tag `v1.0.0` will have the same effect as Lerna's fixed mode (all changed monorepo packages released at same time)
31 changes: 31 additions & 0 deletions bin/cli.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
#!/usr/bin/env node

// Imports.
const getWorkspacesYarn = require("../lib/getWorkspacesYarn");
const multiSemanticRelease = require("../lib/multiSemanticRelease");

// Get directory.
const cwd = process.cwd();

// Catch errors.
try {
// Get list of package.json paths according to Yarn workspaces.
const paths = getWorkspacesYarn(cwd);

// Do multirelease (log out any errors).
multiSemanticRelease(paths, {}, { cwd }).then(
() => {
// Success.
process.exit(0);
},
error => {
// Log out errors.
console.error(`[multi-semantic-release]:`, error);
process.exit(1);
}
);
} catch (error) {
// Log out errors.
console.error(`[multi-semantic-release]:`, error);
process.exit(1);
}
28 changes: 28 additions & 0 deletions lib/blork.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
const { existsSync, lstatSync } = require("fs");
const { checker, check, add, ValueError } = require("blork");
const { Signale } = require("signale");
const { Writable } = require("stream");
const { WritableStreamBuffer } = require("stream-buffers");

// Get some checkers.
const isAbsolute = checker("absolute");
const isObject = checker("object");
const isString = checker("string");

// Add a directory checker.
add(
"directory",
v => isAbsolute(v) && existsSync(v) && lstatSync(v).isDirectory(),
"directory that exists in the filesystem"
);

// Add a writable stream checker.
add(
"stream",
// istanbul ignore next (not important)
v => v instanceof Writable || v instanceof WritableStreamBuffer,
"instance of stream.Writable or WritableStreamBuffer"
);

// Exports.
module.exports = { checker, check, ValueError };
24 changes: 24 additions & 0 deletions lib/cleanPath.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
const { normalize, isAbsolute, join } = require("path");
const { check } = require("blork");

/**
* Normalize and make a path absolute, optionally using a custom CWD.
* Trims any trailing slashes from the path.
*
* @param {string} path The path to normalize and make absolute.
* @param {string} cwd=process.cwd() The CWD to prepend to the path to make it absolute.
* @returns {string} The absolute and normalized path.
*
* @internal
*/
function cleanPath(path, cwd = process.cwd()) {
// Checks.
check(path, "path: path");
check(cwd, "cwd: absolute");

// Normalize, absolutify, and trim trailing slashes from the path.
return normalize(isAbsolute(path) ? path : join(cwd, path)).replace(/[/\\]+$/, "");
}

// Exports.
module.exports = cleanPath;

0 comments on commit 942ef94

Please sign in to comment.