Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Per-package version flags for independently versioned publishing #389

Closed
traviskaufman opened this issue Oct 24, 2016 · 14 comments
Labels

Comments

@traviskaufman
Copy link
Contributor

We've been using Lerna to develop Material Design Lite v2, and it's been working out extremely well for us. We independently version all of our components, with packages/material-design-lite mirroring the overall version of our monorepo.

As we near our initial release, we're starting to focus on release infrastructure. While lerna takes care of pretty much all the heavy lifting around tagging/publishing for us, one thing I've noticed is that while you can programmatically specify a singular overarching version for all packages via --repo-version, there doesn't seem to be a way to specify per-package versions in the same manner.

Just at alpha, we already have ~20 packages. Entering versions for each one at a prompt could be quite time-consuming and error-prone for us, especially when trying to determine the optimal version change for an update spanning multiple commits for multiple packages. I've written some ideas on ways we can minimize the work we have to do for this in an issue comment, but that still involves manually entering a version.

I was wondering if the team would consider adding a flag similar to --repo-version, perhaps something along the lines of --package-versions, which could be used to programmatically specify versions for each package.

For example:

lerna publish -i --package-versions 'mdl-button=0.1.0 mdl-drawer=0.3.2 ...'

or

lerna publish -i --package-version 'mdl-button=0.1.0' --package-version 'mdl-drawer=0.3.2' ...

or

lerna publish -i --package-versions '{"mdl-button": "0.1.0", "mdl-drawer": "0.3.2", ... }'

and/or

lerna publish -i --package-versions new-versions.json

This would allow teams like us to programmatically determine package versions, and then provide them to lerna.

I've (crappily) sketched out what a high-level implementation for this could look like below:

  • Let packagesToUpdate be the collection of packages returned from UpdatedPackagesCollector
  • If --package-versions is specified
    • If not independent, error
    • If --repo-version also specified, error
    • Else
      • let packageVersions be an object where each key is a package and each value is a version
      • For each (package, version) specified in --package-versions
        • If package cannot be found in packages dir, error
        • Else
          • If packageVersions[package] is already defined, error (?)
          • Else
            • Assign packageVersions[package] = version
      • Let packagesNeedingPrompt be the intersection of packagesToUpdate and the keys of packageVersions
      • Prompt for each remaining package needing a version
  • Continue with publishing

I'd be happy to help out with this as it would help out a lot with our release workflow. Another (much simpler) alternative could be to add support for omitting auto-tagging and assume the end user has already manually tagged all sub-packages. This, however, could be more error-prone than a solution integrated directly into lerna.

As an FYI, we've also looked at lerna-semantic-release, but we don't use cz-lerna-changelog (we use cz-conventional-changelog), there isn't much docs on how exactly the releases work, and we're still a bit wary of fully-automated releases until the project stabilizes.

@jamiebuilds
Copy link
Contributor

We've discussed this before, @hzoo do you remember what APIs were suggested for this?

@jamiebuilds jamiebuilds changed the title [feature request] Per-package version flags for independently versioned publishing [Feature] Per-package version flags for independently versioned publishing Nov 16, 2016
@hzoo
Copy link
Contributor

hzoo commented Nov 16, 2016

Might of been a while ago so I forgot - just that users wanted to specify versions before hand for CI and I wanted to use something like semantic release as well (github labels/commit message)

@traviskaufman
Copy link
Contributor Author

traviskaufman commented Nov 16, 2016

if it helps at all, we've frankenstein-ed together some scripts to assist us with this manually. We basically print out what we believe the new versions should be, which we could then use to manually update lerna. I'd love to be able to turn these tables into a machine-readable format which we could pass off to lerna, whether that's by formatting them as CLI flags or some other method.

@jamiebuilds
Copy link
Contributor

The other option we have here is to allow people to update the versions in their package.json's manually without running a publish or making any git tags and have a flag in Lerna that will use those versions during publish. It could have validation to ensure that the results of lerna updated have unpublished versions in their package.json files.

It could even be automated by a separate command:

$ lerna pre-publish
# interactive version updates
# in CI:
$ lerna publish # maybe doesn't need flag?
# no need for interactive bits

@vampolo
Copy link

vampolo commented Feb 27, 2017

Any update/workaround on this?

@evocateur
Copy link
Member

No progress that I'm aware of. Does #607 solve this issue?

@vampolo
Copy link

vampolo commented Mar 8, 2017

It might. but i'm not sure how.
The workflow i would like to achieve is to have devs be able to bump versions manually and then, after CI tests and validation, run lerna publish in CI. This will publish the packages that changed.

koddsson pushed a commit to koddsson/apis that referenced this issue May 2, 2017
This is probably going to change in the future but for now I need this
to publish from CI. Possible fixes for this is to implement this[1] or
force conventional commits.

Conventional commits might not be that bad actually so I'll think about
it a bit more

[1]: lerna/lerna#389 (comment)
[2]: https://github.com/lerna/lerna#--conventional-commits
@noherczeg
Copy link
Contributor

@vampolo Using --skip-git in combination with --skip-npm works better?

@GeorgeTaveras1231
Copy link

GeorgeTaveras1231 commented Jan 26, 2018

Would love to see this implemented. Unfortunately, I don't have the time to implement this myself, however, I wanted to share a somewhat hacky solution to programmatically update independently versioned packages. I created a POC script which given a map of package names pointing to a number representing the location of the bump strategy on the prompt, will programmatically answer the prompt. Needless to say, this is very hacky, as arbitrary changes to how the prompt is presented could break this script.

import childPty from 'child_pty';
import { EOL as ENTER } from 'os';

const PATCH = '1';
const MINOR = '2';
const MAJOR = '3';

const publish = childPty.spawn('yarn', 'lerna publish'.split(' '));

/* package to bump strategy map */
const updateMap = {
  '@xo-union/sdk-membership': MINOR,
  '@xo-union/store-conversations-redux': MAJOR,
  '@xo-union/store-membership-redux': PATCH,
  '@xo-union/store-membership': MINOR,
  '@xo-union/tk-component-core-provider': MAJOR,
  '@xo-union/tk-component-header-nav-redux': MINOR,
  '@xo-union/tk-component-membership-modal-redux': PATCH,
  '@xo-union/tk-component-vendor-card': MAJOR,
  'union-release': PATCH
};

const alreadyAnswered = {};

publish.stdout.on('data', (data) => {
  const string = data.toString('utf8');

  const waitingForAnswer = string.indexOf('Select a new version for') >= 0;

  if (waitingForAnswer) {
    const packageName = string.match(/Select a new version for ([^\s]+)/)[1];

    if (!alreadyAnswered[packageName]) {
      const key = updateMap[packageName];
      publish.stdin.write(key);
      publish.stdin.write(ENTER);

      alreadyAnswered[packageName] = true;
    }
  }

  if (string.indexOf('Are you sure you want to publish the above changes?') >= 0) {
    /* Start accepting input, to allow user to confirm */
    process.stdin.on('data', (data) => publish.stdin.write(data));

    /**
    if continous deployment:
     publish.stdin.write('y');
     publish.stdin.write(ENTER);
    **/
  }

  process.stdout.write(data);
});


publish.on('close', function () {
  process.stdin.pause();
});

@evocateur
Copy link
Member

evocateur commented Jan 26, 2018 via email

@GeorgeTaveras1231
Copy link

@evocateur You are right but adopting conventional commits means forcing developers into the conventional commit workflow (for both developers and reviewers) that could cause confusion and production issues if not monitored carefully. We are weighing that option against an option to use a simpler method of storing the next-release metadata in YAML files, which has the benefit of being simpler to review (as our developers primarily focus on reviewing files in PRs), and simpler to edit in the case of mistakes.

Depending on who you speak to, knowing how to navigate git fluently enough to not cringe at the thought of editing old commits, should be a requirement for all developers -- but from my own experience, I've found that many (especially new) developers really struggle with this skill.

@evocateur
Copy link
Member

evocateur commented Jan 26, 2018 via email

@stale
Copy link

stale bot commented Dec 27, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Dec 27, 2018
@stale stale bot closed this as completed Jan 3, 2019
@lock
Copy link

lock bot commented Apr 3, 2019

This thread has been automatically locked because there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Apr 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

7 participants