Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update semver usage in cdk packages #3711

Open
KnisterPeter opened this issue Aug 19, 2019 · 11 comments

Comments

@KnisterPeter
Copy link
Contributor

commented Aug 19, 2019

General Issue

Semver defined in internal packages.

The Question

All CDK packages use semver for internal dependencies. That means e.g. @aws-cdk/aws-cloudfront defines @aws-cdk/aws-iam as dependecies with the version of itself but with a caret.

E.g. version 1.2.0 of aws-cloudfront dependes on ^1.2.0 of aws-iam which is wrong since it requires exaclty 1.2.0.

This could easily leads to bugs which are hard to debug/understand.

@skinny85

This comment has been minimized.

Copy link
Contributor

commented Aug 19, 2019

I confirm this is a problem:

$ cat package.json 
{
	"dependencies": {
		"@aws-cdk/core": "1.3.0"
	}
}

$ cat node_modules/\@aws-cdk/core/package.json | grep version
  "version": "1.3.0"

$ cat node_modules/\@aws-cdk/cx-api/package.json | grep version
  "version": "1.4.0"

There is no way to downgrade to 1.3.0 after 1.4.0 has been released.

@carlosrfernandez

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2019

@statik Well technically there is... But it is a pain, I had to do it...

You have first uninstall -g aws-cdk, then npm i -g aws-cdk@1.3.0.

Then edit the packages.json and the packages-lock.json files in your project...

Manually replacing 1.4.0 for 1.3.0, the packages-lock.json has to be reverted (I used git's history to locate the previous version to the upgrade) because it contains hashes of the 1.4.0 packages and these will not match the 1.3.0 versions.

after those files are updated (and commited) try again.

@skinny85

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2019

@carlosrfernandez thanks for posting that workaround, but I think I speak for everyone on the team when I say that experience does not meet our CDK usability bar :(

@carlosrfernandez

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2019

@skinny85 I was merely pointing out that it is possible to roll back. And in our case we had no alternative than to apply this workaround.

I am in total agreement with you that the desired developer experience should be much better.

@skinny85

This comment has been minimized.

Copy link
Contributor

commented Aug 27, 2019

Some implementation notes/questions below.


So, I believe the correct way to model this in package.json is the following:

  1. dependencies should be pinned to an exact version; so, it should be "@aws-cdk/core": "1.5.0", not "@aws-cdk/core": "^1.5.0" as it is today.
  2. peerDepedencies should remain open to any compatible major version, the way it is today. That allows customers to try and use the core libraries with third-party libraries by changing the versions in their package.json. So, it should remain "@aws-cdk/core": "^1.5.0", not be changed to "@aws-cdk/core": "1.5.0" like in dependencies.

If that proposal seems OK to everybody, this requires changes in the following places:

  1. The JSII build tools assume dependency and peerDependency are identical. We would have to loosen this requirement; maybe just check that the dependencies version satisfies peerDependency version, like we do in pkglint in the CDK?
  2. We should also change the way versions of local dependencies are inferred in JSII (always with a caret).
  3. After those change in JSII, we would have to modify the sync-peer-deps.js script - probably also checking for dependencies version satisfying peerDependency version instead of a strict equality.
@eladb

This comment has been minimized.

Copy link
Contributor

commented Aug 27, 2019

Just to make sure I understand the problem. It is only caused when users choose to pin their dependencies, correct? If they use a caret dependency, they should also get the latest compatible one.

EDIT: I think we still have an issue with transitive dependencies.

@KnisterPeter

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2019

The main problem are transitive dependencies. Right now the contract in cdk is to use the same minor version off all packages. With a carret this semantic is violated.
You may consider to use a tilde to get latest bugfixes on all transitive dependencies.

@rix0rrr

This comment has been minimized.

Copy link
Contributor

commented Aug 27, 2019

You may consider to use a tilde to get latest bugfixes on all transitive dependencies.

Still not good enough for intra-CDK dependencies. We might release a bug in any version (even a patch version), and with any ^ or ~ users won't have a way to roll back from that.

@KnisterPeter

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2019

Thats right, but with a bug in a bug-release another release should be easy and fast.
But sticking to fixed versions all along is also good enough.

@RomainMuller

This comment has been minimized.

Copy link
Contributor

commented Aug 27, 2019

The fix-peer-deps.js script shouldn't be needed anymore - jsii will do this automatically for you unless you ask it not to (by passing --no-fix-peer-dependencies).

@skinny85

This comment has been minimized.

Copy link
Contributor

commented Aug 27, 2019

PR in JSII: aws/jsii#747

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.