Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Created release script that updates all ckeditor5 dependencies to the latest version. #737

Merged
merged 29 commits into from
Jan 7, 2022
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
11fa1c9
Added updatePackageVersions script
przemyslaw-zan Dec 13, 2021
68536f7
Restructure
przemyslaw-zan Dec 14, 2021
7c3f456
Added committing of changes, formatted the output log.
przemyslaw-zan Dec 14, 2021
9344acc
Applied review requests and refractored the script.
przemyslaw-zan Dec 16, 2021
7090e1c
Refractoring
przemyslaw-zan Dec 20, 2021
8b85dfc
Fixed display of a wrong amount of files commited.
przemyslaw-zan Dec 21, 2021
0c5b3ca
Extracted similar loops to a function.
przemyslaw-zan Dec 21, 2021
b0910e9
Displaying diff in the console
przemyslaw-zan Dec 21, 2021
1d07d15
Updated docs, rearanged arguments taken by the function.
przemyslaw-zan Dec 21, 2021
0f71e49
Updated dry run diff display to work gradually based on user input.
przemyslaw-zan Dec 27, 2021
c3a5d12
Fixed logic.
przemyslaw-zan Dec 27, 2021
49cb082
Added tests
przemyslaw-zan Dec 27, 2021
b273e6e
Applied review requests
przemyslaw-zan Dec 27, 2021
e30a9ad
Review requests.
przemyslaw-zan Dec 29, 2021
395e133
Added option to display all changes at once.
przemyslaw-zan Jan 3, 2022
2e83188
Changed order of the conditions to simplify the logic.
przemyslaw-zan Jan 3, 2022
d0e4536
Updated commit message and docs.
przemyslaw-zan Jan 3, 2022
cd941be
Refractoring.
przemyslaw-zan Jan 4, 2022
86aefbf
Changed name to updateCKEditor5Dependencies.
przemyslaw-zan Jan 4, 2022
24a24af
Updated function name in tests.
przemyslaw-zan Jan 4, 2022
c67c7d0
Added more tests.
przemyslaw-zan Jan 4, 2022
176e40b
Added more tests.
przemyslaw-zan Jan 4, 2022
0aabbee
Full code coverage.
przemyslaw-zan Jan 5, 2022
55eae45
Updated tests for `index.js`.
przemyslaw-zan Jan 5, 2022
441bc44
Fixed breaking other test files.
przemyslaw-zan Jan 5, 2022
7efde0a
Merge branch 'master' into ci/1123
przemyslaw-zan Jan 5, 2022
4ae946e
Updated yarn.lock
przemyslaw-zan Jan 5, 2022
1898110
Minor refactoring.
pomek Jan 7, 2022
6aa65a5
Adjusted tests to new API.
pomek Jan 7, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/ckeditor5-dev-env/lib/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@ module.exports = {
return require( './release-tools/tasks/generatechangelogformonorepository' )( ...args );
},

updatePackageVersions( ...args ) {
return require( './release-tools/tasks/update-package-versions' )( ...args );
},

createPotFiles( ...args ) {
return require( './translations/createpotfiles' )( ...args );
},
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,245 @@
/**
* @license Copyright (c) 2003-2021, CKSource - Frederico Knabben. All rights reserved.
* For licensing, see LICENSE.md.
*/

'use strict';

const chalk = require( 'chalk' );
const { diffLines: diff } = require( 'diff' );
const { execSync } = require( 'child_process' );
const fs = require( 'fs' );
const glob = require( 'glob' );
const readline = require( 'readline' );
const { sep, posix } = require( 'path' );

/**
* For all directories passed as an argument, all `@ckeditor/ckeditor5-*` and `ckeditor5` dependencies will be updated to the latest
pomek marked this conversation as resolved.
Show resolved Hide resolved
* version. If `commit: true` is added, changes will be committed as well.
*
* See https://github.com/cksource/ckeditor5-internal/issues/1123
pomek marked this conversation as resolved.
Show resolved Hide resolved
*
* @param {Array<Object>} pathsToUpdate Array of objects, where each object needs to have `path` value with an absolute path to the
* repository that is to be updated, as well as optional `commit` flag, that when set to true, will commit the changes from that path.
* @param {Boolean} dryRun Prevents the script from committing or changing anything, and instead shows list of files that would be
* changed.
*/
module.exports = function updatePackageVersions( pathsToUpdate, dryRun ) {
pomek marked this conversation as resolved.
Show resolved Hide resolved
const totalResult = { found: 0, updated: 0, toCommit: 0, differences: [] };
const pathsToCommit = [];

console.log( '\n📍 ' + chalk.blue( 'Updating CKEditor 5 dependencies...\n' ) );

for ( const pathToUpdate of pathsToUpdate ) {
const path = pathToUpdate.path.split( sep ).join( posix.sep );
pomek marked this conversation as resolved.
Show resolved Hide resolved

console.log( `Looking for package.json files in '${ path }'...` );
pomek marked this conversation as resolved.
Show resolved Hide resolved

const result = updateDirectory( path, dryRun );

totalResult.found += result.found;
totalResult.updated += result.updated;

if ( result.differences ) {
totalResult.differences.push( ...result.differences );
}
przemyslaw-zan marked this conversation as resolved.
Show resolved Hide resolved

if ( !result.found ) {
console.log( 'No files were found.\n' );
} else if ( !result.updated ) {
console.log( `${ chalk.bold( result.found ) } files were found, but none needed to be updated.\n` );
} else {
console.log( `Out of ${ chalk.bold( result.found ) } files found, ${ chalk.bold( result.updated ) } were updated.\n` );

if ( pathToUpdate.commit ) {
totalResult.toCommit += result.updated;
pathsToCommit.push( path );
}
}
}

if ( dryRun ) {
console.log( chalk.yellow( 'DRY RUN mode - press any key to display next file diff, or Q to exit.' ) );

if ( !totalResult.differences.length ) {
console.log( chalk.yellow( 'The script has not changed any files.' ) );
process.exit();
}

readline.emitKeypressEvents( process.stdin );
process.stdin.setRawMode( true );
pomek marked this conversation as resolved.
Show resolved Hide resolved

process.stdin.on( 'keypress', ( str, key ) => {
if ( key.name === 'q' ) {
console.log( chalk.yellow( 'Manual exit.' ) );
process.exit();
}

const nextDiff = totalResult.differences.shift();
const formattedDiff = formatDiff( nextDiff.content );
przemyslaw-zan marked this conversation as resolved.
Show resolved Hide resolved

console.log( chalk.underline( nextDiff.file ) );

for ( const line of formattedDiff ) {
console.log( line );
}

if ( !totalResult.differences.length ) {
console.log( chalk.yellow( 'No more files.' ) );
process.exit();
}

console.log( chalk.yellow( 'Any key - Next | Q - Exit' ) );
} );
} else {
if ( pathsToCommit.length ) {
console.log( '\n📍 ' + chalk.blue( 'Committing the changes...\n' ) );

for ( const path of pathsToCommit ) {
const execOptions = {
stdio: 'inherit',
cwd: path
};

console.log( `${ chalk.green( '+' ) } ${ path }` );

execSync( `git add ${ path }`, execOptions );
execSync( 'git commit -m "Internal: Updated all CKEditor 5 dependencies ' +
'in `packages/*` to the latest version. [skip ci]"', execOptions );
pomek marked this conversation as resolved.
Show resolved Hide resolved
}

console.log( '\n📍 ' + chalk.green( `Successfully committed ${ totalResult.toCommit } files!\n` ) );
}

if ( totalResult.updated ) {
console.log( '\n📍 ' + chalk.green( `Updated total of ${ totalResult.updated } files!\n` ) );
} else {
console.log( '\n📍 ' + chalk.green( 'No files needed an update.\n' ) );
}
}
};

/**
* Updates `@ckeditor/ckeditor5-*` and `ckeditor5` dependencies in the specified directory to the latest version.
pomek marked this conversation as resolved.
Show resolved Hide resolved
* Latest version is taken from the `version` property from the root of the `package.json` file, since that value
* should already have been bumped at this point of release process.
*
* @param {String} pathToUpdate Directory containing files to update.
* @param {Boolean} dryRun If set to true, diff of changes that would be made is calculated, and included in the returned object. Without
* this flag, file is updated normally.
* @returns {updateResult}
*/
function updateDirectory( pathToUpdate, dryRun ) {
const globPattern = pathToUpdate + '/*/package.json';
const packageJsonArray = glob.sync( globPattern );

if ( !packageJsonArray.length ) {
return { found: 0, updated: 0, differences: [] };
}

let updatedFiles = 0;
const differences = [];

for ( const file of packageJsonArray ) {
const currentFileData = fs.readFileSync( file, 'utf-8' );
const parsedData = JSON.parse( currentFileData );

updateObjectProperty( parsedData, 'dependencies' );
updateObjectProperty( parsedData, 'devDependencies' );

const newFileData = JSON.stringify( parsedData, null, 2 ) + '\n';

if ( currentFileData !== newFileData ) {
updatedFiles++;

if ( dryRun ) {
differences.push( {
file,
content: diff( currentFileData, newFileData, { newlineIsToken: true } )
} );
} else {
fs.writeFileSync( file, newFileData, 'utf-8' );
}
}
}

return { found: packageJsonArray.length, updated: updatedFiles, differences };
}

/**
* Updates the CKEditor 5 dependencies, except the `*-dev` and `*-inspector` ones, in the provided `package.json` file.
*
* @param {Object} parsedPkgJson Object to update.
* @param {String} propertyName Name of the property to update.
*/
function updateObjectProperty( parsedPkgJson, propertyName ) {
// Update only the CKEditor 5 dependencies, except the *-dev and *-inspector.
const regex = /^@ckeditor\/ckeditor5-(?!dev|inspector)|^ckeditor5$/;
pomek marked this conversation as resolved.
Show resolved Hide resolved
const version = parsedPkgJson.version;

for ( const dependency in parsedPkgJson[ propertyName ] ) {
if ( !regex.test( dependency ) ) {
continue;
}

parsedPkgJson[ propertyName ][ dependency ] = `^${ version }`;
}
}

/**
* Takes in raw changelog for a single file generated by `diff` library, and returns formatted array of strings, containing
* human-readable, line by line changelog of a file, with removals and additions colored. If the file has any longer parts without changes,
* these will be partially hidden.
*
* @param {Array<Object>} diff Array of changes for a single file generated by `diff` library.
* @returns {Array<String>} Formatted changelog split into single lines.
*/
function formatDiff( diff ) {
const formattedDiff = [];
const regex = /(?<=":) (?=")/;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this regexp is needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to split these lines and merge them back together in order to pretty print the diff. Indentations are also kept in these lines, so there are more than one space. I suppose i could split using last space of a string instead, but this seems safer.

    "@ckeditor/ckeditor5-core": "^31.1.0",

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if, for whatever reason, the script changes something else in the package.json file, other than the CKEditor 5 dependency versions? Now, only changes to the dependency versions are displayed in red and green colors, but other changes that could occur in the file are not highlighted in the console.

The --dry-run flag should display all the changes that the script has made, to be sure that running it without this flag will not surprise us.

If the dependency version has changed, it can be highlighted as it is now (only the version, not the full line). But when something else has changed that is not expected, let the entire changed line be highlighted.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, i've actually taken this into account, but then i forgot to color the rest of changes, so good thing You pointed that out 😅

Single removals and additions are simple enough, they are just colored lines. But how do we want to show replacements?

a) Replacement on the same line

b) Replacement on the same line, but trim the whitespaces off the addition

c) Replacement on the line below 

I'd suppose option b) is preferable, so this is what I'll do for now.


for ( let i = 0; i < diff.length; i++ ) {
const current = diff[ i ];
const next = diff[ i + 1 ];
const currentLines = current.value.split( '\n' );

const validReplaceSequence = current.removed && next.added && regex.test( current.value ) && regex.test( next.value );
pomek marked this conversation as resolved.
Show resolved Hide resolved

if ( validReplaceSequence ) {
// Adding removals followed by additions (formatted).
formattedDiff.push( [
current.value.split( regex )[ 0 ],
' ',
chalk.red( current.value.split( regex )[ 1 ] ),
chalk.green( next.value.split( regex )[ 1 ] )
].join( '' ) );

i++;
} else if ( !current.added && !current.removed && currentLines.length > 8 ) {
// Cutting out the middle of a long streak of unchanged lines.
const shortenedLines = [
...currentLines.slice( 0, 3 ),
chalk.gray( `[...${ currentLines.length - 7 } lines without changes...]` ),
...currentLines.slice( -4 )
].join( '\n' );

formattedDiff.push( shortenedLines );
} else {
// Adding everything else that does not need formatting.
formattedDiff.push( current.value );
}
}

// This turns the array from random chunks containing newlines, to uniform set of single lines.
return formattedDiff.join( '' ).split( '\n' );
}

/**
* Contains information about the way in which the files were processed.
*
* @typedef {Object} updateResult
* @property {Number} found amount of files found
* @property {Number} updated amount of files updated
* @property {Array<Object>} differences array of objects, where each object has string `file` containing path to the file, as well as
* array of objects `content` returned by the `diff` library, that describes changes made to each file.
przemyslaw-zan marked this conversation as resolved.
Show resolved Hide resolved
*/
2 changes: 2 additions & 0 deletions packages/ckeditor5-dev-env/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,14 @@
"conventional-commits-filter": "^2.0.6",
"conventional-commits-parser": "^3.1.0",
"del": "^5.1.0",
"diff": "^5.0.0",
"fs-extra": "^9.0.0",
"git-raw-commits": "^2.0.7",
"glob": "^7.1.6",
"inquirer": "^7.1.0",
"minimatch": "^3.0.4",
"mkdirp": "^1.0.4",
"mock-fs": "^5.1.2",
przemyslaw-zan marked this conversation as resolved.
Show resolved Hide resolved
"parse-github-url": "^1.0.2",
"request": "^2.88.2",
"semver": "^7.3.2"
Expand Down
28 changes: 27 additions & 1 deletion packages/ckeditor5-dev-env/tests/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ describe( 'dev-env/index', () => {
releaseSubRepositories: sandbox.stub(),
generateChangelogForSinglePackage: sandbox.stub(),
generateChangelogForMonoRepository: sandbox.stub(),
bumpVersions: sandbox.stub()
bumpVersions: sandbox.stub(),
updatePackageVersions: sandbox.stub()
}
};

Expand All @@ -51,6 +52,7 @@ describe( 'dev-env/index', () => {
mockery.registerMock( './release-tools/tasks/generatechangelogforsinglepackage', releaseTools.generateChangelogForSinglePackage );
mockery.registerMock( './release-tools/tasks/releasesubrepositories', releaseTools.releaseSubRepositories );
mockery.registerMock( './release-tools/tasks/generatechangelogformonorepository', releaseTools.generateChangelogForMonoRepository );
mockery.registerMock( './release-tools/tasks/update-package-versions', releaseTools.updatePackageVersions );

tasks = proxyquire( '../lib/index', {
'@ckeditor/ckeditor5-dev-utils': {
Expand Down Expand Up @@ -133,6 +135,30 @@ describe( 'dev-env/index', () => {
} );
} );

describe( 'updatePackageVersions()', () => {
it( 'should update versions in package.json files', () => {
przemyslaw-zan marked this conversation as resolved.
Show resolved Hide resolved
stubs.release.updatePackageVersions.returns( 'OK.' );

const output = tasks.updatePackageVersions(
[
{ path: 'foo/packages', commit: true },
{ path: 'bar/packages', commit: false }
],
process.argv.includes( '--dry-run' )
);

sinon.assert.calledOnce( stubs.release.updatePackageVersions );
sinon.assert.alwaysCalledWithExactly( stubs.release.updatePackageVersions,
[
{ path: 'foo/packages', commit: true },
{ path: 'bar/packages', commit: false }
],
process.argv.includes( '--dry-run' )
);
expect( output ).to.equal( 'OK.' );
} );
} );

describe( 'uploadPotFiles()', () => {
it( 'should upload translations', async () => {
stubs.translations.uploadPotFiles.resolves( { status: true } );
Expand Down