Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[shopsys] Add Releaser to automate release process #623

Merged
merged 36 commits into from Jan 11, 2019

Conversation

Projects
None yet
6 participants
@TomasVotruba
Copy link
Contributor

commented Nov 20, 2018

Q A
Description, reason for the PR US-5075 - automation of the release process
New feature Yes
BC breaks No
Fixes issues No
Standards and tests pass Yes
Have you read and signed our License Agreement for contributions? Yes

There are 3 stages available:

  1. release-candidate
  2. release
  3. after-release

For dry run, to see the list of steps only, run

vendor/bin/monorepo-builder release <release-number> --dry-run --stage <stage>

Note: If you want to see the names of the workers that are responsible for each step, simply set verbose mode by adding -v.

No changes to your code are performed when using the dry-run.

To perform the actual release, just ommit the --dry-run argument and follow instructions.

@TomasVotruba TomasVotruba requested a review from vitek-rostislav Nov 20, 2018

@TomasVotruba TomasVotruba force-pushed the tv-releaser branch 2 times, most recently from 2f1d6cc to cd1189a Nov 21, 2018

Show resolved Hide resolved CHANGELOG.md Outdated

@TomasVotruba TomasVotruba force-pushed the tv-releaser branch 2 times, most recently from ef2e1f8 to 75f7be6 Nov 21, 2018

Show resolved Hide resolved composer.json Outdated

@TomasVotruba TomasVotruba force-pushed the tv-releaser branch from 62b1884 to 14128ba Nov 21, 2018

@vitek-rostislav
Copy link
Contributor

left a comment

Hi, I have added a few notes, lease check them out. @boris-brtan will now do the review as well.

What we agreed on todays call:

  • we want to convert the whole release process from the google doc checklist to instances of ReleaseWorkerInterface - possibly some of them will only provide instructions that need to be performed manually (e.g. StopMergingToMasterReleaseWorker)
  • it would be nice to have a possibility to list all the workers with their priorities ordered by the priority, it would be useful to see all the steps e.g. when adding a new step to the process
  • each step that requires a human action will wait for confirmation (something like "[Manual] Tell team to stop merging to master branch. Press enter if you wish to continue with the release process")
@vitek-rostislav

This comment has been minimized.

Copy link
Contributor

commented Nov 22, 2018

Oh, one more idea - how should this work after I send the rc-xxx branch for review? Because it might mean that I will continue on my work another day so I need to keep the process running? Or is it possible to split the release command into "pre review" and "post review" part? Or even better, could I somehow specify, where to start?

@boris-brtan
Copy link
Contributor

left a comment

commit message typo Symplify

Show resolved Hide resolved composer.json Outdated
@boris-brtan
Copy link
Contributor

left a comment

oooooOOOoooo, seems like this will be powerful tool if it will work. Can spare time and problems during releases

Show resolved Hide resolved CHANGELOG.md Outdated
Show resolved Hide resolved packages/releaser/src/FileManipulator/UpgradeFileManipulator.php Outdated
Show resolved Hide resolved packages/releaser/src/FileManipulator/UpgradeFileManipulator.php Outdated
Show resolved Hide resolved packages/releaser/src/IntervalEvaluator.php Outdated
Show resolved Hide resolved packages/releaser/src/IntervalEvaluator.php Outdated
Show resolved Hide resolved ...es/releaser/src/ReleaseWorker/CheckPackagesTravisBuildsReleaseWorker.php Outdated

@TomasVotruba TomasVotruba force-pushed the tv-releaser branch from 14128ba to 84d0d0f Nov 22, 2018

@vitek-rostislav

This comment has been minimized.

Copy link
Contributor

commented Nov 22, 2018

Hi, I have checked those fixups, most of them are ok, thank you! 👍 I have added a few more comments and also, I tried running the commands vendor/bin/monorepo-builder release v7.1 --dry-run but I get following errors:

In FileLoader.php line 168:
                                                                                                                                                                                                           
  Expected to find class "Shopsys\Releaser\FileProvider\DockerComposeFileProvider" in file "/var/www/html/packages/releaser/src/FileProvider/DockerComposeFileProvider.php" while importing services from  
   resource "../src", but it was not found! Check the namespace prefix used with the resource in /var/www/html/packages/releaser/config/services.yaml (which is being imported from "/var/www/html/monore  
  po-builder.yml").                                                                                                                                                                                        
                                                                                                                                                                                                           

In FileLoader.php line 161:
                                                                                                                                                                                                           
  Expected to find class "Shopsys\Releaser\FileProvider\DockerComposeFileProvider" in file "/var/www/html/packages/releaser/src/FileProvider/DockerComposeFileProvider.php" while importing services from  
   resource "../src", but it was not found! Check the namespace prefix used with the resource.  

Could you please check it? 🙂

@TomasVotruba

This comment has been minimized.

Copy link
Contributor Author

commented Nov 22, 2018

@vitek-rostislav That autoload issue should be resolved now

@TomasVotruba

This comment has been minimized.

Copy link
Contributor Author

commented Nov 22, 2018

@vitek-rostislav

Or is it possible to split the release command into "pre review" and "post review" part? Or even better, could I somehow specify, where to start?

This will be solved by --stage prepare and --stage after-release. Configurable as any stage name you want (it's a string).
I'll complete that tomorrow. Along with Confirmable

@TomasVotruba TomasVotruba force-pushed the tv-releaser branch from e5ca9ae to 09675d4 Nov 22, 2018

@vitek-rostislav

This comment has been minimized.

Copy link
Contributor

commented Nov 23, 2018

Hi @TomasVotruba, please do not forget to move the whole structure from \packages\releaser as we do not want to have it as a monorepo package (at least not now). I would simply move it to \releaser, what do you think @boris-brtan?

@TomasVotruba

This comment has been minimized.

Copy link
Contributor Author

commented Nov 23, 2018

Hey, I'd move it to at least one nested directory, like /utils/releaser or /local-packages/releaser.

Having /releaser, /doc-generator, /bla-bla dirs in the root is very confusing.

@vitek-rostislav

This comment has been minimized.

Copy link
Contributor

commented Nov 23, 2018

You are right, /utils/releaser seems ok to me 👍

@boris-brtan

This comment has been minimized.

Copy link
Contributor

commented Nov 23, 2018

@vitek-rostislav OK for me

@vitek-rostislav

This comment has been minimized.

Copy link
Contributor

commented Nov 23, 2018

Hi, update after call:

  • we will not be setting travis packages in configuration, rather we will get all the shopsys packages from travis directly
  • we will not use any special interface for resolution whether the worker should wait for human input, we will rather use SymfonyStyle in the worker itself for interaction in work() method
  • where it makes sense, it would be nice to perform git operations (creating branch, making commits, ...) automatically as well 🙂
@vitek-rostislav

This comment was marked as outdated.

Copy link
Contributor

commented Nov 23, 2018

Hi, here is the ultimate list of the steps 😉 I recommend to copy paste the raw content to https://dillinger.io/ or some other MD previewer as the github comments thread is not wide enough 🙂

Step description (including info whether is manual) Worker Stage Interaction
Check Travis build status for all packages CheckPackagesTravisBuildsReleaseWorker release-candidate When anything is reported, it should wait until I confirm that everything is resolved
Create branch "rc-version-number" CreateBranchReleaseWorker release-candidate -
[Manual] Prepare "Release highlights" post on https://blog.shopsys.com CheckReleaseBlogPostReleaseWorker release-candidate Wait for confirm
[Manual] Tell team to stop merging to master branch StopMergingToMasterReleaseWorker release-candidate Wait for confirm
Validate "require" and "require-dev" version format for all packages ValidateRequireFormatInComposerJsonReleaseWorker release-candidate When anything is reported, it should wait until I confirm that everything is resolved
Make sure that "conflict" versions in all composer.json files are closed interval ValidateConflictsInComposerJsonReleaseWorker release-candidate When anything is reported, it should wait until I confirm that everything is resolved
Check new release of "doctrine/orm" package and propose upgrade of our fork ("shopsys/doctrine-orm") if necessary CheckNewDoctrineReleaseReleaseWorker release-candidate When anything is reported, it should wait until I confirm that everything is resolved
Dump new translations with "php phing dump-translations" and commit them DumpTranslatiosReleaseWorker release-candidate 1. Do nothing when there are no changes in the po files. 2. Commit changes with "dump translations" commit message automatically when there are only deletions in the po files. 3. Wait for confirm that developer added new translations and commited them when there are new translations dumped in the po files
Resolve TODO comments in *.md files ResolveDocsTodoReleaseWorker release-candidate When anything is reported, it should wait until I confirm that everything is resolved
Dump new features to CHANGELOG.md, clean from placeholders and manually check everything is ok UpdateChangelogReleaseWorker release-candidate Wait for confirm
Update UPGRADE.md from/to headline with "vX.X" version and polish the upgrading notes UpdateUpgradeReleaseWorker release-candidate Tell user to polish the upgrading notes and wait for confirmation
Update micro-service references in all docker-composer.yml.dist from "latest" to "v7.1" version UpdateDockerComposeToVersionReleaseWorker release-candidate Commit automatically with "all shopsys Docker images are now used in X.X version instead of the latest"
Set mutual package dependencies to "^7.1" version SetMutualDependenciesToVersionReleaseWorker release-candidate Commit automatically with "all shopsys packages have now dependency on X.X version of all other shopsys packages instead of dev-master" commit message
Test your branch locally - release-candidate This should run php phing composer-dev standards tests automatically and wait for confirmation that everything was fixed when there were any issues
[Manual] Send the branch for review and tests - release-candidate Wait for confirm (this is the last step of release-candidate stage)
Check the release date of "v7.1" version is "2018-11-23" in CHANGELOG.md CheckChangelogForTodaysDateReleaseWorker release Automatically commit if necessary
[Manual] Merge branch into master - release Wait for confirm
Create and push a git tag - release Automatically
Check new version (Tom, please see the checklist for all the instructions - some of them could be automated) - after-release Wait for confirm
Update micro-service references in all docker-compose.yml.dist from "v7.1" version to "latest" UpdateDockerComposeToLatestReleaseWorker after-release Wait form confirm (Here we should create commit to master so I would prefer to do so manually
Set mutual package dependencies to "dev-master" version SetMutualDependenciesToDevMasterReleaseWorker after-release Wait form confirm (Here we should create commit to master so I would prefer to do so manually
[Manual] Create release notes for the new release on Github (link the changelog here) - after-release Wait for confirm
[Manual] Measure the performance on performator (see https://docs.google.com/document/d/1VRQOl_c2KkDekMUkLPwencUVhE3UPtvkQQSywNtjyX8/edit#heading=h.2h92hrp89r2b) - after-release Wait for confirm
[Manual] Ensure "Release highlights" is released - after-release Wait for confirm
[Manual] Post info to slack channels - #news in public Slack, #group_ssfw_news in internal Slack - link the "release highlights here" - after-release Wait for confirm
[Manual] Be happy - the new version of Shopsys Framework is released! - after-release Last step :)
@vitek-rostislav

This comment was marked as outdated.

Copy link
Contributor

commented Nov 23, 2018

Maybe this will be helpfull for displaying the whole table 🙂
automated-release.html.zip

@TomasVotruba

This comment was marked as outdated.

Copy link
Contributor Author

commented Nov 23, 2018

I thought you'd send me Google Sheets, as we use Google alraedy. Could that be done?

@TomasVotruba TomasVotruba force-pushed the tv-releaser branch 2 times, most recently from 710cf8b to 63c07a1 Nov 24, 2018

TomasVotruba and others added some commits Dec 11, 2018

renamed usages of class InterdependencyUpdater to DependencyUpdater t…
…o reflect changes in Symplify/MonorepoBuilder

- minimum required version of Symplify packages is now 5.2.17
DockerComposeFilesProvider: exclude project-base/var/postgres-data an…
…d utils/releaser folders from seacrhing for docker-compose files
TestYourBranchLocallyReleaseWorker: now it is possible to see the out…
…put of phing targets when something goes wrong and re-run the checks
added custom ComposerJsonFilesProvider as it is not possible to exclu…
…de certain folders in the orignal provider from symplify/monorepo-builder package
fixed automated committing
- AbstractShopsysReleaseWorker::hasChangesToCommit(): fixed reversed return value
- the value of commit message parameter is now backslashed using addslashes() function
- docs: added info about the necessity of mounting gitconfig into the php-fpm container
composer.json files in monorepo: "keywords" section is now formatted …
…properly

- Symplify\MonorepoBuilder\FileSystem\JsonFileManager is used when the composer.json files are modified during the release process
- the JsonFileManager formats the "keywords" section the same way
use custom DependencyUpdater as it is necessary to keep untouched the…
… dependency on shopsys/coding-standards in http-smoke-testing package

- http-smoke-testing package is dependend on obsolete version of coding-standards, not on dev-master version as all the other monorepo packages
monorepo utils folder is now checked for coding standards violations …
…in "standards", "standards-diff", "standards-fix" and "standards-fix-diff" phing commands
phpstan phing target: memory consumption is now unlimitted
- memory consumption increased by adding the monorepo utils folder in 2de4303
- see http://heimdall:8080/job/tv-releaser/44/console
UpdateUpgradeReleaseWorker now reflects the new structure of upgradin…
…g files (introduced in #699)

- monorepo upgrade file is updated
- links in general upgrade file are updated
- current unreleased upgrade file is updated and renamed to released version number
- new unreleased upgrade file is created from a template
tests of utils are now run as well when using tests-static monorepo p…
…hing task

- DockerComposeFileManipulatorTest fixed
releasing-a-new-version.md renamed to releasing-a-new-version-of-shop…
…sys-framework.md to avoid confusion

- the first sentence modified as well to make it clearer that the article describes releasing of a new version of Shopsys Framework
improved descriptions of release workers to make them clearer
- each step that requires manual input from a developer has now "[Manually]" in the description
switched last two steps of release-candidate stage
- first, we should build the project-base on Heimdall and after that is succesfull, branch should be sent to review

@vitek-rostislav vitek-rostislav force-pushed the tv-releaser branch from a76ebb9 to 085cd61 Jan 11, 2019

@vitek-rostislav vitek-rostislav merged commit 2abf7e4 into master Jan 11, 2019

@vitek-rostislav vitek-rostislav deleted the tv-releaser branch Jan 11, 2019

@TomasVotruba

This comment has been minimized.

Copy link
Contributor Author

commented Jan 16, 2019

👏 I look forward to using this in practise

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.