Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross-platform CI scripts #21081

Merged
merged 9 commits into from Sep 14, 2020
Merged

Conversation

aminya
Copy link
Contributor

@aminya aminya commented Jul 16, 2020

Description of the change

  • This uses Powershell 7 as a cross-platform shell for CI scripts:

    • PowerShell 7 allows using the same syntax and code across all the operating systems.
    • This simplifies CI scripts. For example, previously CMD was used in Windows, which is an old shell with limited functionality.
    • The old linux_trusty container did not detect pwsh, so we use the Linux image provided by Microsoft, and we use Clang 9 that is provided in it.
  • This refactors CI scripts into separate files (templates):

    • It also allows using parallelization of the tests without copy-pasting the same code. For example:

      • In macOS, the same templates are reused in the test phase.
      • In Windows x86, the templates are used for parallelization of tests in this PR
    • This makes it easier to manage CI code. For example, we can quickly compare the publish step for all operating systems without going through big files and trying to find the relevant steps.

Previously, the CI scripts were written back in ~2017. At that time, Azure Templates had not been introduced yet, and because of that using PowerShell was not suitable yet. Now in 2020, we can benefit from these and improve the CI. This is also a requirement for the future Test Parallelization pull request.

Verification

The CI passes. Tested in atom-ide-community

Release Notes

N/A

### Includes
This includes #21023.
This supersedes #21057.

### Thing you should do
Because this includes #21023, it requires you to set SHOULD_SIGN=true as an environment variable inside Azure.

Fork Pull request

atom-community#46

@sadick254
Copy link
Contributor

sadick254 commented Aug 17, 2020

@aminya Great job 👍

Since this is a huge refactor I would recommend splitting it into multiple PR's(1 for each template). That way it becomes easy to review and track the impact the changes have on our CI.

Am happy to assist with that if need be.

@aminya
Copy link
Contributor Author

aminya commented Aug 17, 2020

@sadick254 You're welcome!

The commits are granular, so they can be grouped together and cherry-picked if needed.

Start by merging #21023 and #21057. Although the code is overwritten later in this PR, the functionality is the same and it also helps to clear the commit list from this PR (first 5 commits). Merging these two helps to cherry-pick the remaining commits.

@sadick254
Copy link
Contributor

sadick254 commented Aug 25, 2020

@aminya I see this now has conflicts that need to be resolved.

@aminya aminya force-pushed the CI_templates_4upstream branch 4 times, most recently from 331182a to 9e2a7f9 Compare Aug 25, 2020
@aminya
Copy link
Contributor Author

aminya commented Aug 25, 2020

@aminya I see this now has conflicts that need to be resolved.

@sadick254 I rebased this! The next step is merging #21057

@sadick254
Copy link
Contributor

sadick254 commented Aug 26, 2020

@aminya #21057 has been merged. You might need to rebase again.

@aminya aminya force-pushed the CI_templates_4upstream branch 3 times, most recently from 0024492 to 0f9398a Compare Aug 26, 2020
@aminya aminya mentioned this pull request Aug 26, 2020
@aminya aminya force-pushed the CI_templates_4upstream branch 2 times, most recently from e88a7dc to 0ceda6c Compare Aug 26, 2020
@aminya
Copy link
Contributor Author

aminya commented Aug 26, 2020

@sadick254 Thanks! I cherry-picked the preparation template. The next step is merging #21221

@aminya aminya force-pushed the CI_templates_4upstream branch 3 times, most recently from b887d00 to 7c59464 Compare Aug 28, 2020
@aminya aminya mentioned this pull request Aug 28, 2020
@aminya aminya force-pushed the CI_templates_4upstream branch from 7c59464 to 446692d Compare Aug 28, 2020
@aminya
Copy link
Contributor Author

aminya commented Aug 28, 2020

@sadick254 This is rebased again. The next one is #21226

@aminya aminya force-pushed the CI_templates_4upstream branch 2 times, most recently from 08934ca to 60528b7 Compare Aug 29, 2020
@aminya aminya force-pushed the CI_templates_4upstream branch from 618cbd6 to a579fb9 Compare Sep 5, 2020
@aminya aminya mentioned this pull request Sep 5, 2020
@aminya aminya force-pushed the CI_templates_4upstream branch 2 times, most recently from ce128ff to 63626df Compare Sep 5, 2020
@aminya
Copy link
Contributor Author

aminya commented Sep 5, 2020

@sadick254 The next one is #21262

@aminya aminya force-pushed the CI_templates_4upstream branch 2 times, most recently from aa45e57 to 67d733b Compare Sep 11, 2020
@aminya
Copy link
Contributor Author

aminya commented Sep 11, 2020

@sadick254 This is rebased. I think this can go in one PR. It has the test-template + miscellaneous changes (in the last 5 commits).

Let me know if you want me to break this into two PR.

@aminya aminya force-pushed the CI_templates_4upstream branch from 67d733b to 4a03165 Compare Sep 11, 2020
@sadick254
Copy link
Contributor

sadick254 commented Sep 11, 2020

@aminya Breaking it to two PR's would be great.

@aminya aminya force-pushed the CI_templates_4upstream branch from 4a03165 to 5234311 Compare Sep 11, 2020
@aminya aminya mentioned this pull request Sep 11, 2020
@sadick254
Copy link
Contributor

sadick254 commented Sep 11, 2020

This needs another rebasing. @aminya

@aminya aminya force-pushed the CI_templates_4upstream branch from 5234311 to 9a56ebd Compare Sep 11, 2020
We already install the npm version we want in previous steps

#140

Co-Authored-By: DeeDeeG <DeeDeeG@users.noreply.github.com>
@aminya aminya force-pushed the CI_templates_4upstream branch from 0221509 to 5eb5058 Compare Sep 11, 2020
@aminya aminya force-pushed the CI_templates_4upstream branch from 25f2cc4 to a1ca3e7 Compare Sep 11, 2020
DeeDeeG and others added 5 commits Sep 11, 2020
We can use this to run `get-release-version.js`
with the `--nightly` flag on the "Atom Nightly" pipeline.
We no-longer run the Linux build in an old Ubuntu 14.04 "Trusty Tahr"
Docker container. We run builds directly in a virtualized Ubuntu 16.04
"Xenial Xerus" image now.
This old bug in the CI inexplicably went away at some point.

Co-Authored-By: DeeDeeG <DeeDeeG@users.noreply.github.com>
@aminya aminya force-pushed the CI_templates_4upstream branch from a1ca3e7 to ea1283f Compare Sep 11, 2020
@aminya
Copy link
Contributor Author

aminya commented Sep 11, 2020

@sadick254 Done! Now, this can be merged.

Copy link
Contributor

@sadick254 sadick254 left a comment

This looks great. Thanks @aminya and @DeeDeeG for the great collaboration on this one. 👍

@sadick254 sadick254 merged commit 9d65b79 into atom:master Sep 14, 2020
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants