New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arcade layout #31560

Merged
merged 3 commits into from Dec 10, 2018

Conversation

Projects
None yet
3 participants
@tmat
Copy link
Member

tmat commented Dec 5, 2018

Build output (/artifacts)

The layout of build output is documented in Arcade docs and is the same for all repositories on Arcade SDK. Some directories are optional, e.g. repositories that do not build VS components naturally do not create VSSetup and VSSetup.obj directories.
Repositories may put additional content under VSSetup/{config}, beyond what’s documented in the Arcade SDK.
All builds outputs go under artifacts directory in the repository root.

VSSetup directory

Roslyn adds the following to VSSetup:

  • DevDivPackages: packages inserted to VS CoreXT package feed and other files inserted to VS enlistment
  • Insertion\bootstrapper, Insertion\BranchInfo, Insertion\OptProf – files needed for training of partial NGEN optimization (aka OptProf, aka IBC). This will likely change soon as our OptProf infrastructure is still evolving.
  • Installer: The installer zip (Roslyn_Preview.zip) goes here
  • VSIXes that are inserted to VS are placed under Insertion directory, those that are not inserted to VS are placed directly under VSSetup, e.g. RoslynDeployment.vsix.
  • Projects that build shipping VS components produce VSIXes of 2 flavors:
    o “Experimental”: placed to VSSetup/{config} directory
    o “SystemComponent”: placed to VSSetup/{config}/Insertion

VSSetup directory is published by official build to Azure Pipelines Artifact Store and picked up from there by Roslyn Insertion Tool. Everything needed for insertion must be places to this directory.

Engineering directory (/eng)

This directory contains scripts, targets and configuration used by the build.
The most interesting files for a Roslyn developer are:

  • cibuild.cmd/.sh
    Runs CI build (both PR validation and official builds).
  • test-build-correctness.cmd
    Validates consistency and correctness of the build.
  • test-determinism.cmd
    Validates that our build is deterministic.
  • generate-compiler-code.cmd
    Regenerates generated files used in the compilers (syntax trees, etc.).

You can run the above scripts directly from your local clone. No arguments are required, although you might want to specify configuration if Debug is not what you want to test. The scripts are designed to execute exactly the same logic as on CI machine, modulo real signing and publishing. These run in validation-only mode when executed in non-official build (locally or during PR validation).

Common engineering directory (/eng/common)

This directory contains bootstrapper scripts that are common across all Arcade repositories. The Arcade engineering system will overwrite these periodically, so don’t make any permanent changes here, they will disappear without notice.

Scripts directory (/scripts)

This directory is used for helper scripts that are manually executed and not essential for CI build.

.dotnet, .tools, .packages

These directories might be created in the repository root under certain circumstances. The bootstrapper script uses them to store downloaded .NET Core (when the required version is not found on PATH), vswhere (when building from environment that is not initialized with VS env variables) and restored NuGet packages (on CI when not using global nuget cache), respectively.

Official build artifacts publishing

The official build does not publish to cpvsbuild share anymore. Instead it publishes

  1. VS Setup Components
    These are VSIX packages that are used by VS installer built to artifacts/VSSetup/{config}/Insertion directory
  2. CoreXT packages
    Packages used by VS build – mapped to ExternalAPIs and Tools in Razzle
  3. Artifact VSSetup
    Entire artifacts/VSSetup/{config} directory.
  4. Artifact Packages
    Entire artifacts/packages/{config} directory

Fixes #23547

@tmat tmat requested review from dotnet/roslyn-compiler as code owners Dec 5, 2018

@tmat tmat referenced this pull request Dec 5, 2018

Merged

Arcade build scripts #31511

@tmat tmat force-pushed the tmat:Layout branch from b51f9f1 to 54a1ef2 Dec 6, 2018


if ($launch) {
$devenvExe = Join-Path $env:VSINSTALLDIR 'Common7\IDE\devenv.exe'
&$devenvExe /rootSuffix RoslynDev

This comment has been minimized.

@jaredpar

jaredpar Dec 6, 2018

Member

&$devenvExe /rootSuffix RoslynDev [](start = 7, length = 34)

Should use Exec-Process here as the exit code isn't validated.

This comment has been minimized.

@tmat

tmat Dec 6, 2018

Member

Not sure why we have it but I'll update it to use Exec-Process

@jaredpar

This comment has been minimized.

Copy link
Member

jaredpar commented Dec 6, 2018

tar czf mono-${VERSION}.tar.gz mono

Why do we still have a scripts directory at all after this change?


Refers to: scripts/build_mono.sh:16 in 54a1ef2. [](commit_id = 54a1ef2, deletion_comment = False)

@jaredpar
Copy link
Member

jaredpar left a comment

:shipit:

@tmat

This comment has been minimized.

Copy link
Member

tmat commented Dec 6, 2018

Why do we still have a scripts directory at all after this change?

I moved all scripts that are not used in build/CI there. These can definitely use some clean up.

PublicApi
UploadAzureZip
GenerateSdkPackages
build_mono.sh
check-for-loc-changes.cmd
check-for-loc-changes.ps1
cleanup_perf.ps1
clean_benchview_tools.cmd
crossgen.sh
edit-designers.ps1
generate-badges.ps1
generate-compiler-code.cmd
generate-compiler-code.ps1
install_benchview_tools.cmd
README.md
Run-TestsWithProcessWatchdog.ps1
run_perf.ps1
sort-unshipped.cmd
test-nuget-cache.ps1

@tmat tmat force-pushed the tmat:Layout branch 4 times, most recently from 89881e1 to 0913d4e Dec 7, 2018

@tmat tmat force-pushed the tmat:Layout branch from 77d36dc to 4eae118 Dec 9, 2018

@tmat tmat changed the title WIP: Arcade layout Arcade layout Dec 10, 2018

@tmat tmat force-pushed the tmat:Layout branch from 4ff156d to c841b3b Dec 10, 2018

@tmat tmat force-pushed the tmat:Layout branch from e042dfd to 97c5806 Dec 10, 2018

@tmat tmat merged commit d7e4939 into dotnet:master Dec 10, 2018

2 of 3 checks passed

roslyn-integration-CI in progress
Details
license/cla All CLA requirements met.
Details
roslyn-CI #20181210.6 succeeded
Details

@tmat tmat deleted the tmat:Layout branch Dec 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment