Add guidance for repo owners

Documentation/sourcebuild-in-repos/README.md

@@ -0,0 +1,12 @@
+# Repo owner's handbook for source-build
+
+These docs aim to be a collection of useful information for repo owners who work
+with and participate in source-build.
+
+## Index
+
+* [Adding new dependencies](new-dependencies.md)
+* [Updating dependencies](update-dependencies.md)
+* [Considerations when adding features](adding-features.md)
+* [General notes on the source-build build](build-info.md)
+* [Adding new repositories](new-repo.md)

Documentation/sourcebuild-in-repos/adding-features.md

@@ -0,0 +1,43 @@
+# New features
+
+New features are great!  They also have source-build considerations though.
+.NET is no longer just multi-platform but also multi-distribution: there's
+a Linux SDK produced by Microsoft, but there's also a Linux SDK produced
+by Red Hat, one produced by Fedora, and one that anyone in the community
+can build himself from source.
+
+## Things to consider
+
+New features, or expansions of current features, should act sensibly
+across Windows, Linux, and OSX.  This also involves taking into account
+the limitations and conventions of the different platforms - for instance,
+on Linux, it's typical for the .NET SDK to be installed by root and
+therefore be unchangeable by normal users, so installing global tools
+needs to take into account the fact that the user might not be able to
+add anything to the SDK directories (see
+[this docker global tools issue](https://github.com/dotnet/dotnet-docker/issues/520)
+and [a similar installer issue](https://github.com/dotnet/installer/issues/7069)).
+
+New features also need to be compatible across all distributions of
+.NET - for instance, Fedora and Debian cannot distribute *any*
+non-open-source code, even samples, tests, or minor additions to
+licenses like restricting the field that the code can be used in.
+This includes any code or packages that are used to build product
+code as well.  Microsoft generally prefers the MIT license.
+
+One example of a licensing issue that source-build found was with
+dotnet-check and dotnet-format: these tools were brought into the
+product build but had a dependency that was not licensed compatibly -
+even without the source-build aspect.  We had to scramble to replace
+the dependency before the release.  Dependencies need to be carefully
+checked for license compatibility as well.
+
+## Resources
+
+Fedora's approved open-source licenses can be found
+[on their wiki](https://fedoraproject.org/wiki/Licensing:Main#Good_Licenses),
+or  you can check on the [OSI-approved list of licenses](https://opensource.org/licenses/alphabetical).
+
+If you would like some of our distribution maintainer partners to
+review your new feature you can ping @dotnet/distro-maintainers
+on a PR or issue.

Documentation/sourcebuild-in-repos/build-info.md

@@ -0,0 +1,50 @@
+# Source-build build info
+
+This is a collection of notes about how source-build can differ in general
+from your repo's build and what kind of issues that can create.
+
+## Single-version and single-RID build
+
+Source-build is required to build on a single machine with no internet
+access.  This means that we build targeting a single RID, usually the
+non-portable RID for the build machines (like rhel.7-x64).  We do
+support building portable (linux-x64) as well - this is useful for
+bootstrapping new distributions.
+
+Source-build cannot build with any *prebuilts*.  This is our term for
+any package that comes from *outside the current source-build*.  This means
+that everything that ships out of source-build and everything that is used to
+build those shipping products must come from the source-build in progress.
+Packages from nuget.org, Microsoft builds, or other unrelated source-builds
+cannot be used in source-build except in a limited bootstrapping process.
+
+Source-build supplies a *previously-source-built* set of packages for this
+bootstrapping process.  This is one way we have of breaking cycles in the
+build.  However, none of these packages can make it to the final build output.
+This also means that your repo should be buildable with the immediately
+previous version of the SDK than you are building for; i.e., if you are
+building for 6.0.103, everything should be buildable with the 6.0.102 SDK.
+
+We also only build one version of each repo.  This means that if your repo
+turns production of some packages on and off, for instance, if you only
+produce packages if they are changed, source-build will need a workaround
+to force all packages to be produced.  Additionally, we can only supply
+one version of each package to a repo.  This is injected into the
+`$({PackageName}PackageVersion)` variables, e.g. SystemReflectionMetadataPackageVersion.
+One exception is reference-only packages -
+[dotnet/source-build-reference-packages](https://github.com/dotnet/source-build-reference-packages)
+produces multiple versions and these can be hard-coded or use a
+`$(<PackageName>ReferenceVersion)` property or similar if you don't
+need source-build to change them.
+
+## Platform-specific packages
+
+Packages that require components or packages only available on some other
+operating system than the building OS cannot be built in source-build.
+These should use `<ExcludeFromSourceBuild>true</ExcludeFromSourceBuild>` or
+other options to be excluded from the source-build.  For instance, if a
+project depends on a package that can only be built on Windows, it will need
+to be disabled or worked around in source-build.  As an example,
+[Roslyn removes](https://github.com/dotnet/roslyn/blob/b999a65c8b0feeccb2b58da3d7a6e80e5f08feab/src/Workspaces/Core/Portable/Storage/PersistentStorageExtensions.cs#L23)
+a small performance improvement when building for source-build because it
+requires a component that isn't available.

Documentation/sourcebuild-in-repos/new-dependencies.md

@@ -0,0 +1,118 @@
+# Adding a new dependency
+
+## Basics
+
+When adding a new dependency, there are a few steps common between any type
+of dependency.  If you use darc, `darc add-dependency` will take care of
+this process for you.  Otherwise:
+
+1. Add the package and a default version to eng/Versions.props.  The default
+  version property should be named `<PackageName>PackageVersion`,
+  e.g. `SystemCollectionsImmutablePackageVersion`.
+1. Add the package and a repo reference to eng/Version.Details.xml.  This
+  should include a SourceBuild metadata entry, looking something like:
+
+```xml
+   <Dependency Name="Microsoft.Net.Compilers.Toolset"
+                  Version="4.2.0-3.22205.5"
+                  CoherentParentDependency="Microsoft.NET.Sdk">
+      <Uri>https://github.com/dotnet/roslyn</Uri>
+      <Sha>0167599e0e1634ea3ed8d0e41390a3c0d9b3e4e9</Sha>
+      <SourceBuild RepoName="roslyn" ManagedOnly="true" />
+    </Dependency>
+```
+
+1. Set up dependency flow from the foreign repo to your repo.
+
+In general, you should aim to use one version of each package.  If you are
+using a package as reference-only, it is possible to use multiple versions,
+but only one implementation version of each package will be used -
+source-build will override it to the version that is being built in this
+version of the SDK.
+
+The source-build metadata is important - this tells source-build which repo
+package contains the specific nupkg you want.
+
+Another uncommon case that can cause problems is when the repo `<Dependency>`
+version and the source-built intermediate package version don't match.
+In this case, a direct dependency should be added to the source-build
+intermediate package and the CoherentParentDependency attribute should be
+set to the repo that consumes the dependency. For an example,
+see [installer's F# dependency](https://github.com/dotnet/installer/blob/ba1739a2363b1062f03ea386ec67174c6468d3b2/eng/Version.Details.xml#L128).
+You can find the version needed by running `darc get-build`
+or using [BAR](https://aka.ms/bar).
+
+## Adding dependencies
+
+Source build classifies dependencies in the following ways
+
+1. .NET - a dependency on a component from the .NET org - e.g. dotnet/runtime
+1. Microsoft - a dependency on a component from the Microsoft org - e.g. microsoft/vstest
+1. External - a dependency on a component outside of Microsoft/.NET - e.g. JamesNK/Newtonsoft.Json
+
+The following checklist can be used to determine how to handle each type of
+dependency and the nuances it may have.
+
+1. Are you already using a package from the same repo and is the new
+  dependency already source-built?  You can determine this by checking if
+  the repo has a SourceBuild leg building, e.g. runtime's is
+  `runtime-dev-innerloop (Build Linux x64 release SourceBuild)`.
+
+    1. This is the simplest case.  You can add new dependencies from repos
+      you are already using freely.  Note that you need to use the same version
+      for all packages from the repo.
+1. Is the repo already built in source-build including the specific
+  package you want?
+    1. Add the dependency using `darc add-dependency` as normal, then
+      add the [source-build metadata](#Basics) as above.
+1. Is the repo already built in source-build but the specific package is not?
+    1. There's probably an issue with source-building this package.  Please
+      talk to a [source-build team member](https://github.com/orgs/dotnet/teams/source-build-internal)
+      about why that is and whether we can fix it.
+1. Is this a repo that uses Arcade to build?
+    1. Does the foreign repo depend on your repo, directly or indirectly?
+      i.e. would adding the dependency create a cycle?
+        1. This isn't necessarily a deal-breaker - it can sometimes be worked
+          around with reference-only packages.  Please contact a
+          [source-build team member](https://github.com/orgs/dotnet/teams/source-build-internal)
+          to discuss.
+    1. Does the foreign repo publish to BAR?
+        1. If not, please contact them to get them publishing to BAR
+          in an appropriate channel.
+    1. If neither of these caveats apply you should be in good shape.
+      Follow the instructions under "Basics" above.
+1. Dependencies that have no code (e.g. SDKs with just props and targets) can
+  usually be added using the [source-build text-only-package process](https://github.com/dotnet/source-build-reference-packages/tree/main/src/textOnlyPackages/src).
+  If the package is not already included there, you can [open a PR](https://github.com/dotnet/source-build-reference-packages/pulls)
+  or [file an issue](https://github.com/dotnet/source-build/issues/new/choose)
+  to include it.
+1. Source-build has in the past used Arcade shims to allow non-Arcade repos
+  to build appropriate packages for source-build.  Please
+  [log an issue](https://github.com/dotnet/source-build/issues/new/choose)
+  to determine if this is a workable approach for your foreign repo.
+1. We build some external dependencies in the [dotnet/source-build-externals](https://github.com/dotnet/source-build-externals)
+  repo.  Good targets for this generally have very few if any dependencies and
+  very simple build processes.
+  [Please log an issue](https://github.com/dotnet/source-build/issues/new/choose)
+  to get the process started.
+
+## Deciding what version to use
+
+- If you are using the package as reference-only and want the version to be
+  pinned, use a literal version number in the csprojs that reference the
+  project.  You can also set up a reference-only package version variable
+  in eng/Versions.props, for instance `<PackageNameReferenceOnly>1.2.3</PackageNameReferenceOnly>`
+  in addition to `<PackageNamePackageVersion>4.5.6</PackageNamePackageVersion>`.
+  Also verify that the package is available in
+  [source-build-reference-packages](https://github.com/dotnet/source-build-reference-packages),
+  and if not, [file a source-build issue](https://github.com/dotnet/source-build/issues).
+- If you are using the package in the actual build or want the version
+  to be updated whenever the foreign repo publishes to your channel, use
+  the version number property set up in eng/Versions.props.  When performing
+  a source-build, the version number will get updated to the current
+  source-built version.
+- If you are using an external or non-Arcade package, please coordinate as
+  much as possible with other teams using that package.  Each
+  package-version is essentially maintained as a separate concern, so
+  something like repo A requiring Newtonsoft.Json 9.0.1 and repo B requiring
+  12.0.2 essentially doubles the source-build team's work.