Skip to content

Add a doc describing how to add new RIDs to source-build #2103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 10, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Documentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ artifacts/${ARCHITECTURE}/Release/dotnet-sdk-${SDK_VERSION}-${RUNTIME_ID}.tar.gz

For example, building a 3.1.105 SDK on an x64 (aka x86\_64) platform running Fedora 32 will produce `artifacts/x64/Release/dotnet-sdk-3.1.105-fedora.32-x64.tar.gz`.

If you run into an error about an unknown `RuntimeIdentifier`, see [Adding a new Runtime Identifier to source-build](add-new-rid.md).

If you are interested in "installing" this SDK system wide or making a Linux package out of the SDK, please see [Packaging and Installation](packaging-installation.md).

`./build.{cmd|sh}` accepts a number of parameters to control the build.
Expand Down
147 changes: 147 additions & 0 deletions Documentation/add-new-rid.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
# Adding a new Runtime Identifier (RID) to source-build

This document describes how to add a new Runtime Identifier (also
called runtime id or even just RID) to source-build. You might find
this useful if you see an error like this when building source-build:

```
error NETSDK1083: The specified RuntimeIdentifier 'foo.bar-x64' is not recognized.
```

# Assumptions

- .NET (coreclr, source-build, etc) already supports this new OS.

For example, this could be a new version of a Linux distribution
that's already known to work. Or it could be a fork of another Linux
distribution which is expected to work

- source-build computes the RID correctly

Source-build computes the RID, at least on Linux, by looking at
`/etc/os-release` and running `${VERSION}.${VERSION_ID}-$(uname
-m)`. If the result of this computation itself is wrong (for example
a rolling Linux distro doesn't use `VERSION_ID`), the steps in this
doc wont be sufficient.

- We just need to add things to the RID graph

This isn't a brand new platform, just a slight tweak to an existing
one.

# Step-by-step fix

This error message is generally from the rolysn build:

```
error NETSDK1083: The specified RuntimeIdentifier 'foo.bar-x64' is not recognized.
```

It means that the SDK that's being used to build roslyn doesn't have
the RID in it's RID graph. We can fix that.

The RID is generally present in 2 places:

1. `RuntimeIdentifierGraph.json` in a built-SDK

2. `Microsoft.NETCore.Platforms/runtime.json` in the source
repositories (dotnet/runtime or dotnet/corefx)

Generally fixing the second one is sufficient.

Here's how to fix it.

1. Grab the source code of dotnet/runtime or dotnet/corefx

For .NET Core (3.1 or earlier):

```
git clone https://github.com/dotnet/corefx
cd corefx
git checkout release/$version
```

For .NET (5 or later):


```
git clone https://github.com/dotnet/runtime
cd runtime
git checkout release/$version
```

2. Add your new RID to `Microsoft.NETCore.Platforms`'s
`runtimeGroups.props` file

If you are adding a new Linux distro, the change might look
something like this:

```diff
--- a/pkg/Microsoft.NETCore.Platforms/runtimeGroups.props
+++ b/pkg/Microsoft.NETCore.Platforms/runtimeGroups.props
@@ -40,6 +40,11 @@
<TreatVersionsAsCompatible>false</TreatVersionsAsCompatible>
</RuntimeGroup>

+ <RuntimeGroup Include="exherbo">
+ <Parent>linux</Parent>
+ <Architectures>x64</Architectures>
+ </RuntimeGroup>
+
<RuntimeGroup Include="fedora">
<Parent>linux</Parent>
<Architectures>x64;arm64</Architectures>
```

If you are just adding a new version, it might look like this:

```diff
--- a/pkg/Microsoft.NETCore.Platforms/runtimeGroups.props
+++ b/pkg/Microsoft.NETCore.Platforms/runtimeGroups.props
@@ -43,7 +43,7 @@
<RuntimeGroup Include="fedora">
<Parent>linux</Parent>
<Architectures>x64;arm64</Architectures>
- <Versions>23;24;25;26;27;28;29;30;31;32;33;34</Versions>
+ <Versions>23;24;25;26;27;28;29;30;31;32;33;34;35</Versions>
<TreatVersionsAsCompatible>false</TreatVersionsAsCompatible>
</RuntimeGroup>
```

Then build dotnet/runtime or dotnet/corefx with the
`/p:UpdateRuntimeFiles=true` flag:

```
./build.sh /p:UpdateRuntimeFiles=true

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this should be
./build.sh -- /p:UpdateRuntimeFiles=true

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the dotnet/runtime release/5.0 branch, ./build.sh --help says:

...
Command line arguments starting with '/p:' are passed through to MSBuild.
...

Have you run into any cases where -- was required?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defenitly yes

dotnet-v5.0.104-SDK$ ./build.sh --help
usage: ./build.sh [options]
options:
  --with-ref-packages <dir>          use the specified directory of reference packages
  --with-packages <dir>              use the specified directory of previously-built packages
  --with-sdk <dir>                   use the SDK in the specified directory for bootstrapping
use -- to send the remaining arguments to MSBuild
dotnet-v5.0.104-SDK$ ./build.sh --with-sdk ../bootstrap/.dotnet /p:UpdateRuntimeFiles=true 
Unrecognized argument '/p:UpdateRuntimeFiles=true'
usage: ./build.sh [options]
options:
  --with-ref-packages <dir>          use the specified directory of reference packages
  --with-packages <dir>              use the specified directory of previously-built packages
  --with-sdk <dir>                   use the SDK in the specified directory for bootstrapping
use -- to send the remaining arguments to MSBuild

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I think we might be talking about different things.

I think you are looking at source-build's support/tarball/build.sh (which becomes ./build.sh when you create the tarball). That needs -- to separate normal arguments from msbuild arguments.

The step here is for passing the /p:UpdateRuntimeFiles flag to dotnet/runtime's build.sh, which doesn't need that: https://github.com/dotnet/runtime/blob/fb2db274ae16120b16c64167cc00538b2acaf155/eng/common/build.sh#L40

Do you know where I could have made that clearer in this doc?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank You very match

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be after "1. Grab the source code" add something like "(Not this repository) because I completly miss it :(

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Does the new commit make it clearer?

```

Building should result in a patch that updates
`runtimeGroups.props`, but also `runtime.json` and
`runtime.compatiblity.json`.

3. Commit the results and generate a patch:

```
git commit -a ...
git format-patch -1 HEAD
```

Here's an [example
result](https://src.fedoraproject.org/rpms/dotnet3.1/raw/43e957aabcbcef730c9433b817be947a7820ae67/f/corefx-43032-fedora-35-rid.patch)
of a real patch generated though this approach.

You might also want to create a PR to get this change included into
the main .NET repositories. That will save you the trouble of
having to update this patch yourself in the future.

4. Add that patch to source-build

For .NET Core 3.1 or earlier, add the patch to `patches/corefx/`

For .NET 5, add the patch to `patches/runtime/`

5. Build source-build

Use your regular build steps, such as `./build.sh`. source-build
will apply the new patch and build a new SDK with the new RID
added.