Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reproducible build of 1.1.12.9 did not match #6315

Closed
achow101 opened this issue Sep 6, 2021 · 39 comments
Closed

Reproducible build of 1.1.12.9 did not match #6315

achow101 opened this issue Sep 6, 2021 · 39 comments
Labels

Comments

@achow101
Copy link

achow101 commented Sep 6, 2021

I attempted to perform a reproducible build of 1.1.12.9 on Windows but the resulting binaries did not match what was published. I think the issue is likely due to a different SDK version than what was used to build the release as many dlls seem to come from the system doing the build. However no specific SDK was specified in the instructions.

@yahiheb yahiheb added the debug label Sep 6, 2021
@yahiheb
Copy link
Collaborator

yahiheb commented Sep 6, 2021

The latest Wasabi v1.1.12.9 was release on the 3rd Jun.
The latest version of the SDK at that time was .NET 5.0.6 which can be found here https://dotnet.microsoft.com/download/dotnet/5.0.
I assume that is the one that was used .

@kiminuo
Copy link
Collaborator

kiminuo commented Sep 9, 2021

So Andrew is right.

I have tried to create the same build and ultimately I failed too. However, I believe the following steps make sense:

Now only a few files are different and I have verified that the differences are inconsequential (if I didn't miss anything). I'm saying that just for the record & debugging purposes, not to imply that it's OK for customers.

With .NET 5, I can see that a lot more changes than with .NET Core 3.1 build.

@molnard Could you possibly show dotnet --info output from the machine where you build Wasabi Wallet releases, please?

@eriknylund
Copy link

eriknylund commented Sep 12, 2021

I can confirm I see the same issues, which are a bit related to the issues we were having as we tested and prepared for the v1.1.12 release a year ago. I managed to build v1.1.12.9 but I end up with a diff in WalletWasabi.dll and WalletWasabi.Gui.dll on Windows and in WalletWasabi.Gui.deps.json on macOS. I tried building the two earlier releases and see the same diff. I went back and rebuilt the v1.1.12 release and the binaries match, so I'll continue building the releases in between to see how it goes.

I can also see that the automated pipeline was last successful on July 23rd: https://dev.azure.com/zkSNACKs/Wasabi/_build/results?buildId=57559&view=results so that may also be a point in time worth checking if something changed.

Here's the steps from my v1.1.12.9 repro build on Windows 10:

PS C:\Users\erik\Downloads> Get-FileHash -Path .\Wasabi-1.1.12.9.msi

Algorithm       Hash                                                                   Path
---------       ----                                                                   ----
SHA256          91CA39A9CB8ADA3B8F125D55358DB8FE3EE605B02496ABFD094C77551A79458F       C:\Users\erik\Downloads\Wasab...
erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.12.9))
$ git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"
diff --git a/win7-x64/WalletWasabi.Gui.dll b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll
index c8d2eea..f71542e 100644
Binary files a/win7-x64/WalletWasabi.Gui.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll differ
diff --git a/win7-x64/WalletWasabi.dll b/C:/Program Files/WasabiWallet/WalletWasabi.dll
index 1cdfe16..59a75e7 100644
Binary files a/win7-x64/WalletWasabi.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.dll differ

Diff on macOS, ignoring file permission with -G.

erik@MacBookPro151-BigSur-1152 Downloads % gpg --verify Wasabi-1.1.12.9.dmg.asc 
gpg: assuming signed data in 'Wasabi-1.1.12.9.dmg'
gpg: Signature made Wed Jun  2 17:39:35 2021 EEST
gpg:                using RSA key 6FB3872B5D42292F59920797856348328949861E
gpg: Good signature from "zkSNACKs <zksnacks@gmail.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 6FB3 872B 5D42 292F 5992  0797 8563 4832 8949 861E

erik@MacBookPro151-BigSur-1152 Downloads % shasum -a 256 Wasabi-1.1.12.9.dmg 
1e44aa928d6001be593d6665432082d85da02f762f8b6e7e473612728167b9b4  Wasabi-1.1.12.9.dmg

erik@MacBookPro151-BigSur-1152 dist % ls -la
total 0
drwxr-xr-x    6 erik  staff    192 Sep 12 18:28 .
drwx------@  11 erik  staff    352 Sep 12 18:26 ..
drwxr-xr-x@   3 erik  staff     96 Jun  2 17:28 Wasabi Wallet.app
drwxr-xr-x  398 erik  staff  12736 Sep 12 18:23 linux-x64
drwxr-xr-x  400 erik  staff  12800 Sep 12 18:23 osx-x64
drwxr-xr-x  539 erik  staff  17248 Sep 12 18:22 win7-x64

Without the -G. argument:

erik@MacBookPro151-BigSur-1152 dist % git diff --no-index osx-x64/ Wasabi\ Wallet.App/Contents/MacOS/ 
diff --git a/osx-x64/AvalonStudio.Shell.Extensibility.dll b/Wasabi Wallet.App/Contents/MacOS/AvalonStudio.Shell.Extensibility.dll
old mode 100755
new mode 100644
diff --git a/osx-x64/AvalonStudio.Shell.dll b/Wasabi Wallet.App/Contents/MacOS/AvalonStudio.Shell.dll
old mode 100755
new mode 100644
diff --git a/osx-x64/AvalonStudio.Utils.dll b/Wasabi Wallet.App/Contents/MacOS/AvalonStudio.Utils.dll
old mode 100755
new mode 100644 

Adding -G. to ignore file permissions:

erik@MacBookPro151-BigSur-1152 dist % git diff -G. --no-index osx-x64/ Wasabi\ Wallet.App/Contents/MacOS/       
diff --git a/osx-x64/WalletWasabi.Gui.deps.json b/Wasabi Wallet.App/Contents/MacOS/WalletWasabi.Gui.deps.json
old mode 100755
new mode 100644
index 5f4ff06..e2c9d52
--- a/osx-x64/WalletWasabi.Gui.deps.json
+++ b/Wasabi Wallet.App/Contents/MacOS/WalletWasabi.Gui.deps.json       
@@ -5113,20 +5113,8 @@
       "any",
       "base"
     ],
-    "osx.10.16-x64": [
-      "osx.10.16",
-      "osx.10.15-x64",
-      "osx.10.15",
-      "osx.10.14-x64",
-      "osx.10.14",
-      "osx.10.13-x64",
-      "osx.10.13",
-      "osx.10.12-x64",
-      "osx.10.12",
-      "osx.10.11-x64",
-      "osx.10.11",
-      "osx.10.10-x64",
-      "osx.10.10",
+    "osx.11-x64": [^M
+      "osx.11",^M
       "osx-x64",
       "osx",
       "unix-x64",
@@ -5136,20 +5124,8 @@
     ],
     "osx.11.0-x64": [
       "osx.11.0",
-      "osx.10.16-x64",
-      "osx.10.16",
-      "osx.10.15-x64",
-      "osx.10.15",
-      "osx.10.14-x64",
-      "osx.10.14",
-      "osx.10.13-x64",
-      "osx.10.13",
-      "osx.10.12-x64",
-      "osx.10.12",
-      "osx.10.11-x64",
-      "osx.10.11",
-      "osx.10.10-x64",
-      "osx.10.10",
+      "osx.11-x64",^M
+      "osx.11",^M
       "osx-x64",
       "osx",
       "unix-x64",

@eriknylund
Copy link

eriknylund commented Sep 12, 2021

v1.1.12.6:

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.12.6))
$ git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"
diff --git a/win7-x64/WalletWasabi.Gui.dll b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll
index a8a7558..5ffe2c6 100644
Binary files a/win7-x64/WalletWasabi.Gui.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll differ
diff --git a/win7-x64/WalletWasabi.dll b/C:/Program Files/WasabiWallet/WalletWasabi.dll
index e953074..412de43 100644
Binary files a/win7-x64/WalletWasabi.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.dll differ

v1.1.12.5:

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.12.5))
$ git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"
diff --git a/win7-x64/WalletWasabi.Gui.dll b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll
index 89d9af9..c661201 100644
Binary files a/win7-x64/WalletWasabi.Gui.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll differ
diff --git a/win7-x64/WalletWasabi.dll b/C:/Program Files/WasabiWallet/WalletWasabi.dll
index 49abee2..6fa344b 100644
Binary files a/win7-x64/WalletWasabi.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.dll differ

v1.1.12.4 (SDK changed to 3.1.403!):
@molnard Looks like it was actually built with SDK 3.1.404 rather than SDK 3.1.403 🤔

--- a/win7-x64/WalletWasabi.Gui.deps.json
+++ b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.deps.json
@@ -15,666 +15,666 @@
           "Dock.Avalonia.Themes.Metro": "0.9.9",
           "System.Runtime": "4.3.1",
           "WalletWasabi": "1.1.12.4",
-          "runtimepack.Microsoft.NETCore.App.Runtime.win-x64": "3.1.9",
-          "runtimepack.Microsoft.AspNetCore.App.Runtime.win-x64": "3.1.9"
+          "runtimepack.Microsoft.NETCore.App.Runtime.win-x64": "3.1.10",
+          "runtimepack.Microsoft.AspNetCore.App.Runtime.win-x64": "3.1.10"
         },

v1.1.12.3 (SDK 3.1.403):

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.12.3))
$ git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"
diff --git a/win7-x64/WalletWasabi.Gui.dll b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll
index 25d6d54..75149e6 100644
Binary files a/win7-x64/WalletWasabi.Gui.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll differ
diff --git a/win7-x64/WalletWasabi.dll b/C:/Program Files/WasabiWallet/WalletWasabi.dll
index 463b141..ab34153 100644
Binary files a/win7-x64/WalletWasabi.dll and b/C:/Program Files/Wasabi

v1.1.12.2 (SDK 3.1.403):

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.12.2))
$ git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"

NO diff here. So we can see that things started misbehaving after v1.1.12.2.

v1.1.12.1 (SDK 3.1.403):

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.12.1))
$ git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"

No diff on v1.1.12.1 either.

@eriknylund
Copy link

eriknylund commented Sep 12, 2021

Attempted to build v1.1.12.4 with the newer SDK 3.1.404:

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Packager ((v1.1.12.4))
$ dotnet --list-sdks
3.1.404 [C:\Program Files\dotnet\sdk]

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.12.4))
$ git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"
diff --git a/win7-x64/WalletWasabi.Gui.dll b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll
index eec517d..9b9ada8 100644
Binary files a/win7-x64/WalletWasabi.Gui.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll differ
diff --git a/win7-x64/WalletWasabi.dll b/C:/Program Files/WasabiWallet/WalletWasabi.dll
index 8a7e089..c0165b4 100644
Binary files a/win7-x64/WalletWasabi.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.dll differ

Output diff looks more consistent with the other builds, still not as expected.

@molnard
Copy link
Collaborator

molnard commented Sep 13, 2021

@kiminuo

[.NET SDK (reflecting any global.json):
 Version:   5.0.301
 Commit:    ef17233f86

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19042
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.301\

Host (useful for support):
  Version: 5.0.7
  Commit:  556582d964

.NET SDKs installed:
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.1.802 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]
  3.0.100 [C:\Program Files\dotnet\sdk]
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.201 [C:\Program Files\dotnet\sdk]
  3.1.302 [C:\Program Files\dotnet\sdk]
  3.1.403 [C:\Program Files\dotnet\sdk]
  3.1.407 [C:\Program Files\dotnet\sdk]
  5.0.100 [C:\Program Files\dotnet\sdk]
  5.0.301 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]](`url`)

@molnard
Copy link
Collaborator

molnard commented Sep 13, 2021

@achow101 @eriknylund thank you for bringing this up. We are taking deterministic builds seriously - however, it is possible that I made a mistake somewhere. I will take a look at it later this week. In the current master, det builds are verified automatically but the backport version is manual.

You will need Windows!

you need git package, Windows 10, and the version of .NET Core SDK

https://github.com/zkSNACKs/WalletWasabi/blob/master/WalletWasabi.Documentation/Guides/DeterministicBuildGuide.md#1-assert-correct-environment

I am building Wasabi on Windows with WSL. Unfortunately, .NET Core is building different binaries on different OS. Maybe did it this way, then disregard this comment.

Backport (current releases)

Dotnet version can be found in the Release notes
According to that Wasabi v1.1.12.9 - Updates was built with .NET Core 3.1.407-win-x64..

macOS

However, there are some extra notes, for example on macOS, it is impossible to have deb build because of code signing.
#4110

As a walkaround, we agreed to provide unsigned binaries, that one can download, verify and use Wasabi.

Wasabi 2.0

In progress: #5409

Answers

@eriknylund Looks like it was actually built with SDK 3.1.404 rather than SDK 3.1.403 Impossible, I checked the release environment and SDK is not installed.

@eriknylund
Copy link

eriknylund commented Sep 13, 2021

I am building Wasabi on Windows with WSL. Unfortunately, .NET Core is building different binaries on different OS. Maybe did it this way, then disregard this comment.

I built it on a clean Windows 10 with the latest updates, installing only Git and the SDK, plus the relevant MSI package to diff against.

I could give WSL a go to see if it makes a difference, but I would perhaps need more specific details on OS to make it match. Also it makes the setup of the build machine more complicated. I wasn't able to build it at all on macOS on my first try, ran into an exception with paths.

@eriknylund
Copy link

eriknylund commented Nov 4, 2021

I tried building v1.1.13.0 and see a similar pattern to my attempt at v1.1.12.4 #6315 (comment) where @molnard later verified that the scenario is impossible because that SDK version is not installed. If I use the SDK in the release notes (.NET Core 3.1.407-win-x64) I get a big diff pointing to SDK 3.1.412 being used for the build. If I retry with SDK 3.1.412 I get the smaller diff in only WalletWasabi.Gui.dll and WalletWasabi.dll, a diff consistent with every build since v1.1.12.2.

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.13.0))
$ dotnet --list-sdks
3.1.412 [C:\Program Files\dotnet\sdk]

erik@DESKTOP-E0BTURI MINGW64 ~/WalletWasabi/WalletWasabi.Gui/bin/dist ((v1.1.13.0))
$ git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"
diff --git a/win7-x64/WalletWasabi.Gui.dll b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll
index f5efb30..0633cb3 100644
Binary files a/win7-x64/WalletWasabi.Gui.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll differ
diff --git a/win7-x64/WalletWasabi.dll b/C:/Program Files/WasabiWallet/WalletWasabi.dll
index d3ec3c1..c58828e 100644
Binary files a/win7-x64/WalletWasabi.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.dll differ

The references to SDK 3.1.412 can be seen in the installed version by opening C:\Program Files\WasabiWallet\WalletWasabi.Gui.runtimeconfig:

{
  "runtimeOptions": {
    "tfm": "netcoreapp3.1",
    "includedFrameworks": [
      {
        "name": "Microsoft.NETCore.App",
        "version": "3.1.18"
      },
      {
        "name": "Microsoft.AspNetCore.App",
        "version": "3.1.18"
      }
    ],
    "configProperties": {
      "System.Globalization.Invariant": true
    }
  }
}

The version number 3.1.18 above points to https://github.com/dotnet/core/blob/main/release-notes/3.1/3.1.18/3.1.18.md?WT.mc_id=dotnet-35129-website - The .NET Core Runtime 3.1.18, .NET Core SDK 3.1.412

Could it somehow be that the runtime on the build env is 3.1.18, but the SDK used for the build is as specified in the release notes? 🤔 In case the build environment get regular updates and the runtime is updated as a part of it...

$ dotnet --list-runtimes
Microsoft.AspNetCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

UPDATE: I tried simulating that by installing the runtime using the installer, then manually replacing the SDK with the older one. The build output is the same as just installing the older SDK. I added the full diff here in case anyone's interested: https://gist.github.com/eriknylund/121169e33c0619742ab4c8379c60255d

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 4, 2021

I agree with @eriknylund, I think 3.1.412 .NET SDK was used to do the actual build as I got the smallest diff when installing .NET 3.1.18 from https://dotnet.microsoft.com/download/dotnet/3.1. The specified SDK got me much larger diff as Erik observed.

Anyway, now WalletWasabi.Gui.dll and WalletWasabi.Gui.dll differ. I took a look at WalletWasabi.dll and all differences are basically similar to this one in WalletWasabi\Blockchain\Transactions\TransactionFactory.cs:

image

So for some reason some temporary variables are created for a function call. I have tried to find out why that may happen but at the moment I don't know so I will try to ask somebody.

@eriknylund
Copy link

eriknylund commented Nov 4, 2021

I agree with @eriknylund, I think 3.1.412 .NET SDK was used to do the actual build as I got the smallest diff when installing .NET 3.1.18 from https://dotnet.microsoft.com/download/dotnet/3.1. The specified SDK got me much larger diff as Erik observed.

Anyway, now WalletWasabi.Gui.dll and WalletWasabi.Gui.dll differ. I took a look at WalletWasabi.dll and all differences are basically similar to this one in WalletWasabi\Blockchain\Transactions\TransactionFactory.cs:

image

So for some reason some temporary variables are created for a function call. I have tried to find out why that may happen but at the moment I don't know so I will try to ask somebody.

Thanks for verifying my steps @kiminuo 👍 I can also confirm I had a deeper look into the DLLs and see the same thing. The code looks the same, the variables are sometimes suffixed with 1 and in some cases the arguments are extracted to temporary variables.

This is also a part that stood out for me, do you also see this @kiminuo ?
image

UPDATE: Never mind, it was in there only at the very bottom. My bad!

@eriknylund
Copy link

eriknylund commented Nov 4, 2021

In case someone else is curious, this is how I approached it, by using dotPeek to reverse the DLLs and export them to projects, and then diff those with WinMerge using this technique https://stackoverflow.com/questions/590863/tool-to-compare-dlls-and-disassemble-the-differences

The diff for the two DLLs can also be found here: https://gist.github.com/eriknylund/cc7c4b0a34d0c701aa6dc20895e5802c

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 4, 2021

So some random thoughts and guesses:

  • Deterministic build might not fully work in .NET Core 3.1 perfectly (this may be fixed in .NET 5 or .NET 6, so we would just wait)
  • David should probably try to do the build on a different machine and compare that he can actually himself verify the build is deterministic. I believe that this is a good first step given that me and @eriknylund observe the same differences in DLL files.

@molnard
Copy link
Collaborator

molnard commented Nov 4, 2021

@adamPetho please go through this

  • Deterministic build switch
  • Try to add the used version into the software for the user to be able to see. Idea: Environment.Version or RuntimeInformation source
  • Add this information to somewhere like the About.

@molnard
Copy link
Collaborator

molnard commented Nov 4, 2021

I just collect some information about the client, maybe useful for something later.

image

image

WalletWasabi.Gui.runtimeconfig.json

{
  "runtimeOptions": {
    "tfm": "netcoreapp3.1",
    "includedFrameworks": [
      {
        "name": "Microsoft.NETCore.App",
        "version": "3.1.18"
      },
      {
        "name": "Microsoft.AspNetCore.App",
        "version": "3.1.18"
      }
    ],
    "configProperties": {
      "System.Globalization.Invariant": true
    }
  }
}

image

@molnard
Copy link
Collaborator

molnard commented Nov 4, 2021

The machine that I am using for the release.

PS C:\Users\User\Desktop> dotnet --info

.NET SDK (reflecting any global.json):
 Version:   5.0.402
 Commit:    e9d3381880

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19042
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.402\

Host (useful for support):
  Version: 5.0.11
  Commit:  f431858f8b

.NET SDKs installed:
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.1.802 [C:\Program Files\dotnet\sdk]
  2.2.401 [C:\Program Files\dotnet\sdk]
  3.0.100 [C:\Program Files\dotnet\sdk]
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.201 [C:\Program Files\dotnet\sdk]
  3.1.302 [C:\Program Files\dotnet\sdk]
  3.1.403 [C:\Program Files\dotnet\sdk]
  3.1.407 [C:\Program Files\dotnet\sdk]
  5.0.100 [C:\Program Files\dotnet\sdk]
  5.0.402 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.13 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

When using the packager, from console output:
Microsoft (R) Build Engine version 16.11.1+3e40a09f8 for .NET

@eriknylund
Copy link

Thanks @molnard !

I'll try to see if I can replicate this setup and if that has any effect.

The same information from my side:

$ dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.1.412
 Commit:    751fd182ef

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19043
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.412\

Host (useful for support):
  Version: 3.1.18
  Commit:  5d3919d34e

.NET Core SDKs installed:
  3.1.412 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

$ dotnet build
Microsoft (R) Build Engine version 16.7.2+b60ddb6f4 for .NET

@eriknylund
Copy link

eriknylund commented Nov 4, 2021

Progress! Now I can build the Wasabi DLLs without diff, but for some reason I ended up on runtime 3.1.20 instead of 3.1.18 so need to figure out that part to get a complete working setup.

@eriknylund
Copy link

Alright, so here's a setup that should be reproducible.

Microsoft (R) Build Engine version 16.7.2+b60ddb6f4 for .NET from SDK 5.0.402 is required to produce the correct DLLs, but this has a side effect (for me) that runtime is set to 3.1.20 instead of 3.1.18, being the highest patch version of 3.1 at the time of wriitng. This can be seen in your .nuget folder after running dotnet restore:

image

A workaround for this is to force the runtime framework version to 3.1.18 by adding the <RuntimeFrameworkVersion>3.1.18</RuntimeFrameworkVersion> element in WalletWasabi.Packager\WalletWasabi.Packager.csproj, WalletWasabi.Gui\WalletWasabi.Gui.csproj and WalletWasabi\WalletWasabi.csproj:

  <PropertyGroup>
    <RuntimeFrameworkVersion>3.1.18</RuntimeFrameworkVersion>
...
  </PropertyGroup>

With this in place, running dotnet restore the 3.1.18 folder is the only version and the project can be reproducibly built.

image

git clone --depth 1 --branch v1.1.13.0 https://github.com/zkSNACKs/WalletWasabi.git
cd WalletWasabi/WalletWasabi.Packager
dotnet nuget locals all --clear
# edit the three .csproj files before proceeding to the next step
dotnet restore
dotnet build
dotnet run -- --onlybinaries

cd ../WalletWasabi.Gui/bin/dist
git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet"
# no diff! 🎉 

@molnard I would expect the same thing to happen on your end when you build a release, if you perform the dotnet nuget locals all --clear and dotnet restore steps? Otherwise it may depend on what the latest patch version was at the time of the release and would explain why the runtimepack.Microsoft.NETCore.App.Runtime.win-x64 version has changed between versions.

This is my setup that produces a reproducible build.

$ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.402
 Commit:    e9d3381880

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19043
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.402\

Host (useful for support):
  Version: 5.0.11
  Commit:  f431858f8b

.NET SDKs installed:
  3.1.412 [C:\Program Files\dotnet\sdk]
  5.0.402 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

These steps can probably be applied to repro build 1.1.12.3-9 as well.

@kiminuo are you able to validate this approach?

@eriknylund
Copy link

Here's a convenient Powershell script to force the runtime to 3.1.18 for the three project files: https://gist.github.com/eriknylund/b714e51584ad29c7d596d2db6be3a7a2

Run it from the Wasabi root folder.

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 5, 2021

@eriknylund Nice work. Thank you for debugging it like this. I will try to replicate your findings.

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 5, 2021

@eriknylund So I created https://github.com/kiminuo/WalletWasabi/tree/feature/2021-11-05-v1.1.13-deterministic-build based on your instructions - is it correct?

Now, with this I got one step closer as WalletWasabi.dll is on my computer the same as the officially built one. However, there is still two differences in WalletWasabi.Gui.dll on my computer vs the one in the official build:

image
image

I'm not entirely sure what those bytes represent though. It's not a difference in C# code though.

Could you please tell me if my commit in the git branch is correct and whether it works for you or not?

@eriknylund
Copy link

@kiminuo I ended up not doing any changes in global.json, could that be the reason? Since WasabiWallet.dll builds okay I assume the build engine is correct. I'll try running your branch to see what happens.

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 5, 2021

@kiminuo I ended up not doing any changes in global.json, could that be the reason?

I have added that file by design so that only specified .NET SDK can be used for the build. That removes a lot of uncertainty whether a correct/intended .NET SDK is used or not.

@eriknylund
Copy link

@kiminuo I ended up not doing any changes in global.json, could that be the reason? Since WasabiWallet.dll builds okay I assume the build engine is correct. I'll try running your branch to see what happens.

@kiminuo git diff doesn't report any diff building your branch. What tool did you use to diff the two files (from picture above) and did it also show in the git diff?

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 5, 2021

git diff doesn't report any diff building your branch.

So it's only on my computer. That's good and bad 😆

What tool did you use to diff the two files (from picture above)

https://www.cjmweb.net/vbindiff/VBinDiff-Win32

and did it also show in the git diff?

So git diff --no-index "win7-x64" "C:\Program Files\WasabiWallet" reports for me:

warning: CRLF will be replaced by LF in WalletWasabi.Gui/bin/dist/win7-x64/Extensions/DummyHack.txt.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in C:/Program Files/WasabiWallet/Extensions/DummyHack.txt.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in C:/Program Files/WasabiWallet/Legal/Assets/LegalDocuments.txt.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in WalletWasabi.Gui/bin/dist/win7-x64/SOS_README.md.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in C:/Program Files/WasabiWallet/SOS_README.md.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in C:/Program Files/WasabiWallet/TorDaemons/digests.txt.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in WalletWasabi.Gui/bin/dist/win7-x64/WalletWasabi.Gui.deps.json.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in C:/Program Files/WasabiWallet/WalletWasabi.Gui.deps.json.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in WalletWasabi.Gui/bin/dist/win7-x64/WalletWasabi.Gui.runtimeconfig.json.
The file will have its original line endings in your working directory
warning: CRLF will be replaced by LF in C:/Program Files/WasabiWallet/WalletWasabi.Gui.runtimeconfig.json.
The file will have its original line endings in your working directory
diff --git a/win7-x64/WalletWasabi.Gui.dll b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll
index 43ec37a20..0633cb315 100644
Binary files a/win7-x64/WalletWasabi.Gui.dll and b/C:/Program Files/WasabiWallet/WalletWasabi.Gui.dll differ

Maybe it's because of those CRLFs vs LFs and some checksum in DLL is different or something. But the thing is that the diff images above show that my bytes are different and not that the DLL is larger/smaller.

@eriknylund
Copy link

eriknylund commented Nov 5, 2021

Maybe it's because of those CRLFs vs LFs and some checksum in DLL is different or something. But the thing is that the diff images above show that my bytes are different and not that the DLL is larger/smaller.

Yup, looks like it's the good old CRLF vs LF issue for all text files. However a CRLF in deps.json and runtimeconfig.json shouldn't cause this 🤔

Could you double check your build engine 16.7.2+b60ddb6f4 16.11.1+3e40a09f8 and Windows version OS Version: 10.0.19043 to make sure they match?

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 5, 2021

dotnet --info reports:

.NET SDK (reflecting any global.json):
 Version:   5.0.402
 Commit:    e9d3381880

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19043
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.402\

Host (useful for support):
  Version: 6.0.0-rc.2.21480.5
  Commit:  6b11d64e7e

.NET SDKs installed:
  3.1.407 [C:\Program Files\dotnet\sdk]
  3.1.412 [C:\Program Files\dotnet\sdk]
  5.0.400 [C:\Program Files\dotnet\sdk]
  5.0.402 [C:\Program Files\dotnet\sdk]
  6.0.100-rc.2.21505.57 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-preview.7.21378.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 6.0.0-rc.2.21480.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.30 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-preview.7.21377.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.0-rc.2.21480.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.18 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.11 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.0-preview.7.21378.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.0-rc.2.21501.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET runtimes or SDKs:
  https://aka.ms/dotnet-download

Build engine seems different:

PS C:\WasabiWallet\WalletWasabi\WalletWasabi.Packager> dotnet build
Microsoft (R) Build Engine version 16.11.1+3e40a09f8 for .NET

@eriknylund
Copy link

eriknylund commented Nov 5, 2021

Build engine seems different:

PS C:\WasabiWallet\WalletWasabi\WalletWasabi.Packager> dotnet build
Microsoft (R) Build Engine version 16.11.1+3e40a09f8 for .NET

Thanks. Exactly, since you have 6.0.0-rc.2.21480.5 installed you will get the build engine from that and this could introduce the diff you see in the Gui.dll. Are you able to uninstall it to downgrade to 5.0.11 and get the build engine to match?

UPDATE: Scratch that, I copy pasted the wrong engine. I am also on the same version. It could be worth a try to remove the 6.0 SDK and see if it has any effect.

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 5, 2021

Well, but you know we have that Global.json file which should be respected so my inclination is more like to store output of Wasabi Packager to see what SDK was actually used. I don't think I'm on a bad SDK as otherwise I would expect much bigger diff since .NET 6 has new SDK and I guess it will introduce new changes as there are many and many improvements and optimizations.

@eriknylund
Copy link

Well, but you know we have that Global.json file which should be respected so my inclination is more like to store output of Wasabi Packager to see what SDK was actually used. I don't think I'm on a bad SDK as otherwise I would expect much bigger diff since .NET 6 has new SDK and I guess it will introduce new changes as there are many and many improvements and optimizations.

I agree that would be ideal. However, from my experimentation with different SDKs it seems the build engine does not respect those definitions, rather the newest available is always used. In this case it doesn't seem to differ and have an impact. To verify, I installed the latest SDK 6.0.100-rc2.21505.57 and I can confirm the diff on your branch is still clean so we can rule that out. What seems to be important is that precisely 16.11.1+3e40a09f8 is used for this build to be reproducible.

I'm curious why the CRLFs appear. I've been using Git bash to perform all build steps, is it perhaps treating the line endings differently? 🤔

@molnard
Copy link
Collaborator

molnard commented Nov 5, 2021

Let me summarize it

To be able to reproduce the build:

  • You need to know the Build engine (in this case 16.11.1+3e40a09f8) that can be seen when I am publishing the package or entering dotnet build.
  • You need to know SDK (in this case v3.1.407) that can be extracted WalletWasabi.Gui.runtimeconfig.json "tfm": "netcoreapp3.1" and the release machine SDK list. It will always use the latest SDK with v3.1.x.

Anything else is important?

@eriknylund
Copy link

Let me summarize it

To be able to reproduce the build:

* You need to know the **Build engine** (in this case 16.11.1+3e40a09f8) that can be seen when I am publishing the package or entering dotnet build.

* You need to know SDK (in this case v3.1.407) that can be extracted `WalletWasabi.Gui.runtimeconfig.json "tfm": "netcoreapp3.1"` and the release machine SDK list. It will always use the latest SDK with v3.1.x.

Anything else is important?

Correct. Also, until something like the PR from @kiminuo is in place, you need to know which runtime version was used. This should be the latest 3.1.x but it wasn't .20 rather .18 here, so you have to force it to .18 by explicitly adding the element in the three .csproj files.

@kiminuo
Copy link
Collaborator

kiminuo commented Nov 6, 2021

@eriknylund

I'm curious why the CRLFs appear. I've been using Git bash to perform all build steps, is it perhaps treating the line endings differently? 🤔

For some reason, I had in my .git/config file autocrlf=input setting (that's only on my machine, nobody else should encounter the issue really). After removing that setting I can build deterministically with no diff! Very good news.

So to summarize it:

  1. Install .NET Core 3.1.18 from here
  2. install .NET SDK 5.0.11 (that has .NET SDK 5.0.402) from here
  3. Clone the branch https://github.com/kiminuo/WalletWasabi/tree/feature/2021-11-05-v1.1.13-deterministic-build (credit to @eriknylund)
  4. Follow https://github.com/zkSNACKs/WalletWasabi/blob/master/WalletWasabi.Documentation/Guides/DeterministicBuildGuide.md
  5. Your deterministic build should be verified.

I agree that would be ideal. However, from my experimentation with different SDKs it seems the build engine does not respect those definitions, rather the newest available is always used. In this case it doesn't seem to differ and have an impact. To verify, I installed the latest SDK 6.0.100-rc2.21505.57 and I can confirm the diff on your branch is still clean so we can rule that out. What seems to be important is that precisely 16.11.1+3e40a09f8 is used for this build to be reproducible.

global.json is most certainly respected by dotnet build tools and correct .NET SDK is picked if it is available (if it isn't, the build fails). At this point, global.json is used by many projects and many projects rely on its functionality so I would be very surprised if it doesn't work properly. Whether with global.json being respected, there is still room for non-determinism in the build, I'm not sure but it's certainly conceivable that it is.

@molnard
Copy link
Collaborator

molnard commented Nov 8, 2021

We are here: https://bitcoinbinary.org/ 👍 🙏

@molnard
Copy link
Collaborator

molnard commented Nov 13, 2021

This is a very important feature but only a few users will do it - so it can be manual.

I suggest the following:

  • Create a text file (BUILDINFO.txt) that collects all the information available when releasing the software - like:
    • Information from dotnet --info
    • Build engine (like 16.11.1+3e40a09f8), can be string-extracted from stdout after dotnet publish.
    • Commit hash of the actually built source code.
  • Put this file next to the executable, for all platforms.
  • Implement this into the master and the backport
  • Create PR to update the det-build documentation that describes how to use this file.

Remarks

  • BUILDINFO file is the same for all OS, so it should be generated once, I would prefer after all the dotnet publish command went through.
  • BUILDINFO should be generated even if running the packager in onlybinaries mode.

@adamPetho can you go through this?

@adamPetho
Copy link
Collaborator

@adamPetho can you go through this?

Yep!

@stale
Copy link

stale bot commented Jan 14, 2022

This has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 14, 2022
@yahiheb yahiheb removed the stale label Jan 14, 2022
@molnard
Copy link
Collaborator

molnard commented Jan 17, 2022

We sorted this out - see above.

@molnard molnard closed this as completed Jan 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

6 participants