Reproducible Builds

[mkg20001]

Currently when running pkg twice the hashsum of the generated binaries does not match.

Example:

console.log("hello world")

pkg main.js -t node8.0.0-linux-x64
mv main A
pkg main.js -t node8.0.0-linux-x64
mv main B
sha256sum A B

Produces the following hashes:

dfcc0d60144098ac9de15fca1202e1f9d440f1fe53bbf97fbdb2b8ab0dda42c3  A
c271faf859a68552e5362f62ad24ab35bfc07cf20582537601d77b9ac5afb129  B

running it again creates other hashes

[igorklopov]

Fixed in pkg@4.2.0. Please upgrade!

[igorklopov]

Regression in v8 version 6.0 (nodejs 8.3.0). Reopening

[mkg20001]

Any progress on this?

[USSliberty]

+1 On this!

[mkg20001]

@igorklopov Progress?

Still not reproducible:

> pkg@4.3.0
> Targets not specified. Assuming:
  node8-linux-x64, node8-macos-x64, node8-win-x64
1a741607fd0ffcf49227dfebc829f597993f2b5d5a01e77fb2bdc74cbf78d3ab  test.js
fff411a32aaedba31079244952a49e238bc3e0c9ff93caa3c86013a5684a40a8  test-linux
82189abb072ba71218b1261a9a36041e8a017911fa7f9d33885e2c161da5b811  test-macos
e5b9f1f20cdfe57d9e9915c680c76381e08193084aec1b5d6a68bd57a761f259  test.sh
5060638ba2f0891acf8e58da32d78e3e8a70bfad3cf3b33e1c844091f75288ac  test-win.exe
> pkg@4.3.0
> Targets not specified. Assuming:
  node8-linux-x64, node8-macos-x64, node8-win-x64
1a741607fd0ffcf49227dfebc829f597993f2b5d5a01e77fb2bdc74cbf78d3ab  test.js
7eed29798a8ef5e9209b03dae72f6f6c8cc4762c98b0d573df20efaa1181420a  test-linux
3f27e11f56392790c6ce368342bdaf7c47151c88794cb6f181b9f0a6fd201f7a  test-macos
e5b9f1f20cdfe57d9e9915c680c76381e08193084aec1b5d6a68bd57a761f259  test.sh
67923995534abf90d4c441466ad028424abcc4ad3081956223c420a9d3e3f2e4  test-win.exe
> pkg@4.3.0
> Targets not specified. Assuming:
  node8-linux-x64, node8-macos-x64, node8-win-x64
1a741607fd0ffcf49227dfebc829f597993f2b5d5a01e77fb2bdc74cbf78d3ab  test.js
d994d4f30f813334267e3fb8d1704a9913b4f478d1e03a210b888f1c8f3cf468  test-linux
dec5236142c95c0529e2890a3510650e594803a541e409cc324ebd6080b7f119  test-macos
e5b9f1f20cdfe57d9e9915c680c76381e08193084aec1b5d6a68bd57a761f259  test.sh
ba266872f0519c41cb5caac40ff2fd5cc3469d903f3db6281f7f606e66e1248e  test-win.exe
> pkg@4.3.0
> Targets not specified. Assuming:
  node8-linux-x64, node8-macos-x64, node8-win-x64
1a741607fd0ffcf49227dfebc829f597993f2b5d5a01e77fb2bdc74cbf78d3ab  test.js
d36419c8fd41b71902f67ac8635c7826b7768782ff056b2b65f22d85bfe58271  test-linux
1326406608d5975010858dfdc4172e5975d42c03131e2c61f260fe82331f0146  test-macos
e5b9f1f20cdfe57d9e9915c680c76381e08193084aec1b5d6a68bd57a761f259  test.sh
c1bf9348e032418c61fe93680ff3133c7c9bbccd71d682485566b1f084e4fe5a  test-win.exe
> pkg@4.3.0
> Targets not specified. Assuming:
  node8-linux-x64, node8-macos-x64, node8-win-x64
1a741607fd0ffcf49227dfebc829f597993f2b5d5a01e77fb2bdc74cbf78d3ab  test.js
e7dab8fd7110a14b9f4286cc00adcb00acffd4c9e0d927c74f81c121b1c0b06e  test-linux
ab45feb2f6d4787a9dd89e569d7209198f8ece4d0734acad091c4c2bf9369360  test-macos
e5b9f1f20cdfe57d9e9915c680c76381e08193084aec1b5d6a68bd57a761f259  test.sh
2e1aeca7cbd95a960ea231525e373509ed8fcd6ec5a45b111b29cc2c871233c2  test-win.exe

Edit: Here is a diff

--- first-bin	2018-02-15 16:11:36.667193005 +0100
+++ second-bin	2018-02-15 16:11:54.419266505 +0100
@@ -2163346,8 +2163346,8 @@
 21698e0 4836 6e61 6c64 4965 534e 5f30 4f36 6a62
 21698f0 6365 4574 4545 f100 de04 00c0 0000 6400
 2169900 8431 5d12 0000 0000 0000 8e00 a438 05b1
-2169910 0000 0000 0000 6000 0003 3b00 6cf9 c366
-2169920 364e 00e8 0000 5000 0000 e080 0004 0080
+2169910 0000 0000 0000 6000 0003 5c00 f5e3 d8c0
+2169920 7a83 009f 0000 5000 0000 e080 0004 0080
 2169930 0000 0080 0000 0080 0000 0080 0000 0100
 2169940 934c fc0e 0189 9220 00c5 0000 0600 0000
 2169950 0000 0000 4300 0000 0000 0000 0000 0000
@@ -2163358,22 +2163358,22 @@
 21699a0 0189 9230 00c4 0000 0a00 0000 0000 0000
 21699b0 c100 0041 0000 0000 0500 0000 0000 0000
 21699c0 0000 0000 0000 0000 0000 0000 0100 9110
-21699d0 62c2 90d1 0056 0000 0000 0000 0700 0000
-21699e0 6500 7078 726f 7374 0100 9110 16c2 8e2b
-21699f0 00cb 0000 0000 0000 0700 0000 7200 7165
-2169a00 6975 6572 0100 9110 bac2 fe55 0073 0000
+21699d0 06c2 99a0 00a1 0000 0000 0000 0700 0000
+21699e0 6500 7078 726f 7374 0100 9110 0ec2 3cf0
+21699f0 007b 0000 0000 0000 0700 0000 7200 7165
+2169a00 6975 6572 0100 9110 0ac2 017a 00c0 0000
 2169a10 0000 0000 0600 0000 6d00 646f 6c75 0065
-2169a20 0100 9114 6ec3 43ab 00bc 0000 0000 0000
+2169a20 0100 9114 56c3 ebb7 005a 0000 0000 0000
 2169a30 0a00 0000 5f00 665f 6c69 6e65 6d61 0065
-2169a40 0000 0000 0100 9114 7ac3 2912 0042 0000
+2169a40 0000 0000 0100 9114 42c3 a6e4 0003 0000
 2169a50 0000 0000 0900 0000 5f00 645f 7269 616e
 2169a60 656d 0000 0000 0000 c000 0000 0000 0000
 2169a70 0000 0e85 3990 2801 c03a 0000 0000 0017
 2169a80 0000 1401 c08c 0000 0000 0003 0000 1001
-2169a90 c291 7c4a 1f27 0000 0000 0000 0000 0007
-2169aa0 0000 6f63 736e 6c6f 0065 1001 c291 966a
-2169ab0 de63 0000 0000 0000 0000 0003 0000 6f6c
-2169ac0 0067 0000 0000 1401 c391 028e 4296 0000
+2169a90 c291 642a 7b5f 0000 0000 0000 0000 0007
+2169aa0 0000 6f63 736e 6c6f 0065 1001 c291 676e
+2169ab0 b538 0000 0000 0000 0000 0003 0000 6f6c
+2169ac0 0067 0000 0000 1401 c391 109e f6ff 0000
 2169ad0 0000 0000 0000 000b 0000 6548 6c6c 206f
 2169ae0 6f57 6c72 0064 0000 0000 019e 8b10 00c2
 2169af0 0000 0a00 0000 0100 0216 0b68 1510 0c00

e7dab8fd7110a14b9f4286cc00adcb00acffd4c9e0d927c74f81c121b1c0b06e  test-linux2
7f98f18ce6034f6b043be98b1b25798687d09d4f4637af767cc567e576d7ec5f  test-linux

[whexberg]

+1

[leerob]

Hello! We welcome any and all contributions to try and address this. Thank you!

[whexberg]

Hello! We welcome any and all contributions to try and address this. Thank you!

TLDR Using the '--no-bytecode' flag creates reproducible builds, but also puts your source code directly in the executable. Running without the flag compiles your source code to bytecode, which makes it a little harder to get the source, but builds are not reproducible.

@leerob Actually, contributing sounded fun, so yesterday, I spent almost the entire day digging through the code. I narrowed it down to 1 file, and figured out that the bytes change when you precompile to bytecode. So, I thought to myself "Hey self, we could make a big contribution by adding a flag. Maybe something like 'no-bytecode'" Then I high-fived myself, for coming up with such a great idea, reverted everything in git to start afresh, went to index.js to add my new flag, and there I discovered a flag called 'no-bytecode'. I decided to try it out, and wouldn't you know it, the builds are in fact reproducible. So, I just thought I'd commend you guys on a job well done. You implemented my awesome idea before I even knew it was my awesome idea. However, I didn't notice it in the documentation. Did I miss it, or is it not documented?

All joking aside, though, this is a great project, you guys are doing a great job, and I'd love to contribute if/when you guys need help.

[hipstersmoothie]

If it isn't in the docs we would def appreciate a pr ❤️

[leerob]

Yes absolutely! I'm happy to help review and get it merged.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove the stale label or comment or this will be closed in 5 days. To ignore this issue entirely you can add the no-stale label

[github-actions]

This issue is now closed due to inactivity, you can of course reopen or reference this issue if you see fit.