Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: upgrade Nix from 2.3.12 to 2.6.1 #12980

Merged
merged 2 commits into from
Apr 11, 2022
Merged

ci: upgrade Nix from 2.3.12 to 2.6.1 #12980

merged 2 commits into from
Apr 11, 2022

Conversation

jakubgs
Copy link
Member

@jakubgs jakubgs commented Jan 6, 2022

Due to changes in how Nix handles Git refs we need to specify
refs/tags/ prefix in package.json to avoid the following error:

fatal: couldn't find remote ref refs/heads/v2.0.3-status-v6
error: program 'git' failed with exit code 128

I also had to rewrite some logic in nix/scripts/source.sh in order to take account of single-user and multi-user installations. We default to multi-user for Darwin, but not for any other OS due to discovered issues with nix-daemon socket on Arch and open file limits.

I also rewrote nix/scripts/setup.sh and /nix/scripts/purge.sh to support different types of installations. Both single-user and multi-user, as some operating systems have issues with multi-user installations.

Resolves: #12832
Depends on: https://github.com/status-im/status-jenkins-lib/pull/37
Issue: NixOS/nix#5291

Related changes:

@jakubgs jakubgs added the nix label Jan 6, 2022
@jakubgs jakubgs self-assigned this Jan 6, 2022
@status-github-bot status-github-bot bot added this to REVIEW in Pipeline for QA Jan 6, 2022
@status-im-auto
Copy link
Member

status-im-auto commented Jan 6, 2022

Jenkins Builds

Click to see older builds (236)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ bf59de1 #1 2022-01-06 18:58:14 ~12 min ios 📦ipa 📲
✔️ bf59de1 #1 2022-01-06 19:02:42 ~16 min android-e2e 📦apk 📲
✔️ bf59de1 #1 2022-01-06 19:02:45 ~16 min android 📦apk 📲
✔️ 4e52ec1 #2 2022-01-10 15:34:25 ~10 min ios 📦ipa 📲
✔️ 4e52ec1 #2 2022-01-10 15:39:18 ~15 min android-e2e 📦apk 📲
✔️ 4e52ec1 #2 2022-01-10 15:39:18 ~15 min android 📦apk 📲
✔️ ec851fa #3 2022-01-11 10:53:51 ~12 min ios 📦ipa 📲
✔️ ec851fa #3 2022-01-11 10:55:51 ~14 min android 📦apk 📲
✔️ ec851fa #3 2022-01-11 10:55:56 ~14 min android-e2e 📦apk 📲
✔️ 8155d6e #4 2022-01-11 10:56:56 ~14 min android 📦apk 📲
✔️ 8155d6e #4 2022-01-11 10:58:46 ~16 min android-e2e 📦apk 📲
✔️ 8155d6e #4 2022-01-11 11:00:42 ~18 min ios 📦ipa 📲
✔️ 7e2523f #5 2022-01-11 15:34:31 ~11 min ios 📦ipa 📲
✔️ 7e2523f #5 2022-01-11 15:40:09 ~17 min android 📦apk 📲
✔️ 7e2523f #5 2022-01-11 15:40:23 ~17 min android-e2e 📦apk 📲
✔️ 6904320 #6 2022-01-13 12:25:16 ~25 min android 📦apk 📲
✔️ 6904320 #6 2022-01-13 12:25:58 ~26 min android-e2e 📦apk 📲
6904320 #6 2022-01-13 15:26:55 ~3 hr 27 min ios 📄log
11bd436 #7 2022-01-13 15:41:31 ~13 sec android 📄log
11bd436 #7 2022-01-13 15:41:32 ~14 sec ios 📄log
11bd436 #7 2022-01-13 15:41:39 ~21 sec android-e2e 📄log
b8088e8 #8 2022-01-13 15:42:36 ~13 sec android-e2e 📄log
b8088e8 #8 2022-01-13 15:42:39 ~16 sec ios 📄log
b8088e8 #8 2022-01-13 15:42:46 ~23 sec android 📄log
b8088e8 #9 2022-01-13 15:43:51 ~14 sec ios 📄log
26153b7 #9 2022-01-13 16:44:02 ~14 sec android-e2e 📄log
26153b7 #9 2022-01-13 16:44:13 ~25 sec android 📄log
26153b7 #10 2022-01-13 16:44:19 ~30 sec ios 📄log
26153b7 #11 2022-01-13 16:45:28 ~14 sec ios 📄log
26153b7 #10 2022-01-13 18:03:32 ~17 sec android 📄log
26153b7 #12 2022-01-13 18:08:47 ~16 sec ios 📄log
26153b7 #13 2022-01-13 18:10:34 ~14 sec ios 📄log
26153b7 #14 2022-01-13 18:18:48 ~2 min ios 📄log
b6cd3f4 #11 2022-01-13 19:59:27 ~27 sec android 📄log
b6cd3f4 #10 2022-01-13 19:59:38 ~38 sec android-e2e 📄log
b6cd3f4 #15 2022-01-13 19:59:46 ~45 sec ios 📄log
393b6e6 #11 2022-01-13 20:01:31 ~13 sec android-e2e 📄log
393b6e6 #12 2022-01-13 20:01:42 ~23 sec android 📄log
✔️ 393b6e6 #16 2022-01-13 20:20:49 ~19 min ios 📦ipa 📲
c5dce79 #12 2022-01-13 20:21:33 ~17 sec android-e2e 📄log
c5dce79 #17 2022-01-13 20:21:36 ~19 sec ios 📄log
c5dce79 #13 2022-01-13 20:21:47 ~29 sec android 📄log
0defe33 #14 2022-01-13 20:22:59 ~14 sec android 📄log
0defe33 #13 2022-01-13 20:23:12 ~26 sec android-e2e 📄log
✔️ 0defe33 #18 2022-01-13 20:35:02 ~12 min ios 📦ipa 📲
2df5814 #15 2022-01-13 20:24:46 ~27 sec android 📄log
2df5814 #14 2022-01-13 20:25:09 ~50 sec android-e2e 📄log
f2381a8 #15 2022-01-13 20:26:25 ~19 sec android-e2e 📄log
f2381a8 #16 2022-01-13 20:26:35 ~28 sec android 📄log
✔️ f2381a8 #20 2022-01-13 20:47:39 ~21 min ios 📦ipa 📲
83f7746 #17 2022-01-13 20:38:43 ~32 sec android 📄log
83f7746 #16 2022-01-13 20:38:43 ~32 sec android-e2e 📄log
732a2a7 #17 2022-01-13 20:42:36 ~11 sec android-e2e 📄log
732a2a7 #18 2022-01-13 20:42:36 ~11 sec android 📄log
bfb539e #18 2022-01-13 20:46:23 ~32 sec android-e2e 📄log
bfb539e #19 2022-01-13 20:46:24 ~32 sec android 📄log
b5eda84 #19 2022-01-13 20:49:35 ~15 sec android-e2e 📄log
b5eda84 #20 2022-01-13 20:49:36 ~15 sec android 📄log
d72ca5d #20 2022-01-13 20:52:36 ~13 sec android-e2e 📄log
d72ca5d #25 2022-01-13 20:52:48 ~22 sec ios 📄log
d72ca5d #21 2022-01-13 20:52:48 ~22 sec android 📄log
9df47c5 #21 2022-01-13 20:54:04 ~18 sec android-e2e 📄log
3603ac8 #23 2022-01-13 20:54:34 ~14 sec android 📄log
3603ac8 #22 2022-01-13 20:54:45 ~25 sec android-e2e 📄log
f6d6a94 #24 2022-01-13 20:55:59 ~13 sec android 📄log
f6d6a94 #23 2022-01-13 20:56:13 ~27 sec android-e2e 📄log
7b2e840 #25 2022-01-13 20:58:18 ~21 sec android 📄log
7b2e840 #24 2022-01-13 20:58:32 ~35 sec android-e2e 📄log
7640344 #26 2022-01-13 20:59:45 ~14 sec android 📄log
7640344 #25 2022-01-13 20:59:56 ~25 sec android-e2e 📄log
✔️ d23ecb1 #31 2022-01-13 21:15:28 ~9 min ios 📦ipa 📲
✔️ d23ecb1 #26 2022-01-13 21:24:30 ~18 min android-e2e 📦apk 📲
d23ecb1 #27 2022-01-13 21:25:20 ~19 min android 📄log
✔️ d23ecb1 #28 2022-01-13 21:43:57 ~12 min android 📦apk 📲
✔️ 7f58333 #32 2022-01-13 21:59:49 ~15 min ios 📦ipa 📲
✔️ 7f58333 #27 2022-01-13 22:00:06 ~15 min android-e2e 📦apk 📲
✔️ 7f58333 #29 2022-01-13 22:00:14 ~15 min android 📦apk 📲
✔️ 077f2d3 #33 2022-01-14 00:18:52 ~11 min ios 📦ipa 📲
✔️ 077f2d3 #28 2022-01-14 00:21:34 ~14 min android-e2e 📦apk 📲
✔️ 077f2d3 #30 2022-01-14 00:23:53 ~16 min android 📦apk 📲
✔️ 8d38e3e #34 2022-01-14 00:37:05 ~10 min ios 📦ipa 📲
✔️ 8d38e3e #29 2022-01-14 00:41:39 ~15 min android-e2e 📦apk 📲
✔️ 8d38e3e #31 2022-01-14 00:41:39 ~15 min android 📦apk 📲
c371225 #35 2022-01-14 09:55:23 ~16 sec ios 📄log
c371225 #30 2022-01-14 10:18:25 ~23 min android-e2e 📄log
c371225 #32 2022-01-14 10:30:04 ~34 min android 📄log
✔️ afd4f1a #36 2022-01-14 10:38:08 ~11 min ios 📦ipa 📲
✔️ afd4f1a #33 2022-01-14 10:42:21 ~16 min android 📦apk 📲
✔️ afd4f1a #31 2022-01-14 10:54:43 ~28 min android-e2e 📦apk 📲
✔️ 54310cf #34 2022-01-14 11:39:43 ~17 min android 📦apk 📲
✔️ 54310cf #32 2022-01-14 11:39:49 ~17 min android-e2e 📦apk 📲
✔️ 54310cf #37 2022-01-14 11:40:15 ~17 min ios 📦ipa 📲
✔️ 4e8794f #38 2022-01-17 10:39:35 ~12 min ios 📦ipa 📲
✔️ 4e8794f #35 2022-01-17 10:42:45 ~15 min android 📦apk 📲
✔️ 4e8794f #33 2022-01-17 10:42:54 ~16 min android-e2e 📦apk 📲
✔️ fe85a9d #39 2022-01-19 09:42:38 ~15 min ios 📦ipa 📲
✔️ fe85a9d #36 2022-01-19 09:45:34 ~18 min android 📦apk 📲
✔️ fe85a9d #34 2022-01-19 09:45:34 ~18 min android-e2e 📦apk 📲
✔️ dbebd6d #35 2022-01-19 10:03:14 ~15 min android-e2e 📦apk 📲
✔️ dbebd6d #37 2022-01-19 10:03:14 ~15 min android 📦apk 📲
✔️ dbebd6d #40 2022-01-19 10:05:11 ~17 min ios 📦ipa 📲
✔️ 60c9ae8 #41 2022-01-19 10:05:10 ~16 min ios 📦ipa 📲
8aadf77 #42 2022-01-19 10:07:44 ~4 min ios 📄log
✔️ 8aadf77 #39 2022-01-19 10:18:27 ~15 min android 📦apk 📲
✔️ 8aadf77 #37 2022-01-19 10:18:30 ~15 min android-e2e 📦apk 📲
✔️ f6490a4 #43 2022-01-19 10:40:36 ~10 min ios 📦ipa 📲
✔️ f6490a4 #40 2022-01-19 10:45:40 ~15 min android 📦apk 📲
✔️ f6490a4 #38 2022-01-19 10:45:41 ~15 min android-e2e 📦apk 📲
✔️ b279db6 #44 2022-01-20 11:27:02 ~12 min ios 📦ipa 📲
9a14db4 #45 2022-01-20 11:43:38 ~18 sec ios 📄log
✔️ a3e5a1d #46 2022-01-20 11:58:39 ~14 min ios 📦ipa 📲
✔️ 93bb91c #47 2022-01-20 11:58:39 ~14 min ios 📦ipa 📲
✔️ bd2f40c #49 2022-01-20 12:15:05 ~18 min ios 📦ipa 📲
✔️ 4fef59e #50 2022-01-20 12:37:38 ~12 min ios 📦ipa 📲
✔️ 2730328 #52 2022-01-20 13:00:38 ~10 min ios 📦ipa 📲
✔️ 6432523 #53 2022-01-20 13:18:03 ~11 min ios 📦ipa 📲
b9aef43 #50 2022-01-20 17:51:08 ~4 min android 📄log
b9aef43 #48 2022-01-20 17:51:09 ~4 min android-e2e 📄log
✔️ b9aef43 #54 2022-01-20 17:57:27 ~10 min ios 📦ipa 📲
50a14ed #51 2022-01-21 11:54:51 ~1 min android 📄log
50a14ed #49 2022-01-21 11:54:59 ~1 min android-e2e 📄log
✔️ 50a14ed #55 2022-01-21 12:09:57 ~16 min ios 📦ipa 📲
44b829a #50 2022-01-21 11:56:51 ~3 min android-e2e 📄log
44b829a #52 2022-01-21 11:56:51 ~2 min android 📄log
✔️ 44b829a #56 2022-01-21 12:09:54 ~15 min ios 📦ipa 📲
5790284 #51 2022-01-21 14:32:59 ~2 min android-e2e 📄log
5790284 #53 2022-01-21 14:33:08 ~2 min android 📄log
✔️ 5790284 #57 2022-01-21 14:43:51 ~13 min ios 📦ipa 📲
63e0d9f #52 2022-01-21 14:34:50 ~2 min android-e2e 📄log
63e0d9f #54 2022-01-21 14:34:53 ~2 min android 📄log
✔️ 63e0d9f #58 2022-01-21 14:43:51 ~11 min ios 📦ipa 📲
30d2193 #55 2022-01-26 11:59:00 ~4 min android 📄log
30d2193 #53 2022-01-26 11:59:02 ~4 min android-e2e 📄log
✔️ 30d2193 #59 2022-01-26 12:17:05 ~22 min ios 📦ipa 📲
147a257 #54 2022-01-26 12:18:23 ~2 min android-e2e 📄log
147a257 #56 2022-01-26 12:18:24 ~2 min android 📄log
✔️ 147a257 #60 2022-01-26 12:30:12 ~13 min ios 📦ipa 📲
958c63b #55 2022-01-26 12:21:19 ~1 min android-e2e 📄log
958c63b #57 2022-01-26 12:21:20 ~1 min android 📄log
✔️ 958c63b #61 2022-01-26 12:41:28 ~22 min ios 📦ipa 📲
0271712 #56 2022-01-26 12:37:59 ~2 min android-e2e 📄log
0271712 #58 2022-01-26 12:37:59 ~2 min android 📄log
✔️ 0271712 #62 2022-01-26 12:54:03 ~18 min ios 📦ipa 📲
80de3e4 #57 2022-01-26 12:50:47 ~1 min android-e2e 📄log
80de3e4 #59 2022-01-26 12:50:48 ~1 min android 📄log
✔️ 80de3e4 #63 2022-01-26 13:10:58 ~22 min ios 📦ipa 📲
e6f0fa3 #60 2022-01-26 13:02:06 ~2 min android 📄log
e6f0fa3 #58 2022-01-26 13:02:13 ~2 min android-e2e 📄log
eb7ad0a #61 2022-01-26 13:04:49 ~1 min android 📄log
eb7ad0a #59 2022-01-26 13:04:55 ~1 min android-e2e 📄log
a78e0e1 #60 2022-01-26 13:10:20 ~2 min android-e2e 📄log
a78e0e1 #62 2022-01-26 13:10:20 ~2 min android 📄log
✔️ a78e0e1 #66 2022-01-26 13:26:48 ~18 min ios 📦ipa 📲
2e42a89 #63 2022-01-26 13:17:58 ~1 min android 📄log
2e42a89 #61 2022-01-26 13:18:19 ~2 min android-e2e 📄log
9cf873f #62 2022-01-26 13:25:39 ~1 min android-e2e 📄log
9cf873f #64 2022-01-26 13:27:31 ~3 min android 📄log
✔️ 9cf873f #68 2022-01-26 13:38:49 ~14 min ios 📦ipa 📲
36a13af #63 2022-01-26 13:44:53 ~1 min android-e2e 📄log
36a13af #65 2022-01-26 13:44:59 ~2 min android 📄log
36a13af #64 2022-01-26 13:45:35 ~1 min android-e2e 📄log
✔️ 36a13af #69 2022-01-26 14:00:14 ~17 min ios 📦ipa 📲
706a464 #66 2022-01-26 14:17:37 ~1 min android 📄log
706a464 #65 2022-01-26 14:17:44 ~1 min android-e2e 📄log
✔️ 706a464 #70 2022-01-26 14:29:01 ~13 min ios 📦ipa 📲
f71ec92 #67 2022-01-26 14:22:41 ~1 min android 📄log
f71ec92 #66 2022-01-26 14:22:51 ~2 min android-e2e 📄log
✔️ f71ec92 #71 2022-01-26 14:30:36 ~9 min ios 📦ipa 📲
ce55bf8 #67 2022-01-26 14:37:41 ~5 min android-e2e 📄log
ce55bf8 #68 2022-01-26 14:39:23 ~6 min android 📄log
✔️ ce55bf8 #72 2022-01-26 14:47:59 ~15 min ios 📦ipa 📲
f0398ad #69 2022-01-31 09:08:25 ~1 min android 📄log
f0398ad #68 2022-01-31 09:08:25 ~1 min android-e2e 📄log
✔️ f0398ad #73 2022-01-31 09:18:22 ~11 min ios 📦ipa 📲
38b688a #69 2022-01-31 09:14:03 ~1 min android-e2e 📄log
38b688a #70 2022-01-31 09:14:04 ~1 min android 📄log
5d6a929 #70 2022-01-31 09:16:04 ~1 min android-e2e 📄log
5d6a929 #71 2022-01-31 09:16:06 ~1 min android 📄log
✔️ 5d6a929 #75 2022-01-31 09:30:20 ~16 min ios 📦ipa 📲
83c8d86 #71 2022-02-01 09:05:45 ~2 min android-e2e 📄log
83c8d86 #72 2022-02-01 09:05:47 ~2 min android 📄log
✔️ 83c8d86 #76 2022-02-01 09:17:06 ~13 min ios 📦ipa 📲
9acd278 #72 2022-02-01 09:49:46 ~4 min android-e2e 📄log
9acd278 #73 2022-02-01 09:52:44 ~7 min android 📄log
✔️ 9acd278 #77 2022-02-01 09:57:03 ~11 min ios 📦ipa 📲
39b6ddf #73 2022-02-01 10:16:20 ~1 min android-e2e 📄log
39b6ddf #74 2022-02-01 10:16:25 ~1 min android 📄log
✔️ 39b6ddf #78 2022-02-01 10:25:54 ~11 min ios 📦ipa 📲
d063549 #75 2022-02-01 10:28:03 ~1 min android 📄log
d063549 #74 2022-02-01 10:29:42 ~3 min android-e2e 📄log
✔️ d063549 #79 2022-02-01 10:44:13 ~17 min ios 📦ipa 📲
d063549 #76 2022-02-01 16:20:30 ~1 min android 📄log
1d54b3f #77 2022-02-11 08:18:39 ~2 min android 📄log
1d54b3f #75 2022-02-11 08:18:50 ~2 min android-e2e 📄log
75a8a24 #76 2022-02-17 20:58:41 ~2 min android-e2e 📄log
75a8a24 #78 2022-02-17 20:58:53 ~2 min android 📄log
75a8a24 #81 2022-02-18 18:25:44 ~21 hr ios 📄log
fa74971 #82 2022-03-07 15:48:26 ~41 sec ios 📄log
fa74971 #79 2022-03-07 15:50:16 ~2 min android 📄log
fa74971 #77 2022-03-07 15:50:23 ~2 min android-e2e 📄log
cb10ffd #83 2022-03-07 17:18:04 ~1 min ios 📄log
cb10ffd #78 2022-03-07 17:18:13 ~1 min android-e2e 📄log
cb10ffd #80 2022-03-07 17:18:13 ~1 min android 📄log
cb10ffd #81 2022-03-07 17:25:42 ~1 min android 📄log
cb10ffd #82 2022-03-07 17:28:57 ~1 min android 📄log
cb10ffd #83 2022-03-07 18:26:57 ~2 min android 📄log
cb10ffd #84 2022-03-07 18:38:17 ~1 min android 📄log
cb10ffd #85 2022-03-07 18:39:42 ~1 min android 📄log
cb10ffd #86 2022-03-07 18:40:45 ~1 min android 📄log
cb10ffd #87 2022-03-07 18:45:05 ~1 min android 📄log
cb10ffd #88 2022-03-07 19:01:45 ~16 sec android 📄log
cb10ffd #89 2022-03-07 19:03:59 ~1 min android 📄log
cb10ffd #90 2022-03-07 21:43:12 ~1 min android 📄log
a0220ea #91 2022-03-07 22:30:37 ~1 min android 📄log
a0220ea #79 2022-03-07 22:30:44 ~2 min android-e2e 📄log
✔️ a0220ea #84 2022-03-07 22:44:38 ~15 min ios 📦ipa 📲
a0220ea #92 2022-03-07 22:54:40 ~41 sec android 📄log
a0220ea #93 2022-03-07 22:58:46 ~1 min android 📄log
a0220ea #94 2022-03-07 23:00:19 ~1 min android 📄log
a0220ea #95 2022-03-07 23:07:36 ~1 min android 📄log
a0220ea #96 2022-03-07 23:08:45 ~1 min android 📄log
a0220ea #97 2022-03-07 23:15:08 ~2 min android 📄log
a0220ea #98 2022-03-07 23:15:39 ~1 min android 📄log
a0220ea #100 2022-03-07 23:26:21 ~1 min android 📄log
✔️ a0220ea #99 2022-03-07 23:31:49 ~9 min android 📦apk 📲
✔️ a0220ea #101 2022-03-08 09:21:22 ~7 min android 📦apk 📲
a0220ea #102 2022-03-08 11:49:50 ~1 min android 📄log
✔️ a0220ea #103 2022-03-08 12:09:38 ~7 min android 📦apk 📲
✔️ a86d5a1 #104 2022-03-08 12:18:24 ~8 min android 📦apk 📲
✔️ a86d5a1 #80 2022-03-08 12:18:44 ~8 min android-e2e 📦apk 📲
✔️ a86d5a1 #85 2022-03-08 12:22:29 ~12 min ios 📦ipa 📲
✔️ f6d175d #81 2022-03-08 12:56:48 ~8 min android-e2e 📦apk 📲
✔️ f6d175d #105 2022-03-08 12:58:56 ~10 min android 📦apk 📲
✔️ f6d175d #86 2022-03-08 13:08:16 ~19 min ios 📦ipa 📲
11178eb #106 2022-04-11 09:29:15 ~1 min android 📄log
11178eb #82 2022-04-11 09:29:27 ~1 min android-e2e 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 82c0dde #107 2022-04-11 09:44:08 ~10 min android 📦apk 📲
✔️ 82c0dde #83 2022-04-11 09:44:35 ~10 min android-e2e 📦apk 📲
✔️ 82c0dde #88 2022-04-11 09:51:14 ~17 min ios 📦ipa 📲
✔️ a30541f #84 2022-04-11 10:17:20 ~11 min android-e2e 📦apk 📲
✔️ a30541f #108 2022-04-11 10:18:31 ~13 min android 📦apk 📲
✔️ a30541f #89 2022-04-11 10:18:50 ~13 min ios 📦ipa 📲

@jakubgs
Copy link
Member Author

jakubgs commented Jan 6, 2022

Looks like this will require some changes to packages-check-bot in order let the bot find the tags:

image

jakubgs added a commit to status-im/packages-check-bot that referenced this pull request Jan 10, 2022
The need to reference Git tags with `refs/tags/` prefix is caused by
changes in how Nix `2.4` and newer versions handle Git tags. They expect
tags without full name to be branches, which fails with:
```
fatal: couldn't find remote ref refs/heads/v2.0.3-status-v6
error: program 'git' failed with exit code 128
```

Issue: status-im/status-mobile#12832
PR: status-im/status-mobile#12980

Signed-off-by: Jakub Sokołowski <jakub@status.im>
jakubgs added a commit to status-im/packages-check-bot that referenced this pull request Jan 11, 2022
The need to reference Git tags with `refs/tags/` prefix is caused by
changes in how Nix `2.4` and newer versions handle Git tags. They expect
tags without full name to be branches, which fails with:
```
fatal: couldn't find remote ref refs/heads/v2.0.3-status-v6
error: program 'git' failed with exit code 128
```

Issue: status-im/status-mobile#12832
PR: status-im/status-mobile#12980

Signed-off-by: Jakub Sokołowski <jakub@status.im>
@jakubgs jakubgs force-pushed the ci/nix-upgrade branch 2 times, most recently from ec851fa to 8155d6e Compare January 11, 2022 10:42
jakubgs added a commit to status-im/packages-check-bot that referenced this pull request Jan 11, 2022
Extends `urlRegex` to accept tags specified with full `refs/tags/*` format.
Note the use of `?:` part to make the group not present in the `matche` list.

The need to reference Git tags with `refs/tags/` prefix is caused by
changes in how Nix `2.4` and newer versions handle Git tags. They expect
tags without full name to be branches, which fails with:
```
fatal: couldn't find remote ref refs/heads/v2.0.3-status-v6
error: program 'git' failed with exit code 128
```

Issue: status-im/status-mobile#12832
PR: status-im/status-mobile#12980

Signed-off-by: Jakub Sokołowski <jakub@status.im>
jakubgs added a commit to status-im/packages-check-bot that referenced this pull request Jan 11, 2022
Extends `urlRegex` to accept tags specified with full `refs/tags/*` format.
Note the use of `?:` part to make the group not present in the `matche` list.

The need to reference Git tags with `refs/tags/` prefix is caused by
changes in how Nix `2.4` and newer versions handle Git tags. They expect
tags without full name to be branches, which fails with:
```
fatal: couldn't find remote ref refs/heads/v2.0.3-status-v6
error: program 'git' failed with exit code 128
```

Issue: status-im/status-mobile#12832
PR: status-im/status-mobile#12980

Signed-off-by: Jakub Sokołowski <jakub@status.im>
@jakubgs jakubgs marked this pull request as ready for review January 11, 2022 15:41
@jakubgs jakubgs requested a review from a team January 11, 2022 15:42
@jakubgs
Copy link
Member Author

jakubgs commented Jan 11, 2022

I'd appreciate a manual test, especially on a MacOS machine. Will require make nix-purge first tho.

@jakubgs
Copy link
Member Author

jakubgs commented Jan 13, 2022

Oooh, this might be a problem:

jenkins@maci7-03.ms-eu-dublin.slave.ci:~ % /tmp/nix-2.5.1-x86_64-darwin/install --no-daemon
Error: --no-daemon installs are no-longer supported on Darwin/macOS!

Since the single user(--no-daemon) way is the default for our CI hosts.

@jakubgs
Copy link
Member Author

jakubgs commented Jan 13, 2022

I tried installing in a single user mode anyway by commenting out the check but it seems to fail:

jenkins@maci7-03.ms-eu-dublin.slave.ci:~ % /tmp/nix-2.5.1-x86_64-darwin/install --no-daemon                                          
performing a single-user installation of Nix...
copying Nix to /nix/store..............................................
installing 'nix-2.5.1'
building '/nix/store/4lnxyn7j0kpc4ir2qx39vd5jch28k494-user-environment.drv'...
installing 'nss-cacert-3.66'
building '/nix/store/6wndkfwgrdlc6igsnvwr22c07lbqvv2w-user-environment.drv'...
unpacking channels...
error: failed to extract archive (Cannot extract through symlink /nix/store/q18dv2s6jw2xyrmyy443msjis386ibgv-nixpkgs-22.05pre345560.b458e5133fb/nixpkgs-22.05pre345560.b458e5133fb)
error: builder for '/nix/store/cbrq0nrwrmihl5l3x1m34hwfwks411jf-nixpkgs-22.05pre345560.b458e5133fb.drv' failed with exit code 1
error: program '/nix/store/iq3ra93h9kxmnrw3zlxmqn8ng5w62dra-nix-2.5.1/bin/nix-env' failed with exit code 100
Fetching the nixpkgs channel failed. (Are you offline?)
To try again later, run "nix-channel --update nixpkgs".
modifying /Users/jenkins/.zshrc...

Installation finished!  To ensure that the necessary environment
variables are set, either log in again, or type

  . /Users/jenkins/.nix-profile/etc/profile.d/nix.sh

in your shell.

@jakubgs
Copy link
Member Author

jakubgs commented Jan 13, 2022

These instructions are useful: https://gist.github.com/chriselsner/3ebe962a4c4bd1f14d39897fc5619732

@jakubgs
Copy link
Member Author

jakubgs commented Jan 13, 2022

Another thing is that the current installer appears to not be made for non-interactive installation:

jenkins@maci7-03.ms-eu-dublin.slave.ci:~ % /tmp/nix-2.5.1-x86_64-darwin/install --daemon                                           
Switching to the Multi-user Installer                                             
Welcome to the Multi-User Nix Installation

This installation tool will set up your computer with the Nix package
manager. This will happen in a few stages:

1. Make sure your computer doesn't already have Nix. If it does, I
   will show you instructions on how to clean up your old install.

...

6. Start the Nix daemon.

Would you like to see a more detailed list of what I will do?
[y/n] 

@jakubgs jakubgs force-pushed the ci/nix-upgrade branch 2 times, most recently from 11bd436 to b8088e8 Compare January 13, 2022 15:42
@jakubgs
Copy link
Member Author

jakubgs commented Jan 13, 2022

For some reason the installator is not creating the /etc/profile.d/nix.sh file:

admin@maci7-03.ms-eu-dublin.slave.ci:~ % cat /etc/profile.d/nix.sh
cat: /etc/profile.d/nix.sh: No such file or directory

@jakubgs
Copy link
Member Author

jakubgs commented Jan 13, 2022

The way Nix is sources id from modified /etc/bashrc and /etc/zshrc:

jenkins@maci7-03.ms-eu-dublin.slave.ci:~ % tail -n6 /etc/bashrc 
# Nix
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
  . '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi
# End Nix

@jakubgs
Copy link
Member Author

jakubgs commented Jan 13, 2022

It's worth noting that what needs to be sourced is /nix/var/nix/profiles/default/etc/profile.d/nix.sh and not /nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh.

@jakubgs jakubgs force-pushed the ci/nix-upgrade branch 2 times, most recently from b6cd3f4 to 393b6e6 Compare January 13, 2022 20:01
@jakubgs
Copy link
Member Author

jakubgs commented Feb 17, 2022

I've created an issue in Nix repository outlining the sandbox problem:

@jakubgs jakubgs changed the title ci: upgrade Nix from 2.3.12 to 2.5.1 ci: upgrade Nix from 2.3.12 to 2.6.0 Feb 18, 2022
@jakubgs
Copy link
Member Author

jakubgs commented Mar 7, 2022

The issue with the sandbox is lack of defined truested-users, which should include jenkins: NixOS/nix#6115 (comment)

@jakubgs
Copy link
Member Author

jakubgs commented Mar 7, 2022

Looks like adding jenkins user to trusted-users in https://github.com/status-im/infra-ci/commit/8d6b6b3f did work, but now we get:

[2022-03-07T15:48:15.918Z] error: getting attributes of path ****: Permission denied

So looks like nixbld* users can't access files created by Jenkins credentials systemd.

@jakubgs
Copy link
Member Author

jakubgs commented Mar 7, 2022

Interestingly enough, the file permissions are already quite broad, with read for all users:

18:24:21  + ls -l /tmp/sandbox-file.tmp
18:24:21  -rw-rw-r-- 1 jenkins jenkins 243 Mar  7 17:24 /tmp/sandbox-file.tmp

@jakubgs
Copy link
Member Author

jakubgs commented Mar 7, 2022

I tried changing file permissions but it doesn't seem to work:

19:43:42  + ls -l ****
19:43:42  -r-------- 1 jenkins jenkins 2163 Mar  7 18:43 ****
[Pipeline] sh
19:43:43  + chgrp nixbld ****
19:43:43  + chmod 664 ****
[Pipeline] sh
19:43:44  + ls -l ****
19:43:44  -rw-rw-r-- 1 jenkins nixbld 2163 Mar  7 18:43 ****

@jakubgs
Copy link
Member Author

jakubgs commented Mar 7, 2022

Ah, I see, there it is. The issue is the permissions of the folder:

jenkins@linux-03:~$ ls -l /home/jenkins/workspace/tatus-react_prs_android_PR-12980@tmp/
total 8
drwxrwxr-x 2 jenkins jenkins 4096 Mar  7 19:03 durable-1d67657a
drwx------ 2 jenkins jenkins 4096 Mar  7 19:02 secretFiles

@jakubgs
Copy link
Member Author

jakubgs commented Mar 7, 2022

The issue on iOS with this error:

error: getting status of '/private/tmpxxx-tmp-status-react-cb10ffd99': No such file or directory

Appears to be fixed in Nix 2.6.1, as I described in NixOS/nix#6214.

@jakubgs
Copy link
Member Author

jakubgs commented Mar 7, 2022

For some reason setting 750 isn't enough. I wants 775 which is far too broad.

I need to do more testing to figure out how to narrow down the permissions as much as I can.

@jakubgs
Copy link
Member Author

jakubgs commented Mar 8, 2022

I did some stracing and found this in nix-daemon syscalls:

2291348 newfstatat(AT_FDCWD, "/tmp/sandbox-dir/test", 0x7f96160f9ac0, 0) = -1 EACCES (Permission denied)

And according to the docs for newfstatat:

These functions return information about a file, in the buffer pointed to by statbuf. No permissions are required on the file itself, but—in the case of stat(), fstatat(), and lstat() - execute (search) permission is required on all of the directories in pathname that lead to the file.

But the folder already has execute permissions for bixbld group:

 > ls -l /tmp | grep sandbox-dir
drwxr-x--- 2 jakubgs nixbld     3 Mar  8 10:29 sandbox-dir

So there's something else missing. Maybe the group I'm assuming is necessary is wrong.

@jakubgs
Copy link
Member Author

jakubgs commented Mar 8, 2022

I've managed to fix the permission issue with: https://github.com/status-im/status-jenkins-lib/commit/91a22e59

But I would really prefer if I could narrow down the permissions to just group ones instead of "other".

@jakubgs jakubgs changed the title ci: upgrade Nix from 2.3.12 to 2.6.0 ci: upgrade Nix from 2.3.12 to 2.6.1 Mar 8, 2022
@jakubgs
Copy link
Member Author

jakubgs commented Mar 8, 2022

This change is ready, but since a new release is brewing I'm going to wait with merging this until the release is out.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
@jakubgs jakubgs force-pushed the ci/nix-upgrade branch 2 times, most recently from 11178eb to 82c0dde Compare April 11, 2022 09:33
Due to changes in how Nix handles Git refs we need to specify
`refs/tags/` prefix in `package.json` to avoid the following error:
```
fatal: couldn't find remote ref refs/heads/v2.0.3-status-v6
error: program 'git' failed with exit code 128
```

I also had to rewrite some logic in `nix/scripts/source.sh` in order to
take account of single-user and multi-user installations.
We default to multi-user for Darwin, but not for any other OS due to
discovered issues with `nix-daemon` socket on Arch and open file limits.

I also rewrote `nix/scripts/setup.sh` and `/nix/scripts/purge.sh` to support
different types of installations. Both single-user and multi-user, as some
operating systems have issues with multi-user installations.

Resolves: #12832
Depends on: status-im/status-jenkins-lib#37

Related changes:
* status-im/infra-ci@84947b9f
* status-im/infra-ci@bb98f5f3
* status-im/infra-ci@f75d524d
* status-im/infra-ci@d1fc92cd
* status-im/infra-ci@87c4091e
* status-im/infra-ci@8d6b6b3f
* status-im/infra-ci@c4f13285
* status-im/infra-ci@38ac698d

Signed-off-by: Jakub Sokołowski <jakub@status.im>
@jakubgs jakubgs merged commit 0311983 into develop Apr 11, 2022
@jakubgs jakubgs deleted the ci/nix-upgrade branch April 11, 2022 10:27
Pipeline for QA automation moved this from CONTRIBUTOR to DONE Apr 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Nix 2.4 issues with git tags
5 participants