nix: fortify the nix-update-gradle process

[jakubgs]

This fixes two issues with the nix-update-gradle target:

• It now fails when a JAR is missing which used to be ignored.
• It ignores dependencies that have no JARs, like Eclipse plugins.

This makes the process more robust, since we can see something is missing right away, and a developer may re-run the process to take account of possible temporary networking failures or rate limiting.

It also slims down the size of the deps.json by removing dependencies which contribute no actual JARs or AARs to the build process.

Also limited the number of jobs started by parallel for nix/deps/gradle/url2json.sh to limit possibility of rate limiting.

[status-im-auto]

Jenkins Builds

Click to see older builds (9)

❔	Commit	#️⃣	Finished (UTC)	Duration	Platform	Result
✔️	d7fa4d8	#1	2022-05-17 17:56:55	~9 min	android-e2e	📦apk 📲
✔️	d7fa4d8	#1	2022-05-17 17:57:05	~9 min	android	📦apk 📲
✔️	d7fa4d8	#1	2022-05-17 18:00:10	~12 min	ios	📦ipa 📲
✔️	c6cf95a	#2	2022-05-17 18:18:49	~16 min	android-e2e	📦apk 📲
✔️	c6cf95a	#2	2022-05-17 18:19:17	~17 min	android	📦apk 📲
✔️	c6cf95a	#2	2022-05-17 18:19:46	~17 min	ios	📦ipa 📲
✔️	372942b	#3	2022-05-17 18:30:12	~8 min	android	📦apk 📲
✔️	372942b	#3	2022-05-17 18:30:19	~9 min	android-e2e	📦apk 📲
✔️	372942b	#3	2022-05-17 18:33:07	~11 min	ios	📦ipa 📲

❔	Commit	#️⃣	Finished (UTC)	Duration	Platform	Result
✔️	6601024	#4	2022-05-18 08:22:44	~8 min	android-e2e	📦apk 📲
✔️	6601024	#4	2022-05-18 08:23:04	~9 min	android	📦apk 📲
✔️	6601024	#4	2022-05-18 08:26:07	~12 min	ios	📦ipa 📲
✔️	27219eb	#5	2022-05-18 08:38:06	~8 min	android-e2e	📦apk 📲
✔️	27219eb	#5	2022-05-18 08:39:07	~10 min	android	📦apk 📲
✔️	27219eb	#5	2022-05-18 08:43:06	~13 min	ios	📦ipa 📲

[jakubgs]

Since @flexsurfer was having weird random failures to fetch POMs and JARs:

Done in: 1.615s
Regenerating Nix files...
Found 39 sub-projects...
Found 651 direct dependencies...
Found 1043 dependency URLs...
 ! Failed to fetch: https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-scripting-common/1.4.31/kotlin-scripting-common-1.4.31.pom
 ! Failed to fetch: https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-scripting-compiler-embeddable/1.4.31/kotlin-scripting-compiler-embeddable-1.4.31.pom
 ! Failed to fetch: https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-scripting-compiler-impl-embeddable/1.3.50/kotlin-scripting-compiler-impl-embeddable-1.3.50.pom
make: *** [nix-update-gradle] Error 3

Found 39 sub-projects...
Found 651 direct dependencies...
Found 1043 dependency URLs...
 ! Failed to fetch: https://dl.google.com/dl/android/maven2/androidx/annotation/annotation/1.0.2/annotation-1.0.2.pom
 ! Failed to fetch: https://dl.google.com/dl/android/maven2/androidx/annotation/annotation/1.1.0/annotation-1.1.0.pom

I've added a limit on the number of parallel jobs we start to be 4, to avoid rate-limiting.

[jakubgs]

@flexsurfer checked the logs for the fetch failures and it appears they are timeouts:

warning: error: unable to download 'https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-reflect/1.3.20/kotlin-reflect-1.3.20.pom': Timeout was reached (28); retrying in 353 ms
warning: error: unable to download 'https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-reflect/1.3.50/kotlin-reflect-1.3.50.pom': Timeout was reached (28); retrying in 341 ms
warning: error: unable to download 'https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-reflect/1.4.31/kotlin-reflect-1.4.31.pom': Timeout was reached (28); retrying in 284 ms
warning: error: unable to download 'https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-scripting-common/1.3.50/kotlin-scripting-common-1.3.50.pom': Timeout was reached (28); retrying in 312 ms
warning: error: unable to download 'https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-scripting-common/1.4.31/kotlin-scripting-common-1.4.31.pom': Timeout was reached (28); retrying in 315 ms
warning: error: unable to download 'https://repo.maven.apache.org/maven2/org/jetbrains/kotlin/kotlin-scripting-compiler-embeddable/1.3.50/kotlin-scripting-compiler-embeddable-1.3.50.pom': Timeout was reached (28); retrying in 313 ms

Which might still be rate-limiting - though a 429 error would be more obvious - or it's literally just running out of connections.

[jakubgs]

For future reference, the changes merged here are:

• 372942b - nix: fortify the nix-update-gradle process
• 43f7aaf - nix: limit jobs used by parallel to update Gradle

Damned PR closeing bug in merge script causes diff to disappear due to branch being auto-deleted.