[DEMO] reproducer / visualizer for i2095

[stenolog]

• re ErgoTransactionSpec is unstable #2095

---

The unstable test is

property("transaction with too many inputs should be rejected") {

it is timing-related.

3 observations:

• first execution take more time, it goes down later (cache? )
• the overall speed has increased significantly since v5.0.0, test was not adjusted
• timing behavior on CI is different (Timeout is slower in general, higher numbers)

It looks like halving the Timeout should do it.

Local Machine

sbt   "testOnly *ErgoTransactionSpec -- -z inputs"

v5.0.20

[info] ErgoTransactionSpec:
1: 455 > 155
2: 276 > 149
3: 240 > 145
4: 319 > 145
5: 160 > 144
6: 164 > 144
7: 159 > 150
8: 151 > 145
9: 201 > 150
10: 147 > 145
11: 486 > 257
12: 151 > 152
[info] - transaction with too many inputs should be rejected *** FAILED ***
[info]   151 was not greater than 152 (ErgoTransactionSpec.scala:375)

v5.0.4

[info] ErgoTransactionSpec:
1477 > 150
921 > 150
570 > 141
374 > 149
390 > 143
393 > 140
396 > 140
451 > 148
439 > 162

v5.0.0

[info] ErgoTransactionSpec:
1286 > 148
767 > 150
390 > 143
432 > 177
410 > 141
463 > 142
536 > 144
366 > 150
511 > 152
551 > 142
435 > 147
387 > 143

On CI

master (v5.0.20 +)

(halved the Timeout value)

using "testOnly *ErgoTransactionSpec -- -z inputs"

[info] ErgoTransactionSpec:
1: 419 > 108 (Timeout/2)
2: 265 > 99 (Timeout/2)
3: 256 > 97 (Timeout/2)
4: 191 > 97 (Timeout/2)
7: 128 > 97 (Timeout/2)
10: 140 > 97 (Timeout/2)
11: 126 > 97 (Timeout/2)
12: 141 > 97 (Timeout/2)
13: 141 > 97 (Timeout/2)
14: 155 > 97 (Timeout/2)
15: 169 > 96 (Timeout/2)
16: 186 > 98 (Timeout/2)
17: 127 > 97 (Timeout/2)
18: 129 > 98 (Timeout/2)
19: 156 > 99 (Timeout/2)
20: 127 > 98 (Timeout/2)
21: 123 > 98 (Timeout/2)
22: 158 > 98 (Timeout/2)
23: 124 > 98 (Timeout/2)
24: 143 > 98 (Timeout/2)
25: 124 > 97 (Timeout/2)
26: 177 > 100 (Timeout/2)
27: 156 > 98 (Timeout/2)
28: 130 > 98 (Timeout/2)
29: 128 > 98 (Timeout/2)
30: 166 > 98 (Timeout/2)
31: 161 > 98 (Timeout/2)
32: 161 > 98 (Timeout/2)
33: 153 > 97 (Timeout/2)
34: 125 > 102 (Timeout/2)
35: 129 > 97 (Timeout/2)
36: 167 > 98 (Timeout/2)
37: 173 > 98 (Timeout/2)
38: 134 > 97 (Timeout/2)
39: 128 > 98 (Timeout/2)
40: 181 > 98 (Timeout/2)
41: 182 > 97 (Timeout/2)
42: 124 > 98 (Timeout/2)
43: 128 > 98 (Timeout/2)
44: 125 > 98 (Timeout/2)
45: 178 > 98 (Timeout/2)
46: 127 > 98 (Timeout/2)
47: 130 > 97 (Timeout/2)
48: 127 > 98 (Timeout/2)
49: 131 > 97 (Timeout/2)
50: 129 > 98 (Timeout/2)

using the full ci run

1: 210 > 100 (Timeout/2)
2: 222 > 103 (Timeout/2)
3: 210 > 99 (Timeout/2)
4: 204 > 101 (Timeout/2)
5: 250 > 99 (Timeout/2)
6: 275 > 102 (Timeout/2)
7: 218 > 101 (Timeout/2)
8: 242 > 101 (Timeout/2)
9: 239 > 100 (Timeout/2)
10: 282 > 99 (Timeout/2)
11: 273 > 101 (Timeout/2)
12: 208 > 99 (Timeout/2)
13: 239 > 99 (Timeout/2)
14: 266 > 99 (Timeout/2)
15: 205 > 101 (Timeout/2)
16: 199 > 106 (Timeout/2)
17: 197 > 102 (Timeout/2)
18: 206 > 101 (Timeout/2)
19: 284 > 99 (Timeout/2)
20: 201 > 101 (Timeout/2)
21: 205 > 101 (Timeout/2)
22: 229 > 100 (Timeout/2)
23: 197 > 101 (Timeout/2)
24: 235 > 102 (Timeout/2)
25: 205 > 101 (Timeout/2)
26: 265 > 100 (Timeout/2)
27: 208 > 98 (Timeout/2)
28: 220 > 100 (Timeout/2)
29: 202 > 100 (Timeout/2)
30: 233 > 99 (Timeout/2)
31: 208 > 103 (Timeout/2)
32: 239 > 101 (Timeout/2)
33: 277 > 99 (Timeout/2)
34: 257 > 100 (Timeout/2)
35: 206 > 99 (Timeout/2)
36: 204 > 99 (Timeout/2)
37: 194 > 99 (Timeout/2)
38: 200 > 101 (Timeout/2)
39: 211 > 100 (Timeout/2)
40: 201 > 99 (Timeout/2)
41: 198 > 100 (Timeout/2)
42: 283 > 98 (Timeout/2)
43: 241 > 105 (Timeout/2)
44: 240 > 99 (Timeout/2)
45: 222 > 102 (Timeout/2)
46: 254 > 99 (Timeout/2)
47: 309 > 98 (Timeout/2)
48: 214 > 99 (Timeout/2)
49: 204 > 98 (Timeout/2)
50: 288 > 100 (Timeout/2)
[info] - transaction with too many inputs should be rejected

Run A

https://github.com/ergoplatform/ergo/actions/runs/8022318975/job/21916370871?pr=2120#step:5:8207

1 -209 > 212
[info] - transaction with too many inputs should be rejected *** FAILED ***
[info]   209 was not greater than 212 (ErgoTransactionSpec.scala:373)

Run B

1 -212 > 207
2 -224 > 202
3 -206 > 201
4 -234 > 201
5 -209 > 200
6 -209 > 217
[info] - transaction with too many inputs should be rejected *** FAILED ***
[info]   209 was not greater than 217 (ErgoTransactionSpec.scala:373)

[stenolog]

Closing this DEMO/reproducer PR. Fix is within:

• halve Timeout to avoid random CI failure #2121