Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BLST-based crypto package #4358

Merged
merged 238 commits into from Dec 6, 2023
Merged
Show file tree
Hide file tree
Changes from 212 commits
Commits
Show all changes
238 commits
Select commit Hold shift + click to select a range
dfe442c
add c and asm srouce files from blst
tarakby Jan 18, 2023
635f960
add blst header files and README
tarakby Jan 18, 2023
abe0fe1
cgo directives to compile blst files
tarakby Jan 18, 2023
59be524
update README and clean up C directives
tarakby Jan 24, 2023
56b4b56
remove non code files
tarakby Jan 25, 2023
3c5accd
disable c99 and compile blst src and assembly
tarakby Feb 14, 2023
3c0247c
include blst.h
tarakby Feb 14, 2023
79601b6
tidy go.mod after removing blst package
tarakby Feb 14, 2023
e64cc36
add missing relic flags
tarakby Feb 14, 2023
e51d94e
first iteration of scalar type change
tarakby Feb 17, 2023
7593960
merge feature branch
tarakby Feb 17, 2023
0a066ec
merge keygen updates
tarakby Feb 17, 2023
b99d75b
use new type Fr in BLS simple sig
tarakby Feb 19, 2023
e3f4fee
implement BLS threshold signature with new Fr type
tarakby Feb 25, 2023
7163fc1
Merge pull request #3851 from onflow/tarak/add-blst-src
tarakby Feb 27, 2023
8b49d0a
add Fermat inversion - Lagrange interpolation works with Euclidean in…
tarakby Feb 27, 2023
1b3293f
DKG feldmanVSS with Fr type
tarakby Feb 27, 2023
2bbea26
enable all DKG protocols to work with new Fr type
tarakby Feb 28, 2023
b113d33
uncomment tests
tarakby Mar 3, 2023
7a2617c
renaming and linter errors
tarakby Mar 3, 2023
d7f3d5d
fix gcc compilation issue and remove blst.h
tarakby Mar 3, 2023
4979760
fix double definition
tarakby Mar 3, 2023
90c412f
enable tmate on ci temporarily
tarakby Mar 6, 2023
d96d6fa
test improvement and temporary memory free
tarakby Mar 6, 2023
0593da5
fix memory allocation bug in temp function
tarakby Mar 6, 2023
0e8829e
Revert "enable tmate on ci temporarily"
tarakby Mar 6, 2023
b2ae93c
Merge pull request #3993 from onflow/tarak/2-blst-types
tarakby Mar 6, 2023
52cae3e
g1 and g2 exportable types to cgo
tarakby Mar 8, 2023
e783a42
new Fp and Fp2 tools, but still the mess
tarakby Apr 6, 2023
f025a01
first changes to use new type G2
tarakby Apr 11, 2023
a5738f6
G2 type working for BLS
tarakby Apr 13, 2023
25b7be6
integrate G2 in BLS multi-sig
tarakby Apr 14, 2023
444f755
update BLSBatchVerify with regards to invalid signature format
tarakby Apr 15, 2023
d70883b
fix a bug and minor updates
tarakby Apr 15, 2023
54ee84a
BLS threshold signature works with new G2 type
tarakby Apr 16, 2023
ab67ed4
DKG works with new G2 type
tarakby Apr 16, 2023
693029f
BLS-SPoCK works with new G2 type
tarakby Apr 16, 2023
1a93763
clean up and G2 to E2 renaming
tarakby Apr 17, 2023
5cbfd08
rename some G1/G2 functions to E1/E2
tarakby Apr 17, 2023
8595b05
Merge pull request #4230 from onflow/tarak/g2-blst-type
tarakby Apr 17, 2023
2070aa6
Merge branch 'feature/blst-based-crypto' of github.com:onflow/flow-go…
tarakby Apr 18, 2023
f104505
various renaming in DKG functions
tarakby Apr 18, 2023
6f044c2
renaming and add Fr_generate_poly function
tarakby Apr 18, 2023
acb053f
update C polynomial headers to use degree
tarakby Apr 19, 2023
fdf18be
use pseudo-random randFr, randFrStar and FrPolynomial
tarakby Apr 19, 2023
3d26f0c
unify seed lengths of DKG and other keyGen seed lengths
tarakby Apr 19, 2023
768602c
update randG2 to map to G2 and update membership check in G2 tests
tarakby Apr 20, 2023
2167684
membership check in G2 using BLST
tarakby Apr 20, 2023
2d90a70
update batch verify random coefficients
tarakby Apr 21, 2023
183627b
minor cleanup
tarakby Apr 21, 2023
c033600
fix linter
tarakby Apr 21, 2023
558c149
merge master into feature branch
tarakby Apr 24, 2023
0a2e943
add replaces to test using latest crypto package
tarakby Apr 24, 2023
7bd182a
temp update to makefile to setup crypto with replace statement
tarakby Apr 25, 2023
9ae8df2
mod tidy
tarakby Apr 25, 2023
2fb816c
enable membership check in G2 to fix FVM test
tarakby Apr 25, 2023
54b92e6
fix E2 infinity set and check to be based on projective Z
tarakby Apr 26, 2023
f387ce9
fix warning
tarakby Apr 26, 2023
cbe51a3
disable thread SAN
tarakby Apr 26, 2023
be727d8
add SIGILL handler
tarakby Apr 26, 2023
6bd85a7
fix blst_cgo_init
tarakby Apr 26, 2023
76fcc73
disable ADX instructions in BlST by default as a temp measure
tarakby Apr 27, 2023
e07178d
merge latest master changes
tarakby Apr 27, 2023
2a851b5
uncomment DKG tests
tarakby Apr 28, 2023
26e5636
more logging in FeldmanVSSQ when shares aren't matching computed keys…
tarakby Apr 29, 2023
c590019
merge main feature branch
tarakby May 1, 2023
5e01e46
use new KeyGenSeedMinLen
tarakby May 1, 2023
bbd3c74
fix a bug when zeroring a buffer that is not a multiple of 8 bytes wi…
tarakby May 2, 2023
633f337
Merge pull request #4264 from onflow/tarak/blst-merge-master
tarakby May 3, 2023
187aeb3
Merge branch 'feature/blst-based-crypto' into tarak/replace-relic-rand
tarakby May 3, 2023
d6b4e86
clean ups
tarakby May 3, 2023
9f20a59
clean up encode/decode tests in BLS
tarakby May 3, 2023
f2731a8
add endianness test for maptoFr
tarakby May 3, 2023
70c3c64
add endianness comment
tarakby May 3, 2023
c54e2aa
Merge pull request #4253 from onflow/tarak/replace-relic-rand
tarakby May 3, 2023
21c4686
add blst Go package as an internal test package
tarakby May 8, 2023
98bb9cc
clarify internal blst package structure in comment
tarakby May 8, 2023
d729688
update bls cross-checks
tarakby May 8, 2023
ab2fa14
upgrade rapid package
tarakby May 8, 2023
9d35d3d
fix linter false positives
tarakby May 9, 2023
6031d71
go mod tidy
tarakby May 9, 2023
6b5fd8f
Revert "upgrade rapid package"
tarakby May 9, 2023
e35860f
Revert "go mod tidy"
tarakby May 9, 2023
22b1ef8
fix linter error
tarakby May 9, 2023
f66205c
Merge pull request #4328 from onflow/blst_crossblst_test
tarakby May 9, 2023
0290a98
clean up of multiple pairing
tarakby May 9, 2023
7f86c94
Fp12 and multi-pairing computation
tarakby May 11, 2023
7b7e484
use E1 blst type in Go
tarakby May 12, 2023
7cafb2b
use POINTonEx_mult_glv instead of blst_sign and remove Relic's E1 mult
tarakby May 12, 2023
109b238
E1_read_bytes and E1_write_bytes and their tools
tarakby May 12, 2023
5ae1abb
G1 membership check and connect E1 read/write to the Go layer
tarakby May 12, 2023
8d32cff
map to G1 using BLST and add length sanity check of input hash
tarakby May 12, 2023
9efaddb
remove local SSWU implementation and rely only on BLST map to G1 tools
tarakby May 12, 2023
56081df
clean up membership check macros and delete Bowe's check code
tarakby May 12, 2023
13025e1
refactor bls_verify_ep to use hashed point
tarakby May 12, 2023
8885e5d
add temporary E1_blst_to_relic and use E1 in all bls_core functions -…
tarakby May 12, 2023
9e938a9
implement mapping to Fp to use in map_to_G1
tarakby May 12, 2023
f10d881
fix E1_read_bytes bug and improve debug printing
tarakby May 13, 2023
d3396c6
update BLS threshold signature with E1 points
tarakby May 15, 2023
557d3a7
uncomment BLST cross check tests
tarakby May 15, 2023
878b7e7
spock works with new E1 type
tarakby May 15, 2023
49b01a8
write E1_sum_vector using new E1 type
tarakby May 15, 2023
534c0a8
clean up unsecure mapping to G1/G1 and fix subgroup checks
tarakby May 15, 2023
c338c93
clean up constants pre-computation
tarakby May 15, 2023
3fd0539
add read/write tests for G1 points
tarakby May 15, 2023
29f7489
rename insecure to unsafe
tarakby May 16, 2023
c47c321
fix node info comparison bug in test
tarakby May 16, 2023
184a49d
fix public key comparison bugs in tests
tarakby May 16, 2023
a4fb435
yet another key comparison bug in tests
tarakby May 16, 2023
ff5a0c7
another bug: compare ER bassed on IDs and not the in-mem struct
tarakby May 16, 2023
eae8458
Merge pull request #4355 from onflow/tarak/blst-E1-G1
tarakby May 16, 2023
22db5b0
multi-pairing function used in BLS single signatture verify
tarakby May 23, 2023
ee91d4f
use BLST multi_pairing to verify BLS signature with many messages
tarakby May 23, 2023
e84403d
add test calling multi_pairing with length covering many values mod N…
tarakby May 23, 2023
9167d9e
use BLST multi_pairing to verify BLS SPoCK
tarakby May 23, 2023
cf0aa6b
clean up Fp12 tools
tarakby May 23, 2023
98152ba
clean up Relic tools
tarakby May 23, 2023
b6b90d6
uncomment a test
tarakby May 23, 2023
c94b091
Merge pull request #4377 from onflow/tarak/blst-pairing
tarakby May 24, 2023
3ec1ecf
remove relic tag and delete non-needed files
tarakby May 24, 2023
64112cc
remove relic build scripts
tarakby May 24, 2023
19a21db
remove relic macros and xmd_sha256, remove relic binray from LD flags
tarakby May 24, 2023
e58fe24
update Makefile and dockerignore
tarakby May 24, 2023
8123970
remove Relic mentions in code and README
tarakby May 25, 2023
541df79
update flow-go/README and gitignore
tarakby May 25, 2023
1d0b8f9
remove relic related commands from Makefile/ci/dockerfile
tarakby May 25, 2023
4eabaf1
remove relic tags from go files
tarakby May 25, 2023
9b17f93
remove more relic related tags and code
tarakby May 25, 2023
5721546
remove crypto_setup_gopath
tarakby May 25, 2023
06572e3
update go generate
tarakby May 25, 2023
33c5e0e
remove cmake install from dockerfile
tarakby May 26, 2023
83f42fb
clean up header files in blst_include.h
tarakby May 27, 2023
ec2ceb4
update boolean usage from bool_t to C type bool
tarakby May 27, 2023
fb4ac12
add sanity check scalar mult in G1 and G2
tarakby May 28, 2023
0232a95
use not enough shares error in BLSReconstructThresholdSignature
tarakby May 28, 2023
5fa28df
refactor BLS constants to use internal BLS12_381 length constants
tarakby May 28, 2023
efefb51
Merge pull request #4380 from onflow/tarak/blst-unplug-relic
tarakby May 29, 2023
d2c7cbf
more consolidation of length constants
tarakby May 29, 2023
a888972
cgo supports macros! use C constant macros in go
tarakby May 29, 2023
945f2b9
define new internal ERROR type to abstract BLST_ERROR
tarakby May 30, 2023
06c4ca7
update code base to work for G1 serialization defined as uncompressed
tarakby May 30, 2023
9ed47f1
update code base to work for G2 serialization defined as uncompressed
tarakby May 31, 2023
01b64c5
make sure older compilers recognize uintx_t
tarakby May 31, 2023
7b0a25e
update crypto/Makefile go command
tarakby May 31, 2023
00f66d6
package default build uses ADX
tarakby May 31, 2023
f5c3668
Merge pull request #4397 from onflow/tarak/blst-improv
tarakby May 31, 2023
e998ab6
add ADX detection and cgo flags for all go commands in Makefile/Docke…
tarakby May 31, 2023
5dba1c5
clarify BLST sigill message
tarakby Jun 1, 2023
f471e4b
fix a bug in sigill string
tarakby Jun 1, 2023
d4b873d
update how cgo flag is passed to Dockerfile so that dittos aren't del…
tarakby Jun 1, 2023
fa5177f
add cgo flag to mockgen commands
tarakby Jun 1, 2023
2872fe7
Merge pull request #4405 from onflow/tarak/blst-adx
tarakby Jun 9, 2023
2a87898
remove test assertion strings in favor of PRG seed logging for test r…
tarakby Aug 17, 2023
8ffac58
add interface implementation sanity checks
tarakby Aug 17, 2023
8415a45
add faster scalar mult in E2 for small expos
tarakby Aug 18, 2023
a408dec
more implementation check sanity check
tarakby Aug 18, 2023
65ee3bf
clean up some todos and add global g2 key
tarakby Aug 18, 2023
b379744
address more TODOs
tarakby Aug 18, 2023
9d6c7c7
add c-formatting target
tarakby Aug 18, 2023
30d781c
remove clanf-format config file
tarakby Aug 18, 2023
647c0c2
add crypto code formatting check to CI
tarakby Aug 18, 2023
63cedc8
move c formatting to linter target
tarakby Aug 21, 2023
a8666e4
fix linter error
tarakby Aug 21, 2023
20b89d2
Merge pull request #4645 from onflow/tarak/blst-c-formatting
tarakby Aug 21, 2023
30e5a7b
delete unused fermat inversion
tarakby Aug 22, 2023
8556f69
update README with BLST update steps
tarakby Aug 22, 2023
c1f294c
temp tmate debug and compile with asan
tarakby Aug 24, 2023
bacdb33
tmp
tarakby Aug 24, 2023
c8b643c
add new target for sanitization
tarakby Aug 24, 2023
62c1a16
add sanitizer to ci job
tarakby Aug 24, 2023
e557e8b
add more sanitization flags and restrict sanitization to linux
tarakby Aug 29, 2023
f8bc02b
add sanitization to ci
tarakby Aug 29, 2023
d58b172
disable sanitization for E1_write_bytes because of false positive
tarakby Aug 29, 2023
e280664
split c-sanitize and disable msan from CI - add NO_MSAN macro
tarakby Aug 29, 2023
aa8d79e
disable tmate and format
tarakby Aug 29, 2023
b2302c9
add missing change
tarakby Aug 29, 2023
ceab7e0
fix asan command
tarakby Aug 29, 2023
8314b0c
more details about updating BLST version
tarakby Aug 30, 2023
6bb393c
Merge pull request #4654 from onflow/tarak/blst-sanitize
tarakby Aug 31, 2023
4adb5cf
minor macro improvement
tarakby Sep 2, 2023
e6b29bc
add types sanity check in init()
tarakby Sep 2, 2023
5305b28
add affine conversions for potential public keys that can be used in …
tarakby Sep 2, 2023
afa9f24
clarify some TODOs
tarakby Sep 6, 2023
76850aa
DKG's readVector enforces A to be in G2
tarakby Sep 6, 2023
589d8d7
format
tarakby Sep 6, 2023
14c9e3d
clean up c flags and add instruction to readme
tarakby Sep 6, 2023
140edc3
Merge pull request #4681 from onflow/tarak/blst-misc
tarakby Sep 6, 2023
4c7af01
merge master and fix conflicts
tarakby Sep 6, 2023
12e338e
c format
tarakby Sep 6, 2023
7a268ae
mod tidy
tarakby Sep 6, 2023
7640d94
remove deprecated Seed use
tarakby Sep 6, 2023
5c72468
add temp tmate
tarakby Sep 6, 2023
5011cc6
fix merging issues and delete relic related builds
tarakby Sep 7, 2023
c52da9c
Revert "add temp tmate"
tarakby Sep 7, 2023
e5c0630
update flakey test monitor
tarakby Sep 7, 2023
d1776c8
more clarifications to BLST version update README
tarakby Sep 7, 2023
ecf7021
update BLST source to v0.3.11
tarakby Sep 7, 2023
e478781
Merge pull request #4687 from onflow/tarak/blst-merge-master
tarakby Sep 7, 2023
3d062fc
Merge branch 'feature/blst-based-crypto' into tarak/blst-update-blst
tarakby Sep 7, 2023
241798f
clean up assembly files include in cgo compilations
tarakby Sep 8, 2023
3bea523
update internal/blst files
tarakby Sep 8, 2023
c292bc4
fix a readme typo
tarakby Sep 8, 2023
0d09d55
tmp tmate to debug
tarakby Sep 9, 2023
dce50f9
make start up node time larger to accommodate failing TestClusterSwit…
tarakby Sep 12, 2023
633f152
merge master branch
tarakby Sep 13, 2023
6fb9707
Merge pull request #4693 from onflow/tarak/blst-update-blst
tarakby Sep 13, 2023
ba78ef6
makefile typo
tarakby Sep 21, 2023
8ff9b79
clean up C bls12_381 utils
tarakby Sep 21, 2023
2332a61
clean up threshold and dkg C files - use poly degree in secret sharing
tarakby Sep 21, 2023
2cd3d28
format c files
tarakby Sep 21, 2023
262c3e0
rename G2_ prefix to E2_
tarakby Sep 21, 2023
7184bc5
Merge pull request #4740 from onflow/tarak/blst-cleanup
tarakby Sep 21, 2023
dc28e03
move godoc closer to the type definition
tarakby Oct 10, 2023
627d682
add E1 random point multiplication benchmark
tarakby Oct 11, 2023
cf8667b
remove StopTimer in bench
tarakby Oct 11, 2023
5f89c65
comment updates and reformat
tarakby Oct 11, 2023
0587bc6
fix non-freed memory in error case
tarakby Oct 12, 2023
f64d5ea
use a common logic to detect ADX support and set the crypto flag
tarakby Oct 20, 2023
6c34ae3
add noop target for empty ci operations
tarakby Oct 20, 2023
b180269
cleaning and typos
tarakby Oct 20, 2023
6a42f1e
merge master branch
tarakby Oct 21, 2023
5f2509d
Merge branch 'master' into jordan/merge-master-blst
jordanschalm Oct 23, 2023
7ad6a7a
make tidy
jordanschalm Oct 23, 2023
f45e546
fix overwritten test settings
tarakby Oct 23, 2023
34f18b3
merge master and fix conflicts
tarakby Nov 15, 2023
99b1237
remove crypto setup
tarakby Nov 15, 2023
867b2cf
merge master
tarakby Nov 20, 2023
fbfecc1
update graceful stop duration to 1s in integration upgrade test
tarakby Nov 21, 2023
c990e22
Merge branch 'master' into jordan/merge-master-blst
tarakby Nov 24, 2023
91f529c
Merge pull request #4863 from onflow/jordan/merge-master-blst
tarakby Nov 24, 2023
4839835
merge main feature branch
tarakby Nov 24, 2023
cd2d74e
slow down block rate in integration access cohort1 test
tarakby Nov 24, 2023
500f205
Merge pull request #4817 from onflow/tarak/blst-review
tarakby Nov 28, 2023
f713b2e
Merge branch 'master' into feature/blst-based-crypto
tarakby Dec 5, 2023
769ad67
slow down block production in bft tests
tarakby Dec 5, 2023
f7dac6c
slow down block rate in SN test
tarakby Dec 5, 2023
e3e29f0
fix SN integration test bug
tarakby Dec 5, 2023
bea7bf6
Merge pull request #5111 from onflow/tarak/blst-fix-integration-test
tarakby Dec 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 3 additions & 6 deletions .github/workflows/bench.yml
Expand Up @@ -44,19 +44,16 @@ jobs:
go-version: "1.20"
cache: true

- name: Build relic
run: make crypto_setup_gopath

- name: Run benchmark on current branch
run: |
(for i in {1..${{ steps.settings.outputs.benchmark_repetitions }}}; do go test ./fvm ./engine/execution/computation --bench . --tags relic -shuffle=on --benchmem --run ^$; done) | tee new.txt
(for i in {1..${{ steps.settings.outputs.benchmark_repetitions }}}; do go test ./fvm ./engine/execution/computation --bench . -shuffle=on --benchmem --run ^$; done) | tee new.txt

- name: Checkout base branch
run: git checkout ${{ github.event.pull_request.base.sha }}

- name: Run benchmark on base branch
run: |
(for i in {1..${{ steps.settings.outputs.benchmark_repetitions }}}; do go test ./fvm ./engine/execution/computation --bench . --tags relic -shuffle=on --benchmem --run ^$; done) | tee old.txt
(for i in {1..${{ steps.settings.outputs.benchmark_repetitions }}}; do go test ./fvm ./engine/execution/computation --bench . -shuffle=on --benchmem --run ^$; done) | tee old.txt

# see https://trstringer.com/github-actions-multiline-strings/ to see why this part is complex
- name: Use benchstat for comparison
Expand Down Expand Up @@ -85,7 +82,7 @@ jobs:

This branch with compared with the base branch ${{ github.event.pull_request.base.label }} commit ${{ github.event.pull_request.base.sha }}

The command `(for i in {1..${{ steps.settings.outputs.benchmark_repetitions }}}; do go test ./fvm ./engine/execution/computation --bench . --tags relic -shuffle=on --benchmem --run ^$; done)` was used.
The command `(for i in {1..${{ steps.settings.outputs.benchmark_repetitions }}}; do go test ./fvm ./engine/execution/computation --bench . -shuffle=on --benchmem --run ^$; done)` was used.

<details>
<summary>Collapsed results for better readability</summary>
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/cd.yml
Expand Up @@ -17,8 +17,6 @@ jobs:
go-version: "1.20"
- name: Checkout repo
uses: actions/checkout@v2
- name: Build relic
run: make crypto_setup_gopath
# Provide Google Service Account credentials to Github Action, allowing interaction with the Google Container Registry
# Logging in as github-actions@dl-flow.iam.gserviceaccount.com
- name: Docker login
Expand Down
40 changes: 13 additions & 27 deletions .github/workflows/ci.yml
Expand Up @@ -41,8 +41,10 @@ jobs:
with:
go-version: ${{ env.GO_VERSION }}
cache: true
- name: Build relic
run: make crypto_setup_gopath
- name: Install C formatter
run: sudo apt-get install -y clang-format
- name: Run C formatter and sanitizer for ./crypto
run: make -C crypto c-format && make -C crypto c-sanitize
- name: Run go generate
run: go generate
working-directory: ${{ matrix.dir }}
Expand All @@ -51,10 +53,11 @@ jobs:
with:
# Required: the version of golangci-lint is required and must be specified without patch version: we always use the latest patch version.
version: v1.51
args: -v --build-tags relic
args: -v
working-directory: ${{ matrix.dir }}
# https://github.com/golangci/golangci-lint-action/issues/244
skip-cache: true


tidy:
name: Tidy
Expand All @@ -72,18 +75,6 @@ jobs:
- name: code sanity check
run: make code-sanity-check

shell-check:
name: ShellCheck
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v3
- name: Run ShellCheck
uses: ludeeus/action-shellcheck@203a3fd018dfe73f8ae7e3aa8da2c149a5f41c33
with:
scandir: './crypto'
ignore: 'relic'

create-dynamic-test-matrix:
name: Create Dynamic Test Matrix
runs-on: ubuntu-latest
Expand Down Expand Up @@ -144,18 +135,15 @@ jobs:
matrix:
include:
- name: crypto
make1: -C crypto setup
make2: unittest
setup:
retries: 1
race: 1
- name: insecure
make1: install-tools
make2: test
setup: install-tools
retries: 3
race: 0
- name: integration
make1: install-tools
make2: test
setup: install-tools
retries: 3
race: 0
runs-on: ubuntu-latest
Expand All @@ -168,16 +156,16 @@ jobs:
go-version: ${{ env.GO_VERSION }}
cache: true
- name: Setup tests (${{ matrix.name }})
run: make ${{ matrix.make1 }}
run: make ${{ matrix.setup }}
- name: Run tests (${{ matrix.name }})
env:
RACE_DETECTOR: ${{ matrix.race }}
uses: nick-fields/retry@v2
with:
timeout_minutes: 25
max_attempts: ${{ matrix.retries }}
# run `make2` target inside each module's root
command: VERBOSE=1 make -C ${{ matrix.name }} ${{ matrix.make2 }}
# run test target inside each module's root
command: VERBOSE=1 make -C ${{ matrix.name }} test
- name: Upload coverage report
uses: codecov/codecov-action@v3
with:
Expand Down Expand Up @@ -217,8 +205,6 @@ jobs:
with:
go-version: ${{ env.GO_VERSION }}
cache: true
- name: Build relic
run: make crypto_setup_gopath
- name: Docker build
run: make docker-build-flow docker-build-flow-corrupt
- name: Run tests
Expand All @@ -244,7 +230,7 @@ jobs:
with:
go-version: ${{ env.GO_VERSION }}
cache: true
- name: Build relic and other tools
- name: install tools
run: make install-tools
- name: Install Flow Client In Docker
# This proved to be more reliable than installing it locally.
Expand Down
15 changes: 5 additions & 10 deletions .github/workflows/flaky-test-monitor.yml
tarakby marked this conversation as resolved.
Show resolved Hide resolved
Expand Up @@ -83,18 +83,15 @@ jobs:
matrix:
include:
- name: crypto
make1: -C crypto setup
make2: unittest
setup:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should have an explicit target to call in crypto/Makefile now that setup target is no longer there.
By leaving this blank, this will execute the first target crypto/Makefile which is c-format. Is this the intention?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it's not the intention. I didn't know the first target is run. I'll add a noop target as you suggested

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm noticing the setup is called using make ${{ matrix.setup }}, so I guess this is only involving ./Makefile and not crypto/Makefile. I'll add the noop target in ./Makefile then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

race: 1
test_category: unit-crypto
- name: insecure
make1: install-tools
make2: test
setup: install-tools
race: 0
test_category: unit-insecure
- name: integration
make1: install-tools
make2: test
setup: install-tools
race: 0
test_category: unit-integration
runs-on: ubuntu-latest
Expand All @@ -107,11 +104,11 @@ jobs:
go-version: ${{ env.GO_VERSION }}
cache: true
- name: Setup tests (${{ matrix.name }})
run: make ${{ matrix.make1 }}
run: make ${{ matrix.setup }}
- name: Run tests (${{ matrix.name }})
env:
RACE_DETECTOR: ${{ matrix.race }}
run: make -es -C ${{ matrix.name }} ${{ matrix.make2 }} > test-output
run: make -es -C ${{ matrix.name }} test > test-output
timeout-minutes: 100
continue-on-error: true
- name: Process test results (${{ matrix.name }})
Expand Down Expand Up @@ -168,8 +165,6 @@ jobs:
with:
go-version: ${{ env.GO_VERSION }}
cache: true
- name: Build relic
run: make crypto_setup_gopath
- name: Docker build
run: make docker-build-flow docker-build-flow-corrupt
- name: Run tests
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/tools.yml
Expand Up @@ -36,8 +36,6 @@ jobs:
uses: actions/checkout@v2
with:
ref: ${{ inputs.tag }}
- name: Build relic
run: make crypto_setup_gopath
- name: Build and upload boot-tools
run: |
make tool-bootstrap tool-transit
Expand Down
2 changes: 0 additions & 2 deletions .gitignore
Expand Up @@ -7,8 +7,6 @@
/cmd/util/util
/cmd/bootstrap/bootstrap

# crypto relic folder
crypto/relic/

# Test binary, build with `go test -c`
*.test
Expand Down