Falcon - Select share + evaluating #4474

gmuraru · 2020-08-25T07:19:25Z

Description

This PR implements the share selection from the Falcon paper and the evaluation part.
Fixes the first point and the last from from #3852

How has this been tested?

Added test to validate it works

Checklist

I have followed the Contribution Guidelines and Code of Conduct
I have commented my code following the OpenMined Styleguide
I have labeled this PR with the relevant Type labels
My changes are covered by tests

gmuraru · 2020-08-26T22:28:46Z

syft/frameworks/torch/tensors/interpreters/replicated_shared.py

@@ -82,13 +88,13 @@ def retrieve_pointers(self):
        return pointers

    def __sum_shares(self, shares):
-        return sum(shares) % self.ring_size
+        return sum(shares) % self.field


@abogaziah changed the name to field to be consistent with the AdditiveSharingTensor + it seems also in CrypTen they use the same name attribute

I do not agree with that, field stands for the whole arithmetic field ]0, Z [, which makes it an incorrect and misleading name for the variable that represents Z only. also, the paper uses "ring size" so I think we should stick to it

omershlo · 2020-08-29T17:33:15Z

syft/frameworks/torch/mpc/falcon.py

+                when selecting between x or y
+
+        Return:
+            x_share or b_share depending on the value of b_share


typo: x_share or y_share

omershlo · 2020-08-29T18:25:34Z

syft/frameworks/torch/mpc/falcon.py

+        c_share_L = c.share(*workers, protocol="falcon", field=field)
+        c_share_2 = c.share(*workers, protocol="falcon", field=2)


I couldn't find the secret sharing scheme you are using (can you share a link?), but if we want to be consistent with the protocol it should be

replicated secret sharing , same as for b

it should be maliciously secure secret sharing

omershlo · 2020-08-29T18:29:21Z

syft/frameworks/torch/mpc/falcon.py

+        field: int, workers: List[BaseWorker]
+    ) -> (ReplicatedSharingTensor, ReplicatedSharingTensor):
+        """
+        TODO: Need to move this in Primitive Store:


this function is returning two different sharing which is not indicated from the name of the function and also makes it less general. why not return a single sharing and call the function twice?

Hmm..in that case I think I can completely remove this function - doing that now

omershlo · 2020-08-29T18:49:20Z

syft/frameworks/torch/mpc/falcon.py

+        If the bit b_share is 0, x_sh is returned
+        If the bit b_share is 1, y_sh is returned
+
+        Args:


as a general comment: The sharing used in ReplicatedSharingTensor (https://github.com/OpenMined/PySyft/pull/3856/files) is for semi-honest adversary and falcon is secure against malicious adversary...

Yep - currently the RST and the Falcon protocol are "combined" inside the RST (the \pi_mul operation is encapsulated in the RST) - if we add more protocols over RST we can have something like we have in the aditive_sharing_tensor.py

@crypto_protocl("fss") def add(self, other): # here we define how our operation over the AST should work if we use the FSS protocol

omershlo · 2020-08-29T18:57:29Z

syft/frameworks/torch/mpc/falcon.py

+        shape = x_share.shape
+
+        pow_sh = (
+            ReplicatedSharingTensor.one_shares(players, field=field, shape=shape) - beta_share * 2


I don't think that you need to share a 1 for that. a much more efficient way will be to do (see Falcon paper, Linear operations):
party1 share = 1-2[beta1]
party2 share = -2[beta2]
party3 share = -2[beta3]

Yep - I left a comment there - basically doing this I think becomes a "little" intrusive with the RST - but it is worth thinking if we can rewrite this - hopefully, we will finish falcon soon and we will start benchmarking it and see where we can "get some juice"

tudorcebere · 2020-08-29T18:56:23Z

syft/frameworks/torch/mpc/falcon.py

+    @staticmethod
+    def _share_random_tensor(
+        field: int, workers: List[BaseWorker]
+    ) -> (ReplicatedSharingTensor, ReplicatedSharingTensor):


(Optional): When using annotations, it is cleaner to use Tuple from typing instead of the tuple syntax from python.

Removed this function

tudorcebere · 2020-08-29T18:58:46Z

syft/frameworks/torch/tensors/interpreters/replicated_shared.py

@@ -33,6 +33,18 @@ def __share_secret(self, plain_text, workers):
        PRZS.setup(workers)
        return shares_map

+    @staticmethod
+    def one_shares(players, field, shape=(1,)):


(Optional): Ret type annotation.

Made an issue for adding types annotations for all stuff related to falcon here

tudorcebere · 2020-08-29T18:59:05Z

syft/frameworks/torch/tensors/interpreters/replicated_shared.py

+        )
+
+    @staticmethod
+    def zero_shares(players, field, shape=(1,)):


(Optional): Ret type annotations

tudorcebere · 2020-08-29T18:59:30Z

syft/frameworks/torch/tensors/interpreters/replicated_shared.py

+        return ReplicatedSharingTensor(
+            plain_text=torch.zeros(shape, dtype=torch.int64), players=players, field=field
+        )
+
    @staticmethod
    def __arrange_workers(workers):


(Optional): Ret type annotations

Will create a ticket for adding annotations to all the elements in the falcon related implementation

syft/frameworks/torch/mpc/falcon/falcon_helper.py

gmuraru · 2020-08-30T15:44:01Z

syft/frameworks/torch/mpc/przs.py

@@ -5,7 +5,7 @@
 import torch
 import syft

-RING_SIZE = 2 ** 32
+MAX_LIMIT = 2 ** 32


This is not really the ring size - is the maximum value for the random value (for the common randomness)

how would you know they going to sum up to ring size if you don't know what is the ring size
( alpha1 + alpha2+ alpha3 )% ring size should be zero

Yep...it is. will redo

codecov · 2020-08-30T17:02:57Z

Codecov Report

Merging #4474 into master will increase coverage by 0.02%.
The diff coverage is 98.41%.

@@            Coverage Diff             @@
##           master    #4474      +/-   ##
==========================================
+ Coverage   94.79%   94.82%   +0.02%     
==========================================
  Files         209      209              
  Lines       21377    21419      +42     
==========================================
+ Hits        20265    20310      +45     
+ Misses       1112     1109       -3

Impacted Files	Coverage Δ
syft/frameworks/torch/mpc/primitives.py	`95.48% <ø> (ø)`
...ks/torch/tensors/interpreters/replicated_shared.py	`92.77% <50.00%> (ø)`
syft/frameworks/torch/mpc/falcon/falcon.py	`100.00% <100.00%> (ø)`
syft/frameworks/torch/mpc/falcon/falcon_helper.py	`97.67% <100.00%> (+2.67%)`	⬆️
test/torch/mpc/falcon/test_falcon.py	`100.00% <100.00%> (ø)`
test/torch/mpc/falcon/test_falcon_helper.py	`100.00% <100.00%> (ø)`
...ft/frameworks/torch/tensors/interpreters/native.py	`91.48% <0.00%> (+0.21%)`	⬆️
syft/execution/plan.py	`94.90% <0.00%> (+0.56%)`	⬆️

abogaziah · 2020-08-30T17:44:04Z

syft/frameworks/torch/mpc/przs.py

@@ -5,7 +5,7 @@
 import torch
 import syft

-RING_SIZE = 2 ** 32
+MAX_LIMIT = 2 ** 32


how would you know they going to sum up to ring size if you don't know what is the ring size
( alpha1 + alpha2+ alpha3 )% ring size should be zero

abogaziah · 2020-08-30T17:44:43Z

syft/frameworks/torch/mpc/falcon/falcon.py

@@ -11,3 +14,71 @@ def conv2d(filters: syft.ReplicatedSharingTensor, image, padding=0):
    output_size = (image_height - filter_height + 2 * padding) + 1
    result = result.view(-1, channels_out, output_size, output_size)
    return result
+
+
+def select_share(


that's a subroutine, it should be moved to the helper class

abogaziah · 2020-08-30T17:45:35Z

syft/frameworks/torch/mpc/falcon/falcon.py

+    x_share: ReplicatedSharingTensor,
+    y_share: ReplicatedSharingTensor,
+) -> ReplicatedSharingTensor:
+    """Performs select share protocol


select share is a subroutine not protocol, it would be better to describe the arguments and the return directly
like
return: x if b=0. y if b=1
and remove the rest of the comment

abogaziah · 2020-08-30T17:50:36Z

syft/frameworks/torch/mpc/falcon/falcon.py

+
+
+def select_share(
+    b_share: ReplicatedSharingTensor,


it's not a share, it's a shared value, it would be better to call them x, y, b

abogaziah · 2020-08-30T17:54:24Z

syft/frameworks/torch/mpc/falcon/falcon.py

+
+    # TODO: Think of a method to do this better (1 - c_share) without being to
+    # "intrusive" with the operation
+    ones_share = ReplicatedSharingTensor.one_shares(players, ring_size)


this is not coherent with the interface please remove it and just use torch.ones.share

abogaziah · 2020-08-30T17:58:21Z

syft/frameworks/torch/mpc/falcon/falcon.py

+    return selected_share
+
+
+def evaluate(


evaluate is like the worst name ever a routine can have 😄 that's basically what every routine in the world does, it evaluates some expression, it doesn't tell you anything about what this routine does.
this also should be moved to the helper class

if you noticed this routine determines the negativity of x, if beta is an even number this will return x, if beta is odd it will return - x, you can choose a name that explains this functionality

Will rename it

abogaziah · 2020-08-30T18:01:54Z

syft/frameworks/torch/mpc/falcon/falcon.py

+    """Performs the computation (-1)^beta * x
+
+    Args:
+        x_share (ReplicatedSharingTensor): share to "evaluate"


please remove the "share" suffix

abogaziah · 2020-08-30T18:05:17Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        value: Union[int, ReplicatedSharingTensor], other: Union[int, ReplicatedSharingTensor]
+    ) -> ReplicatedSharingTensor:
+        """
+        Compute the XOR value between value and other


please remove the docstring, xor is self-documented it doesn't need comments, "you should see every comment as a failure to express yourself with code"

We need to do docstring because if we move to syft_0.3.0 that will fail if we do not have every function documented. For a newcomer, the documentation might help.

abogaziah · 2020-08-30T18:06:50Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+            The XOR computation between value and other
+        """
+
+        if not any(isinstance(x, ReplicatedSharingTensor) for x in (value, other)):


same expression will work on ints, this expression adds not-needed overhead

We should not use FalconHelper for simple ints. We should use int_a ^ int_b

It is a check to make sure we are doing everything correctly - we could remove this after we finish the implementation.

abogaziah · 2020-08-30T18:10:32Z

syft/frameworks/torch/tensors/interpreters/replicated_shared.py

@@ -34,6 +34,18 @@ def __share_secret(self, plain_text, workers):
        PRZS.setup(workers)
        return shares_map

+    @staticmethod
+    def one_shares(players, ring_size, shape=(1,)):


please remove this, we need the interface to be as small as possible, you can just use torch.ones().share()

Yep - tried to follow the model from AST, but we might refactor that

gmuraru · 2020-08-30T20:55:57Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        selected_val = (y - x) * d + x
+        return selected_val
+
+    def negate_cond(


Thinking of a conditional negate - I do not know if this is the best name for it

I know that negate_con doesn't mean conditional negative, do not use abbreviations just use the whole word until you come up with a better name

abogaziah

thank you for the changes, it improved significantly 🎉 just few more changes

abogaziah · 2020-08-31T11:03:21Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+
+
+class FalconHelper:
+    XOR_EXPECTED_RING_SIZE = 2


why is this a global variable thought it's only in xor?

Hmm..this is not a global variable, it is accessed only using the class, but I think you are right, we can move it inside the xor method

abogaziah · 2020-08-31T11:10:14Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        If value and other are both ints we should use the "^" operator.
+
+        Args:
+            value (ReplicatedSharingTensor or Int): RST with ring size of 2 or Int value in {0, 1}


value should be always RST, other could be RST for private xor, or long tensor or int for public xor

Hm, I made it like this to be able to have xor(int, RST) - I can remove it

abogaziah · 2020-08-31T11:15:38Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        ):
+            raise ValueError("If both arguments are RST then they should be in ring size 2")
+
+        # Make sure we have a ReplicatedSharingTensor as value


remove comments before merging to the codebase

abogaziah · 2020-08-31T11:17:55Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        # This should be generated by a crypto provider or create a 0 RST shared value and make
+        # each worker add a random number at an offline step
+        # (Idea): Use the correlated randomness shared at the beginning
+        c = torch.randint(high=2, size=(1,))


should the size be 1 or x.shape?

Reading that part of the paper again - yeah it should be x.shape

abogaziah · 2020-08-31T11:18:20Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        ring_size = x.ring_size
+        players = x.players
+
+        # This should be generated by a crypto provider or create a 0 RST shared value and make


remove comments

Will remove

abogaziah · 2020-08-31T11:22:17Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        selected_val = (y - x) * d + x
+        return selected_val
+
+    def negate_cond(


I know that negate_con doesn't mean conditional negative, do not use abbreviations just use the whole word until you come up with a better name

abogaziah · 2020-08-31T11:32:49Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        players = x.players
+        shape = x.shape
+
+        pow_sh = (


remove abbreviations

abogaziah · 2020-08-31T11:34:04Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        shape = x.shape
+
+        pow_sh = (
+            torch.ones(size=shape).share(*players, protocol="falcon", field=ring_size) - beta * 2


we can use public multiplication and addition here to save some communication rounds: (beta*-2)+1

Yep - we can do. I had a feeling that it would require multiple multiple rounds of communications if we reorder the operations, but the public multiplication is done locally.

abogaziah · 2020-08-31T11:41:50Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        # "intrusive" with the operation
+        ones = torch.ones(size=(1,)).share(*players, protocol="falcon", field=ring_size)
+        if xor_b_c.reconstruct().item():
+            d = ones - c_L


we can use public operations here: d= -1*c_L +1

The same for this.

I'm working on RST to enable executing 1-c_L directly, it should be ready shortly

gmuraru · 2020-08-31T20:30:30Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+            The XOR computation between value and other
+        """
+
+        if not isinstance(value, ReplicatedSharingTensor) or value.ring_size != 2:


I have all those types in place - which look pretty bad - let's keep them in place only to make sure we define the protocol correctly and then remove them

gmuraru · 2020-08-31T20:31:51Z

syft/frameworks/torch/tensors/interpreters/replicated_shared.py

@@ -178,8 +178,8 @@ def __public_linear_operation(self, plain_text, operator):
            shares_map[players[0]][1],
        )
        shares_map[players[-1]] = (
-            operator(shares_map[players[-1]][-1], remote_plain_text[-1]),


P1 has share x1, x2; P2 has x2,x3 and P3 has x3 and x1 - when adding a constant we will have
P1 has share x1 + c, x2, P2 has share x2,x3 (nothing changed) and P3 has x3 and x1 + c

abogaziah

Great work! just two final notes on the code

syft/frameworks/torch/mpc/falcon/falcon_helper.py

abogaziah · 2020-08-31T20:52:04Z

test/torch/tensors/test_replicated_shared.py

@@ -11,6 +11,18 @@ def test_sharing(workers):
    assert type(secret.child) == dict


+def test_share_float(workers):


RST only supports torch.long, deterministic, little, and fast interface is better than a slow wide one, we will use FPT to support floating-point arithmetic (which will pass torch.long to this class)

Hmm...in that case when generating the shares we should raise an exception is the passed Tensor is a different type than long, right?

I added that in my last PR

abogaziah · 2020-08-31T20:55:32Z

test/torch/mpc/falcon/test_falcon_helper.py

 import torch
 from syft.frameworks.torch.mpc.falcon.falcon_helper import FalconHelper

+import itertools


tests should not include any iterations nor conditions (from clean coding book 😄) just simple expressions that tell what the interface does

Yep - I will manually create the TEST_VALS list with all values we except

abogaziah · 2020-08-31T21:05:25Z

test/torch/mpc/falcon/test_falcon_helper.py

-    assert (FalconHelper.xor(y, y).reconstruct() == torch.tensor(0)).all()
-    assert (FalconHelper.xor(y, x).reconstruct() == torch.tensor(1)).all()
+
+    with pytest.raises(ValueError) as e:


last request: let's move the assertion inside the body of the code, the tests should only test the interface and helper functions, we can use assertions in code during construction it would make it easier to find and remove them before moving it to production
for example, put (assert x.ring_size == 2) inside the xor function instead of if statements and remove the corresponding test case
also better manage whitespace, related expressions should be in a continuous chunk

It is not ok to have asserts in the "real" code because they can be removed if you run with the -O or -OO flag.

yes that's the point, you should remove the asserts before pushing to production, asserts makes it easier to do that if you want your checks not to be moved to production to improve performance

gmuraru · 2020-09-01T04:23:26Z

syft/frameworks/torch/tensors/interpreters/replicated_shared.py

@@ -48,7 +48,7 @@ def __arrange_workers(workers):

    def generate_shares(self, plain_text, number_of_shares=3):


If generate_shares is public shouldn't we move the __validation function here?
I will also remove line 51

it's not part of the interface, I left it public just to test it

Co-authored-by: Muhammed Abogazia <33666625+abogaziah@users.noreply.github.com>

abogaziah

Awesome!! thank you for the PR and the changes ❤🎉

gmuraru · 2020-09-01T12:29:56Z

Awesome!! thank you for the PR and the changes

We make a good team! 👍

gmuraru · 2020-09-01T12:30:53Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        """
+
+        assert (
+            isinstance(value, ReplicatedSharingTensor) and value.ring_size == 2


Q: This looks kind of ugly but this is how black formatted it :\

no problem :D

gmuraru · 2020-09-01T12:31:17Z

syft/frameworks/torch/mpc/falcon/falcon_helper.py

+        assert (
+            isinstance(value, ReplicatedSharingTensor) and value.ring_size == 2
+        ), "First argument should be a RST with ring size 2"
+        assert any(


Q: Do you prefer this to be split into 3 asserts? (one for each type?) @abogaziah

there's no difference, anyone would work

* Add select share * Black formatting * Ring size to field * Add evaluation * Remove print * Basic operations + tests * Remove falcon test * Fix flake8 * Rename test * Comments fix * Comments fix * Remove the changes from przs * Remove changes from przs test * More changes from przs * Fix flake8 * Add select share and determine sign * Apply suggestions from code review Co-authored-by: Muhammed Abogazia <33666625+abogaziah@users.noreply.github.com> * Formatting * Test fix * Remove notebook * Cleanup xor and tests * Fix formatting * Remove itertools Co-authored-by: Muhammed Abogazia <33666625+abogaziah@users.noreply.github.com>

gmuraru requested review from LaRiffle and abogaziah August 25, 2020 07:19

gmuraru requested review from a team as code owners August 25, 2020 07:19

gmuraru requested review from replomancer and tudorcebere August 25, 2020 07:19

gmuraru mentioned this pull request Aug 26, 2020

Replicated Sharing Tensor: Multiplication, Matrix multiplication, and Convolution #4015

Merged

7 tasks

gmuraru force-pushed the gm-falcon-select-share branch from af7d52d to a5206f2 Compare August 26, 2020 22:24

gmuraru commented Aug 26, 2020

View reviewed changes

gmuraru changed the title ~~[WIP] Falcon - Select share~~ [WIP] Falcon - Select share + evaluating Aug 26, 2020

omershlo reviewed Aug 29, 2020

View reviewed changes

tudorcebere reviewed Aug 29, 2020

View reviewed changes

gmuraru force-pushed the gm-falcon-select-share branch from a846a11 to 2422fb9 Compare August 30, 2020 15:42

gmuraru commented Aug 30, 2020

View reviewed changes

syft/frameworks/torch/mpc/falcon/falcon_helper.py Show resolved Hide resolved

gmuraru commented Aug 30, 2020

View reviewed changes

gmuraru changed the title ~~[WIP] Falcon - Select share + evaluating~~ Falcon - Select share + evaluating Aug 30, 2020

abogaziah suggested changes Aug 30, 2020

View reviewed changes

gmuraru requested a review from abogaziah August 30, 2020 20:50

gmuraru commented Aug 30, 2020

View reviewed changes

abogaziah suggested changes Aug 31, 2020

View reviewed changes

gmuraru added 5 commits August 31, 2020 21:20

Add select share

e586c0a

Black formatting

cc3cb5b

Ring size to field

c727aeb

Add evaluation

5d0f64a

Remove print

f249eb4

gmuraru added 10 commits August 31, 2020 21:21

Remove falcon test

639cbc6

Fix flake8

bc44ad7

Rename test

2dd5063

Comments fix

4bf71c7

Comments fix

a1da00b

Remove the changes from przs

7bd00e5

Remove changes from przs test

adf5540

More changes from przs

afec37a

Fix flake8

36a1b74

Add select share and determine sign

65e9baf

gmuraru force-pushed the gm-falcon-select-share branch from e3a1250 to 65e9baf Compare August 31, 2020 20:26

gmuraru requested a review from abogaziah August 31, 2020 20:29

gmuraru commented Aug 31, 2020

View reviewed changes

abogaziah reviewed Aug 31, 2020

View reviewed changes

gmuraru commented Sep 1, 2020

View reviewed changes

Apply suggestions from code review

5dc4ef5

Co-authored-by: Muhammed Abogazia <33666625+abogaziah@users.noreply.github.com>

gmuraru requested a review from abogaziah September 1, 2020 04:24

gmuraru added 3 commits September 1, 2020 07:29

Formatting

49d60f6

Test fix

6742f16

Remove notebook

791bae4

abogaziah approved these changes Sep 1, 2020

View reviewed changes

Cleanup xor and tests

4ad9c52

gmuraru commented Sep 1, 2020

View reviewed changes

gmuraru added 2 commits September 1, 2020 16:33

Fix formatting

b02091b

Remove itertools

6563288

gmuraru merged commit c6a5e91 into OpenMined:master Sep 1, 2020

		c_share_L = c.share(*workers, protocol="falcon", field=field)
		c_share_2 = c.share(*workers, protocol="falcon", field=2)

		@@ -11,6 +11,18 @@ def test_sharing(workers):
		assert type(secret.child) == dict


		def test_share_float(workers):

		@@ -48,7 +48,7 @@ def __arrange_workers(workers):

		def generate_shares(self, plain_text, number_of_shares=3):

Falcon - Select share + evaluating #4474

Falcon - Select share + evaluating #4474

Conversation

gmuraru commented Aug 25, 2020 • edited

Description

How has this been tested?

Checklist

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Aug 30, 2020 • edited

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abogaziah Aug 30, 2020 • edited

Choose a reason for hiding this comment

gmuraru Aug 30, 2020 • edited

Choose a reason for hiding this comment

abogaziah Aug 30, 2020 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abogaziah left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abogaziah left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gmuraru commented Aug 25, 2020 •

edited

codecov bot commented Aug 30, 2020 •

edited

abogaziah Aug 30, 2020 •

edited

gmuraru Aug 30, 2020 •

edited

abogaziah Aug 30, 2020 •

edited