[Sort Multi Bit] Upgrade to malicious + semi honest ipa now calls multi bit sort #396

richajaindce · 2023-01-17T13:53:48Z

No description provided.

andyleiserson · 2023-01-18T22:01:43Z

This will have some small conflicts with #387. https://github.com/andyleiserson/ipa/commits/multi_bit may be useful for resolving them. Commits b80bfe4 and 173ac7b are the substantive changes, and the commit between those two is a merge with the conflicts resolved.

(One thing I am doing in #387 is removing the MaliciousUpgradeInput step. Since each of the upgrades is done with a separate MaliciousValidator, it's not necessary to narrow the context used for the upgrade -- an upgrade context is created as part of the malicious validator creation.)

andyleiserson · 2023-01-18T22:06:04Z

src/protocol/sort/generate_permutation.rs

@@ -274,7 +276,7 @@ where
    assert_eq!(sort_keys.len(), num_bits as usize);

    let upgraded_sort_keys = m_ctx
-        .upgrade_vector(&MaliciousUpgradeInput, sort_keys[0].clone())
+        .upgrade_vector(&MaliciousUpgradeInput(0), sort_keys[0].clone())


Not sure if there's a reason these functions take a reference to sort_keys. It looks like the callers have an owned Vec anyways, and the upgrade functions want to take the raw input by value, so it may be possible to change the argument type to this function and get rid of this clone and several like it.

Callers own the vector but it is used in the next step after sorting. We use the sort keys to sort and then use them to apply sort on combined_match_keys_and_sidecar_data.
Great to see upgrade_vector go away! The only reason I included (0) was to run it across bits in malicious_multi

richajaindce · 2023-01-19T04:47:23Z

This will have some small conflicts with #387. https://github.com/andyleiserson/ipa/commits/multi_bit may be useful for resolving them. Commits b80bfe4 and 173ac7b are the substantive changes, and the commit between those two is a merge with the conflicts resolved.

(One thing I am doing in #387 is removing the MaliciousUpgradeInput step. Since each of the upgrades is done with a separate MaliciousValidator, it's not necessary to narrow the context used for the upgrade -- an upgrade context is created as part of the malicious validator creation.)

Let me wait for you to land the code and then I will make the changes. Since there might be some code-restructure I can re-use from your PR

benjaminsavage · 2023-01-19T06:36:06Z

src/protocol/sort/generate_permutation.rs

@@ -274,7 +276,7 @@ where
    assert_eq!(sort_keys.len(), num_bits as usize);

    let upgraded_sort_keys = m_ctx
-        .upgrade_vector(&MaliciousUpgradeInput, sort_keys[0].clone())
+        .upgrade_vector(&MaliciousUpgradeInput(0), sort_keys[0].clone())
        .await?;
    let bit_0_permutation =
        bit_permutation(m_ctx_0.narrow(&BitPermutationStep), &upgraded_sort_keys).await?;


Shouldn't you be removing this?

benjaminsavage · 2023-01-19T06:38:48Z

src/protocol/sort/generate_permutation_opt.rs

+    let mut malicious_validator = MaliciousValidator::new(sh_ctx.narrow(&MaliciousUpgradeContext));
+    let mut m_ctx = malicious_validator.context();
+    let m_ctx_0 = m_ctx.narrow(&Sort(0));
+    assert_eq!(sort_keys.len(), num_bits as usize);
+
+    let last_bit_num = std::cmp::min(num_multi_bits, num_bits);
+
+    let upgraded_sort_keys = try_join_all((0..last_bit_num).zip(repeat(m_ctx.clone())).map(


Lots of duplicate code here between this and the semi-honest version. This introduces the risk of a bug when they get out of sync.

benjaminsavage · 2023-01-19T06:41:39Z

src/protocol/sort/generate_permutation_opt.rs

+        let (randoms_for_shuffle0, randoms_for_shuffle1, revealed) = (
+            revealed_and_random_permutations
+                .randoms_for_shuffle
+                .0
+                .as_slice(),
+            revealed_and_random_permutations
+                .randoms_for_shuffle
+                .1
+                .as_slice(),
+            revealed_and_random_permutations.revealed.as_slice(),
+        );


Is it possible to do this destructuring when revealed_and_random_permutations is originally created back up on line 184?

richajaindce · 2023-01-20T02:24:36Z

src/protocol/sort/generate_permutation_opt.rs

+    let upgraded_sort_keys = try_join_all(
+        (0..last_bit_num)
+            .zip(repeat(m_ctx.clone()))
+            .map(|(i, m_ctx)| async move { m_ctx.upgrade(sort_keys[i as usize].clone()).await }),


@andyleiserson this still will fail since we are using the same context across bits for the same record with the error "Generated randomness for index '0' twice using the same key 'protocol/run-0/sort/malicious_upgrade_context/upgrade_input' ". Any ideas how to resolve this without narrowing context?

My thinking (based on an earlier revision of this PR) was to add an upgrade impl for Vec<Vec<Replicated>> and then use that to upgrade everything at once rather than doing multiple upgrades.

Does that make sense? I'm not sure if something changed since I looked at it or if there's some other reason I missed that that won't work.

The code was earlier written to just work on one vector at a time and so I am doing some refactoring to let the sort code take in vec<vec<vec>> so that each vec<vec> can be processed in one iteration

richajaindce · 2023-01-20T04:42:06Z

src/protocol/sort/generate_permutation.rs

 ) -> Result<Vec<Replicated<F>>, Error>
 where
    F: Field,
 {
+    let mut malicious_validator = None;
+    let sort_ctx: &dyn Context<F> = if malicious_setting {


@akoshelev need your help in checking how to make this work. It would be nice to have malicious and semi honest sort work from the same function since we can avoid duplication and mistakes in upgrades by this approach.

Code still does not compile because we need share upgrade

martinthomson · 2023-01-20T05:21:03Z

src/protocol/sort/generate_permutation.rs

+        let malicious_validator = Some(MaliciousValidator::new(
+            ctx.narrow(&MaliciousUpgradeContext),
+        ));
+        let ctx = malicious_validator.unwrap().context();


Why not:

Suggested change

let malicious_validator = Some(MaliciousValidator::new(

ctx.narrow(&MaliciousUpgradeContext),

));

let ctx = malicious_validator.unwrap().context();

let malicious_validator = MaliciousValidator::new(

ctx.narrow(&MaliciousUpgradeContext),

);

let ctx = malicious_validator.context();

You added an Option<T> wrapper and then unwrapped it immediately.

We were trying to combine the two sorts i.e. malicious and semi honest in the same function hence introduced an option for malicious_validator. Will revisit later

richajaindce · 2023-01-20T12:50:55Z

Closing this since this needs a bit of a refactor to use upgrade on a slice of num_multi_bits per record

richajaindce added 3 commits January 17, 2023 20:46

Multi bit is replacing bit permutation

baf120d

Merge branch 'main' into multi_bit

408c96f

Malicious upgrade of multi bit gen perm

7bc13d8

richajaindce force-pushed the multi_bit branch 3 times, most recently from b8bf517 to 1bc381a Compare January 18, 2023 02:37

richajaindce requested a review from benjaminsavage January 18, 2023 02:37

Remove prss changes

d413dda

andyleiserson reviewed Jan 18, 2023

View reviewed changes

martinthomson mentioned this pull request Jan 18, 2023

Sorting might take values #400

Open

benjaminsavage reviewed Jan 19, 2023

View reviewed changes

Merge branch 'main' into multi_bit

98fabe4

richajaindce force-pushed the multi_bit branch from 1bc381a to 98fabe4 Compare January 20, 2023 02:22

richajaindce marked this pull request as draft January 20, 2023 02:22

richajaindce commented Jan 20, 2023

View reviewed changes

Non working context handling

1edd422

richajaindce commented Jan 20, 2023

View reviewed changes

Generic generate permutation

207e9a3

Code still does not compile because we need share upgrade

martinthomson reviewed Jan 20, 2023

View reviewed changes

richajaindce closed this Jan 20, 2023

richajaindce deleted the multi_bit branch February 1, 2023 06:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Sort Multi Bit] Upgrade to malicious + semi honest ipa now calls multi bit sort #396

[Sort Multi Bit] Upgrade to malicious + semi honest ipa now calls multi bit sort #396

richajaindce commented Jan 17, 2023

andyleiserson commented Jan 18, 2023 •

edited

Loading

andyleiserson Jan 18, 2023

richajaindce Jan 19, 2023

richajaindce commented Jan 19, 2023

benjaminsavage Jan 19, 2023

benjaminsavage Jan 19, 2023

benjaminsavage Jan 19, 2023

richajaindce Jan 20, 2023

andyleiserson Jan 20, 2023

richajaindce Jan 21, 2023

richajaindce Jan 20, 2023

martinthomson Jan 20, 2023

richajaindce Jan 20, 2023

richajaindce commented Jan 20, 2023

[Sort Multi Bit] Upgrade to malicious + semi honest ipa now calls multi bit sort #396

[Sort Multi Bit] Upgrade to malicious + semi honest ipa now calls multi bit sort #396

Conversation

richajaindce commented Jan 17, 2023

andyleiserson commented Jan 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richajaindce commented Jan 19, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

richajaindce commented Jan 20, 2023

andyleiserson commented Jan 18, 2023 •

edited

Loading