-
-
Notifications
You must be signed in to change notification settings - Fork 35
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When Client and Server items are in some certain sizes, server.CreateSetupMessage raise ValueError: basic_string::resize #143
Comments
This is a bug with the |
great thanks |
This has been fixed in https://github.com/OpenMined/PSI/releases/tag/v1.0.1 |
@s0l0ist Nick, seem when the fpr is small 1e-11, it still fails. I updated the notebook: But I did try with larger fpr=1e-9, the code seems to work |
Yep, looks like smaller false positive rate causes the backing GCS datastructure to try and allocate too much memory. This is 'fixed' in the sense that you must use the So I admit, the bug still exists in GCS, but the workaround is to use bloom filters. setup = dup(
duplicate, s.CreateSetupMessage(fpr, len(client_items), server_items, psi.DataStructure.BLOOM_FILTER), psi.ServerSetup()
) |
Let me try this. I don't mind extra argument parameter. |
This works:
|
Ah yes, I put the param outside of the function. Oops. |
@s0l0ist Nick, https://github.com/chesterxgchen/psi_mpc_related/blob/main/psi/psi_intersect.ipynb Compare Cell 11 and 14. Cell 11 is with extra argument, Cell 14 is not. In Cell 11, the resulting intersect is 19 items which is larger than the Server Items of 17. Noticed that few larger numbers is much larger than max number of the item from server. Cell 14 is without extra argument, the result is correct. |
Looking at the example code, it looks like the client has many more values than the server. This will yield much higher chances of receiving a false positive value in return and isn't the intended way of how to use this library regardless of the backing data structure used. The protocol is asymmetric in the sense that both client and server sets can be different sizes, but the real use case is for when the client has much fewer items in its set and wants to check against a server with many more items in its set. If you flip the client and server set sizes, you should get more reasonable results in expectation of the FPR you've set. |
Nick, All the examples I showed in the notebook ( client and server items) are not arbitrarily chosen, they are actually from test failure cases, which I simplified to put notebook |
I see, yeah this is a bug. I'll investigate. |
There was a bug that produced more erroneous results than normal; however, it is quite possible for false positives to manifest themselves in the form of intersection index values that are out of range of the original client set. Checkout the my comments in the PR to handle those cases. |
Description
When client items size and server items size are in certain values, the PSI code throw basic_string::resize Value Error.
For example, using the tests.py https://github.com/OpenMined/PSI/blob/master/private_set_intersection/python/tests.py#L28
if we change the client items and server items range into the following values:
client_items = ["Element " + str(i) for i in range(333334)]
server_items = ["Element " + str(2 * i) for i in range(397)]
You should be able to get the exception
How to Reproduce
check the code in https://github.com/chesterxgchen/psi_mpc_related/blob/main/psi/psi_intersect.ipynb
command Cell No. 5 should re-produce the error.
The code in this notebook is essentially the same as https://github.com/OpenMined/PSI/blob/master/private_set_intersection/python/tests.py#L28
pip install the openmined.psi
start the jupyter notebook with https://github.com/chesterxgchen/psi_mpc_related/blob/main/psi/psi_intersect.ipynb
execute the import the psi packages, and def dup() function, then run cmd 5, to see error
Expected Behavior
No error
Screenshots
System Information
Additional Context
We experience this issue while testing our openmined-based PSI solution
The text was updated successfully, but these errors were encountered: