Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WARNING: high error rate, more details with --display-optimizer-choice? #95

Closed
maxwellgodv opened this issue Jul 10, 2023 · 16 comments
Closed

Comments

@maxwellgodv
Copy link

1bbbb1843a4ed7bd4278b72ad17807e Hello ,What does this mean?
@RomanBredehoft
Copy link
Collaborator

Hello,
Would you be able to give us more info about your issue ? More precisely, would it be possible to :

  • indicate to us which model you are working with
  • set verbose=True when compiling your model (depending what you are doing: compile, compile_torch_model, etc.) and share us the printed details

Many thanks !

@maxwellgodv
Copy link
Author

maxwellgodv commented Jul 11, 2023

After updating the new version of concrete ml, this issue occurred. In previous versions, there was no such warning.
CifarInFhe.ipynb

@RomanBredehoft
Copy link
Collaborator

Thanks for reporting ! We are indeed getting the same warning and are thus currently investigating on what could be the issue 🙂

@kcelia
Copy link
Collaborator

kcelia commented Jul 11, 2023

Hello,
Thanks for reporting the warning!
We have fixed the issue on our end, but in the meantime, you can also make a fix on your side to continue.

def fhe_compatibility(model: Callable, data: DataLoader) -> Callable:
    qmodel = compile_brevitas_qat_model(
        model.to("cpu"),
        # Training
        torch_inputset=data,
        show_mlir=False,
        output_onnx_file="test.onnx",
    )
    return qmodel

Thanks !

@maxwellgodv
Copy link
Author

Hello, thank you for replying to me.
But I still have a problem.
Do you mean that the configuration and p_error parameters are not required?
Can you explain these two parameters to me?

@andrei-stoian-zama
Copy link
Collaborator

Hi, you can find an explanation of p_error in this documentation page. Here the solution proposed is to remove the custom p_error value that this example was setting.

You mention that you still have a problem, can you please elaborate ?

@maxwellgodv
Copy link
Author

Thanks,How fhe_simulation_inference function simulate Homomorphic encryption?Where can I see the details of simulation?

@umut-sahin
Copy link

In Concrete v1, homomorphic encryption wasn't simulated, computation graph was walked and evaluated (and some errors are added before TLUs to kinda simulate p_error). Here are the relevant pieces of code:

With Concrete v2, homomorphic encryption is simulated! We've introduced a new backend to do that and new simulation is using the same compiler architecture, so it's much more accurate. Here is the relevant piece of code:

Hope it helps!

@RomanBredehoft
Copy link
Collaborator

To complement @umut-sahin's answer, Concrete-ML's currently works with Concrete v1 but we are planning on integrating v2 in our next release !

@maxwellgodv
Copy link
Author

So this code does not simulate Homomorphic encryption?It uses Concrete v1 ?cifar_brevitas_finetuning

@RomanBredehoft
Copy link
Collaborator

Yes it does provide some kind of FHE simulation using Concrete v1's technique (graph evaluation + noise simulation), fhe_simulation_inference does not execute the inference in FHE ! The technique is just not as accurate as its second version (documentation: https://docs.zama.ai/concrete/tutorials/simulation), which will come with Concrete ML's next release. Still, you should currently be able to simulate FHE execution with a very good accuracy.

@RomanBredehoft
Copy link
Collaborator

RomanBredehoft commented Jul 21, 2023

Hello @maxwellgodv , if you are satisfied with the above answer, please feel free to close this issue 🙂

@maxwellgodv
Copy link
Author

So is it to compile the quantitative model into an circuit, and then input the data into the circuit for prediction ?And, where can I find documents about circuit.
I don't have much knowledge about circuit.Thanks.

@umut-sahin
Copy link

There is no dedicated page for circuit, but there are pages for everything you can do with a circuit in https://docs.zama.ai/concrete!

Also, you can read the source code, every method is documented https://github.com/zama-ai/concrete/blob/main/frontends/concrete-python/concrete/fhe/compilation/circuit.py 🙂

And of course you can always ask directly to us, see https://www.zama.ai/community 😉

@RomanBredehoft
Copy link
Collaborator

RomanBredehoft commented Jul 27, 2023

Hello again @maxwellgodv, is it okay for you if we close this issue as the current discussion does not concern the initial problem anymore ? If you have more questions, we'll gladly answer you on our community forum as well as on the Concrete ML channel that you can find in FHE.org's discord server 🙂

Thanks again for the report !

@maxwellgodv
Copy link
Author

OK,thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants