Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add PReLU #9

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

add PReLU #9

wants to merge 1 commit into from

Conversation

LehaoLin
Copy link

I am not familiar with circom code.
But I try to write a PReLU.circom, PReLU_test.circom, PReLU.js
I found that PReLU is quite similar to Leaky_relu, but the main difference is that in PReLU, alpha is not a fixed parameter but a learnable parameter, typically initialized randomly and then adjusted through backpropagation during training.

However, I try to set the alpha in the block of the circom template but get error

error[T3001]: Non quadratic constraints are not allowed!
   ┌─ "/Users/linlehao/project/circomlib-ml/circuits/PReLU.circom":17:5
   │
17 │     neg <== isNegative.out * alpha * in;
   │     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ found here

Therefore, I use the same code with LeakyReLU as

// PReLU layer
template PReLU (alpha) {  // alpha is 10 times the actual alpha, since usual alpha is 0.2, 0.3, etc.
    signal input in;
    signal input out;
    //signal input alpha; 
    signal input remainder;
    
    component isNegative = IsNegative();

    isNegative.in <== in;

    signal neg;
    neg <== isNegative.out * alpha * in;

    out * 10 + remainder === neg + 10 * in * (1 - isNegative.out);
}

Hope you can help it more.

@LehaoLin LehaoLin mentioned this pull request Mar 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant