Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement complex-valued constraint parameter #28

Open
j-peetz opened this issue May 20, 2022 · 7 comments
Open

Implement complex-valued constraint parameter #28

j-peetz opened this issue May 20, 2022 · 7 comments
Labels
feature request New feature or request help wanted Extra attention is needed

Comments

@j-peetz
Copy link

j-peetz commented May 20, 2022

Hello @NEGU93, thank you for making this library! I'm attempting to use the arguments "use_bias" and "kernel_constraint" within "ComplexDense" as so:

self.d1 = complex_layers.ComplexDense(dim, use_bias=False, kernel_constraint=ConstantWeights())

However, these arguments seem to have no effect -- there are still biases, and the constraints on the weights are not enforced. That is, the layer performs identically to this:

self.d1 = complex_layers.ComplexDense(dim)

I tried the same arguments with tf.keras.layers.Dense and got the desired behaviors. Please let me know if I misunderstand how to use these. Thanks in advance!

Here's the relevant context, in case it helps:

class ConstantWeights(tf.keras.constraints.Constraint):
    def __call__(self, w):
        tf.keras.backend.set_value(w[0, 0], 3)
        tf.keras.backend.set_value(w[1, 2], 3)
        return w

class MyModel(Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.d1 = complex_layers.ComplexDense(dim, use_bias=False, kernel_constraint=ConstantWeights())

    def call(self, x):
        x = self.d1(x)
        return x

model = MyModel()
@NEGU93
Copy link
Owner

NEGU93 commented May 20, 2022

Hello and thank you for your interest.
It is possible that the kernel_contraint is not implemented but the bias parameter should be working...
Could you confirm me which version of the library are you using? Could you try using the last version and see if the issue is solved?

@j-peetz
Copy link
Author

j-peetz commented May 20, 2022

Thank you for the response. I updated from v1.0.0 (via conda) to v1.2.14 (via pip). Like you said, the bias feature is now working properly. However, the kernel constraint still seems to have no effect.

@NEGU93
Copy link
Owner

NEGU93 commented May 21, 2022

Yes sorry, I need to update conda's version. I will do so next week.

Indeed, the kernel_constraint is not yet implemented.
I may add this feature in the future. I will let you know when (if) I do.

@NEGU93 NEGU93 added the feature request New feature or request label May 21, 2022
@NEGU93
Copy link
Owner

NEGU93 commented May 23, 2022

I have been investigating about this. I could easily do:

  • A real-valued kernel_contraint that applies to both real and imaginary parts (but separately)
  • Two real-valued real_contraint and imag_contraint that applies to them separately.

This is a problem because it will prevent (I think) the idea of making a contraint on the phase or amplitude.
I understand that the ideal would be to be able to define a complex-valued contraint but that will require more work and I am not free.

Does this still work for you? If so, which feature do you think would be better suit for you?

@j-peetz
Copy link
Author

j-peetz commented May 23, 2022

I think having two separate real_constraint and imag_constraint options would be especially useful.

In my specific application, I just needed to set certain weights to 0. And I actually made some modifications that worked for me here: master...j-peetz:master

But I only changed the ComplexDense class. And in general, it'd be nice to be able to specify separate real and imaginary constraints.

Thank you so much for being responsive! Having complex NNs opens up a lot of possibilities, and this library is super helpful.

@NEGU93
Copy link
Owner

NEGU93 commented May 24, 2022

Congrats! I did a pull request of your code.
I leave this issue open as feature requests of:

  • Add Bias contraint
  • Add constrain to other layers (such as Conv)
  • Add separate real_constraint and imag_constraint
  • Best case scenario as complex_constraint. Related to issue 56217

@j-peetz
Copy link
Author

j-peetz commented May 24, 2022

Ok great! Thanks again.

@NEGU93 NEGU93 changed the title ComplexDense arguments have no effect Implement complex-valued constraint parameter Jun 2, 2022
@NEGU93 NEGU93 added the help wanted Extra attention is needed label Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants