ScaledDotProductAttention doesn't apply C.sqrt?

[ghost]

Hello, there

I am very grateful for the cntkx you are developing.

Now I'm trying to use scaled_dot_product_attention, but it doesn't seem like you apply C.sqrt in the follow part, is that a missing or is it not a problem?

cntkx/ops/__init__.py

Line 479 in c2775ea

dk = C.reduce_sum(C.ones_like(query)) # cannot use sequence.last, will conflict with recurrence

cntkx/ops/__init__.py

Line 486 in c2775ea

scaled = C.times_transpose(query, broadcasted_key) / dk

[delzac]

You are absolutely right! Sorry for the bug, i'll fix it right away and push a new release. Thanks for reporting this issue!

[delzac]

Just push out a new build 0.1.33 where the bug is resolved. Please feel free to reopen the issue if there's any other problem!