Add Whisper Backbone

[abheesht17]

Ta-da!

https://colab.research.google.com/drive/1BhEVK2NpOUICgi8ThLt7Qogzk3DqTJjQ?usp=sharing

[abheesht17]

Reverting back to the original since we can't set "trainable" for weights:

AttributeError: Exception encountered when calling layer "transformer_encoder_layer_0" (type WhisperEncoder).

in user code:

    File "/content/keras-nlp/keras_nlp/layers/transformer_encoder.py", line 180, in call  *
        self._build(inputs.shape)
    File "/content/keras-nlp/keras_nlp/models/whisper/whisper_encoder.py", line 41, in _build  *
        self._self_attention_layer._key_dense.bias.trainable = False

    AttributeError: can't set attribute


Call arguments received by layer "transformer_encoder_layer_0" (type WhisperEncoder):
  • inputs=tf.Tensor(shape=(None, None, 384), dtype=float32)
  • padding_mask=None
  • attention_mask=None

[mattdangerw]

This looks great to me! Just some small comments, but let's add tests!

[mattdangerw]

Looks great! Just a last round of comments I think.

[mattdangerw]

Two nits. But let's merge after that!

[mattdangerw]

Let's do NUM_MELS = 80 after the import block but before the class, that is approach we have used before.

[mattdangerw]

Thanks! Excited for this.