Release 0.9.0

Compatible with TensorFlow 2.8 - 2.19

Updated license

Release 0.8.0

Compatible with TensorFlow 2.8 - 2.19

Added

• Added an input_d parameter to LMUCell. This only needs to be specified when hidden_cell=None and input_to_hidden=True; in that scenario it is required in order to accurately set LMUCell.output_size. (#56)

Release 0.7.0

Compatible with TensorFlow 2.4 - 2.13

Changed

• Minimum supported Python version is now 3.8 (3.7 reached end of life in June 2023). (#54)

Release 0.6.0

Compatible with TensorFlow 2.4 - 2.11

Changed

• LMUFeedforward can now be used with unknown sequence lengths, and LMU will use LMUFeedforward for unknown sequence lengths (as long as the other conditions are met, as before). (#52)
• Allow input_to_hidden=True with hidden_cell=None. This will act as a skip connection. (#52)
• Changed order of LMU states so that the LMU memory state always comes first, and any states from the hidden cell come afterwards. (#52)

Fixed

• Fixed errors when setting non-default dtype on LMU layers. (#52)

Release 0.5.0

Compatible with TensorFlow 2.4 - 2.11

Added

• Layers are registered with the Keras serialization system (no longer need to be passed as custom_objects). (#49)

Release 0.4.2

Compatible with TensorFlow 2.1 - 2.9

Added

• Added support for TensorFlow 2.9. (#48)

Release 0.4.1

Compatible with TensorFlow 2.1 - 2.8

Added

• Added support for TensorFlow 2.8. (#46)

Release 0.4.0

Added

• Setting kernel_initializer=None now removes the dense input kernel. (#40)
• The keras_lmu.LMUFFT layer now supports memory_d > 1. keras_lmu.LMU now uses this implementation for all values of memory_d when feedforward conditions are satisfied (no hidden-to-memory or memory-to-memory connections, and the sequence length is not None). (#40)
• Added trainable_theta option, which will allow the theta parameter to be learned during training. (#41)
• Added discretizer option, which controls the method used to solve for the A and B LMU matrices. This is mainly useful in combination with trainable_theta=True, where setting discretizer="euler" may improve the training speed (possibly at the cost of some accuracy). (#41)
• The keras_lmu.LMUFFT layer can now use raw convolution internally (as opposed to FFT-based convolution). The new conv_mode option exposes this. The new truncate_ir option allows truncating the impulse response when running with a raw convolution mode, for efficiency. Whether FFT-based or raw convolution is faster depends on the specific model, hardware, and amount of truncation. (#42)

Changed

• The A and B matrices are now stored as constants instead of non-trainable variables. This can improve the training/inference speed, but it means that saved weights from previous versions will be incompatible. (#41)
• Renamed keras_lmu.LMUFFT to keras_lmu.LMUFeedforward. (#42)

Fixed

• Fixed dropout support in TensorFlow 2.6. (#42)

Release 0.3.1

Changed

• Raise a validation error if hidden_to_memory or input_to_hidden are True when hidden_cell=None. (#26)

Fixed

• Fixed a bug with the autoswapping in keras_lmu.LMU during training. (#28)
• Fixed a bug where dropout mask was not being reset properly in the hidden cell. (#29)

Release 0.3.0

Changed

• Renamed module from lmu to keras_lmu (so it will now be imported via import keras_lmu), renamed package from lmu to keras-lmu (so it will now be installed via pip install keras-lmu), and changed any references to "NengoLMU" to "KerasLMU" (since this implementation is based in the Keras framework rather than Nengo). In the future the lmu namespace will be used as a meta-package to encapsulate LMU implementations in different frameworks. (#24)