Embeddings

[AnFreTh]

This pull request includes significant changes to the testing workflow, attention utilities, embedding layer, base model, and transformer model. The most important changes include adding a new GitHub Actions workflow for unit testing, removing rotary embeddings from the attention utilities, updating the embedding layer to handle additional feature types, and modifying the base and transformer models to support the new embedding layer structure.

Testing Workflow:

• .github/workflows/pr-tests.yml: Added a GitHub Actions workflow to run unit tests on pull requests targeting the develop and master branches. This includes setting up Python, installing dependencies, and running tests using pytest.

Attention Utilities:

• mambular/arch_utils/layer_utils/attention_utils.py: Removed the rotary parameter and related code from the Attention and Transformer classes, simplifying the attention mechanism. [1] [2] [3] [4] [5]

Embedding Layer:

• mambular/arch_utils/layer_utils/embedding_layer.py: Updated the EmbeddingLayer class to handle an additional emb_feature_info parameter and process embedding features. Modified the forward method to integrate these new features. [1] [2] [3] [4] [5] [6] [7]

Base Model:

• mambular/base_models/basemodel.py: Updated the encode method to accept a single data parameter instead of separate num_features and cat_features parameters. [1] [2]

Transformer Model:

• mambular/base_models/ft_transformer.py: Modified the FTTransformer class to accept a tuple of feature information and updated the forward method to handle the new embedding layer structure. [1] [2] [3]

Lightning Wrapper:

• mambular/base_models/lightning_wrapper.py: Adjusted the forward, training_step, validation_step, test_step, and predict_step methods to work with the new embedding layer structure. [1] [2] [3] [4] [5] [6] [7] [8] [9]

[AnFreTh]

Hey @mkumar73 - I added a workflow integrating a few unit tests as a start. Could you have a closer look? It seems unnecessarily slow.