Enhancing the model architecture customization?

[Morphlng]

Hi! We are trying to apply MARLlib to fulfill our task. The observation from our custom environment can be a little too complicated for a simple MLP/CNN encoder. We want to apply a pretrained model to improve the feature extraction.

Furthermore, the decision network, which is currently RNN based or MLP based, can only tune a little parameter in the config file. It would be great if we can directly use a self-designed torch model (and this can solve the problem of loading pretrained model), or at least release the full customization ability like in Ray.

Wondering if there is any plan to enhance these kinds of ability?

[Ivan-Zhong]

Hi there! Thank you for reaching out and sharing your feedback on MARLlib.

The reason why MARLlib does not support highly customizable models for complex data structures, such as NeuralMMO, is due to the need for data sharing in MARLlib's algorithms, which is essential for centralized training methods like MAPPO and QMIX. Achieving a unified pipeline for handling diverse data sharing strategy becomes impractical in this context.

That being said, it is indeed possible to implement more customized models, especially if you are familiar with RLlib and are focusing on specific algorithm in MARLlib. But this may impact other parts of the code and introduce bugs that require fixing.

My suggestion is to begin with independent learning algorithms such as IPPO and customize the model for your environment, as they are quite similar to RLlib. Once you have gained familiarity and confidence, you can then explore more complex algorithms like MAPPO or HAPPO. I welcome any further concerns or questions you may have.