Result changes drastically depending on initialization

[nonconvexopt]

I experienced that compared to Transformers, Mamba model has large variance in performance with respect to model initialization. Did you guys also noticed this?

[tridao]

For language modeling the performance is stable. Which intialization do you mean?

[nonconvexopt]

Just random initialization. I am applying it to financial time series but result has more variance compared to Transformer.