Django axes with msal integration

Is it possible to toggle django axes depending on the authentication backend used, if so how? 
I have a case where I'm using multiple login methods, and two of them are external ones like Microsoft Identity (via msal) and Google OAuth., for Microsoft if axes is enabled it keeps giving me the following error: 'dict' object has no attribute 'axes_locked_out' 

