Notification category __dynamic__ is not being replaced when the new button set is registered. This is adding another copy to the existing list. It sometimes replaces the last registered set for __dynamic__ and sometimes not. The below code needs to look for an existing __dynamic__ entry and remove it before adding the new one.
https://github.com/OneSignal/OneSignal-iOS-SDK/blob/2.4.0/iOS_SDK/OneSignal/OneSignalHelper.m#L703-L704
Unit test should check to make sure it is being replaced as well.
https://github.com/OneSignal/OneSignal-iOS-SDK/blob/2.4.0/iOS_SDK/UnitTests/UnitTests.m#L883