Model Convergence and Losses are not Decreasing

[shree-exofield]

I am trying to train the ellipse-rcnn model on my data, but its not converging at all; not even on the FDDB and dataset_sample.h5 as well.
The loss_ellipse and total_loss only fluctuates. Other losses also show similar pattern at lower scale values.
Would you tell me how to solve this issue?
thereby I will have perfect detection for my elliptical object.