training model for WLP -- stuck in suboptimal solution

[nupoorgandhi]

I'm trying to train the relation extraction model for the Wet Labs Protocol dataset. My loss stays fairly constant, and the model always predicts no relation between each span pair. When I look at the logits I can see that for each relation, the score is pretty much the same for all the examples, so it doesn't seem like anything is being learned. The entity extraction task is working for the WLP dataset, and I am certain that the data format is correct. I have tried learning rates between 1e-3 and 1e-7, and batch sizes 1 and 32. Does anyone have suggestions for how to debug?