Inference is slow when the classes is large.

[Jinming-Su]

@YuwenXiong
When I testing on Mapillary vistas dataset, the classes is set to 66. The post_time is changed from 0.8s (19 classes on cityscapes) to 4.187. Log is shown as listed,

upsnet_end2end_test.py | line 276: Batch 151/2000, data_time:0.017, net_time:0.134, post_time:4.187

Do you know the cause? Is that means the code can not deal with large numbers of classes?