baseline FCN

[yyfyan]

  the baseline FCN can gain 70.03% mIOU in your paper，now I want to realize  the result based your paper.But my result is not good.
  I noticed the paper used LR=0.001,but the git used LR = 0.003
  Can you tell me more details about the training process.Thank you.

[junfu1115]

ref #30

[PangYunsheng8]

I have the same question about the baseline Dilated FCN model. I trained dilated FCN(resnet 101 as backbone) with tensorflow, but only got 65% miou. I found a fcn model in this repo, did you achieve 72.54% miou with it? Could you tell me more details about the baseline dilated fcn model? Thank you very much.

[junfu1115]

@PangYunsheng8 Sorry, I'm not familiar with tensorflow. I suggest you use our framework. We obtain 72.54% without multi-scale training and multi-grid. The epochs is set to 180, cropsize is set to 768, the lr is set to 0.01.

[PangYunsheng8]

Thank you for your reply. Could you tell me the setting of 'output_stride' and 'batch_size' in your dialted FCN? You just upsample the last feature map of Resnet in your dilated FCN and there is no using skip connection？

[junfu1115]

@PangYunsheng8 No skip connection is used. The output_stride is set to 1/8 and batshsize is set to 8 on cityscapes, which are mentioned in the paper.

[cs-heibao]

@PangYunsheng8
hi, I've also implement the PAM and CAM module with tensorflow, when finished the training, the test result seems not good, I do not know whether the coding of PAM and CAM is right? and how the parameter α and β update?

def danet(net_encoder, in_channels, planes, num_classes, gama, beta):
    # =================================== PAM ========================================
    # sa
    # conv5a
    feat1 = Head(net_encoder, in_channels, is_training=True, name='res5feature1')
    m_batchsize, height, width ,C= feat1.shape
    proj_query = PAM_Module(feat1, planes, name='pam1', transpose=True)
    proj_key = PAM_Module(feat1, planes, name='pam2')
    energy = tf.matmul(proj_query, proj_key)
    attention = tf.nn.softmax(energy)
    proj_value = PAM_Module_conv(feat1, planes, name='pam3', reshape=True)
    out = tf.matmul(proj_value, tf.transpose(attention,[0, 2, 1]))
    out = tf.transpose(tf.reshape(out, [m_batchsize, C, height, width]),[0, 2, 3, 1])
    sa_feat = gama*out + feat1
    # conv51
    sa_conv = PAM_Module_conv(sa_feat, planes, kernel_size=3, BN=True, Relu=True, name='pam_conv')
    # conv6
    sa_output = tf.nn.dropout(sa_conv, keep_prob=0.1)
    output = PAM_Module_conv(sa_output,512, num_classes,Out_Classes=True, name='pam_out')
    # =================================== PAM ========================================
    
    # =================================== CAM ========================================
    # conv5c
    feat2 = Head(net_encoder, in_channels, is_training=True, name='res5feature2')
    m_batchsize, height, width, C = feat2.shape
    proj_query = tf.reshape(feat2, [m_batchsize, C, -1])
    proj_key = tf.transpose(tf.reshape(feat2, [m_batchsize, C, -1]), [0, 2, 1])
    energy = tf.matmul(proj_query, proj_key)
    energy_init = tf.ones_like(energy)
    energy_new = tf.reduce_max(energy,-1, keepdims=True)
    energy_new = tf.multiply(energy_init, energy_new)
    energy_new = energy_new - energy
    attention = tf.nn.softmax(energy_new)
    proj_value = tf.reshape(feat2, [m_batchsize, C, -1])
    out = tf.matmul(attention, proj_value)
    out = tf.transpose(tf.reshape(out, [m_batchsize, C, height, width]), [0, 2, 3, 1])
    sc_feat = beta*out + feat2
    # conv52
    sc_conv = PAM_Module_conv(sc_feat, planes, kernel_size=3, BN=True, Relu=True, name='cam_conv')
    # conv7
    sc_output = tf.nn.dropout(sc_conv, keep_prob=0.1)
    sc_output = PAM_Module_conv(sc_output, 512, num_classes, Out_Classes=True, name='cam_out')

    feat_sum = sa_conv + sc_conv

    sasc_output = tf.nn.dropout(feat_sum, keep_prob=0.1)
    output = PAM_Module_conv(sasc_output, 512, num_classes, Out_Classes=True, name='pam_cam')
    return output