bad results using the pretrained model than project website

[jbyu]

Hi, I got bad results with the pretrained model, but the results from project website are good.
Is it different model on the project website or I have to do some pre-processing?
Thank you very much.

[fuenwang]

Hi,

Yes, the image need to be aligned first. You can refer to the following code for pre-processing.

            img = PIL.Image.open(img_stream)
            img = np.asarray(img).astype(np.float32) / 255.0
            
            s = config['exp_args']['visualizer_args']['equi_shape']
            if img.shape[0] != s[0] or img.shape[1] != s[1]:
                print ('Resize', img.shape)
                img = cv2.resize(img, dsize=tuple(s[::-1]), interpolation=cv2.INTER_AREA)

            batch = torch.FloatTensor(img.transpose(2, 0, 1)[None, ...]).to(device)

            ER = LED2Net.Projection.EquirecRotate(config['exp_args']['visualizer_args']['equi_shape'][0])
            olines, vp, views, edges, panoEdge, score, angle = LED2Net.Projection.PanoAlign.panoEdgeDetection(
                        (img*255).astype(np.uint8),
                        qError=0.7,
                        refineIter=3
                    )
            vp = vp[2::-1]
            R_vp = torch.FloatTensor(np.linalg.inv(vp.T).T)[None, ...].to(device)
            r_vp = LED2Net.Conversion.MatrixTools.rotation_matrix_to_angle_axis(R_vp)
            r_x, r_y, r_z = torch.unbind(r_vp, dim=-1)
            r_vp = torch.cat([r_x[..., None], -r_z[..., None], r_y[..., None]], dim=-1)
            batch = ER(batch, r_vp)
            
            img_align = (batch.permute(0, 2, 3, 1)[0, ...].data.cpu().numpy() * 255).astype(np.uint8)

            with torch.no_grad(): pred = model(batch)

[jbyu]

I got the correct results. Thank you so much.