We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Dear authors,
thank you very much for your work. I have a question about the replication of the results from your paper.
First, I evaluate your provided weights, model and config file on the val1 data split, I get the following results:
OLD_test_iter pretrain 2d car --> easy: 0.9277, mod: 0.8439, hard: 0.6785 NEW_test_iter pretrain 2d car --> easy: 0.9342, mod: 0.8377, hard: 0.6742 OLD_test_iter pretrain gr car --> easy: 0.3349, mod: 0.2507, hard: 0.1983 NEW_test_iter pretrain gr car --> easy: 0.3225, mod: 0.2268, hard: 0.1722 OLD_test_iter pretrain 3d car --> easy: 0.2490, mod: 0.2077, hard: 0.1729 NEW_test_iter pretrain 3d car --> easy: 0.2317, mod: 0.1621, hard: 0.1234 OLD_test_iter pretrain 2d pedestrian --> easy: 0.6618, mod: 0.5812, hard: 0.4975 NEW_test_iter pretrain 2d pedestrian --> easy: 0.6896, mod: 0.5670, hard: 0.4756 OLD_test_iter pretrain gr pedestrian --> easy: 0.0628, mod: 0.0512, hard: 0.0483 NEW_test_iter pretrain gr pedestrian --> easy: 0.0471, mod: 0.0391, hard: 0.0321 OLD_test_iter pretrain 3d pedestrian --> easy: 0.0436, mod: 0.0445, hard: 0.0396 NEW_test_iter pretrain 3d pedestrian --> easy: 0.0371, mod: 0.0293, hard: 0.0270 OLD_test_iter pretrain 2d cyclist --> easy: 0.6234, mod: 0.4608, hard: 0.3972 NEW_test_iter pretrain 2d cyclist --> easy: 0.6301, mod: 0.4180, hard: 0.3816 OLD_test_iter pretrain gr cyclist --> easy: 0.0344, mod: 0.0296, hard: 0.0306 NEW_test_iter pretrain gr cyclist --> easy: 0.0295, mod: 0.0168, hard: 0.0168 OLD_test_iter pretrain 3d cyclist --> easy: 0.0293, mod: 0.0270, hard: 0.0262 NEW_test_iter pretrain 3d cyclist --> easy: 0.0263, mod: 0.0149, hard: 0.0148
Then i use your config to train the net, config information as below:
conf.model = 'resnet_dilate' conf.lr = 0.01 conf.max_iter = 40000 conf.use_dropout = True conf.drop_channel = True conf.dropout_rate = 0.5 conf.dropout_position = 'early' # 'early' 'late' 'adaptive' conf.do_test = True conf.lr_policy = 'onecycle' # 'onecycle' # 'cosinePoly' # 'cosineRestart' # 'poly' conf.restart_iters = 5000 conf.batch_size = 2 * 4 conf.base_model = 50 conf.depth_channel = 1 conf.adaptive_diated = True conf.use_seg = False conf.use_corner = False conf.corner_in_3d = False conf.use_hill_loss = False conf.use_rcnn_pretrain = False conf.deformable = False conf.alias = 'Adaptive_block2' conf.result_dir = '_'.join([conf.alias, conf.model + str(conf.base_model), 'batch' + str(conf.batch_size), 'dropout' + conf.dropout_position + str(conf.dropout_rate), 'lr' + str(conf.lr), conf.lr_policy, 'iter' + str(conf.max_iter), datetime.datetime.now().strftime("%Y.%m.%d-%H:%M:%S")]).replace('.', '_').replace(':', '_').replace('-', '_') # solver settings conf.solver_type = 'sgd' conf.momentum = 0.9 conf.weight_decay = 0.0005 conf.snapshot_iter = 5000 conf.display = 50 # sgd parameters conf.lr_steps = None conf.lr_target = conf.lr * 0.00001 # random conf.rng_seed = 2 conf.cuda_seed = 2 # misc network conf.image_means = [0.485, 0.456, 0.406] conf.image_stds = [0.229, 0.224, 0.225] if conf.use_rcnn_pretrain: conf.image_means = [102.9801, 115.9465, 122.7717] # conf.image_means[::-1] conf.image_stds = [1, 1, 1] #conf.image_stds[::-1] if conf.use_seg: conf.depth_mean = [4413.160626995486, 4413.160626995486, 5.426258330316642] conf.depth_std = [3270.0158918863494, 3270.0158918863494, 0.5365540402943388] else: conf.depth_mean = [4413.160626995486, 4413.160626995486, 4413.160626995486] # DORN conf.depth_std = [3270.0158918863494, 3270.0158918863494, 3270.0158918863494] # conf.depth_mean = [8295.013626842678, 8295.013626842678, 8295.013626842678] # PSM # conf.depth_std = [5134.9781439128665, 5134.9781439128665, 5134.9781439128665] # conf.depth_mean = [30.83664619525601, 30.83664619525601, 30.83664619525601] # DISP # conf.depth_std = [19.992999492848206, 19.992999492848206, 19.992999492848206] if conf.depth_channel == 3: conf.depth_mean = [137.39162828, 40.58310471, 140.70854621] # MONO1 conf.depth_std = [33.75859339, 51.479677, 65.254889] conf.depth_mean = [107.0805491, 68.26778312, 133.50751215] # MONO2 conf.depth_std = [38.65614623, 73.59464917, 88.24401221] conf.feat_stride = 16 conf.has_3d = True # ---------------------------------------- # image sampling and datasets # ---------------------------------------- # scale sampling conf.test_scale = 512 conf.crop_size = [512, 1760] conf.mirror_prob = 0.50 conf.distort_prob = -1 # datasets conf.dataset_test = 'kitti_split1' conf.datasets_train = [{'name': 'kitti_split1', 'anno_fmt': 'kitti_det', 'im_ext': '.png', 'scale': 1}] conf.use_3d_for_2d = True # percent expected height ranges based on test_scale # used for anchor selection conf.percent_anc_h = [0.0625, 0.75] # labels settings conf.min_gt_h = conf.test_scale*conf.percent_anc_h[0] conf.max_gt_h = conf.test_scale*conf.percent_anc_h[1] conf.min_gt_vis = 0.65 conf.ilbls = ['Van', 'ignore'] conf.lbls = ['Car', 'Pedestrian', 'Cyclist'] # ---------------------------------------- # detection sampling # ---------------------------------------- # detection sampling conf.fg_image_ratio = 1.0 conf.box_samples = 0.20 conf.fg_fraction = 0.20 conf.bg_thresh_lo = 0 conf.bg_thresh_hi = 0.5 conf.fg_thresh = 0.5 conf.ign_thresh = 0.5 conf.best_thresh = 0.35 # ---------------------------------------- # inference and testing # ---------------------------------------- # nms conf.nms_topN_pre = 3000 conf.nms_topN_post = 40 conf.nms_thres = 0.4 conf.clip_boxes = False conf.test_protocol = 'kitti' conf.test_db = 'kitti' conf.test_min_h = 0 conf.min_det_scales = [0, 0] # ---------------------------------------- # anchor settings # ---------------------------------------- # clustering settings conf.cluster_anchors = 0 conf.even_anchors = 0 conf.expand_anchors = 0 conf.anchors = None conf.bbox_means = None conf.bbox_stds = None # initialize anchors base = (conf.max_gt_h / conf.min_gt_h) ** (1 / (12 - 1)) conf.anchor_scales = np.array([conf.min_gt_h * (base ** i) for i in range(0, 12)]) conf.anchor_ratios = np.array([0.5, 1.0, 1.5]) # loss logic conf.hard_negatives = True conf.focal_loss = 1 conf.cls_2d_lambda = 1 conf.iou_2d_lambda = 0 conf.bbox_2d_lambda = 1 conf.bbox_3d_lambda = 1 conf.bbox_3d_proj_lambda = 0.0 conf.hill_climbing = True # visdom conf.visdom_port = 9891 return conf
But the results are not the same as in the paper or your provided models. And we got different results every time. Our results:
round1: OLD_test_iter pretrain 2d car --> easy: 0.8227, mod: 0.7466, hard: 0.6567 NEW_test_iter pretrain 2d car --> easy: 0.8689, mod: 0.7665, hard: 0.6302 OLD_test_iter pretrain gr car --> easy: 0.3332, mod: 0.2484, hard: 0.2033 NEW_test_iter pretrain gr car --> easy: 0.2925, mod: 0.2058, hard: 0.1627 OLD_test_iter pretrain 3d car --> easy: 0.2604, mod: 0.1910, hard: 0.1689 NEW_test_iter pretrain 3d car --> easy: 0.2060, mod: 0.1459, hard: 0.1150 OLD_test_iter pretrain 2d pedestrian --> easy: 0.6248, mod: 0.4880, hard: 0.4088 NEW_test_iter pretrain 2d pedestrian --> easy: 0.5965, mod: 0.4799, hard: 0.3932 OLD_test_iter pretrain gr pedestrian --> easy: 0.0437, mod: 0.0456, hard: 0.0433 NEW_test_iter pretrain gr pedestrian --> easy: 0.0387, mod: 0.0346, hard: 0.0294 OLD_test_iter pretrain 3d pedestrian --> easy: 0.0382, mod: 0.0403, hard: 0.0359 NEW_test_iter pretrain 3d pedestrian --> easy: 0.0298, mod: 0.0263, hard: 0.0214 OLD_test_iter pretrain 2d cyclist --> easy: 0.4188, mod: 0.2548, hard: 0.2557 NEW_test_iter pretrain 2d cyclist --> easy: 0.4219, mod: 0.2430, hard: 0.2432 OLD_test_iter pretrain gr cyclist --> easy: 0.0396, mod: 0.0227, hard: 0.0227 NEW_test_iter pretrain gr cyclist --> easy: 0.0256, mod: 0.0135, hard: 0.0137 OLD_test_iter pretrain 3d cyclist --> easy: 0.0374, mod: 0.0227, hard: 0.0227 NEW_test_iter pretrain 3d cyclist --> easy: 0.0219, mod: 0.0127, hard: 0.0121 round2: OLD_test_iter pretrain 2d car --> easy: 0.8936, mod: 0.7502, hard: 0.6588 NEW_test_iter pretrain 2d car --> easy: 0.8975, mod: 0.7701, hard: 0.6333 OLD_test_iter pretrain gr car --> easy: 0.3283, mod: 0.2423, hard: 0.1982 NEW_test_iter pretrain gr car --> easy: 0.2892, mod: 0.1931, hard: 0.1571 OLD_test_iter pretrain 3d car --> easy: 0.2591, mod: 0.1890, hard: 0.1639 NEW_test_iter pretrain 3d car --> easy: 0.2069, mod: 0.1415, hard: 0.1061 OLD_test_iter pretrain 2d pedestrian --> easy: 0.5659, mod: 0.4894, hard: 0.4094 NEW_test_iter pretrain 2d pedestrian --> easy: 0.5809, mod: 0.4803, hard: 0.3945 OLD_test_iter pretrain gr pedestrian --> easy: 0.0437, mod: 0.0459, hard: 0.0403 NEW_test_iter pretrain gr pedestrian --> easy: 0.0394, mod: 0.0337, hard: 0.0280 OLD_test_iter pretrain 3d pedestrian --> easy: 0.0392, mod: 0.0380, hard: 0.0334 NEW_test_iter pretrain 3d pedestrian --> easy: 0.0298, mod: 0.0254, hard: 0.0200 OLD_test_iter pretrain 2d cyclist --> easy: 0.5863, mod: 0.3419, hard: 0.3375 NEW_test_iter pretrain 2d cyclist --> easy: 0.5724, mod: 0.3404, hard: 0.3179 OLD_test_iter pretrain gr cyclist --> easy: 0.0752, mod: 0.0477, hard: 0.0473 NEW_test_iter pretrain gr cyclist --> easy: 0.0544, mod: 0.0323, hard: 0.0273 OLD_test_iter pretrain 3d cyclist --> easy: 0.0508, mod: 0.0399, hard: 0.0416 NEW_test_iter pretrain 3d cyclist --> easy: 0.0411, mod: 0.0230, hard: 0.0231 round3: OLD_test_iter pretrain 2d car --> easy: 0.8492, mod: 0.7583, hard: 0.5942 NEW_test_iter pretrain 2d car --> easy: 0.8977, mod: 0.7804, hard: 0.6208 OLD_test_iter pretrain gr car --> easy: 0.3537, mod: 0.2569, hard: 0.2068 NEW_test_iter pretrain gr car --> easy: 0.3130, mod: 0.2161, hard: 0.1703 OLD_test_iter pretrain 3d car --> easy: 0.2734, mod: 0.1939, hard: 0.1767 NEW_test_iter pretrain 3d car --> easy: 0.2269, mod: 0.1524, hard: 0.1224 OLD_test_iter pretrain 2d pedestrian --> easy: 0.5628, mod: 0.4819, hard: 0.4035 NEW_test_iter pretrain 2d pedestrian --> easy: 0.5641, mod: 0.4587, hard: 0.3722 OLD_test_iter pretrain gr pedestrian --> easy: 0.0692, mod: 0.0542, hard: 0.0516 NEW_test_iter pretrain gr pedestrian --> easy: 0.0491, mod: 0.0393, hard: 0.0324 OLD_test_iter pretrain 3d pedestrian --> easy: 0.0527, mod: 0.0507, hard: 0.0450 NEW_test_iter pretrain 3d pedestrian --> easy: 0.0402, mod: 0.0309, hard: 0.0246 OLD_test_iter pretrain 2d cyclist --> easy: 0.4663, mod: 0.3081, hard: 0.2457 NEW_test_iter pretrain 2d cyclist --> easy: 0.4275, mod: 0.2526, hard: 0.2339 OLD_test_iter pretrain gr cyclist --> easy: 0.0603, mod: 0.0455, hard: 0.0455 NEW_test_iter pretrain gr cyclist --> easy: 0.0272, mod: 0.0134, hard: 0.0136 OLD_test_iter pretrain 3d cyclist --> easy: 0.0586, mod: 0.0455, hard: 0.0455 NEW_test_iter pretrain 3d cyclist --> easy: 0.0254, mod: 0.0125, hard: 0.0123
Can you please tell me how can I obtain the same results as in the paper? Thank you!
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Dear authors,
thank you very much for your work. I have a question about the replication of the results from your paper.
First, I evaluate your provided weights, model and config file on the val1 data split, I get the following results:
Then i use your config to train the net, config information as below:
But the results are not the same as in the paper or your provided models. And we got different results every time. Our results:
Can you please tell me how can I obtain the same results as in the paper?
Thank you!
The text was updated successfully, but these errors were encountered: