Initial commit (#516)

python-sdk/nuscenes/eval/detection/README.md

@@ -46,6 +46,16 @@ Submissions will be accepted from November 1 to December 8, 2020.
 Results and winners will be announced at the [5th AI Driving Olympics](https://driving-olympics.ai/) at NeurIPS 2020.
 Note that this challenge uses the same [evaluation server](https://eval.ai/web/challenges/challenge-page/356/overview) as previous detection challenges.
 
+A summary of the results can be seen below. 
+For details, please refer to the [detection leaderboard](https://www.nuscenes.org/object-detection).
+
+| Rank | Team name          | NDS   |
+|---   |---                 |---    |
+|  1   | CenterPoint        | 71.4% |
+|  2   | PointAugmenting    | 71.1% |
+|  3   | MoCa               | 70.9% |
+|  4   | PVC ensemble       | 70.4% |
+
 ### Workshop on Benchmarking Progress in Autonomous Driving, ICRA 2020
 The second nuScenes detection challenge will be held at [ICRA 2020](https://www.icra2020.org/).
 The submission period will open April 1 and continue until May 28th, 2020.

python-sdk/nuscenes/eval/lidarseg/README.md

@@ -46,6 +46,16 @@ Results and winners will be announced at the [5th AI Driving Olympics](https://d
 For more information see the [leaderboard](https://www.nuscenes.org/lidar-segmentation).
 Note that the [evaluation server](https://eval.ai/web/challenges/challenge-page/720/overview) can still be used to benchmark your results.
 
+A summary of the results can be seen below. 
+For details, please refer to the [lidar segmentation leaderboard](https://www.nuscenes.org/lidar-segmentation).
+
+| Rank | Team name          | mIOU  |
+|---   |---                 |---    |
+|  1   | Noah_Kyber         | 0.783 |
+|  2   | Cylinder3D++       | 0.779 |
+|  3   | CPFusion           | 0.777 |
+|  4   | MIT-HAN-LAB        | 0.774 |
+
 ## Submission rules
 ### Lidar segmentation-specific rules
 * The maximum time window of past sensor data and ego poses that may be used at inference time is approximately 0.5s (at most 6 past camera images, 6 past radar sweeps and 10 past lidar sweeps). At training time there are no restrictions.

python-sdk/nuscenes/eval/lidarseg/render.py

@@ -115,6 +115,8 @@ def render_stratified_overall_metrics(self, filename: str, dpi: int = 100) -> No
         ax = fig.add_subplot(111)
         ax.bar(list(stratified_iou.keys()),
                list(stratified_iou.values()), color='grey')
+        ax.spines['right'].set_visible(False)
+        ax.spines['top'].set_visible(False)
         ax.set_xticklabels(self.strata_names, rotation=45, horizontalalignment='right')
         ax.set_ylabel('mIOU', fontsize=15)
         ax.set_ylim(top=1.1)  # Make y-axis slightly higher to accommodate tag.
@@ -133,7 +135,7 @@ def render_stratified_overall_metrics(self, filename: str, dpi: int = 100) -> No
         if self.verbose:
             plt.show()
 
-    def render_stratified_per_class_metrics(self, filename: str, dpi: int = 150) -> None:
+    def render_stratified_per_class_metrics(self, filename: str, dpi: int = 250) -> None:
         """
         Renders the stratified per class metrics.
         :param filename: Filename to save the render as.
@@ -155,6 +157,8 @@ def render_stratified_per_class_metrics(self, filename: str, dpi: int = 150) ->
         for n, (cls, cls_strata) in enumerate(stratified_classes.items()):
             ax = fig.add_subplot(gs[n])
             ax.bar(self.strata_names, cls_strata, color=np.array(self.mapper.coarse_colormap[cls]) / 255)
+            ax.spines['right'].set_visible(False)
+            ax.spines['top'].set_visible(False)
             ax.set_xticklabels(self.strata_names, rotation=45, horizontalalignment='right')
             ax.set_ylabel('IOU', fontsize=3)
             ax.set_ylim(top=1.1)  # Make y-axis slightly higher to accommodate tag.

python-sdk/nuscenes/eval/lidarseg/validate_submission.py

@@ -142,7 +142,7 @@ def validate_submission(nusc: NuScenes,
     parser.add_argument('--verbose', type=bool, default=False,
                         help='Whether to print to stdout.')
     parser.add_argument('--zip_out', type=str, default=None,
-                        help='Whether to print to stdout.')
+                        help='Path to zip the results folder to.')
     args = parser.parse_args()
 
     result_path_ = args.result_path