[esp/bindings_js] add initial support for object nav #307
Conversation
Object and best-view IoU is specified by a URL parameter. If the user presses the spacebar and IoU of object exceeds 50% of best-view the task is successfully completed.
facebook-github-bot
added
the
CLA Signed
|
|
||
| #scope { | ||
| position: absolute; | ||
| border-color: red; |
There was a problem hiding this comment.
You can do
border: 5px dashed red;
| opacity: 0.4; | ||
| top: 50%; | ||
| left: 50%; | ||
| margin-left: -160px; |
There was a problem hiding this comment.
I am not really a fan of absolute values as they don't scale according to screen sizes. But, we can focus on it later.
| @@ -30,6 +30,7 @@ Module.preRun.push(() => { | |||
| let config = {}; | |||
| config.scene = defaultScene; | |||
| buildConfigFromURLParameters(config); | |||
| window.config = config; | |||
There was a problem hiding this comment.
Let's build it in your ObjectSensor again. Polluting the global window object is usually not preferred.
| @@ -3,6 +3,9 @@ | |||
| // LICENSE file in the root directory of this source tree. | |||
|
|
|||
| /*global Module */ | |||
| import ObjectSensor from "./object_sensor"; | |||
|
|
|||
| const IOU_TOLERANCE = 0.5; | |||
| this.semanticObservation = new Module.Observation(); | ||
| this.semanticObjects = this.sim.sim.getSemanticScene().objects; | ||
| this.semanticScene = this.sim.sim.getSemanticScene(); | ||
| this.semanticObjects = this.semanticScene.objects; | ||
|
|
||
| if (window.config.category) { | ||
| const scopeWidth = this.components.scope.offsetWidth; | ||
| const scopeHeight = this.components.scope.offsetHeight; | ||
| const scopeInsetX = (this.components.canvas.width - scopeWidth) / 2; | ||
| const scopeInsetY = (this.components.canvas.height - scopeHeight) / 2; | ||
| const objectSearchRect = { | ||
| left: scopeInsetX, | ||
| top: scopeInsetY, | ||
| right: scopeInsetX + scopeWidth, | ||
| bottom: scopeInsetY + scopeHeight | ||
| }; | ||
| this.objectSensor = new ObjectSensor( | ||
| objectSearchRect, | ||
| this.semanticShape, | ||
| this.semanticScene, | ||
| window.config.category | ||
| ); | ||
| } |
There was a problem hiding this comment.
It might be a good time to move this to its own function?
| } | ||
| } | ||
|
|
||
| console.log(this.objectIds); |
| } | ||
| } | ||
| for (let i = 0; i < objects.size(); i++) { | ||
| const object = objects.get(i); |
There was a problem hiding this comment.
| const object = objects.get(i); | |
| const object = objects.get(i) || {}; |
| for (let i = 0; i < objects.size(); i++) { | ||
| const object = objects.get(i); | ||
| if ( | ||
| object && |
There was a problem hiding this comment.
| object && |
| * @param {String} className - object class to search for | ||
| */ | ||
| computeIOU(observation) { | ||
| const width = this.semanticShape.get(1); |
There was a problem hiding this comment.
Might not make sense to move to utils with objectIds check below.
| } | ||
| } | ||
| } | ||
| if (maskPixelsIn === 0) { |
There was a problem hiding this comment.
Won't this will be automatically captured in statement below. For speed purposes, checking this in the start would be better without even iterating.
There was a problem hiding this comment.
This value is computed in the loop.
There was a problem hiding this comment.
I meant this check is not needed as without this check it will still return zero.
I thought you were saving on rectArea calculation. If that was the case, this can be precomputed by checking top, left, bottom, right of both triangles beforehand.
…h#307) Object and best-view IoU is specified by a URL parameter. If the user presses the spacebar and IoU of object exceeds 50% of best-view the task is successfully completed.
Object and best-view IoU is specified by a URL parameter. If the
user presses the spacebar and IoU of object exceeds 50% of best-view
the task is successfully completed.
Motivation and Context
Required for object nav.
How Has This Been Tested
Verified that object nav is working.
Types of changes
Checklist