KITTI to XVIZ conversion error

[hpanyda]

KITTI to XVIZ converter generates corrupted files which gives error while parsing on browser. Everything works fine with the already converted KITTI data available in XVIZ-data directory.
https://github.com/uber/xviz-data/tree/master/kitti/2011_09_26_drive_0005_sync

Tried with different data streams from the KITTI website but the issue is the same.

4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:8 Uncaught TypeError: Cannot convert undefined or null to object
at Float64Array.set (<anonymous>)
at 4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:8
at p (4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:8)
at 4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:17
at Array.forEach (<anonymous>)
at 4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:17
at t.default (4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:17)
at y (4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:8)
at d (4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:8)
at t.parseXVIZMessageSync (4f8cbb35-d1bd-4b30-983a-dc0efd5cd9db:8)

The converter generates the files without any error

ubuntu@ubuntu1904:~/xviz/examples/converters/kitti$ yarn start -d 
/home/ubuntu/xviz/scripts/../data/kitti/2011_09_26/2011_09_26_drive_0048_sync -o 
/home/ubuntu/xviz/scripts/../data/generated/kitti/2011_09_26/2011_09_26_drive_0048_sync/ -- 
disable-streams=image_00,image_01
yarn run v1.19.1
warning package.json: No license field
$ node ./src/index.js -d 
/home/ubuntu/xviz/scripts/../data/kitti/2011_09_26/2011_09_26_drive_0048_sync -o 
/home/ubuntu/xviz/scripts/../data/generated/kitti/2011_09_26/2011_09_26_drive_0048_sync/ -- 
disable-streams=image_00,image_01
/home/ubuntu/xviz/scripts/../data/kitti/2011_09_26/2011_09_26_drive_0048_sync 
/home/ubuntu/xviz/scripts/../data/generated/kitti/2011_09_26/2011_09_26_drive_0048_sync/
Converting KITTI data at 
/home/ubuntu/xviz/scripts/../data/kitti/2011_09_26/2011_09_26_drive_0048_sync
Saving to 
/home/ubuntu/xviz/scripts/../data/generated/kitti/2011_09_26/2011_09_26_drive_0048_sync/
Matrix4.transformVector has been removed in version 3.0, see upgrade guide for more 
information
(node:30746) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability 
issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
Generate 22 messages in 21452s
Done in 24.09s.

XVIZServer also starts without any errors

ubuntu@ubuntu1904:~/xviz/modules/server/bin$ ./xvizserver -d 
/home/ubuntu/xviz/scripts/../data/generated/kitti/2011_09_26/2011_09_26_drive_0048_sync -- 
port 8081
xvizserver-log: Listening on port 8081

start-streaming-local also does not give any error

ubuntu@ubuntu1904:~/streetscape.gl/examples/get-started$ yarn start-streaming-local --host 
0.0.0.0
yarn run v1.19.1
warning package.json: No license field
$ webpack-dev-server --env.local --env.stream --progress --hot --open --host 0.0.0.0
10% building 1/1 modules 0 activeℹ ｢wds｣: Project is running at http://0.0.0.0:8080/
ℹ ｢wds｣: webpack output is served from /
ℹ ｢wds｣: Content not from webpack is served from /home/ubuntu/streetscape.gl/examples/get- 
started
12% building 21/29 modules 8 active ...e/ubuntu/streetscape.gl/examples/get- 
started/node_modules/source-map- 
 loader/index.js!/home/ubuntu/streetscape.gl/node_modules/react-dom/cjs/react- 
dom.development.jsℹ ｢wdm｣: wait until bundle finished: /
ℹ ｢wdm｣: Hash: 0044c9a648acc9afe3db
Version: webpack 4.41.2
Time: 60732ms
Built at: 10/25/2019 4:54:08 PM
    Asset      Size  Chunks                   Chunk Names
bundle.js  11.6 MiB     app  [emitted]        app
bundle.js.map  12.4 MiB     app  [emitted] [dev]  app
Entrypoint app = bundle.js bundle.js.map
[0] multi (webpack)-dev-server/client?http://0.0.0.0:8080 (webpack)/hot/dev-server.js ./src/app.js 
52 bytes {app} [built]
[../../modules/main/src/index.js] /home/ubuntu/streetscape.gl/modules/main/src/index.js 1.17 KiB 
{app} [built]
[../../modules/monochrome/src/index.js] 
/home/ubuntu/streetscape.gl/modules/monochrome/src/index.js 1.64 KiB {app} [built]
[../../node_modules/react-dom/index.js] /home/ubuntu/streetscape.gl/node_modules/react- 
dom/index.js 1.33 KiB {app} [built]
[../../node_modules/react/index.js] /home/ubuntu/streetscape.gl/node_modules/react/index.js 
190 bytes {app} [built]
[./node_modules/@xviz/parser/dist/esm/index.js] 1.87 KiB {app} [built]
[./node_modules/strip-ansi/index.js] 161 bytes {app} [built]
[./node_modules/webpack-dev-server/client/index.js?http://0.0.0.0:8080] (webpack)-dev- 
server/client?http://0.0.0.0:8080 4.29 KiB {app} [built]
[./node_modules/webpack-dev-server/client/overlay.js] (webpack)-dev-server/client/overlay.js 
3.51 KiB {app} [built]
[./node_modules/webpack-dev-server/client/socket.js] (webpack)-dev-server/client/socket.js 1.53 
KiB {app} [built]
[./node_modules/webpack-dev-server/client/utils/createSocketUrl.js] (webpack)-dev- 
server/client/utils/createSocketUrl.js 2.89 KiB {app} [built]
[./node_modules/webpack-dev-server/client/utils/log.js] (webpack)-dev-server/client/utils/log.js 
964 bytes {app} [built]
[./node_modules/webpack-dev-server/client/utils/reloadApp.js] (webpack)-dev- 
server/client/utils/reloadApp.js 1.59 KiB {app} [built]
[./node_modules/webpack/hot/dev-server.js] (webpack)/hot/dev-server.js 1.59 KiB {app} [built]
[./src/app.js] 8.44 KiB {app} [built]
+ 1727 hidden modules
ℹ ｢wdm｣: Compiled successfully.

[afei26579]

I have the same problem，Has your problem been solved? thanks lot.

[hpanyda]

No, Not solved yet.

[twojtasz]

thanks. This shows an issue arround Float64Array(). I'll give this a try ASAP and see if I can pinpoint the cause.

[patnolan33]

I am getting the exact same errors when converting the KITTI dataset. Is there any update on this? I know there is a a validation tool supposedly able to validate the generated XVIZ data, however I don't see much documentation on how to use it or what it would provide.

This also seems to be related to #534 and #535. I suspect as was stated in #534 the converted data consists of several null values, which I was able to verify using the --json flag for converting as well.

[patnolan33]

Taking a closer look at the generated JSON file using the --json flag, it looks like the LIDAR points are also converted with a bunch of null values. I removed the trajectory stream generation from the converter (so only vehicle pose is converted) and also removed all but the gpsConverter and CameraConverter from the initialize() method code in kitti_converter.js:

this.converters = [
      gpsConverter,
      new CameraConverter(this.inputDir, {
        disabledStreams: this.disabledStreams,
        options: this.imageOptions
      })
  ];

The get-started demo now runs without error in the console window, however the image displays and plots of acceleration and velocity are now missing. Presumably this is because the data doesn't exist in the processed xviz frame? I'm not sure. @twojtasz have you had a chance to dig any deeper on this issue?

EDIT:

Here is a screenshot of what I'm seeing with no LIDAR, Tracklets, trajectory converted. Notice no velocity/acceleration plots and no visible camera windows:

[patnolan33]

So I think I found the issue with the trajectory generation code. In the getPoseTrajectory() method in xviz/builder/xviz-trajectory-helper.js, the return statement is incorrect

Change the return statement to use relativeOffset[0] for X, and similarly for Y and Z (the relativeOffset variable is an array and doesn't have .x, .y, or .z available to it):

function getPoseTrajectory(_ref) {
  var poses = _ref.poses,
      startFrame = _ref.startFrame,
      endFrame = _ref.endFrame;
  var positions = [];
  var iterationLimit = Math.min(endFrame, poses.length);  for (var i = startFrame; i < iterationLimit; i++) {
    positions.push(poses[i].pose);
  }  var startPose = poses[startFrame].pose;
  var worldToStartPoseTransformMatrix = new _math._Pose(startPose).getTransformationMatrix().invert();
  return positions.map(function (currPose) {
    var offset = getGeospatialVector(startPose, currPose);
    var relativeOffset = worldToStartPoseTransformMatrix.transformVector(offset);
    return [relativeOffset[0], relativeOffset[1], relativeOffset[2]];
  });
}

With this fix, the trajectory generation works for the vehicle. Unfortunately, the velocity/acceleration metrics are still missing, as are the camera displays. After digging a bit deeper, the XVIZPanel component is switching its display based on the uiConfig property of the data passed in in the render method:

return uiConfig ? (
  <div>{uiConfig.children && uiConfig.children.map(this._renderItem)}</div>
) : null;

Since this property is undefined, nothing is displayed. I'm not sure at the moment where this property is supposed to be set or why its not currently being set. Has anyone else seen this issue? Here is the props object in xviz-panel.js logged to the console for reference:

{name: "Camera", uiConfig: undefined, log: XVIZFileLoader, style: {…}, components: {…}, …}
    componentProps: {}
    components: {}
    log: XVIZFileLoader {onXVIZMessage: ƒ, onError: ƒ, getCurrentTime: ƒ, getLookAhead: ƒ,       getMetadata: ƒ, …}
    name: "Camera"
    style: {}
    uiConfig: undefined
    __proto__: Object

[speculees]

Also found out that in streetscape.gl modules/core/src/utils/transform.js
I have to add default value to links on line 78 like so
const {origin, links = {}, streams, transforms = {}, vehicleRelativeTransform} = frame;
because links can be undefined

[twojtasz]

I clearly need to jump back on this. I have a Ubuntu setup I will test on tonight. Thanks @speculees @patnolan33 for all your digging. Sorry this has been broken for you.

[twojtasz]

@patnolan33 Huge thanks for tracking this down. The problem was rooted in math.gl being updated and no longer returning a class but a plain array. Previously it had accessors for fields like .x.