Enable Near Mode for K4W #274

Closed
dxli opened this Issue Feb 24, 2012 · 18 comments

4 participants

@dxli

Hi,

I need to set my kinect for windows(K4W) in near mode.

with microsoft kinect SDK, it's done by the following line,

m_pNuiSensor->NuiImageStreamSetImageFrameFlags(m_pDepthStreamHandle, NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE );

Any help is appreciated on how to get the similar done in libfreenect.

Thanks,

dxli

@nneonneo

There's no support for near mode at the moment. I expect the fix is probably a simple register write, but since I don't have a K4W unit, I can't test this out.

Any K4W-endowed hardware hackers willing to find out?

@dxli

Do we have a tool to figure out the transmission through USB?

For example, collect registers set in cases of far-mode and near-mode, then find the difference.

Right now, I can keep everything else the same, and toggle the near-mode on/off by a single line from Windows Kinect SDK 1.0

@nneonneo

You could use a tool like UsbLyzer to dump the communication between the Kinect and your computer. Turn off all the streams to avoid a flood of data, then toggle the near/far mode a few times. In theory, you will be able to capture a relatively small set of USB packets; we can examine those and find the packets corresponding to the near mode functionality.

@dxli

hi nneonneo,

I have tested the UsbLyzer. I have difficulty in turning streams off. Still I keep the flag settings in regular intervals,

while(true){
OutputDebugString(L"change flags\n");
m_pNuiSensor->NuiImageStreamSetImageFrameFlags(m_pDepthStreamHandle, NUI_IMAGE_STREAM_FLAG_SUPPRESS_NO_FRAME_DATA);
Sleep(250);
m_pNuiSensor->NuiImageStreamSetImageFrameFlags(m_pDepthStreamHandle, NUI_IMAGE_STREAM_FLAG_ENABLE_NEAR_MODE);
Sleep(250);
m_pNuiSensor->NuiImageStreamSetImageFrameFlags(m_pDepthStreamHandle, NUI_IMAGE_STREAM_FLAG_DISTINCT_OVERFLOW_DEPTH_VALUES);
Sleep(500);
}

The data file is lzma compressed:

https://github.com/dxli/libfreenect/blob/master/b.ulz.lzma

@nneonneo

Thanks! That's very helpful.

It looks like there are two registers involved. I'm not sure which is which, but here's the sequence:

t=0.35 write 0x0007 to reg 0x0015 (wiki claims this is an apparent no-op)
t=0.45 write 0x0000 to reg 0x02EF (new register, could be near mode enabler)
t=0.70 write 0x001E to reg 0x0015 (default value for this register)
t=0.80 write 0x0190 to reg 0x02EF

t=1.55 write 0x0007 to reg 0x0015
t=1.65 write 0x0000 to reg 0x02EF
t=1.90 write 0x001E to reg 0x0015
t=2.00 write 0x0190 to reg 0x02EF

So, setting the flag seems to take 100ms, and otherwise the looping flag changes are very obvious. I would suggest trying to set reg 0x02EF to 0x0190 in libfreenect and seeing if that will trigger near mode (you may want to write reg 0x0015 first, just in case?)

@nneonneo

The fact that 0x190 = 400 lends credence to the notion that 0x2EF is the near-mode enable, because 400mm is the closest distance that the Kinect sensor will see in near mode, and also the difference between the near-mode minimum distance and the normal-mode minimum distance (800mm).

@dxli

Hi nneonneo,

Now, I got near-mode with libfreenect. however, it's still very puzzling at this point, because I can not get far mode back unless I try the kinect with windows SDK.

I can set the register 0x2ef to 0x320 (800), or simply skip setting it, while still get the kinect camera in near mode.

my code to tweak the 0x2ef is here:

dxli@5cb5051

thanks!

@nneonneo

Try setting 0x2EF to 0x0000, and also try setting 0x0015 to 0x0007 before setting 0x02EF.

@dxli
@nneonneo
@dxli

Hi Robert,

I have tested setting registers early, but still couldn't get far mode back. Here are my changes:

https://github.com/dxli/libfreenect/tree/nearMode

If you want to me to test other ideas, you may send me pull requests or patches.

Thanks a lot already!

dongxu

@nneonneo

Actually, are you certain it is in near mode? It's actually possible that I got this backwards: 7 to 0x15 and 0 to 0x2EF might be near mode enable; 0x1E to 0x15 and 400 to 0x2EF might be near mode disable. I'm basing this on the fact that the enable and disable are 250ms apart in both the code you sent and the UsbLyzer output...

Can you please double-check to make sure the K4W unit is really being set to a near mode? The easiest way to find out if the toggle does anything would be to make a function which toggles the value, then run something like

{
  freenect_stop_depth(dev);
  freenect_set_near_mode(dev, 1);
  freenect_start_depth(dev);
  for(int i=0; i<50; i++) { freenect_process_events(ctx); } // process events normally for 50 frames

  freenect_stop_depth(dev);
  freenect_set_near_mode(dev, 0);
  freenect_start_depth(dev);
  for(int i=0; i<50; i++) { freenect_process_events(ctx); } // process events normally for 50 frames
}

in, e.g. glview. Then, you should be able to see the depth data changing if the settings are working.

@dxli

Hi Robert,

First, I'm 100% sure that the camera is in near mode, because I can reboot to windows and get far mode, and the difference between 400mm and 800mm is beyond any doubt.

I run out of ideas here, after tried all suggestions here. If there are new ideas to test, please send me a pull request or patch.

Thanks,

dxli

@nneonneo

I feel like we missed something. Just a few more things to try off the top of my head:

  1. Are you using DEPTH_REGISTERED or DEPTH_11BIT in libfreenect?
  2. Could you post another UsbLyzer log with the depth data intact, switching between just near and far modes? Ideally, use different delays (e.g. 1200ms, 1900ms) so that it's clear which is which in the output.
  3. The K4W library actually changes the registers (apparently) without stopping the depth stream in-between. I suppose you could try that, though it definitely seems weird.
@renewagner

It turns out that what the above patch did was to enable and then immediately disable near mode again. Even in default mode the Kinect will sense objects as close as 500mm in front of the sensor. You really want to check whether you get depth info when the object is closer than 500mm.

The sequence to enable near mode is:
write 0x0007 to reg 0x0015
write 0x0000 to reg 0x02EF

To disable it:
write 0x001E to reg 0x0015
write 0x0190 to reg 0x02EF

I've implemented this in my libfreenect fork:

renewagner@34f4c1a

Cheers,

Rene

@nneonneo

Cool! The patch looks good. You should submit a pull request for it :)

@piedar piedar modified the milestone: v0.4.0, v0.5.0 Feb 6, 2014
@piedar piedar modified the milestone: v0.5.1, v0.5.0 Apr 29, 2014
@piedar piedar modified the milestone: v0.5.2, v0.5.1 Jul 20, 2014
@piedar piedar referenced this issue Aug 7, 2014
Closed

enable nearmode #407

@piedar piedar modified the milestone: v0.5.2, v0.5.1 Aug 7, 2014
@piedar piedar modified the milestone: v0.5.2, v0.5.1 Nov 3, 2014
@piedar

I ported near mode to master in 75ec2a6, but I cannot be sure it works because I don't have K4W. Please test and report back here!

@piedar

Assumed working. Please file a new issue if it's broken.

@piedar piedar closed this Jan 7, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment