fixed OpenGLDepthPacketProcessor texture upload

[gaborpapp]

fixes #31

[christiankerl]

Great if it works! You have some details how you figured this out?

[gaborpapp]

Following your comments from half a year ago in #31 I used the stage1 debug value to check ptables as you suggested. I noticed that the ptables are black, so I suspected they didn't get uploaded. Since the ptables are stored in integer textures they don't allow filtering. The initial value of GL_TEXTURE_MIN_FILTER is GL_NEAREST_MIPMAP_LINEAR, and GL_LINEAR for GL_TEXTURE_MAG_FILTER, which don't work for integer textures. I suppose filtering is not needed for any of the textures used in Protonect anyway. The glPixelStorei(GL_UNPACK_ALIGNMENT, 1) line is probably not necessary, but I wanted to make sure the data alignment is right for the upload.

[RyanGordon]

Tested on my Macbook Air (Intel GPU) and it works great! I get right around 55Hz for the depth frames. I tried it without the GL_UNPACK_ALIGNMENT line as well and it worked just as well but as you said, probably no harm in keeping it.

Thank you for figuring this out! This explains why switching the tables to floats started to fix much of the problem. I would have never guessed we needed to set different values for GL_TEXTURE_MIN_FILTER/GL_TEXTURE_MAG_FILTER.

[gaborpapp]

Thanks for testing it. I'm glad that it works.

[ricardborras]

Tested on Toshiba Portege Z30 with Intel embedded card. Ir and depth still black. Output of lspci:
00:02.0 VGA compatible controller: Intel Corporation Haswell-ULT Integrated Graphics Controller (rev 0b)
Any test I can make? I have activated by hand do_debug variable but I don't see any changes in output data

[gaborpapp]

You could test if the textures are uploaded properly in test_opengl. You can dump the necessary data with #169. Then switch back to #171, change the last line of examples/protonect/src/shader/stage1.fs to Debug = vec4( vec3( texelFetch( ZTable, uv ).x / 10000.0 ), 1 );. Then you should see grey circles in the OpenGLDepthPacketProcessor window. Something like this:

If you don't see anything try to change the Debug value to some colour, e.g. Debug = vec4( 0, 1, 0, 1 ); and check if it is displayed in the window.

[ricardborras]

Hi gaborapp,

I have tried proposed solution and does not work. Let me explain my steps to see if I'm doing something wrong.
1- I have switched to #169 but running bin/test_opengl_dump complains about not found ptables
2- I have switched to #161 and dumped ptables using CpuDepthPacketProcessor. Dumped tables are on bin folder. Depth and IR are displayed ok
3- Running again test_opengl_dump crashes, with no ptable complain.
4- I have switched to #171, modified Protonect to use debug in OpengldepthPacketProcessor, modified the shader as you suggested (in both ways) and see black window in OpenGLDepthPacketProcessor window.

My impression is that some OpenGL commands still dislike my OpenGL configuration. Running glxinfo, I have (resumed information):
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) Haswell Mobile x86/MMX/SSE2
OpenGL core profile version string: 3.3 (Core Profile) Mesa 10.1.3
OpenGL core profile shading language version string: 3.30
OpenGL core profile context flags: (none)
OpenGL core profile profile mask: core profile
OpenGL version string: 3.0 Mesa 10.1.3
OpenGL shading language version string: 1.30

Any clues?

Thanks

[gaborpapp]

in 1) & 3) you mean test_opengl not test_opengl_dump, don't you?

it seems that your issue is not connected to the texture upload fix. do you see anything dumped on the console when running protonect or test_opengl? maybe about shaders not getting compiled? if you are not seeing even the simple debug colour as described above, there must be something else going on.

[ricardborras]

Yes, I mean test_opengl, sorry for the mistake.
There are no shader errors outputted to the console, but it is clear that some OpenGL capabilities of my graphic card are not enought for the shaders. I will test on another desktop computer with external graphic card.

thanks,

Ricard

[svope]

I have a similar issue as ricardborras and having the same configuration:
OpenGL renderer string: Mesa DRI Intel(R) Haswell Mobile
OpenGL core profile version string: 3.3 (Core Profile) Mesa 10.1.3
OpenGL core profile shading language version string: 3.30
etc..

Using libfreenect2::CpuPacketPipeline() everything works fine (but slow).

I did:

1. dumped data with added --dump option for exporting input data for test_opengl #161
2. ran test_opengl with dumped data and with this output
   

I didn't dig into code really and I'm not very OpenGL educated, but from my understanding, this is only OpenGL/drawing problem related?

[gaborpapp]

OpenGL 3.3 should be fine. I think your issue is caused by something else. Would be worth to check for OpenGL errors and capabilities.

[ThomasLengeling]

I just did a fresh install on a Mac 10.10.3 a MBP Retina (early 2013) with a NVIDIA GeForce 650M. However, only the RGB frame is showing. I also tried this pull request but didn't work, it actually crashes at the beginning, any suggestion?

Thomas

[gaborpapp]

Have you tried the suggestions above? What did you get? Can you see the tables? If not does the shaders compile and link?

[svope]

I tried to make some OpenGL debug, but error codes are not really describing:
file: opengl_depth_packet_proccessor.cpp
function: void allocate(size_t new_width, size_t new_height)
after " glTexImage2D(GL_TEXTURE_RECTANGLE, 0, FormatT::InternalFormat, width, height, 0, FormatT::Format, FormatT::Type, 0);"
OpenGL error 1281

function: void upload()
after "glTexSubImage2D(GL_TEXTURE_RECTANGLE, /level/0, /xoffset/0, /yoffset/0, width, height, FormatT::Format, FormatT::Type, data);"
OpenGL error 1282

function: void run (.....)
after "glClear(GL_COLOR_BUFFER_BIT);"
and
" gl()->glBindVertexArray(square_vao);
glDrawArrays(GL_TRIANGLES, 0, 6);"
and
" *ir = stage1_infrared.downloadToNewFrame(); "
and
" one more time glClear(GL_COLOR_BUFFER_BIT);"
and
" gl()->glBindVertexArray(square_vao);
glDrawArrays(GL_TRIANGLES, 0, 6);"
there is OpenGL error 1286

I tried to google it, but didn't find anything usefull. I'm sorry for the message format, I just put error handling messages after every opengl command which kind of messed up line numbering (dont know any smarter way)

[gaborpapp]

These are error codes GL_INVALID_VALUE​ (0x501), GL_INVALID_OPERATION (0x502), and GL_INVALID_FRAMEBUFFER_OPERATION​ (0x506). https://www.opengl.org/wiki/OpenGL_Error

[ThomasLengeling]

Ok I tried #169, and i got the following

toms-MBP:bin tom$ ./test_opengl
[OpenGLDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 0 from './p00.bin' failed!
[OpenGLDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 1 from './p01.bin' failed!
[OpenGLDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 2 from './p02.bin' failed!
[CpuDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 0 from './p00.bin' failed!
[CpuDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 1 from './p01.bin' failed!
[CpuDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 2 from './p02.bin' failed!
libc++abi.dylib: terminating with uncaught exception of type std::exception: std::exception
Abort trap: 6
toms-MBP:bin tom$ ./test_opengl_dump
[OpenGLDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 0 from './p00.bin' failed!
[OpenGLDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 1 from './p01.bin' failed!
[OpenGLDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 2 from './p02.bin' failed!
[CpuDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 0 from './p00.bin' failed!
[CpuDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 1 from './p01.bin' failed!
[CpuDepthPacketProcessor::loadP0TablesFromFiles] Loading p0table 2 from './p02.bin' failed!
libc++abi.dylib: terminating with uncaught exception of type std::exception: std::exception
Abort trap: 6

The I did #171. crashes at before running, I so tried the ./test_opengl I get the same error the above

[gaborpapp]

test_opengl_dump does not initialize an OpenGLDepthPacketProcessor as far as I know. Are you sure the output is correct? First you should run test_opengl_dump then test_opengl.

[ThomasLengeling]

yeah strange, I do test_openg_dump first and then test_opengl, I get the same error massage on both, I don't know if I'm missing something.

[bellekci]

You are compiling the wrong source at #169 for test_opengl_dump.
You should fix the source of test_opengl_dump in CMakeLists.txt to test_opengl_dump.cpp
That is why people are getting the same output for test_opengl and test_opengl_dump

[gaborpapp]

Oh, sorry about this. Fixed in #169. Thanks for reporting it.

[julienaubert]

I had the same issue (black IR and depth), this fixed it. Thanks. (Still getting pink in RGB though). I'm on a macbook pro 13" early 2013 retina, Intel HD Graphics 4000 1024 MB.

[bellekci]

I tried on Ubuntu with Intel Haswell Mobile GPU and it did not help. Still getting black output.

[floe]

Since this fixes the issue for at least some configurations and doesn't negatively affect the implementations where it was already working, I'm merging the PR.

[vedanshu95]

Hi,

I am also facing the same problem in ubuntu 14.04. i have installed kernel 13.16 but still i am facing the same problem. I am unable to see dept and ir image. only rgb image shows up.