Showing RGBD images using graph_objects.Surface?

[kmader]

Not an issue exactly, but I am trying to show RGBD images (with Depth) using surface and the 'hack' I found to do it now was to convert the image into 8-bit GIF, use the 8-bit index as the value, color the image by value and use the gif 8-bit palette as the colorscale. Is there an easier way, or should I make a tutorial in case someone else wants to do the same thing?

from PIL import Image
img_as_8bit = lambda x: np.array(Image.fromarray(x).convert('P', palette='WEB', dither=None))
dum_img = Image.fromarray(np.ones((3,3,3), dtype='uint8')).convert('P', palette='WEB')
idx_to_color = np.array(dum_img.getpalette()).reshape((-1, 3))
colorscale=[[i/255.0, "rgb({}, {}, {})".format(*rgb)] for i, rgb in enumerate(idx_to_color)]
trace=go.Surface(
           z=depth_image,
        surfacecolor=img_as_8bit(rgb_image),
        cmin=0, 
        cmax=255,
        colorscale=colorscale
          )
fig = go.Figure(
    data=[trace],
    layout_title_text="3D Face Mesh"
)
fig

[nicolaskruchten]

This is pretty amazing, nicely done! We are, in fact, working on a way to make this much, much easier via a new image trace type. Here's the prototype in Javascript-land, and i'll be available in Python in 4ish weeks: https://codepen.io/antoinerg/pen/OJJPxJK

[nicolaskruchten]

Well, let me reel in my enthusiasm: the image trace type is 2-d not surface ;) Still, this is pretty cool, and we can maybe imagine generalizing surface like this someday.

[kmader]

Thanks, I'll get a public domain RGBD image and try to make a quick notebook out of it and make a PR to one of the examples

[kmader]

Here is a self-contained notebook using public data
https://www.kaggle.com/kmader/show-rgb-d-image-in-3d-plotly

[bsridatta]

Well, let me reel in my enthusiasm: the image trace type is 2-d not surface ;) Still, this is pretty cool, and we can maybe imagine generalizing surface like this someday.

@nicolaskruchten So can we have 2D RGB in a 3D plot say, alongside a scatter plot like in the picture below? I couldn't find a way to do so, I have tried the approach by kmader but looking for a better res image. Thanks

[valerie-vallet]

@kmader I am precisely interested by your notebook https://www.kaggle.com/kmader/show-rgb-d-image-in-3d-plotly.
I could opening it yesterday, to copy and paste stuff and run a test, without success. Today it is not accessible. Could you please check the website, and check whether your example still works with the latest plotly library?
Thanks for your help

[kmader]

it seems to still work for me, but it is a very heavy site, so maybe try chrome? or you can go down to output and save the .html to load locally

[valerie-vallet]

@kmader OK the website is back! I still cannot make your script work. Could you post the image you used in the test, so that I can try to reproduce exactly your script ?
Thanks a lot.

[kmader]

@valerie-vallet I fixed the script to use kaleido instead of orca so it should be easier to get working. The notebook is downloadable from kaggle as is the dataset used to run it although it should really work with any (not too high) resolution images

[valerie-vallet]

@kmader Thanks Kevin for updating the script. It works!