Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GGUI depth information and color information or imwrite broken #8463

Open
sarahhaha opened this issue Jan 24, 2024 · 0 comments
Open

GGUI depth information and color information or imwrite broken #8463

sarahhaha opened this issue Jan 24, 2024 · 0 comments

Comments

@sarahhaha
Copy link

Describe the bug
The GGUI depth information and color information give inaccurate information to the actual scene rendering

To Reproduce

if mode == 'rgbd':
            frame = 'first_last' #'all' #'first_last'
            foodname = 'spaghetti1'
            filename = f'frame_{self.i:05d}.png'
            start_frame = 100
            last_frame = 270

            if frame == 'all' or (frame == 'first_last' and (self.i == start_frame or self.i == last_frame)):
                depth = self.window.get_depth_buffer_as_numpy()
                depth = depth * depth * depth * 100 * 100
                # Save the depth image
                ti.tools.imwrite(depth, f'/home/empriselab/FluidLab/fluidlab/data/depth/{foodname}_{filename}')
                
                img = (self.window.get_image_buffer_as_numpy())[:, :, :3]
                img = (img * 255).astype(np.uint8)
                # Save the rgb image
                ti.tools.imwrite(img, f'/home/empriselab/FluidLab/fluidlab/data/rgb/{foodname}_{filename}')
                
                #sleep(0.1)
                self.window.destroy()
                self.init_window(mode)
                self.window.show()

            return None

Log/Screenshots
These are images produced by get_depth_buffer_as_numpy(), get_image_buffer_as_numpy() and ti.tools.imwrite()
spaghetti1_frame_00270

spaghetti1_frame_00270

Meanwhile, this is the actual GGUI scene video.

fast6-2023-12-22_00.37.38-2024-01-23_22.30.04.mp4

The ply export from GGUI works well. This is the ply exported from this scene and imported into blender.

fast6-2023-12-22_00.37.38-2024-01-23_22.32.06.mp4

Full log

$python fluidlab/run.py --cfg_file configs/exp_spaghetti.yaml --record
[Taichi] version 1.5.0, llvm 15.0.4, commit 7b885c28, linux, python 3.7.13
[Taichi] Starting on arch=cuda
===>  TaichiEnv created.
===>      501 particles of spaghetti     mesh added.
===>      501 particles of spaghetti     mesh added.
===>      501 particles of spaghetti     mesh added.
===>      501 particles of spaghetti     mesh added.
===>      501 particles of spaghetti     mesh added.
===>      501 particles of spaghetti     mesh added.
===>      501 particles of spaghetti     mesh added.
===>      501 particles of spaghetti     mesh added.
===>      501 particles of spaghetti     mesh added.
===>  SpaghettiEnv built successfully.
/home/empriselab/miniconda3/envs/fluidlab/lib/python3.7/site-packages/taichi/lang/struct.py:247: DeprecationWarning: fill() on ti.Struct is deprecated, and it will be removed in Taichi v1.6.0.
  DeprecationWarning)
/home/empriselab/FluidLab/fluidlab/fluidengine/simulators/mpm_simulator.py:371: DeprecationWarning: Calling builtin function "max" in Taichi scope is deprecated, and it will be removed in Taichi v1.6.0.Please use "ti.max" instead.
  S_new[d, d] = min(max(self.particles[f, p].S[d, d], 1 - 2e-3), 1 + 3e-3)
/home/empriselab/FluidLab/fluidlab/fluidengine/simulators/mpm_simulator.py:371: DeprecationWarning: Calling builtin function "min" in Taichi scope is deprecated, and it will be removed in Taichi v1.6.0.Please use "ti.min" instead.
  S_new[d, d] = min(max(self.particles[f, p].S[d, d], 1 - 2e-3), 1 + 3e-3)
/home/empriselab/FluidLab/fluidlab/fluidengine/simulators/mpm_simulator.py:394: DeprecationWarning: Operator "is not" in Taichi scope is deprecated, and it will be removed in Taichi v1.6.0. Please avoid using it.
  if ti.static(self.agent is not None):
===> New target generated and dumped to /home/empriselab/FluidLab/fluidlab/assets/targets/Spaghetti-v0.pkl.

Additional comments
ti diagnose
[Taichi] version 1.5.0, llvm 15.0.4, commit 7b885c2, linux, python 3.7.13


** Taichi Programming Language **


Docs: https://docs.taichi-lang.org/
GitHub: https://github.com/taichi-dev/taichi/
Forum: https://forum.taichi.graphics/

Taichi system diagnose:

python: 3.7.13 (default, Mar 29 2022, 02:18:16)
[GCC 7.5.0]
system: linux
executable: /home/empriselab/miniconda3/envs/fluidlab/bin/python
platform: Linux-5.15.0-91-generic-x86_64-with-debian-bullseye-sid
architecture: 64bit
uname: uname_result(system='Linux', node='Taughannock', release='5.15.0-91-generic', version='#101~20.04.1-Ubuntu SMP Thu Nov 16 14:22:28 UTC 2023', machine='x86_64', processor='x86_64')
locale: en_US.UTF-8
PATH: /home/empriselab/miniconda3/envs/fluidlab/bin:/opt/ros/noetic/bin:/home/empriselab/miniconda3/condabin:/home/empriselab/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/empriselab/bin
PYTHONPATH: ['/home/empriselab/miniconda3/envs/fluidlab/bin', '/home/empriselab/Workspace/premanip_robot/premanip_ws/devel/lib/python3/dist-packages', '/opt/ros/noetic/lib/python3/dist-packages', '/home/empriselab/miniconda3/envs/fluidlab/lib/python37.zip', '/home/empriselab/miniconda3/envs/fluidlab/lib/python3.7', '/home/empriselab/miniconda3/envs/fluidlab/lib/python3.7/lib-dynload', '/home/empriselab/miniconda3/envs/fluidlab/lib/python3.7/site-packages', '/home/empriselab/FluidLab', '/home/empriselab/premanip_ws/src/deoxys_control/deoxys']

No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal

import: <module 'taichi' from '/home/empriselab/miniconda3/envs/fluidlab/lib/python3.7/site-packages/taichi/init.py'>

cc: False
cpu: True
metal: False
opengl: True
cuda: True
vulkan: True

glewinfo not available: [Errno 2] No such file or directory: 'glewinfo': 'glewinfo'

Tue Jan 23 22:38:50 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 545.23.08 Driver Version: 545.23.08 CUDA Version: 12.3 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 NVIDIA GeForce RTX 3060 On | 00000000:01:00.0 On | N/A |
| 0% 50C P8 15W / 170W | 1435MiB / 12288MiB | 5% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 1269 G /usr/lib/xorg/Xorg 53MiB |
| 0 N/A N/A 2667 G /usr/lib/xorg/Xorg 518MiB |
| 0 N/A N/A 451715 G /usr/bin/gnome-shell 64MiB |
| 0 N/A N/A 487169 G /usr/lib/xorg/Xorg 85MiB |
| 0 N/A N/A 796283 G ...seed-version=20240119-130050.604000 33MiB |
| 0 N/A N/A 887282 G ...sion,SpareRendererForSitePerProcess 110MiB |
| 0 N/A N/A 1080348 G /usr/lib/firefox/firefox 150MiB |
| 0 N/A N/A 1095402 G /usr/lib/thunderbird/thunderbird 7MiB |
| 0 N/A N/A 1106955 C+G ...ads/blender-3.6.8-linux-x64/blender 187MiB |
+---------------------------------------------------------------------------------------+

[Taichi] version 1.5.0, llvm 15.0.4, commit 7b885c2, linux, python 3.7.13

[Taichi] version 1.5.0, llvm 15.0.4, commit 7b885c2, linux, python 3.7.13
[Taichi] Starting on arch=x64

[Taichi] version 1.5.0, llvm 15.0.4, commit 7b885c2, linux, python 3.7.13
[Taichi] Starting on arch=opengl

[Taichi] version 1.5.0, llvm 15.0.4, commit 7b885c2, linux, python 3.7.13
[Taichi] Starting on arch=cuda

[Taichi] version 1.5.0, llvm 15.0.4, commit 7b885c2, linux, python 3.7.13


** Taichi Programming Language **


Docs: https://docs.taichi-lang.org/
GitHub: https://github.com/taichi-dev/taichi/
Forum: https://forum.taichi.graphics/

                               TAICHI EXAMPLES                                    

────────────────────────────────────────────────────────────────────────────────────
0: ad_gravity 25: laplace 50: physarum
1: circle_packing_image 26: laplace_equation 51: poisson_disk_sampling
2: comet 27: mandelbrot_zoom 52: print_offset
3: cornell_box 28: marching_squares 53: rasterizer
4: diff_sph 29: mass_spring_3d_ggui 54: regression
5: euler 30: mass_spring_game 55: sdf_renderer
6: explicit_activation 31: mass_spring_game_ggui 56: simple_derivative
7: export_mesh 32: mciso_advanced 57: simple_texture
8: export_ply 33: mgpcg 58: simple_uv
9: export_videos 34: mgpcg_advanced 59: snow_phaseField
10: fem128 35: minimal 60: stable_fluid
11: fem128_ggui 36: minimization 61: stable_fluid_ggui
12: fem99 37: mpm128 62: stable_fluid_graph
13: fractal 38: mpm128_ggui 63: taichi_bitmasked
14: fractal3d_ggui 39: mpm3d 64: taichi_dynamic
15: fullscreen 40: mpm3d_ggui 65: taichi_logo
16: game_of_life 41: mpm88 66: taichi_ngp
17: gui_image_io 42: mpm88_graph 67: taichi_sparse
18: gui_widgets 43: mpm99 68: texture_graph
19: implicit_fem 44: mpm_lagrangian_forces 69: tutorial
20: implicit_mass_spring 45: nbody 70: two_stream_instability
21: initial_value_problem 46: odop_solar 71: vortex_rings
22: jacobian 47: oit_renderer 72: waterwave
23: karman_vortex_street 48: patterns
24: keyboard 49: pbf2d
────────────────────────────────────────────────────────────────────────────────────
42
Running example minimal ...
[Taichi] Starting on arch=x64
42.0

Running time: 0.24s

Consider attaching this log when maintainers ask about system information.

Running time: 9.73s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Untriaged
Development

No branches or pull requests

1 participant