Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Texture support for raw Mesh3D logging #4894

Merged
merged 30 commits into from
Feb 2, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
84fd0cb
update fbs files for mesh texture support
Wumpf Jan 23, 2024
85e3991
extension fixup
Wumpf Jan 23, 2024
c401fe7
re_types tests fixup
Wumpf Jan 23, 2024
9742012
codegen
Wumpf Jan 23, 2024
ce87aee
mesh visualizer picks albedo texture now
Wumpf Jan 23, 2024
8c51435
mesh & material python extension update
Wumpf Jan 23, 2024
6e31d15
python raw mesh example now uses new albedo texture support
Wumpf Jan 23, 2024
9aa916b
update rust raw_mesh example
Wumpf Jan 23, 2024
a5fac6a
introduce new texcoord2d component
Wumpf Jan 23, 2024
a76d34f
clarify texture limitation
Wumpf Jan 23, 2024
e53b16f
crude data ui
Wumpf Jan 23, 2024
fb42ed5
spelling exception
Wumpf Jan 23, 2024
d305578
reexport wgpu in external namespace
Wumpf Jan 25, 2024
acf9f66
better texture_key for meshes
Wumpf Jan 25, 2024
bb26200
fbs doc fixes
Wumpf Jan 25, 2024
10d178a
Merge remote-tracking branch 'origin/main' into andreas/log-textured-…
Wumpf Jan 25, 2024
1570e5a
fix missing dependency - did this break on main?
Wumpf Jan 25, 2024
12b9b8d
fix python serialization for no texture present
Wumpf Jan 25, 2024
acd27b7
missing unreleased attribute
Wumpf Jan 25, 2024
fbc140d
missing newline
Wumpf Jan 25, 2024
6b3c097
fix typo in texcoord2d cpp
Wumpf Jan 25, 2024
6693722
fix and update mesh3d python tests
Wumpf Jan 25, 2024
92bf597
Merge remote-tracking branch 'origin/main' into andreas/log-textured-…
Wumpf Jan 30, 2024
9b0107b
improvements to rgba32 rust extensions
Wumpf Feb 1, 2024
e5f1d37
revert material, put albedo texture on the mesh archetype instead
Wumpf Feb 1, 2024
b630dfd
fix c++ build
Wumpf Feb 1, 2024
3ae6938
fix typo that I didn't cause
Wumpf Feb 1, 2024
4229db9
fix too dark mesh - don't convert the trimesh material if you don't h…
Wumpf Feb 1, 2024
d89fed9
fix python raw_mesh example not picking up albedo factor
teh-cmc Feb 2, 2024
cf2f654
review
teh-cmc Feb 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 5 additions & 3 deletions crates/re_renderer/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -79,10 +79,12 @@ pub use self::file_resolver::{
};
pub use self::file_server::FileServer;

// Re-export used color types.
// Re-export used color types directly.
pub use ecolor::{Color32, Hsva, Rgba};
// Re-export wgpu
pub use wgpu;

pub mod external {
pub use wgpu;
}

// ---------------------------------------------------------------------------

Expand Down
2 changes: 1 addition & 1 deletion crates/re_space_view_spatial/src/mesh_cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ use crate::mesh_loader::LoadedMesh;

// ----------------------------------------------------------------------------

#[derive(Debug, PartialEq, Eq, Hash)]
#[derive(Debug, PartialEq, Eq, Hash, Clone)]
pub struct MeshCacheKey {
pub versioned_instance_path_hash: VersionedInstancePathHash,
pub media_type: Option<MediaType>,
Expand Down
2 changes: 1 addition & 1 deletion crates/re_space_view_spatial/src/mesh_loader.rs
Original file line number Diff line number Diff line change
Expand Up @@ -233,7 +233,7 @@ fn mesh_texture_from_tensor_data(
Ok(re_renderer::resource_managers::Texture2DCreationDesc {
label: "mesh albedo texture from tensor data".into(),
data,
format: re_renderer::wgpu::TextureFormat::Rgba8UnormSrgb,
format: re_renderer::external::wgpu::TextureFormat::Rgba8UnormSrgb,
Wumpf marked this conversation as resolved.
Show resolved Hide resolved
width,
height,
})
Expand Down
11 changes: 6 additions & 5 deletions crates/re_space_view_spatial/src/visualizers/meshes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -104,15 +104,16 @@ impl Mesh3DVisualizer {
let outline_mask_ids = ent_context.highlight.index_outline_mask(InstanceKey::SPLAT);

let mesh = ctx.cache.entry(|c: &mut MeshCache| {
let key = MeshCacheKey {
versioned_instance_path_hash: picking_instance_hash.versioned(primary_row_id),
media_type: None,
};
c.entry(
&ent_path.to_string(),
MeshCacheKey {
versioned_instance_path_hash: picking_instance_hash.versioned(primary_row_id),
media_type: None,
},
key.clone(),
AnyMesh::Mesh {
mesh: &mesh,
texture_key: re_log_types::hash::Hash64::hash(primary_row_id).hash64(),
texture_key: re_log_types::hash::Hash64::hash(&key).hash64(),
},
ctx.render_ctx,
)
Expand Down
3 changes: 2 additions & 1 deletion crates/re_space_view_text_log/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@ all-features = true
re_data_store.workspace = true
re_data_ui.workspace = true
re_entity_db.workspace = true
re_log.workspace = true
re_log_types.workspace = true
re_log.workspace = true
re_query_cache.workspace = true
re_renderer.workspace = true
re_tracing.workspace = true
re_types.workspace = true
Expand Down
5 changes: 2 additions & 3 deletions crates/re_types/definitions/rerun/components/texcoord2d.fbs
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,8 @@ namespace rerun.components;
/// A 2D texture UV coordinate.
///
/// Texture coordinates specify a position on a 2D texture.
/// A range from 0-1 in covers the entire texture in the respective dimension.
/// The behavior for values outside of this range depends on the visualization/renderer,
/// but will most commonly repeat the texture.
/// A range from 0-1 covers the entire texture in the respective dimension.
/// Unless configured otherwise, the texture repeats outside of this range.
/// Rerun uses top-left as the origin for UV coordinates.
///
/// 0 U 1
Expand Down
2 changes: 1 addition & 1 deletion crates/re_types/definitions/rerun/datatypes/material.fbs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ table Material (
/// Optional albedo texture.
///
/// Used with `vertex_texcoords` on `Mesh3D`.
/// Currently supports only RGB & RGBA sRGBA textures, ignoring alpha.
/// Currently supports only sRGB(A) textures, ignoring alpha.
/// (meaning that the tensor must have 3 or 4 channels and use the `u8` format)
albedo_texture: rerun.datatypes.TensorData (nullable, order: 200);
}
4 changes: 2 additions & 2 deletions crates/re_types/src/components/material.rs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 5 additions & 6 deletions crates/re_types/src/components/texcoord2d.rs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions crates/re_types/src/datatypes/material.rs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion crates/re_viewer_context/src/gpu_bridge/tensor_to_gpu.rs
Original file line number Diff line number Diff line change
Expand Up @@ -548,7 +548,7 @@ fn pad_and_narrow_and_cast<T: Copy + Pod>(
pub fn texture_height_width_channels(tensor: &TensorData) -> anyhow::Result<[u32; 3]> {
use anyhow::Context as _;

let Some([mut height, width, channel]) = tensor.image_height_width_channels() else {
let Some([mut height, mut width, channel]) = tensor.image_height_width_channels() else {
anyhow::bail!("Tensor with shape {:?} is not an image", tensor.shape);
};
height = match tensor.buffer {
Expand Down
5 changes: 2 additions & 3 deletions docs/content/reference/types/components/texcoord2d.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 2 additions & 3 deletions rerun_cpp/src/rerun/components/texcoord2d.hpp

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion rerun_cpp/src/rerun/datatypes/material.hpp

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 2 additions & 3 deletions rerun_py/rerun_sdk/rerun/components/texcoord2d.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 8 additions & 2 deletions rerun_py/rerun_sdk/rerun/datatypes/material.py

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 5 additions & 3 deletions rerun_py/rerun_sdk/rerun/datatypes/material_ext.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ class MaterialExt:

@staticmethod
def native_to_pa_array_override(data: MaterialArrayLike, data_type: pa.DataType) -> pa.Array:
from . import Material, Rgba32Type
from . import Material, Rgba32Type, TensorDataType

# If it's a sequence of a single Material, grab the first one
if isinstance(data, collections.abc.Sequence):
Expand All @@ -35,9 +35,11 @@ def native_to_pa_array_override(data: MaterialArrayLike, data_type: pa.DataType)
type=Rgba32Type().storage_type,
)
if data.albedo_texture is not None:
albedo_texture = TensorDataExt.native_to_pa_array_override(data.albedo_texture, field_albedo_texture.type)
albedo_texture = TensorDataExt.native_to_pa_array_override(
data.albedo_texture, TensorDataType().storage_type
)
else:
albedo_texture = pa.array([None], type=field_albedo_texture.type)
albedo_texture = pa.nulls(1, type=TensorDataType().storage_type)

return pa.StructArray.from_arrays(
arrays=[albedo_factors, albedo_texture],
Expand Down