New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ShapeFrame and ShapeNode #608

Merged
merged 52 commits into from Mar 18, 2016

Conversation

Projects
None yet
6 participants
@jslee02
Member

jslee02 commented Feb 12, 2016

This is a work-in-progress pull request for implementing ShapeFrame/ShapeNode as discussed in #394.

@jslee02 jslee02 added this to the DART 6.0.0 milestone Feb 12, 2016

/// allow it to render again.
void setHidden(bool _hide=true);
/// Notify that the color (rgba) of this shape has updated
virtual void notifyColorUpdate(const Eigen::Vector4d& color);

This comment has been minimized.

@mxgrey

mxgrey Feb 13, 2016

Member

Do shapes have Color anymore? I thought Shapes were now purely geometric information, and that color information would be in the VisualAddon of the ShapeNode. Or is this just WIP code that's going to be removed later?

@mxgrey

mxgrey Feb 13, 2016

Member

Do shapes have Color anymore? I thought Shapes were now purely geometric information, and that color information would be in the VisualAddon of the ShapeNode. Or is this just WIP code that's going to be removed later?

This comment has been minimized.

@jslee02

jslee02 Feb 13, 2016

Member

I thought so too, but I realized MeshShape has color properties (colors for the vertices) in it. This function gets called when the color of VisualAddon is changed so that MeshShape can handle it. By default, it does nothing for other shapes. This function is a compromised solution, so I'm open to any better way for this case.

@jslee02

jslee02 Feb 13, 2016

Member

I thought so too, but I realized MeshShape has color properties (colors for the vertices) in it. This function gets called when the color of VisualAddon is changed so that MeshShape can handle it. By default, it does nothing for other shapes. This function is a compromised solution, so I'm open to any better way for this case.

This comment has been minimized.

@mxgrey

mxgrey Feb 13, 2016

Member

There has been some discussion about the possibility of MeshShape being changed into something that only holds geometric data. If we do that, then we could have it so that the texture and color data gets automatically put into the VisualAddon as it's parsed with assimp. I think this would be also be nice for moving away from assimp being a core dependency (instead it would only be an io dependency).

That said, I don't know what the timeline should be for implementing something like that. We could probably hold off on it until version 7.1.

@mxgrey

mxgrey Feb 13, 2016

Member

There has been some discussion about the possibility of MeshShape being changed into something that only holds geometric data. If we do that, then we could have it so that the texture and color data gets automatically put into the VisualAddon as it's parsed with assimp. I think this would be also be nice for moving away from assimp being a core dependency (instead it would only be an io dependency).

That said, I don't know what the timeline should be for implementing something like that. We could probably hold off on it until version 7.1.

This comment has been minimized.

@jslee02

jslee02 Feb 13, 2016

Member

Yeah, the change sounds good to me. If I remember correctly, there is ongoing work of Mesh class (for moving away from assimp) by @mkoval. Once it's done we can remove this function. Maybe a note for this here would be good.

@jslee02

jslee02 Feb 13, 2016

Member

Yeah, the change sounds good to me. If I remember correctly, there is ongoing work of Mesh class (for moving away from assimp) by @mkoval. Once it's done we can remove this function. Maybe a note for this here would be good.

This comment has been minimized.

@jslee02

jslee02 Feb 13, 2016

Member

Okay, I didn't remember correctly. 😭 @mklingen was working on it at here. Ref: #453.

@jslee02

jslee02 Feb 13, 2016

Member

Okay, I didn't remember correctly. 😭 @mklingen was working on it at here. Ref: #453.

This comment has been minimized.

@mkoval

mkoval Feb 14, 2016

Collaborator

I very much do not want DART in the business of defining a canonical set of material properties. I am cautiously optimistic about @PyryM's suggestion:

I think materials should just be key/value stores that map string keys to [vec4 | texture | string] values.

It is straightforward to allow the user to store arbitrary vertex properties. We could simply allow the user to store a std::vector<T> with size() equal to the number of vertices. I am less sure about providing an API to store textures. Hopefully @PyryM can clarify a few points:

  • Assimp supports spherical, cylindrical, and uv mapping. Are all three used in practice?
  • I also found some references to uvw-mapping. How does this differ from uv mapping?
  • Are uv-coordinates stored as separate vertex properties or attached to the image asset?
  • How are uv-coordinates associated with the corresponding assert?
  • Are all textures 24-bit RGBA? Is it common to use a different bit depth or number of channels?

My only other concern is that VisualAddon is attached to ShapeFrame, not Shape. These properties only apply to MeshShape and are intimately related to the mesh, e.g. arrays of vertex properties must have the same length as the number of vertices in the mesh.

Alternatively, we could take the simpler route I suggested in #453:

MeshNode should contain the bare minimum amount of information necessary to perform collision checking and a dynamical simulation. Additionally, DART should provide an Uri to the original resource that can be loaded for visualization. The user is responsible for loading any additional information, e.g. textures and material properties, that is necessary only for visualization.

This has two downsides. First, every mesh is loaded from disk twice: once for collision detection and once for visualization. Second, we cannot programmatically construct a textured or vertex-colored mesh. This latter limitation bothers me more than the former - @mklingen and I independently ran into this in our research when rendering a color map.

@mkoval

mkoval Feb 14, 2016

Collaborator

I very much do not want DART in the business of defining a canonical set of material properties. I am cautiously optimistic about @PyryM's suggestion:

I think materials should just be key/value stores that map string keys to [vec4 | texture | string] values.

It is straightforward to allow the user to store arbitrary vertex properties. We could simply allow the user to store a std::vector<T> with size() equal to the number of vertices. I am less sure about providing an API to store textures. Hopefully @PyryM can clarify a few points:

  • Assimp supports spherical, cylindrical, and uv mapping. Are all three used in practice?
  • I also found some references to uvw-mapping. How does this differ from uv mapping?
  • Are uv-coordinates stored as separate vertex properties or attached to the image asset?
  • How are uv-coordinates associated with the corresponding assert?
  • Are all textures 24-bit RGBA? Is it common to use a different bit depth or number of channels?

My only other concern is that VisualAddon is attached to ShapeFrame, not Shape. These properties only apply to MeshShape and are intimately related to the mesh, e.g. arrays of vertex properties must have the same length as the number of vertices in the mesh.

Alternatively, we could take the simpler route I suggested in #453:

MeshNode should contain the bare minimum amount of information necessary to perform collision checking and a dynamical simulation. Additionally, DART should provide an Uri to the original resource that can be loaded for visualization. The user is responsible for loading any additional information, e.g. textures and material properties, that is necessary only for visualization.

This has two downsides. First, every mesh is loaded from disk twice: once for collision detection and once for visualization. Second, we cannot programmatically construct a textured or vertex-colored mesh. This latter limitation bothers me more than the former - @mklingen and I independently ran into this in our research when rendering a color map.

This comment has been minimized.

@psigen

psigen Feb 14, 2016

Collaborator

Regarding the method in #453:

The user is responsible for loading any additional information, e.g. textures and material properties, that is necessary only for visualization.

Consider loading an object from a file URI but then changing the object on disk (I was trying to adjust some objects in a scene while I had it open in rviz). There is zero guarantee that each loading operation will have the same result, especially if assets are lazily-loaded, which can lead to out-of-sync visuals or even more pathological consistency issues.

This might even specifically be intentional in cases where an object is being dynamically updated, such as serving a generated model that results from perception or some other process at an http URI.

We also currently have to run an HTTP proxy server and rewrite URIs to implement dart_rviz, because we have the problem that URIs that are resolvable in the originating DART process aren't necessarily resolvable in the scope of the visualization (because it can be run over a network).

So, while it is very semantically clean, it does have some significant downsides.

@psigen

psigen Feb 14, 2016

Collaborator

Regarding the method in #453:

The user is responsible for loading any additional information, e.g. textures and material properties, that is necessary only for visualization.

Consider loading an object from a file URI but then changing the object on disk (I was trying to adjust some objects in a scene while I had it open in rviz). There is zero guarantee that each loading operation will have the same result, especially if assets are lazily-loaded, which can lead to out-of-sync visuals or even more pathological consistency issues.

This might even specifically be intentional in cases where an object is being dynamically updated, such as serving a generated model that results from perception or some other process at an http URI.

We also currently have to run an HTTP proxy server and rewrite URIs to implement dart_rviz, because we have the problem that URIs that are resolvable in the originating DART process aren't necessarily resolvable in the scope of the visualization (because it can be run over a network).

So, while it is very semantically clean, it does have some significant downsides.

This comment has been minimized.

@PyryM

PyryM Feb 14, 2016

Assimp supports spherical, cylindrical, and uv mapping. Are all three used in practice?

Spherical and cylindrical are ways of automatically assigning uv coordinates. I wouldn't bother trying to directly support them since any 3d editing program can compute these mappings and export them as regular uv coordinates.

Are uv-coordinates stored as separate vertex properties or attached to the image asset?
I also found some references to uvw-mapping. How does this differ from uv mapping?

Texture coordinates are vertex attributes, they're part of the mesh data like vertex positions and normals. Typical texture mapping, to simplify a lot, works like color = my_texture[u,v]. You can have 3d textures (a 3d array compared to a 2d array for a normal texture) in which case color = my_texture[u,v,w]. But the names and meanings of all the vertex attributes are just conventions, and shaders are free to interpret the attributes however they want.

Are all textures 24-bit RGBA? Is it common to use a different bit depth or number of channels?

32 bit RGBA8/BGRA8 is the most common, and probably the only format that's likely to be encountered in this application. It might be worth supporting float textures (R32F or RGBA32F) to make it convenient to visualize procedurally generated data without having to squash or encode it into RGBA8.

@PyryM

PyryM Feb 14, 2016

Assimp supports spherical, cylindrical, and uv mapping. Are all three used in practice?

Spherical and cylindrical are ways of automatically assigning uv coordinates. I wouldn't bother trying to directly support them since any 3d editing program can compute these mappings and export them as regular uv coordinates.

Are uv-coordinates stored as separate vertex properties or attached to the image asset?
I also found some references to uvw-mapping. How does this differ from uv mapping?

Texture coordinates are vertex attributes, they're part of the mesh data like vertex positions and normals. Typical texture mapping, to simplify a lot, works like color = my_texture[u,v]. You can have 3d textures (a 3d array compared to a 2d array for a normal texture) in which case color = my_texture[u,v,w]. But the names and meanings of all the vertex attributes are just conventions, and shaders are free to interpret the attributes however they want.

Are all textures 24-bit RGBA? Is it common to use a different bit depth or number of channels?

32 bit RGBA8/BGRA8 is the most common, and probably the only format that's likely to be encountered in this application. It might be worth supporting float textures (R32F or RGBA32F) to make it convenient to visualize procedurally generated data without having to squash or encode it into RGBA8.

This comment has been minimized.

@mkoval

mkoval Feb 17, 2016

Collaborator

This discussion moved to #612. In the meantime, should we remove this function?

@mkoval

mkoval Feb 17, 2016

Collaborator

This discussion moved to #612. In the meantime, should we remove this function?

This comment has been minimized.

@jslee02

jslee02 Feb 17, 2016

Member

I think we should keep this function as far as MeshShape contains color data and we want to use and update it.

@jslee02

jslee02 Feb 17, 2016

Member

I think we should keep this function as far as MeshShape contains color data and we want to use and update it.

Show outdated Hide outdated dart/dynamics/BodyNode.h
Show outdated Hide outdated dart/dynamics/BodyNode.h
@mxgrey

This comment has been minimized.

Show comment
Hide comment
@mxgrey

mxgrey Mar 9, 2016

Member

I've finished reviewing. All of my comments which are relevant to this pull request have been addressed. I've also gone ahead and made some changes to the API for accessing and creating ShapeNodes. In particular, I've added With to all functions that operate on a subset of ShapeNodes that contain a particular Addon type. I've also removed the AddonAdder struct and replaced it with two void-returning variadic functions which are now located in common/AddonManager.h.

I only have one remaining concern: It seems that soft bodies are very very broken right now. If we're okay with soft bodies being broken on master, then I'm fine with merging in this pull request. I do think we should try to get them at least looking believable before the major release, though.

Member

mxgrey commented Mar 9, 2016

I've finished reviewing. All of my comments which are relevant to this pull request have been addressed. I've also gone ahead and made some changes to the API for accessing and creating ShapeNodes. In particular, I've added With to all functions that operate on a subset of ShapeNodes that contain a particular Addon type. I've also removed the AddonAdder struct and replaced it with two void-returning variadic functions which are now located in common/AddonManager.h.

I only have one remaining concern: It seems that soft bodies are very very broken right now. If we're okay with soft bodies being broken on master, then I'm fine with merging in this pull request. I do think we should try to get them at least looking believable before the major release, though.

@jslee02

This comment has been minimized.

Show comment
Hide comment
@jslee02

jslee02 Mar 10, 2016

Member

Could you take a look at osgDart as well? I made it just work (I believe ) but there are some stopgaps. For example, EntityNode is replaced by ShapeFrameNode but WorldNode is still using EntityNode.

I'm looking at soft bodies now.

Member

jslee02 commented Mar 10, 2016

Could you take a look at osgDart as well? I made it just work (I believe ) but there are some stopgaps. For example, EntityNode is replaced by ShapeFrameNode but WorldNode is still using EntityNode.

I'm looking at soft bodies now.

@mxgrey

This comment has been minimized.

Show comment
Hide comment
@mxgrey

mxgrey Mar 10, 2016

Member

The drag and drop features in osgDart seem to be working fine. Unfortunately, the diff for those files isn't showing up for me on Github. If anyone can recommend a good git diff viewing desktop application, I would appreciate it. Otherwise, I can comb through the raw code some time soon.

Member

mxgrey commented Mar 10, 2016

The drag and drop features in osgDart seem to be working fine. Unfortunately, the diff for those files isn't showing up for me on Github. If anyone can recommend a good git diff viewing desktop application, I would appreciate it. Otherwise, I can comb through the raw code some time soon.

@psigen

This comment has been minimized.

Show comment
Hide comment
@psigen

psigen Mar 10, 2016

Collaborator

On Ubuntu, I generally use meld.

Collaborator

psigen commented Mar 10, 2016

On Ubuntu, I generally use meld.

@mxgrey

This comment has been minimized.

Show comment
Hide comment
@mxgrey

mxgrey Mar 10, 2016

Member

I use meld for merging, but I don't like looking back and forth from side to side when it comes to comparing two files. I really like the linear way that Github lays it out. The standard console tool lays it out similarly, but isn't very pretty. I probably won't be able to find what I want, so I'll just have to settle for meld or the console. It would be nice if Github offered an offline version of their viewer.

Member

mxgrey commented Mar 10, 2016

I use meld for merging, but I don't like looking back and forth from side to side when it comes to comparing two files. I really like the linear way that Github lays it out. The standard console tool lays it out similarly, but isn't very pretty. I probably won't be able to find what I want, so I'll just have to settle for meld or the console. It would be nice if Github offered an offline version of their viewer.

jslee02 added some commits Mar 10, 2016

Change BodyNodePtr::set() to unconditionally set mPtr to _ptr and inc…
…rement/decrement reference count on its skeleton. This is necessary to store a valid WeakShapeNodePtr to SoftBodyNode during the SoftBodyNode is being created.
@jslee02

This comment has been minimized.

Show comment
Hide comment
@jslee02

jslee02 Mar 11, 2016

Member

It would be nice if Github offered an offline version of their viewer.
👍

I created two dummy branches (with and without osgDart changes) to help reviewing osgDart here. It doesn't include commit-wise changes of the original history, but you can see the final diff.

Also, I resolved the instability of the soft bodies in the last commits.

Member

jslee02 commented Mar 11, 2016

It would be nice if Github offered an offline version of their viewer.
👍

I created two dummy branches (with and without osgDart changes) to help reviewing osgDart here. It doesn't include commit-wise changes of the original history, but you can see the final diff.

Also, I resolved the instability of the soft bodies in the last commits.

@psigen

This comment has been minimized.

Show comment
Hide comment
@psigen

psigen Mar 12, 2016

Collaborator

@mxgrey you can possibly try the vertical diff mode in vimdiff?
http://stackoverflow.com/a/244766

I actually find it hard to use the unified vertical diff, because it makes it really tricky to compare large block diffs, but I think several difftools have flags to go into that mode.

Collaborator

psigen commented Mar 12, 2016

@mxgrey you can possibly try the vertical diff mode in vimdiff?
http://stackoverflow.com/a/244766

I actually find it hard to use the unified vertical diff, because it makes it really tricky to compare large block diffs, but I think several difftools have flags to go into that mode.

@mxgrey

This comment has been minimized.

Show comment
Hide comment
@mxgrey

mxgrey Mar 14, 2016

Member

I know this originally used to be an Entity*, but is there any reason not to use a ShapeFrame* now that only ShapeFrames are able to contain shapes? I feel that using ShapeFrame* now would be an overall enhancement without any downside.

Member

mxgrey commented on osgDart/DefaultEventHandler.h in 4c99fb8 Mar 14, 2016

I know this originally used to be an Entity*, but is there any reason not to use a ShapeFrame* now that only ShapeFrames are able to contain shapes? I feel that using ShapeFrame* now would be an overall enhancement without any downside.

This comment has been minimized.

Show comment
Hide comment
@jslee02

jslee02 Mar 14, 2016

Member

If I understand correctly, ownerEntity is for a pointer to the target object of drag-and-drop. If we restrict this variable to ShapeFrame* then we wouldn't be able to have drag-and-drop class whose target object doesn't inherit ShapeFrame. However, we have DragAndDrop whose target object is BodyNode which doesn't inherit ShapeFrame.

Member

jslee02 replied Mar 14, 2016

If I understand correctly, ownerEntity is for a pointer to the target object of drag-and-drop. If we restrict this variable to ShapeFrame* then we wouldn't be able to have drag-and-drop class whose target object doesn't inherit ShapeFrame. However, we have DragAndDrop whose target object is BodyNode which doesn't inherit ShapeFrame.

This comment has been minimized.

Show comment
Hide comment
@mxgrey

mxgrey Mar 14, 2016

Member

I'm pretty sure it's a pointer to the Entity that has been clicked on, which (from now on) should always be a ShapeFrame, because only ShapeFrames get rendered now. At least that's what I had intended the PickInfo to mean. I can take a closer look at how the PickInfo is getting created to check whether or not this is the case.

Member

mxgrey replied Mar 14, 2016

I'm pretty sure it's a pointer to the Entity that has been clicked on, which (from now on) should always be a ShapeFrame, because only ShapeFrames get rendered now. At least that's what I had intended the PickInfo to mean. I can take a closer look at how the PickInfo is getting created to check whether or not this is the case.

This comment has been minimized.

Show comment
Hide comment
@jslee02

jslee02 Mar 14, 2016

Member

I assumed that DragAndDrop::mEntity is changed to ShapeFrame*. Changing the type of ownerEntity to ShapeFrame* doesn't have any technical problem if we're fine with comparing Entity* and ShapeFrame* for picking.

Member

jslee02 replied Mar 14, 2016

I assumed that DragAndDrop::mEntity is changed to ShapeFrame*. Changing the type of ownerEntity to ShapeFrame* doesn't have any technical problem if we're fine with comparing Entity* and ShapeFrame* for picking.

mxgrey added some commits Mar 14, 2016

@mxgrey

This comment has been minimized.

Show comment
Hide comment
@mxgrey

mxgrey Mar 15, 2016

Member

I've pushed some fixes to the way Addons are cloned, and I've restructured osgDart. Soft bodies seem to be working again after @jslee02 patched it.

I think this pull request is good to merge now 👍

Member

mxgrey commented Mar 15, 2016

I've pushed some fixes to the way Addons are cloned, and I've restructured osgDart. Soft bodies seem to be working again after @jslee02 patched it.

I think this pull request is good to merge now 👍

@jslee02

This comment has been minimized.

Show comment
Hide comment
@jslee02

jslee02 Mar 15, 2016

Member

👍 looks good to me too. Will merge this once #607 is merged.

Member

jslee02 commented Mar 15, 2016

👍 looks good to me too. Will merge this once #607 is merged.

jslee02 added some commits Mar 18, 2016

Merge remote-tracking branch 'origin/master' into shapenode
Conflicts:
	dart/dynamics/BodyNode.h
	dart/dynamics/EndEffector.h
	dart/dynamics/Skeleton.h
	dart/dynamics/detail/SpecializedNodeManager.h

jslee02 added a commit that referenced this pull request Mar 18, 2016

Merge pull request #608 from dartsim/shapenode
[WIP] ShapeFrame and ShapeNode

@jslee02 jslee02 merged commit 26db3c7 into master Mar 18, 2016

2 of 4 checks passed

continuous-integration/appveyor/branch AppVeyor build failed
Details
continuous-integration/appveyor/pr Waiting for AppVeyor build to complete
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@jslee02 jslee02 deleted the shapenode branch Mar 23, 2016

@jslee02 jslee02 changed the title from [WIP] ShapeFrame and ShapeNode to ShapeFrame and ShapeNode Mar 24, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment