color(), material() and part() #1608

Open
doug-moen opened this Issue Mar 25, 2016 · 24 comments

Projects

None yet

4 participants

@doug-moen
Contributor

This feature request generalizes our export mechanism so that we can export models with:

  • multiple named parts (which are allowed to overlap). For AMF, this means multiple <object> elements. For DXF, and perhaps also SVG with Inkscape extensions, this means multiple layers. For STL and OFF, this means multiple output files are created by a single export. [Marius: multi-body ASCII STL appears to be reasonably well supported by external software. We could support this as a user-selectable export option.]
  • multiple colors and materials. The goal is to support 3D printing with multiple extruders, so colors are volumetric, rather than applied to the surface of a mesh. A single 3D part may have multiple non-overlapping volumes, each volume having a different color or material. A single 2D part may have multiple non-overlapping areas, each area having a different color. For AMF, this means each <object>/part has multiple <volume> elements. For DXF and SVG, each layer has multiple non-overlapping shapes, each of which can have a different solid color. For STL and OFF, this means multiple output files are created by a single export, one for each color/material.

The reason for bundling these three features together into one FR is to show how they interact with each another, and with the CSG operators. I also want to make the point that these 3 features are not special cases of some more general "metadata" feature. Different kinds of "metadata" interact with the CSG operators in different ways, depending on the role of that "metadata" within the system. For example, OpenSCAD2 has an "object" data type, which can be interpreted as a way to attach "metadata" to a shape. But the purpose and use of that metadata is different, and CSG operators must ignore OpenSCAD2 object attributes, while they must pay attention to the color(), material() and part() attributes. The CSG operators interact with the part() attribute differently than they interact with the color() and material() attributes.

part("name")shape associates the same part name with all of the components of shape, overriding any previously specified part names. If no part name is specified, the part name defaults to "". So square(10) is equivalent to part("")square(10).

The concept of a 2D or 3D shape is generalized so that each shape has zero or more distinctly named parts, which may overlap. (As a special case, the empty OpenSCAD script produces the empty shape, which has no parts.) The CSG operations are generalized so that each distinct named part contained within the children list is operated on separately, and thus retains its identity.

For example,

union {
   part("p1") cube(10);
   part("p1") sphere(5);
   part("p2") cylinder(h=15,r=2);
}

This produces a new shape containing two parts named "p1" and "p2". The cube and sphere are unioned together into a single composite shape, since they belong to the same part. However, the cylinder is not unioned with anything, it retains its identity since it belongs to a separate part.

As a future extension, we'll be able to import an AMF file containing multiple <object> elements, and preserve the identity of each <object> as a separate part. Likewise for importing a DXF file with multiple layers.

A part name is zero or more characters chosen from the set [a-zA-Z0-9_]. That's intended to be a lowest common denominator design that will be compatible with all export formats.

Part names are included in the CSG tree: they must be, since they change the semantics of CSG operations. Part names should appear in the *.csg output.

The color() operator already exists, but now it is extended so that it affects the F6 rendering semantics. Shapes are generalized so that each part contains one or more disjoint regions, each region having a different colour. The CSG operators are generalized so that, when an operator is passed two shape arguments within the same part that have different colours, then the result is computed as some sort of composite, usually containing multiple regions with different colours. But these semantics vary from one operation to the next.

Colours are volumetric, and are considered to apply to each geometric point within a region, both at the surface and within the interior. If no colour is specified, then it defaults to undef. For example, cube(10) is equivalent to color(undef)cube(10). You can remove all colours from a shape using color(undef)shape. This distinction matters for export: if a region has no colour, then no colour is specified for that region in the exported file.

For example, the union operator is extended so that the order of the shape arguments matters. If two shape arguments overlap, and belong to the same part, but have different colours, then the colour of the shape closest to the beginning of the argument list takes precedence for the colour of the overlapping region in the final result.

The material() attribute is far less important than part() or color(). I'm only suggesting it because the AMF file format allows you to specify a colour, a material, or both, for each volume in an object. The material attribute doesn't provide any benefit for any other currently supported export file format. We could leave it out without losing much. Different colours can be used as a surrogate for different materials. I'll update this paragraph later based on the discussion.

@kintel
Member
kintel commented Mar 25, 2016

Thanks for the write-up doug - I'm interested in moving this forward and this is a good first step.
A few comments:

  • multi-body STL (ASCII only) appear to be reasonably well supported by external software. We should consider doing the same, as a user-selectable export option. Similarly for the future extension of preserving part names on import.
  • Have you considered alternative management of hierarchical naming? e.g.
MyModule() {
  part("p1") cube();
  part("p2") sphere();
}
part("ObjectA") translate([-10,0,0]) MyModule();
part("ObjectB") translate([10,0,0]) MyModule();
  • This has some similarities to marker nodes (#952). Should we extend the .csg file format to include part names?
@doug-moen
Contributor

@kintel:

Part names are included in the CSG tree: they must be, since they change the semantics of CSG operations. Part names should appear in the *.csg output.

However, marker nodes (#952) are different from part names. The key is that marker nodes have no effect on CSG semantics. Marker nodes satisfy the same use case as OpenSCAD2 object attributes. They permit model- and application-specific annotations to be added to the node of a model: annotations which have no meaning to built-in operations. For example, a wheel component could have metadata giving the wheel radius, the width, the rim thickness, the number of spokes.

Hierarchical part names sound cool, but it's an extra complication that needs to be justified by use cases. It's something that could be added later: I've rigidly restricted the part namespace; a hierarchy separator character could be added later as an extension. Also, we'd need to look at each import and export format and see how hierarchical part names map in each case.

@t-paul
Member
t-paul commented Mar 26, 2016

Why does the actual use of the meta data need to influence the syntax? All meta data will need to be in the CSG tree anyway. Some would be only exported like the markers or BOM information and other can be handled either by the core (color), or the GUI (parameters, ...). In all other cases we try to move into the direction of unifying things, it seems strange not to do that for the meta data.

@doug-moen
Contributor

@t-paul: I don't really understand your comment "Why does the actual use of the meta data need to influence the syntax". What syntax? And why do you think that my proposal fails to move in the direction of unifying things? After all, I've unified #1041 and #1044 into a single more general concept of "part".

@t-paul
Member
t-paul commented Mar 27, 2016

Using the part() syntax is only possible for module instantiations, not for variable, function or modules definitions. At least I don't see how that would be possible.

@doug-moen
Contributor

I don't agree that the proposed color() and part() operators have anything to do with metadata. [Maybe you could claim that the old color() operator is metadata, since it has no effect on the model, only on the preview. But the new color() operator proposed here isn't metadata.]

The AMF file format has support for metadata via a <metadata> element. You can specify a textual description, author, copyright information, etc. This is clearly metadata, since it is information about the data, and it has no semantic effect on the model. It has no effect on what actually gets manufactured.

Similarly, if we can tag a module definition with a human readable explanation of what the module does, that is also metadata, since it has no effect on the operation of the module.

But in this proposal, color() and part() convey semantic information that is part of the model. They directly affect what gets manufactured. They aren't metadata. A corollary is that they do need to be included in the *.csg file.

@doug-moen
Contributor

@t-paul: The part() operator is a module; it takes a shape as an argument, and returns another shape. Ditto for color(). Semantically, they have the same domain and range as other modules like translate().

@doug-moen
Contributor

t-paul: Are you suggesting that the part() operator should be applicable to variable, function or modules definitions? If so, this makes no sense to me. In my mind, part() is a geometric operation. It sounds like you may have a counterproposal that is radically different from mine.

@kintel
Member
kintel commented Mar 27, 2016

Some thoughts on the confusion of model attributes vs. metadata. I'm not sure if this really addresses the confusion/comments, but it's an attempt:

Essentially, part, color and material are all used to attach some sort of attribute to a node in the CSG tree. This proposal suggests some semantics for how this affects the underlying geometry, possibly hierarchically.

Now, we could view part, color and material as pure attributes, and also allow other metadata attributes. We then define that some attribute names have built-in behavior which affects the design in various ways. As a side note: One could argue that all transformation nodes are really just attributes as well. This could be useful in the future if we want to associate transformations with objects in a GUI.

This leads me to two issues:

  • How do we attach such other metadata attributes to CSG tree nodes?
  • How do we attach attributes to non-CSG tree nodes (e.g. modules, functions, variables)?

I think @t-paul's comments relate to these two issues.

..so the question to @doug-moen would be: Can we (later on) solve those two issues in a clean way, allowing us to move forward without doing it all at the same time?

@doug-moen
Contributor

@kintel: The proposed API for the part(), color() and material() modules is a pure functional interface that is completely abstract. This means the interface is independent of the underlying implementation. The underlying implementation could change in the future, and we wouldn't be forced to change the API or cause old code to stop working.

As soon as we extend OpenSCAD to allow arbitrary attributes (name/value pairs) to be attached to any entity, then the problem of data abstraction arises. There is now a distinction between stored and computed attributes. "Stored" attributes are stored directly in the data structure as name/value pairs, while "computed" attributes are computed from the values of other attributes when they are requested. We could decide to expose the underlying implementation, by requiring the user to use different syntax for stored vs computed attributes. Or we could hide the underlying implementation by providing an abstract, pure functional interface for accessing the attributes.

@kintel
Member
kintel commented Mar 28, 2016

@doug-moen I don't quite follow you on what a "computed attribute" is..

@t-paul
Member
t-paul commented Mar 28, 2016

Yes, I'm not talking about the behavior, e.g. how color will be handled by the CSG operations.

In my view the color is a property of a specific node / mesh, and has no node identity in itself. Using part() and the already existing color() it create a new node and applies it's properties to the child nodes (that's what's visible to the user writing the code, yes we could hide this internally, but that's not my point). I'd prefer to see in the code and enforced by syntax that the information is only attached to a node, just like the '#' modifier right now.

The second point is currently coming from the Customizer topic which wants to annotate variables. I don't know if it's possible to handle param(desc = "box width", values = [1:6]) size = 3;. Also having no way to directly distinguish meta data from other things makes it more complicated to read and understand the code.

Also I think all meta data must go into the CSG file, that's the whole points of most of the meta data attributes (e.g. for the marker nodes). We will handle some of it internally, e.g. the GUI would handle the export related values. Basically every processor will just ignore all the attributes it does not know.

@kintel
Member
kintel commented Mar 28, 2016

@t-paul Just a small comment on modifier characters: My current view is that this is syntactical sugar for setting a modifier attribute on a node. We could offer a full syntax for the same if we offer a general purpose attribute module.

@kintel
Member
kintel commented Mar 28, 2016

Since customization UI is a much larger topic, I'd like to explore if we could decouple this proposal from the customization parameter annotations and let these concepts peacefully co-exist. In the future, they could potentially use the same underlying mechanism, or even use the same language front-end.

I think moving all these features forward in sync is a bit utopic with the current resource situation. ..so if we can see a clean break I think we should go for it.

@t-paul
Member
t-paul commented Mar 28, 2016

Yes, the modifier # could be a short-cut, but what I mean is, it's attached to the node, which is why it's not possible to write # { cube(); sphere(); } it needs a node to attach to, e.g. union and becomes an attribute of that node.

@doug-moen
Contributor

@t-paul wrote "Yes, the modifier # could be a short-cut, but what I mean is, it's attached to the node, which is why it's not possible to write # { cube(); sphere(); } it needs a node to attach to, e.g. union and becomes an attribute of that node."

See, this is an example of something that I consider a bug in OpenSCAD. As a user, I don't care about what a "node" is in the underlying C++ implementation. I just want things to work. The fact that this doesn't work because "it's not a node" is not something I want to hear. This is something that I had proposed to fix in the OpenSCAD2 design document.

Similarly, I don't want to be forced to care about how colours and parts are implemented internally.

I prefer the language to be simple, general, high level, abstract, and not shove implementation details into my face when it's not necessary.

@kintel
Member
kintel commented Mar 28, 2016

@t-paul ah, ok, now I understand what you meant an hour ago.

@doug-moen
Contributor

@t-paul wrote "Yes, the modifier # could be a short-cut, but what I mean is, it's attached to the node, which is why it's not possible to write # { cube(); sphere(); } it needs a node to attach to, e.g. union and becomes an attribute of that node."

Right now, I can write

color("red") {square(10);circle(5);}

and it works.

From your comments about color(), "I'd prefer to see in the code and enforced by syntax that the information is only attached to a node, just like the '#' modifier right now.", it seems you would like to introduce a new syntax for color in which the above code will no longer work.

@kintel
Member
kintel commented Mar 28, 2016

I tend to agree with @doug-moen. Just because part doesn't create a traditional CSG node doesn't mean that it behaves differently than a CSG node from a user's perspective. The only place a user gets a whiff of what a "node" is, is when we save to a .csg file.

You can think of it as creating a "part node" and add the underlying nodes as children (this might very well also be the cleanest way of implementing it in the current codebase).

This is an implementation detail, until we write to file. When writing to file we need to transform our "node" concept into something compatible with each file format.

@t-paul
Member
t-paul commented Mar 28, 2016

Well, I guess we will just not agree on what's easier to read. That's not a big deal.

Back to the main question "how do I do that for module definitions and variables"? Having an annotation style syntax like a number of newer languages provide seems both useful and works for all cases I see right now.

@doug-moen
Contributor

The original implementation of color() did not support an alpha channel, and did not support symbolic color names. So the original interface was color([r,g,b]). When we upgraded the language to support those things, we extended color() so that you can write color("red"), color("red",0.5) and color([r,g,b,a]). The original interface still works for backward compatibility.

If we stop using an abstract, functional interface for specifying colour, and use "attribute" syntax instead, then does it become impossible to enhance the interface in a backward compatible way, as we did for the color module?

@kintel
Member
kintel commented Mar 28, 2016

@doug-moen I my mind, attribute specification is still strongly rooted in the current syntax, making these two equivalent (note: attribute syntax popped off the top of my head for the sake of the argument):

   color([r,g,b]) someShape();
   attribute('color', [r,g,b]) someShape();

-> does that make sense?

@t-paul:

When we have a fully featured attribute syntax, including the proposed way of attaching any user-defined object dict ("POJO") to a shape, I think we would be close to satisfying @t-paul's request for a Customizer UI syntax. To make that complete, we need a way of attaching attributes to non-shape objects, which shouldn't be a problem as these attributes are really only currying the underlying value. We might, in addition, need some syntactical sugar for making this all more readable (e.g. the proposed javadoc-style syntax).

See a few messages above: If we can agree that this feature is orthogonal (for now) to the Customizer UI stuff, that would make it a lot easier to push this forward.

@bobc
bobc commented Apr 11, 2016

FWIW, I am entirely behind @doug-moen 's proposal. In my experimental Carve-CSG fork I implemented the same semantics, and I believe is the only way it can work.

It is important to realise that once you go to a volumetric representation, the color/material is as fundamental part of the node as the geometry. The two cannot be separated, therefore it is not possible to process it as an meta-data attribute, it is the data! Unless you also regard the geometry as just another attribute.

It is easy to get lost in semantics, but unless you can get over the mental hurdle of what volumetric semantics implies then no progress to implementation can be made.

One slight quibble, I think that AMF specifies non-overlapping volumes. In practice, this may not be too much of a problem, because slicers already handle overlapping STL files by using a priority scheme. So unless they are very strict, they could/are likely to apply priority scheme to AMF objects as well.

Otherwise, I'm fully with it.

@doug-moen
Contributor

Thanks, Bob, I based the color() proposal on your work. Maybe your fork can
be used to create the color() implementation.

In AMF, there is a two level hierarchy. There are multiple named objects
that can overlap. Within an object, there are multiple non-overlapping
volumes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment