Proposal: simpler LayerStack and DerivedLayers

[simbilod]

The current way to deal with "derived layers" (i.e. physical levels on the chip that do not map 1:1 to GDS design layers) is very convoluted. It requires defining etch and grow levels, with flags for what is etched into what, which borders on process emulation. This also does not generalize well to other common mapping between GDS layers and physical elements on the chip, for instance marker layers modifying shared process steps.

I propose instead, at the gdsfactory level, to only consider the final fabricated levels of the chip. These layers should be easily defined from arbitrary combinations of GDS layers. An example could look something like this:

    layer1 = GDSLayer(layer=(1, 0))
    layer2 = GDSLayer(layer=(2, 0))

    ls = LayerStack(
        layers={
            "layerlevel_layer1": LayerLevel(layer=layer1, thickness=10, zmin=0),
            "layerlevel_layer2": LayerLevel(layer=layer2, thickness=10, zmin=10),
            "layerlevel_and_layer": LayerLevel(
                layer=layer1 & layer2,
                thickness=10,
                zmin=0,
                derived_layer=GDSLayer(layer=(3, 0)),
            ),
            "layerlevel_xor_layer": LayerLevel(
                layer=layer1 ^ layer2,
                thickness=10,
                zmin=0,
                derived_layer=GDSLayer(layer=(4, 0)),
            ),
            "layerlevel_not_layer": LayerLevel(
                layer=layer1 - layer2,
                thickness=10,
                zmin=0,
                derived_layer=GDSLayer(layer=(5, 0)),
            ),
            "layerlevel_or_layer": LayerLevel(
                layer=layer1 | layer2,
                thickness=10,
                zmin=0,
                derived_layer=GDSLayer(layer=(6, 0)),
            ),
            "layerlevel_composed_layer": LayerLevel(
                layer=layer1 - (layer1 ^ layer2),
                thickness=10,
                zmin=0,
                derived_layer=GDSLayer(layer=(7, 0)),
            ),
        }
    )

    # Test with simple component
    import gdsfactory as gf

    c = gf.Component()

    rect1 = c << gf.components.rectangle(size=(10, 10), layer=(1, 0))
    rect2 = c << gf.components.rectangle(size=(10, 10), layer=(2, 0))
    rect2.dmove((5, 5))
    c.show()

Now get the component with derived layers according to the stack above:

    c = get_component_with_derived_layers(c, ls)
    c.show()

(3,0) is the intersection:

(4,0) is the exclusive or:

(5,0) is a substraction:

(6,0) is union:

We can also compose the operations; (7,0) substracts the X0R from layer 1:

We can revisit process emulation and/or simulation in gplugins.

If you think this is a good idea I can make tests and fix visualization scripts

[sourcery-ai]

We've reviewed this pull request using the Sourcery rules engine. If you would also like our AI-powered code review then let us know.

[sourcery-ai]

issue (code-quality): Inline variable that is immediately returned [×2] (inline-immediately-returned-variable)

[simbilod]

I'm also removing all the clutter from as discussed in #2556

[joamatab]

looks great, notice there are some conflicts

[joamatab]

looks great!

[simbilod]

get_klayout_3d_script needs fixing, but it should be easy to convert the boolean operations in the layerstack to klayout script booleans

[simbilod]

there's issues with Pydantic to_json, will check again later

[simbilod]

there's issues with Pydantic to_json, will check again later

the pydantic serialization error is fixable by allowing layers to be int, is this acceptable behavior @joamatab @sebastian-goeldi ?

[simbilod]

here

[simbilod]

and here

[sebastian-goeldi]

there's issues with Pydantic to_json, will check again later

the pydantic serialization error is fixable by allowing layers to be int, is this acceptable behavior @joamatab @sebastian-goeldi ?

The problem with int is, it's not a good identifier, even LayerEnum is questionable in this case. Anything that should live with layers outside or directly tied info, should use something more akin to gf.kdb.LayerInfo, so tuple with layer datatype or a string with layer name are better choices.

An int should be stricly used for layout purposes (it's even not so good to pass it through function calls tbh as they will be serialized as ints)

To get tuple from LayerEnum:
tuple(LAYER.WG) for example

To get name:
LAYER.WG.name

To get layerenum from tuple:
LAYER(gf.kcl.layer(*layer_tuple)) (or layer_tuple[0],layer_tuple[1] instead o *... of course.

Get LayerEnum by name:
LAYER[layer_name]

[HelgeGehring]

Great work @simbilod !

Just some ideas:
Why not do a common class between BaseModel and DerivedLayer/LogicalLayer? (AbstractLayer?)
That one could implement and /... so we would only have that code once (as they would need to do exactly the same)
Then, DerivedLayer/LogicalLayer could both implement an abstract function of AbstractLayer called getShapes(component), which would either return the polygons for a LogicalLayer or do the operation for a DerivedLayer. That way we wouldn't need an extra function to derive the Polygons from the Layers.

[joamatab]

when will this PR be ready to review? we plan to release gdsfactory8 in the coming days

[simbilod]

I've implemented @HelgeGehring 's suggestions, it still works exactly as I showed above

@joamatab this is ready to review. I've added a NotImplementedError on the klayout 3d script for now. The only sketchy thing is the int type allowed for layer object to fix pydantic issues. Also the commit history is a bit messy, we can rebase against main if you want.

[sourcery-ai]

We have skipped reviewing this pull request. It looks like we've already reviewed this pull request.

[simbilod]

Some export tests are still failing, probably just need to call layer.layer in some places. I can look tomorrow

[joamatab]

Sounds good,

will this work with the plugins? like gmsh?

im having lots of issues with the gdsfactory8 brach in gplugins

i think it would be great to use klayout instead of shapely there if possible

[simbilod]

Sounds good,

will this work with the plugins? like gmsh?

im having lots of issues with the gdsfactory8 brach in gplugins

i think it would be great to use klayout instead of shapely there if possible

The meshing should mostly be fine, it only relies on get_component or get_component_with_derived_layers, so if those work it will be OK. We might need to add a layer.layer here and there at most.

[codecov]

Codecov Report

Attention: Patch coverage is 63.31361% with 62 lines in your changes missing coverage. Please review.

Project coverage is 68.63%. Comparing base (bc4dca4) to head (92b7895).

Files	Patch %	Lines
gdsfactory/technology/layer_stack.py	45.91%	51 Missing and 2 partials ⚠️
gdsfactory/export/to_stl.py	84.84%	2 Missing and 3 partials ⚠️
gdsfactory/export/to_3d.py	88.46%	1 Missing and 2 partials ⚠️
gdsfactory/component.py	66.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2805      +/-   ##
==========================================
- Coverage   69.05%   68.63%   -0.42%     
==========================================
  Files         296      296              
  Lines       15215    15235      +20     
  Branches     2225     2229       +4     
==========================================
- Hits        10506    10457      -49     
- Misses       4113     4183      +70     
+ Partials      596      595       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sebastian-goeldi]

I think for layers in general, I think we should just pass them as strings (i.e. layer name).

The longer the more I think it's easier to do it that way. Getting them can be as easy as kcl.get_layer. This also allows for easier and more meaningful (de)serialization of layers as the index itself has next to no meaning (except during construction)