remove gdspy dependency from gdsfactory

[joamatab]

gdsfactory is built on top of gdspy

The main issues are:

• extra dependencies makes the package harder to install and sometimes hard to follow and document
• gdspy is no longer developed (only in maintenance mode). gdstk is supposed to be the successor of gdspy
• gdspy requires conda install for windows. Now you can install gdsfactory through mamba/conda but it would be nice to only depend on one package manager such as pip

We have some options

1. switch to klayout API

• Klayout has faster booleans
• Klayout builds wheels for windows pip install klayout. So no more need to conda install gdsfactory for windows users.
• Klayout is already an optional dependency
• Klayout API is not super well documented

See @amccaugh work on the phidl klayout backed
@sebastian-goeldi
@thomaslima

See zeropdk

2. switch to gdstk

• gdstk is an improved version of gdspy

3. switch to rust

https://github.com/dan-fritchman/Layout21/

What do you think?

@thomasdorch
@tvt173
@flaport
@basnijholt

[thomasdorch]

I'm in favor of switching to the KLayout backend

[thomaslima]

I obviously support the klayout option but disclaimer is I don't use gdsfactory yet as I focus on zeropdk. That said, next 0.27 version will support static type checking with mypy and docstring inspection from vscode, which should be helpful for you guys. See KLayout/klayout#1125

[alexsludds]

I would be in favor of migrating to the gdstk backend. My experience working with both gdspy and gdstk is that the APIs are similar, give the same layout results within ~1nm (so folks existing silicon is back compatible) and the compile times are much faster in gdstk.

[tvt173]

I agree with @alexsludds . I think a migration to gdstk will be much more straightforward and there should be limited breaking changes because of the similarity between the two. i'm a big fan of klayout also and appreciative of the work from @thomaslima to make it easier to use the python API! but I'm not sure it makes sense to try and migrate gdsfactory over

[sebastian-goeldi]

It's not straight forward to just swap phidl & gdspy for klayout. You have some fundamental differences in the design approach:

1. In klayout cells are always coupled to a layout object. The layout object takes care of tracking layers (which in klauout are addressed by indexes and not tuples as in gdsfactory)
2. The geometric API is very different. In klayout a shape is independent of the cell. It can be transformed and copied without an attachement to a layer or cell. This gives great flexibility at the cost of a bit overhead in regard to addressing them and defining them in cells.

These can be partially addressed

1. Can be made gdsfactory compatible by implicitly using one layout object by default, but there is one potential pitfall. If we have two klayout cells that live in different layout objects, they aren't directly compatible and have to be explicitly copied from one layout object to another (not hard but not like in gdsfactory where you can do from component_module import component; component_reference = parent_component << component)
2. If we would build wrappers around klayout shapes for backwards compatibility, we will lose a lot of the flexibility klayout offers and also we would specifically lose klayouts (great) distinction between database unit objects (int polygons/boxes/paths) vs the float based ones (micrometer based ones).

I think long term it would make sense to step towards going with klayout, but I agree that gdstk is most likely the more sensible approach for now.

The main reason I am mentioning long term klayout is that I believe the yaml placer uses klayout for placing cells (correct me if I'm wrong @joamatab ). If that's still the case, there is a problem when merging multiple (existing) partial gds. Without explicitly checking for duplicates, it silently merged them when usingklayout 0.26.9. Since 0.27.x you will get a quite cryptic error with a cell duplicate name. If you are aware, it's usually easy to fix, but not obvious when encountering it the first time. This beckmes even more fun when the application and python module have different versions.

[thomaslima]

Hi @sebastian-goeldi I am curious about your comment about merging partial gds. I have to deal with this too and need to manually check for cell duplicates. Would you mind sharing an example and an expected behavior? I can try to implement a patch, or at least give a better error message.

BTW Here's my code for loading a specific cell from a gds file: https://github.com/lightwave-lab/zeropdk/blob/master/zeropdk/klayout_extend/layout.py

[sebastian-goeldi]

Hi @thomaslima . I don't have an example directly at hand. I will try to get a minimal working example running when I get back from vacation ;). But I think the proper solution would be to run either a full strm2gds (I think that's the name of the routine) on the partial layouts or run the python equivalent with the LayoutDiff. With the diff object we can just throw errors/warnings if there's a difference between the two. If we don't care about names of cells (which I believe many do, including me), we could also just use conflict_resolution rename.

[alexsludds]

Hey folks,
I posted this on a relevant thread on the phidl package, but I thought it important to put here too.
As part of converting gdsfactory away from gdspy I think it's important to move phidl away from gdspy to gdstk. I made a brief first pass at converting the phidl backend from gdspy to gdstk:
phidl with gdstk backend

It's not bug free, but it does generate some basic geometry. I need folks help with handling dependencies on gds/oas generation.
In particular, if you run this example code (borrowed from the phidl examples):
example code

You will see it tries to create three waveguides of different shapes. However, in the layout all waveguides have the same shape, equal to the last waveguide that was placed.

This is because of how I tried to implement the write_gds function:
write_gds function

I spent a few hours trying different ways to implement phidl's write_gds and have it be compatible with gdstk, but I can't get the dependency handling correct. I'm hoping this might be really obvious to someone else.

Best,
-Alex

[joamatab]

thank you Alex, it looks good, I fixed some more tests

Another alternative is to create an intermediate format (such as shapely) and use klayout to read and write gds files

this is what Helge does in gdshelpers

@HelgeGehring
@jenshnielsen
@Dominik-Vogel

[joamatab]

For example, the VLSIR project uses protocol buffers as an intermediate representation, and it has a rust package from converting to procol buffer to GDS

dan-fritchman/Layout21#22

[joamatab]

After the gdstk segmentation issues

Here is an MVP with klayout backend

@growly is also looking into using VLSIR as a backend

[joamatab]

We were able to get rid of segmentation faults in gdstk by avoiding inheritance from gdstk.Polygon and gdstk.Label

gdsfactory 6.0.0 will be released soon using gdstk backend 🎉

we also have an MVP of a klayout_backend on a branch that we will further investigate