Export to gEDA PCB format for gerber/etc... processing

[erichVK5]

If the developers are amenable, I would like to write a plugin for exporting to gEDA PCB/pcb-rnd format.

This will make artistic, freeform board design, and importantly, gerber generation, much easier for FidoCadJ users. gEDA PCB/pcb-rnd also allows export in a variety of other formats.

The plugin would export copper layers into respective gEDA design layers, and silkscreen layers, embedding the necessary footprints, and would duplicate lines, boxes, arcs and pads.

Boxes would become four lines.

Design rules, rendering and clearance calculations are challenging with ellipses, so non circular arcs would be exported broken up into line segments.

Similarly, beziers and splines would be exported as line segments too.

Standard library footprints would be exported as footprints that reproduce the silk elements subject to the bezier/spline/ellpise constraints mentioned above, and would approximate the rounded rectangular pin shapes as closely as possible. The SMD rectangular pad shapes would be identical.

Sane soldermask clearances would be exported as a default.

gEDA pcb-rnd, and its progenitor, PCB, can be run headless, allowing simple script driven batch processing to gerbers by users as well.

I will proceed accordingly if there are no objections or other suggestions!

[DarwinNE]

Hi @erichVK5,
that is a great idea, thanks!
Have you seen how the export is implemented in FidoCadJ? You basically have to implement the export interface:

src/net/sourceforge/fidocadj/export/ExportInterface.java

There are also some minor changes to do in the ExportGraphic class and elsewhere (I can do that if you want to).

There is a mechanism so that the splines (complex curves) can be automatically exported as polygons. However, this is not available for ovals and for Béziers, that should be handled by your code.

I would like to release version 0.24.7 at the beginning of April. It should be nice if such a feature can be tested a bit with the preliminary versions by our beta testers, therefore towards the end of February. However, do not be afraid to propose some pull requests of some small amount of code we can discuss about. Just, please do not leave the thing half done once you started working on it... There may also be the need of a little bit of support in the future if a bug is found or if something needs to be improved.

BTW, please follow the coding rules described in the README file. If you need any information, just ask and I will be happy to help if I can!

In the meanwhile, I will be working on issue #99

Kind regards and thanks again,

D.

[erichVK5]

Yes, I have had a look at the interface.

I will get started and see how it goes.

We are keen to implement export of PCB design elements from pcb-rnd as well in due course, into FidoCadJ format, to allow a reasonably lossless two way interchange to occur, subject to caveats re: non circular arcs. This would allow tricks like importation of converted gerbers from pcb-rnd into FidoCadJ for rework or modification.

Now that I think of it, there is a gerber parser in the wild in Java that I used in my translate2geda converter, which may be of interest as an import option for FidoCadJ.

Regards,

Erich.

[DarwinNE]

That's fine, @erichVK5
Do not hesitate to ask if you need technical information about the code.

BTW, concerning a two-way lossles conversion, one of the limitations may be the low resolution employed by FidoCadJ, which may be limiting in some situations.

Kind regards,
D.

[erichVK5]

Export from FidoCadJ of all graphical elements (minus footprints) to a pcb-rnd (.pcb) layout is now working. Here are some preliminary screenshots:

http://imgur.com/a/HG5f3

Footprint (macro) export should be done in the next few days.

The lower image/screenshot is the result of exporting a design from FidoCadJ->pcb-rnd and back to FidoCadJ

Text inevitably loses a bit of its original nature, and the circuit layer is ignored.

This export from pcb-rnd to FidoCadJ was achieved with a newly implemented export_fidocadj module in the gEDA PCB fork pcb-rnd.

Export from pcb-rnd to FidoCadJ also suffers the expected loss of resolution going from a centimil to a 5mil grid, seen best with the beziers and cubic splines. pcb-rnd also support arcs that are less than a full circle, making lossless export to FidoCadJ of these difficult.

Nevertheless, a viable pathway for exporting FidoCadJ design to gerber and a number of other formats, including g-code, via pcb-rnd, is nearly complete.

Regards,

Erich.

[DarwinNE]

Hi @erichVK5
it seems a very nice stuff.
The best thing would be that I can review your code progressively by means of successive pull requests. Before doing that, be sure that you run Checkstyle on every code contribution you include in a pull request, so that we are sure that the coding rules are followed and your code will be included seamlessly in the existing base. It would probably be also a good idea to run PMD so to have a look at the results. Some of the existing export classes may tend to fall in the "god class" category, but there is not much else (I really would not like to add other issues there).
Have a look at chapter 4 of the README.md file for information about Checkstyle and PMD.

I will probably be able to do a code review at the very end of February/beginning of March so that we can work on your pull requests starting from that date.

For the macros (footprints) you can ask to FidoCadJ to split them automatically, using the primitives.

Moreover, there are some automatic tests that I usually employ to see if I have broken something in the export phase:

https://github.com/DarwinNE/FidoCadJ/tree/master/test/export

There is really not much but a bunch of quite simple scripts.
Can you add a test for your export code once you finished implementing it?

Kind regards and thanks again a lot,

D.

P.S. you can post pictures directly here on GitHub, with a drag&drop, it works very well.

If it can be useful, here is the test pattern I employ for testing the export facilities:

[FIDOCAD]
FJC C 1.5
FJC A 0.35
FJC B 0.5
LI 5 50 90 50 0
FCJ 0 0 3 2 4 0
LI 5 45 90 45 0
FCJ 0 0 3 2 3 0
LI 5 40 90 40 0
FCJ 0 0 3 2 2 0
EP 5 5 15 15 0
EV 20 5 30 15 0
EV 50 5 60 15 0
FCJ 2 0
EV 65 5 75 15 0
FCJ 3 0
EV 80 5 90 15 0
FCJ 4 0
RP 5 20 15 30 0
RV 20 20 30 30 0
RV 50 20 60 30 0
FCJ 2 0
RV 65 20 75 30 0
FCJ 3 0
RV 80 20 90 30 0
FCJ 4 0
LI 100 5 185 5 0
FCJ 3 0 3 1 0 0
LI 100 15 185 15 0
FCJ 3 0 3 1 2 0
LI 100 20 185 20 0
FCJ 3 0 3 1 3 0
LI 100 25 185 25 0
FCJ 3 1 3 1 0 0
LI 100 35 185 35 0
FCJ 3 1 3 1 2 0
LI 100 40 185 40 0
FCJ 3 1 3 1 3 0
LI 100 45 185 45 0
FCJ 3 2 3 1 0 0
LI 100 55 185 55 0
FCJ 3 2 3 1 2 0
LI 100 60 185 60 0
FCJ 3 2 3 1 3 0
LI 100 65 185 65 0
FCJ 3 3 3 1 0 0
LI 100 75 185 75 0
FCJ 3 3 3 1 2 0
LI 100 80 185 80 0
FCJ 3 3 3 1 3 0
BE 15 80 10 55 50 55 35 85 0
BE 25 80 20 55 60 55 45 85 0
FCJ 0 0 0 0 2 0
BE 30 80 25 55 65 55 50 85 0
FCJ 0 0 0 0 3 0
BE 35 80 30 55 70 55 55 85 0
FCJ 0 0 0 0 4 0
SA 70 60 0
SA 70 65 0
SA 80 60 0
SA 80 65 0
SA 80 70 0
SA 70 70 0
PL 10 110 90 110 5.0 0
PL 10 120 90 120 6.0 0
PL 10 90 90 90 3.0 0
PL 10 100 90 100 4.0 0
RP 100 85 110 95 0
PA 15 135 10 10 2 0 0
PA 30 135 10 10 2 1 0
PA 45 135 10 10 2 2 0
BE 130 125 50 150 135 175 80 125 0
FCJ 1 0 3 1 0 0
BE 145 125 65 150 150 175 95 125 0
FCJ 2 0 3 1 0 0
PP 0 165 45 150 0 185 50 160 0
PV 40 165 85 150 40 185 90 160 0
PV 120 165 165 150 120 185 170 160 0
FCJ 2 0
PV 160 165 205 150 160 185 210 160 0
FCJ 3 0
PV 200 165 245 150 200 185 250 160 0
FCJ 4 0
TY 220 10 4 3 0 0 0 * Text
TY 220 10 4 3 0 4 0 * Text
TY 220 20 4 8 0 0 0 * Text
TY 220 20 4 8 0 4 0 * Text
TY 220 25 8 3 0 0 0 * Text
TY 220 25 8 3 0 4 0 * Text
TY 220 35 8 3 0 0 0 * Text
TY 225 40 8 3 20 0 0 * Text
TY 230 45 8 3 40 0 0 * Text
TY 235 50 8 3 60 0 0 * Text
TY 240 55 8 3 80 0 0 * Text
TY 245 60 8 3 90 0 0 * Text
TY 215 40 8 3 20 4 0 * Text
TY 210 45 8 3 40 4 0 * Text
TY 205 50 8 3 60 4 0 * Text
TY 200 55 8 3 80 4 0 * Text
TY 220 35 8 3 0 4 0 * Text
TY 195 60 8 3 90 4 0 * Text
EV 200 60 215 70 0
EV 225 60 240 70 0
EV 215 70 225 80 0
BE 205 80 215 90 225 90 235 80 0
BE 190 60 195 115 245 110 250 60 0
EP 205 65 210 70 0
EP 230 65 235 70 0
TY 230 130 4 3 0 4 0 * Text
TY 230 130 4 3 90 4 0 * Text
TY 230 130 4 3 180 4 0 * Text
TY 230 130 4 3 270 4 0 * Text
CV 0 170 130 195 135 200 110 240 115 245 140 215 145 195 100 0
CP 1 125 150 135 135 175 145 145 130 135 150 0
CV 0 70 80 90 80 100 105 115 125 105 105 0
FCJ 3 0 3 2 0 0
CV 0 240 5 260 25 240 30 0
FCJ 2 2 3 1 0 0
LI 15 190 230 190 0
FCJ 3 0 -3 2 0 0
LI 15 195 230 195 0
FCJ 3 1 -3 2 0 0
LI 15 200 230 200 0
FCJ 3 2 -3 2 0 0
LI 15 205 230 205 0
FCJ 3 3 -3 2 0 0
CV 0 15 225 90 215 100 235 135 210 0
FCJ 3 0 -3 2 0 0
CV 0 20 230 95 220 105 240 140 215 0
FCJ 3 1 -3 2 0 0
CV 0 25 235 100 225 110 245 145 220 0
FCJ 3 2 -3 2 0 0
CV 0 30 240 105 230 115 250 150 225 0
FCJ 3 3 -3 2 0 0
BE 170 220 155 240 210 240 200 220 0
FCJ 3 0 -3 2 0 0
BE 200 220 200 220 200 220 200 220 0
BE 195 230 195 230 195 230 195 230 0
BE 175 225 160 245 215 245 205 225 0
FCJ 3 1 -3 2 0 0
BE 180 230 165 250 220 250 210 230 0
FCJ 3 2 -3 2 0 0
BE 185 235 170 255 225 255 215 235 0
FCJ 3 3 -3 2 0 0
RP 105 90 115 100 1
RP 110 95 120 105 2
RP 115 100 125 110 3
RP 120 105 130 115 4
RP 125 110 135 120 5
RP 130 105 140 115 6
RP 135 100 145 110 7
RP 140 95 150 105 8
RP 145 90 155 100 9
RP 150 85 160 95 10
RP 155 90 165 100 11
RP 160 95 170 105 12
RP 165 100 175 110 13
RP 170 105 180 115 14
RP 175 110 185 120 15

[erichVK5]

I'm fairly happy with the degree of support for primitives and footprint elements now. I will start cleaning up the code, and try your export pattern, as suggested, and perhaps sort out a test file.

gEDA footprints do not support text within them, so text is not exported in them.

Ellipses are not well supported, and these have been converted into either polylines or polygons within the layout

Ellipse (other than circle) support has not been provided for exported footprints.

FidoCadJ SMD footprints quote layer 1 for the copper pads, but the exporter assumes layer 2 is actually intended, and proceeds accordingly . They can always be shifted in gEDA PCB if needed.

Text on the layout is exported, but needs to be a reasonable size to render properly on a PCB exported to gerber, i.e. 50mil high or larger.

The exporter currently substitutes rectangular pads for RoundRect pads.

Rotation of elements (i.e. text, footprints) other than 0,90,180,270 is not yet supported.

I will also generate a test board and export to gerber as a proof of concept.

Cheers,

Erich.

[erichVK5]

I've sent a git diff and the new export module.

Here are some gerbers being viewed in gerbv, the gEDA gerber viewer, that were exported from pcb-rnd after being exported from FidoCadj to gEDA PCB/pcb-rnd (.pcb) format:

Incidentally, pcb-rnd also provides a pathway for g-code export, for milling or conceivably eggbot use.

I have not had a chance to provide test outputs yet.

The round trip is now quite functional, with a few caveats:

Steps:

1. (optional) produce a board layout with a rectangle indicating the intended outer margins of the board boundary, and export to FidoCadJ from the current svn version of pcb-rnd, which now supports export to FidoCadJ.

2. Create the tracks on the copper layers. The "circuit" layer artwork in FidoCadJ will end up on the back silk layer on the design in pcb-rnd. The "silkscreen" layer artwork in FidoCadJ will end up on the front silk on the design exported to pcb-rnd. Front and back copper layers map to the front and back in pcb-rnd. The inner layers (optional, additional) map from the bottom up in the stack up in pcb-rnd. SMD pads end up on top copper by default during export from FidoCadJ.

In the absence of an explicit outline layer in FidoCadJ, any of the inner copper layers, or the silk or circuit layer, can be used as the outline layer, which then needs to be manually renamed to "outline" in the exported .pcb file before loading into pcb-rnd or gEDA PCB.

3. Rotation has not yet been implemented for exported footprints. The user can easily rotate the footprints with the rotate tool in pcb-rnd into their desired positions if required. Similarly, pcb-rnd and gEDA PCB do not support text at an angle other than 0, 90, 180, and 270 degrees, although utilities such as FootprintTextForPCB will allow angled text to be added as a footprint.

4. Once the design looks OK in pcb-rnd, the design can be exported to gerber and then checked in gerbv.

Things that are still worth implementing:

on the FidoCadJ side:

automatic rotation of footprints on export to pcb-rnd

possibilities on the pcb-rnd side:

a library of FidoCadJ standard lib parts to allow parts to be placed that retain their positions on export to FidoCadJ. This should allow design rules checks (DRC) and such like, for more complicated designs, if pin numbering and naming can be retained during the round trip for netlisting/DRC purposes in pcb-rnd.

[erichVK5]

I have fine tuned the text definitions exported from pcb-rnd to FidoCadJ format, and have fine tuned the export of text definitions from FidoCadJ to pcb-rnd. The round trip testing results are pretty good given the inevitable quantisation errors that occur with changes in coordinate systems:

[erichVK5]

Footprints libraries seem to be exporting nicely as well, so there should be some compatible gEDA PCB/pcb-rnd footprint libraries available soon to improve the pcb-rnd -> FidoCadJ ->pcb-rnd workflow

And here are some Semiconductor library footprints exported to pcb-rnd, and thence to gerber...

[DarwinNE]

Hi @erichVK5
That is /really/ impressive!

Now, let us proceed to integrate your code. You said you sent a git diff, but it would be probably more convenient to fork the FidoCadJ repository on your Github account (if you have not done it already), then apply the differences there and finally open a pull request.
In this way, we can discuss about your code in the pull request, review it and then integrate it.

Cheers,

D.

[erichVK5]

Done. Let me know if I have neglected to include anything that stops it from building.

[DarwinNE]

Hi @erichVK5
it seems to be a truly wonderful work, I had a superficial look at the code and I accepted the pull request. I will try it and review in a more detailed way in the next week.
In the meanwhile, do you think it would be possible to add a (simple) test for export? BTW, my tests can be improved without doubt, if you have some ideas about that, do no hesitate to open another issue here. If you want to update the documentation too, do not hesitate.

BTW, I know what kind of effort is required to implement what you did, so in behalf of the FidoCadJ community, I really would like to thank you for your work!

D.

[erichVK5]

My pleasure! Your modular and well thought out code, plus the Eagle export module framework, made it much easier to implement.

Test suites aren't my forte. I could devise a layout which exercises the key supported features currently to serve as a reference; would this help?

Cheers,

Erich.

[DarwinNE]

Hi again,
test suites are not my forte either, however, they are useful to avoid gross errors.
Presently, the tests for export work in a very simple way, by checking that the result is conformal to a certain model. For example, here is the command to launch an export from the command line:

echo "  SCR (script for CadSoft Eagle)"
java -jar ../../jar/fidocadj.jar -n -c r2 scr scr/test_out.scr original/test_pattern.fcd >output_scr.txt

It is presently contained in the test_export.sh script (in the test directory).
And here is the check that the export is "correct" (i.e. similar to a certain model), in the check.sh file:

if diff -I "# Created by" scr/test_out.scr scr/ref/test_out.scr >results_scr.txt
then
  echo "scr     OK"
  rm results_scr.txt output_scr.txt
else
  test_fail=1
  printf "\033[1mscr export is not conformal to the model. Please check results_scr.txt\033[0m\n"
fi

It is a very basic test where a line with the comment (that may vary if the version of FidoCadJ changes) is ignored.
Probably, if I would have to rewrite this code now, I would put the two things in the same file, one for each file format to allow to separately test every format, but that is not really an issue for the moment. There is also a script that promotes the current version of the test output as the reference for future comparisons. The test results should not be put under version control, but the references must be.

BTW, is there a specific library for gEDA available? One of the problem we had with the Eagle export is that the original library was provided once and never completed or supported, so it is now quite outdated.

Cheers,
D.

EDIT: I forgot! Your code compiles perfectly and I could export a test drawing, however I do not use gEDA and I could not yet test it.

[erichVK5]

The gEDA PCB fork, pcb-rnd, is the one with the FidoCadJ export module

http://repo.hu/projects/pcb-rnd/

svn checkout svn://repo.hu/pcb-rnd/trunk
cd trunk
./configure --debug
make
cd src
./pcb-rnd ~/path/to/testExportFromFidoCadJ.pcb

will get you up and running pretty quickly.

For extra points there are headless batch processing CLI arguments that will load and export a layout in whatever other format, i.e. gerber.

I will try and do an export test script tonight.

I will have a look at the documentation and see if there is anything I can do to add value.

I still intend to implement footprint rotation in the exporter to gEDA pcb-rnd/PCB. There is a bit of extraneous code that can be trimmed too.

There's no gEDA PCB library as such in Java, but the format is very simple and text based, and the project's commitment to adhering to stable formats which remain backwards compatible means that the FidoCadJ export module should continue to work for years. We can still load layouts generated in the 1990s!

The only residual issue is conversion of the standard lib footprints to gEDA compatible .fp files. The pad descriptions in the macros are not listed in the usual pin number order, i.e. for DIL sockets. If there is no desire to alter the standard library footprints such that pads are described in pin order, to allow automated numbering of pins on conversion/export, I will need to implement a lookup table or similar. The advantage of correct pin numbering is the ability to easily apply netlists for design rule checks in PCB.

Cheers,

Erich.

[DarwinNE]

Quite impressive!
I will give it a try as soon as I can.
I also saw your post to the gEDA list and the Hackaday.io project. Very nice!

Cheers,

D.

[erichVK5]

The first ever batch of boards made using the pcb-rnd -> FidoCadJ -> pcb-rnd toolchain arrived:

Next up will be populating the boards!!

[DarwinNE]

Hi @erichVK5

Great stuff!

[DarwinNE]

Hi @erichVK5
your code is included in the preliminary version I just released for testing:

https://github.com/DarwinNE/FidoCadJ/releases/tag/v0.24.7beta

Thanks again for the work you did.
Cheers,
D.

[erichVK5]

So far, so good!!!

[erichVK5]

All done!!

Now to get back to other things!!

[DarwinNE]

Great project! I am very happy that everything worked fine!

Do you think you can implement an automatic test? BTW, I can do that, it is not a problem for me.

[erichVK5]

It might save reinventing the wheel if you have a template you can use. It is not something I am terribly familiar with and my learning curve might delay your code tidying.

My time would probably be best applied to getting elements to rotate if required on export.

[DarwinNE]

Hi @erichVK5,
I missed the notification about your last message, sorry for the delay.
In any case, you may have a look at the (very simple, almost simplistic) way the current tests are implemented. Basically, an export operation is done in a script and then the result is compared with a reference result. A script is available to update the reference if needed.
I tried to describe the technique in one of my previous messages:

#132 (comment)

If there is anything that is not clear, feel free to ask.

Cheers,

D.

P.S. in any case, I agree with you about the priority on the element rotation!

[DarwinNE]

I am closing this Issue now, do not hesitate to re-open it if there is something to be done to complete it.