[Feature] Support for KiCad 8.0 (undocumented) worksheet files

[BetCvR6]

Hello,

I use Kicad v8.0.1 under Windows 10 and a docker image of KiBot v1.6.5.
I call Kibot from project directory with command "kibot" without parameter.

When I use KiBot to export schematic to pdf with pdf_sch_print (see TEST 1 in config file), my custom worksheet not appears in output document and KiBot uses default Kicad worksheet. But when I export schematic directly from kicad with Plot, generated pdf is correct and use my custom worksheet.

root@c08f9e8af560:/project# kibot
Using config file: config.kibot.yaml
Using SCH file: project_test.kicad_sch
- 'Schematic to PDF' (pdf schematic) [pdf_sch_print]

With a custom worksheet created with Kicad 8.0.1, when I try to export pcb layers to pdf with pcb_print (see TEST 2 in config file), KiBot returns the following messages and none pdf is generated :

root@c08f9e8af560:/project# kibot
Using config file: config.kibot.yaml
Using SCH file: project_test.kicad_sch
- 'Layers to PDF with Kicad 8.0.1 worksheet' (pdf pcb) [pcb_print]
WARNING:(W086) Unsupported worksheet version, loading could fail (kibot - worksheet.py:499)
ERROR:In section 'pdf pcb' (pcb_print): Error reading `/project/worksheet/custom_worksheet_PCB.kicad_wks` (generator is not a Symbol `pl_editor` (<class 'str'>)) (kibot.gs - gs.py:816)

With a custom worksheet created with Kicad 7.0, I get the same error (see TEST 3 in config file).

root@c08f9e8af560:/project# kibot
Using config file: config.kibot.yaml
Using SCH file: project_test.kicad_sch
- 'Layers to PDF with Kicad 7.0 worksheet' (pdf pcb) [pcb_print]
WARNING:(W086) Unsupported worksheet version, loading could fail (kibot - worksheet.py:499)
ERROR:In section 'pdf pcb' (pcb_print): Error reading `/project/worksheet/custom_worksheet_PCB.kicad_wks` (generator is not a Symbol `pl_editor` (<class 'str'>)) (kibot.gs - gs.py:816)

However, if I use option "sheet_reference_layout" to set a path to custom worksheet (see TEST 4" in config file) it works and generated pdf uses my custom worksheet!

root@c08f9e8af560:/project# kibot
Using config file: config.kibot.yaml
Using SCH file: project_test.kicad_sch
- 'Layers to PDF with Kicad 7.0 worksheet passed in parameter' (pdf pcb) [pcb_print]
Found 0 unique warning/s (2 total, 2 filtered)

I think even if this last test works, it's just a way to get around the problem to load custom worksheets. I made several tests but without success. Please let me know if you need any further information.

I add an archive in attachment :

• schematic and pcb files
• pdf generated from Kicad (in ./pdf/fromKicad folder)
• pdf generated from Kibot (in ./pdf/fromKiBot folder)
• config.kibot.yaml file
• custom worksheets (in ./worksheet folder)

Thanks for your help.

project_test.zip

[set-soft]

Hi @BetCvR6 !
I don't have much time today, will look it in deep tomorrow. But I'll suggest you a couple of things:

1. Don't comment the outputs, just use "kibot OUTPUT_NAME" to run one of them without running the others
2. Use debug output (i.e. -vvvv) you'll understand some of the problems

Looking at the debug output you'll see that even when KiCad GUI tolerates DOS backslashes the command line doesn't:

 > DEBUG:Executing: kicad-cli sch export pdf -o '/home/salvador/0Data/Eccosur/0Test/KiBot/#607/project_test/pdf/fromKiBot/project_test.pdf' -t _builtin_default --no-background-color '/home/salvador/0Data/Eccosur/0Test/KiBot/#607/project_test/project_test.kicad_sch' (eeschema_do.kiauto.file_util - file_util.py:386)
> DEBUG:- Output from command: 16:50:42: Drawing sheet 'worksheet\custom_worksheet.kicad_wks' not found.
> Plotted to '/home/salvador/0Data/Eccosur/0Test/KiBot/#607/project_test/pdf/fromKiBot/project_test.pdf'.
> Done.
>  (eeschema_do.kiauto.file_util - file_util.py:382)

If you look at the project file you'll see:

   "page_layout_descr_file": "worksheet\\custom_worksheet.kicad_wks"

A very bad idea, use regular slashes, like you did with the PCB:

"page_layout_descr_file": "worksheet/custom_worksheet_PCB.kicad_wks"

[BetCvR6]

Hi Salvador,

Thanks for your feedback and your suggestions. Right, sorry I didn't think risks with backslashes...

I started by remove comments for all outputs but I've the same behaviour for each generated output.

I've also made some tests with debug option and I've got same message you mentioned above. Hence, it's certainly because kibot doesn't find my custom worksheet that it loads the default worksheet.
DEBUG:- Output from command: 08:23:34: Drawing sheet 'worksheet\custom_worksheet.kicad_wks' not found.

For pcb_print outputs, generation still fails and Kibot reports warning :
WARNING:(W086) Unsupported worksheet version, loading could fail (kibot - worksheet.py:499)

And with debug option, the following complementary error :

ERROR:Trace stack: (kibot.gs - gs.py:802)
  File "/usr/lib/python3/dist-packages/kibot/kiplot.py", line 493, in run_output
    out.run(get_output_dir(out.dir, out))
  File "/usr/lib/python3/dist-packages/kibot/out_base.py", line 214, in run
    self.options.run(target)
  File "/usr/lib/python3/dist-packages/kibot/out_pcb_print.py", line 1305, in run
    self.generate_output(output)
  File "/usr/lib/python3/dist-packages/kibot/out_pcb_print.py", line 1253, in generate_output
    self.plot_frame_internal(pc, po, p, len(pages)+1, len(self.pages))
  File "/usr/lib/python3/dist-packages/kibot/out_pcb_print.py", line 520, in plot_frame_internal
    raise KiPlotConfigurationError('Error reading `{}` ({})'.format(self.layout, error))
ERROR:In section 'pdf_pcb_2' (pcb_print): Error reading `/project/worksheet/custom_worksheet_PCB.kicad_wks` (generator is not a Symbol `pl_editor` (<class 'str'>)) (kibot.gs - gs.py:816)

For pcb_print outputs, I tried to replace backslashes by slashes to specify worksheet path (with option sheet_reference_layout). The following tests performed correctly:
sheet_reference_layout: "./worksheet/custom_worksheet_PCB_Kicad_7.0.kicad_wks"
and
sheet_reference_layout: ".//worksheet//custom_worksheet_PCB_Kicad_7.0.kicad_wks"

But Kibot returns an error because it fails to find worksheet with this test:
sheet_reference_layout: "worksheet\\custom_worksheet_PCB_Kicad_7.0.kicad_wks"

ERROR:Trace stack: (kibot.gs - gs.py:802)
  File "/usr/lib/python3/dist-packages/kibot/kiplot.py", line 460, in config_output
    out.config(None)
  File "/usr/lib/python3/dist-packages/kibot/out_base.py", line 162, in config
    super().config(parent)
  File "/usr/lib/python3/dist-packages/kibot/optionable.py", line 232, in config
    self._perform_config_mapping()
  File "/usr/lib/python3/dist-packages/kibot/optionable.py", line 196, in _perform_config_mapping
    v.config(self)
  File "/usr/lib/python3/dist-packages/kibot/out_pcb_print.py", line 416, in config
    raise KiPlotConfigurationError("Missing page layout file: "+self.sheet_reference_layout)
ERROR:In section 'pdf_pcb_3' (pcb_print): Missing page layout file: /project/worksheet\custom_worksheet_PCB_Kicad_7.0.kicad_wks (kibot.gs - gs.py:816)

Is there a way to pass a path with slashes to kibot?

Thanks.

[set-soft]

Thanks for your feedback and your suggestions. Right, sorry I didn't think risks with backslashes...

I started by remove comments for all outputs but I've the same behaviour for each generated output.

I've also made some tests with debug option and I've got same message you mentioned above. Hence, it's certainly because kibot doesn't find my custom worksheet that it loads the default worksheet. DEBUG:- Output from command: 08:23:34: Drawing sheet 'worksheet\custom_worksheet.kicad_wks' not found.

This message is from KiCad!
Just replacing the \ by a / you get the correct WKS.

For pcb_print outputs, generation still fails and Kibot reports warning : WARNING:(W086) Unsupported worksheet version, loading could fail (kibot - worksheet.py:499)

I think this message is clear: Unsupported worksheet version. I have to investigate what's in the new format and add support.

[set-soft]

Hi @BetCvR6 !

The 4492611 patch (+ d6d48a5) adds a workaround for backslashes. So the first test passes without changes in the project file.

The 2c08f3b add support for the worksheet v20231118 you included in the example. I can't be sure that all files will load because the current KiCad documentation is dated 2023-04-13, so this format isn't documented.

The 3rd test: you mention you used a KiCad 7 WKS, but this is used only in the 4th test. I guess you used other files, but I also guess you didn't save them to disk and hence it failed.

So now all the tests are working

[BetCvR6]

Hi Salvador,

Greats! Thanks a lot for the quick update!
Sorry but I've one last newbee question... how can I install and use this last dev version ? I saw docker image kicad_auto_full is updated on 1.6.6. I install it on docker but I don't know what command line use to launch it?

For kibot 1.6.5 docker image I use this :
docker run -it --rm -v %cd%:/project kibot:1.6.5 /bin/bash

Thanks in advance

[set-soft]

Hi @BetCvR6 !

You'll find information about the docker images in the docs: https://kibot.readthedocs.io/en/latest/usage_with_ci_cd.html

For KiCad 8 the last dev images are fetched using the ghcr.io/inti-cmnb/kicad8_auto_full:dev name. After doing a pull (docker pull ghcr.io/inti-cmnb/kicad8_auto_full:dev when using the command line) just use this name instead of kibot:1.6.5