SCH/KICAD_PCB difference tools from kicad-tools

[mdeweerd]

I updated the schematic and pcb difference tools from kicad-tools, where the maintainer indicated that he'ld be happy to see all functionnality implemented on top of KiBot and archive kicad-tools.

These scripts work in my setup if they are locally present in my repository.
Before going any further, I propose that we discuss how this could go into KiAuto (and later Kibot).

This is from the README_diffscripts.md:

Requirements:

• In KiAuto docker image:
  • apt install -f ghostscript

# See https://stackoverflow.com/questions/52998331/imagemagick-security-policy-pdf-blocking-conversion
IMAGICK_POLICY=/etc/ImageMagick-*/policy.xml
# Only modify once
egrep -q "coder.*read.*PDF" ${IMAGICK_POLICY} || perl -i -p -e 's@(\</policymap\>)@<policy domain="coder" rights="read | write" pattern="PDF" />\n$1@;' ${IMAGICK_POLICY}

• runInKiAuto
  Script that runs a command (parameters) in Docker image "setsoft/kicad_auto:dev"
  TODO: change this call in the scripts (environment variable/...)

How this works:

• setup_git.sh
  Set up git repository so that git diff on *.kicad_pcb and *.sch calls the relevant scripts.
  This script currently expects those scripts to be in the project directory.

• git-diff-sch-local.sh

  • Called by git diff *.kicad_sch", receives the git parameters.
  • Copies git files to directory accessible from Docker constainer.
  • Executes git-diff-sch.sh in docker container.
  • Launch image viewer on difference image

• schematic-diff.sh

  • Can be used as standalone script (for instance, to compare using kibot script).
  • Generates "montage.png" showing the differences.
  • Used gen-schematic-img.sh to generate the images for a schematic version.

• gen-schematic-img.sh
  Generate images for a schematic

• git-diff-pcb-local.sh

  • Called by git diff *.kicad_sch", receives the git parameters.
  • Copies git files to directory accessible from Docker constainer.
  • Executes git-diff-pcb.sh in docker container.
  • Launch image viewer on difference image

• pcb-diff.sh

  • Can be used as standalone script (for instance, to compare using kibot script).
  • Generates "montage.png" showing the differences.
  • Used gen-pcb-img.sh to generate the images for a pcb version.

• gen-schematic-img.sh
  Generate images for a pcb.
  The list of layers is fixed in the script. Should be a configuration setting (environment variable/...).

Other thoughts:

The "difference" images are using a "stereo" method where the output is black if both sides are the same and red or magenta otherwise.
That is an interesting output, but it would also be interesting to generate a real difference file where the average sum of pixel values helps determine the differencE.

For the schematic difference, there is the "issue" of local schematic components. I guess that the methods already used for kibot/kiauto could improve that.

This could evolve to be cable to compare a current version to some golden version (precomputed images), usefull for kibot runs (preflight) to ensure nothing changed compared to the golden output by putting an upper limit on the pixel differences.

Temporary locations and cleanup needs improvement.

Possibly exporting to SVG and converting to PNG from SVG might avoid the requirement to install ghostscript if the output is good enough.

Could be transformed to "python" scripts (still using imagick tools).

[set-soft]

Hi @mdeweerd !

I completely fogot about it. The worst part is that the PCB part is already solved since may 2020. I already adapted @obra's idea in the KiCad PCB diff project.

This tool is much better than calling KiAuto because it uses the Python API to plot the layers. It also implements a cache, very useful to avoid regenerating the files over and over when comparing using git.

Today I adapted it to support KiCad 6 and I'm adding it to the kicad_auto image.

[set-soft]

Ok, and now it supports SCH files using KiAuto.

[mdeweerd]

Great!

[set-soft]

Hi @mdeweerd !
Ok, now KiBot supports it. I'm quite sure some stuff will need adjusts, but is a good starting point.
The tests are checking just a few changes, but they are comparing two files outside a repo, a file against another in the repo and two files from the repo.

Notes:

• Perhaps @obra can archive kicad-tools
• I renamed the tool KiDiff

[mdeweerd]

I'ld love to start testing it but I still have too many other things to do.