[BUG] Parts are being incorrectly grouped

[andreasWallner]

I have a resistor in my schematic that differs only in tolerance specification from some others. During generation of the BOM this resistor is grouped together with the others though.

I tried two options of specifying the tolerance, both failed (see screenshot at the end):

• specify the Value field as 12k 1% (whereas the other resistors have a Value of 12k)
• add a separate field Tolerance with a value of 1% (the other resistors don't have that field)

I tried both using the default group_fields as well as manually specifying group_fields.

To Reproduce
Issue can be reproduced by running kitbot (w/o parameters) on https://github.com/andreasWallner/ft232-syncfifo-pmod.

Log output:

Using config file: project.kibot.yaml
Using SCH file: FT232-SYNCFIFO-PMOD.kicad_sch
- 'Board View export' (basic_boardview) [boardview]
- 'Download the datasheets' (basic_download_datasheets) [download_datasheets]
WARNING:(W013) `REF**` component in board, but not in schematic (kibot - kiplot.py:297)
- 'Gerbers compatible with PCBWay' (PCBWay_gerbers) [gerber]
- 'Drill files compatible with PCBWay' (PCBWay_drill) [excellon]
- 'Gerbers compatible with JLCPCB' (JLCPCB_gerbers) [gerber]
- 'Drill files compatible with JLCPCB' (JLCPCB_drill) [excellon]
- 'Pick and place file, JLCPCB style' (JLCPCB_position) [position]
- 'BoM for JLCPCB' (JLCPCB_bom) [bom]
WARNING:(W009) KiCad config without environment.vars section (kibot - config.py:356)
WARNING:(W004) Field conflict: (R3,R6,R8,R9,R10) [lcsc] : 'C25752' <- 'C269386' (in R10) (kibot - bom.py:299)
- 'Information about the run' (basic_info) [info]
- 'PCB' (basic_pcb_print_pdf) [pcb_print]
- 'PCB 2D render in SVG format, using jlcpcb-green-enig style' (basic_pcbdraw_svg_jlcpcb_green_enig_top) [pcbdraw]
- 'Simple design report' (report_simple) [report]
- 'Full design report' (report_full) [report]
- 'ZIP file for PCBWay' (PCBWay) [compress]
- 'ZIP file for JLCPCB' (JLCPCB) [compress]
Found 3 unique warning/s (5 total)

The resistor in question is R10, the BOM that is generated for me:

"Comment","Designator","Footprint","LCSC Part #"
...
"12k","R3,R6,R8,R9,R10","R_0402_1005Metric","C25752 C269386"
...

Expected behavior
R10 should have a separate BOM entry with part number C269386.

Screenshots

Environment (please complete the following information):
Where are you running KiBot:
I'm running KiBot on Ubuntu 22.04.2, installed via the official repo.

KiBot installation checker

Core:
Linux: 5.15.0.82 (Linux ThinkStation-C20 5.15.0-82-generic #91-Ubuntu SMP Mon Aug 14 14:14:14 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux)
Python: 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0]
KiCad: 7.0.6-7.0.6~ubuntu22.04.1
Kibot: 1.6.3

Modules:
Colorama: 0.4.4
LXML: 4.9.1
Lark: 1.1.5
PyYAML: 6.0.0 (6.0)
QRCodeGen: Ok
Requests: 2.31.0
XLSXWriter: 3.0.3
Xvfbwrapper: Ok
markdown2: 2.4.8
mistune: 2.0.4
numpy: 1.24.3

Tools:
Bash: 5.1.16 (GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu))
Blender: Not available
Ghostscript: 9.55.0
Git: 2.34.1 (git version 2.34.1)
ImageMagick: 6.9.11.60 (Version: ImageMagick 6.9.11-60 Q16 x86_64 2021-01-25 https://imagemagick.org)
Interactive HTML BoM: 2.7.0 (v2.7.0)
KiBoM: 1.9.0 (KiBOM Version: 1.9.0)
Failed to run /usr/local/bin/pcbnew_do, error 1
KiCad Automation tools (kiauto): *UNKNOWN*
KiCad PCB/SCH Diff (kidiff): Not available
KiCost: 1.1.18 (KiCost v1.1.18)
Failed to run /usr/bin/kikit, error 1
KiKit: *UNKNOWN*
OpenSCAD: 2021.1.0 (OpenSCAD version 2021.01)
Pandoc: 2.9.2.1 (pandoc 2.9.2.1)
RAR: 5.50.0 (RAR 5.50   Copyright (c) 1993-2017 Alexander Roshal   11 Aug 2017)
RSVG tools: 2.52.5 (rsvg-convert version 2.52.5)
Xvfb: Ok (xvfb-run)

* Blender not installed or too old
  Install the `blender` package, i.e.: `sudo apt-get install blender`
  - Mandatory for `blender_export`

* KiCad Automation tools not installed or too old
  Visit: https://github.com/INTI-CMNB/KiAuto
  Download it from: https://github.com/INTI-CMNB/KiAuto/releases
  This tool might be automatically downloaded by KiBot.
  - Mandatory for: `dxf_sch_print`, `gencad`, `hpgl_sch_print`, `netlist`, `pdf_pcb_print`, `pdf_sch_print`, `ps_sch_print`, `render_3d`, `run_drc`, `run_erc`, `step`, `svg_pcb_print`, `svg_sch_print`, `update_xml`, `vrml`
  - Optional to:
    - Compare schematics for `diff` (v2.2.0)
    - Show KiAuto installation information for `info` (v2.0.0)
    - Print the page frame in GUI mode for `pcb_print` (v1.6.7)

* KiCad PCB/SCH Diff not installed or too old
  Visit: https://github.com/INTI-CMNB/KiDiff
  Download it from: https://github.com/INTI-CMNB/KiDiff/releases
  This tool might be automatically downloaded by KiBot.
  - Mandatory for `diff`

* KiKit not installed or too old
  Visit: https://github.com/yaqwsx/KiKit
  Download it from: https://github.com/yaqwsx/KiKit/releases
  This tool might be automatically downloaded by KiBot.
  - Mandatory for: `panelize`, `stencil_3d`, `stencil_for_jig`
  - Optional to separate multiboard projects for general use

(interestingly I do have kiauto and kikit installed via apt:

% sudo apt install kiauto
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
kiauto is already the newest version (2.2.7-1).
The following packages were automatically installed and are no longer required:
  dh-elpa-helper libllvm13:i386 librhash0
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 92 not upgraded.
% sudo apt install kikit
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
kikit is already the newest version (1.3.0-8).
kikit set to manually installed.
The following packages were automatically installed and are no longer required:
  dh-elpa-helper libllvm13:i386 librhash0
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 92 not upgraded.

[set-soft]

Hi @andreasWallner !

Before anything: If you have LCSC part numbers for all your components I suggest grouping them using it. This will solve issues like "I have 12k 1% automotive grade and 12k 1% industrial grade ..."

Now: The recommended use is to separate the 1% from the 12k, you can even do it using KiBot filters. I don't consider what you describe as a bug. You say "I tried both using the default group_fields as well as manually specifying group_fields." but you don't show what you tried. If you separated the tolerance in a different field, and you only defined it for the 1% resistor, them you must disable merge_blank_fields. This option is enabled by default for compatibility with KiBoM (the original BoM behavior comes from KiBoM implementation). This option is documented.

[andreasWallner]

Thanks, that solved my issue:

Before anything: If you have LCSC part numbers for all your components I suggest grouping them using it. This will solve issues like "I have 12k 1% automotive grade and 12k 1% industrial grade ..."

Doing that now, works as expected - not sure why I didn't think of that when I was already playing with group_fields

Now: The recommended use is to separate the 1% from the 12k, you can even do it using KiBot filters. I don't consider what you describe as a bug. You say "I tried both using the default group_fields as well as manually specifying group_fields." but you don't show what you tried.

Sorry - didn't put it into the description, only in the linked repo.
I used

    group_fields:
      - Part
      - 'Part Lib'
      - Value
      - Footprint
      - 'Footprint Lib'
      - Tolerance

If you separated the tolerance in a different field, and you only defined it for the 1% resistor, them you must disable merge_blank_fields. This option is enabled by default for compatibility with KiBoM (the original BoM behavior comes from KiBoM implementation). This option is documented.

Switching merge_blank_fields off also fixes my issue - I didn't get that when reading the docs, sorry.

Closing this, but I'd still like to ask/understand where the original behavior that I saw comes from: Which setting/behavior causes the values 12k and 12k 1% to be considered equal (and merge components even if Value is in group_fields)? I had suspected normalize_values, but that is turned off (on default)...

[set-soft]

but I'd still like to ask/understand where the original behavior that I saw comes from: Which setting/behavior causes the values 12k and 12k 1% to be considered equal (and merge components even if Value is in group_fields)? I had suspected normalize_values, but that is turned off (on default)...

The KiBoM idea is to merge components in the same group even when some (optional) field is filled in one of the components, but not in the others. In this way you could fill the digikey P/N of one 12k resistor and propagate it to all the others, no need to edit all of them.

This is a way to see the situation, the other is your case, but in this case you can: fill the digikey P/N for one of the 12k resistors and also for the particular "12k 1%", all the other "12k" will be grouped with "12k" (assuming you group by value AND digikey P/N).

Now an extra note: in the original KiBoM "12k" and "12k 1%" doesn't match their value. But for KiBot both are 12 kilo ohms resistors. If 1% is really important (and not juist garbage) you should separate it. As I said before: you can even ask KiBot to try to separate it parsing the value. In this case KiBot will understand this is the tolerance.

[andreasWallner]

Thanks!