Group by geometry/design: `representationId` issues

[ShaMan123]

I am looking into grouping instances of the same design.
@agviegas suggested using representationId, see ref below.
However I am facing numerical instability in ifc files exported from Tekla.
I ran the following to group instances using fragment's representationId:

    const model = new SingleThreadedFragmentsModel("MODEL", buffer);
    const ids = model.getItemsWithGeometry();
    const items = await model.getItems(ids);
    // TODO: some IFCPLATE entities are the same but are assembled with different bolts
    // workaround by using the tag, should find a better solution
    const tags = new Map(
      [...items].map(([id, { data }]) => [
        id,
        parseTag(data.Tag?.value) ?? parseTag(data.Name?.value),
      ])
    );
    const idsByGeometryMap = new Map<string, number[]>();
    model.getItemsGeometry(ids).forEach((geometries, index) => {
      const id = ids[index];
      const key = `${geometries.map((g) => g.representationId).join(",")}_${tags.get(id)}`;
      let entry = idsByGeometryMap.get(key);
      if (!entry) {
        entry = [];
        idsByGeometryMap.set(key, entry);
      }
      entry.push(id);
    });
    model.dispose();

Numerical instability

This has revealed that Tekla is exporting ifc data with rounding issues, causing instances to vary because of insignificant floating points.
I have tried to configure the ifc importer but that doesn't seem to make a difference.

importer.webIfcSettings.TOLERANCE_SCALAR_EQUALITY = 10;

How can this be improved/workedaround?

Splitting

Another issue I wish to surface is that when splitting an ifc it seems that the localId assigned as the representationId is not unique across splits. This needs discussion.
Shouldn't the splitter split the ifc such that all instances of the same design reside in the same split? If not I think representationId must be unique across all splits or else splitting is causing a damage in performance causing fragments to generate redundant geometries/BIMMesh instances.

Ref

2. Getting representationId while reading the IFC line by line.

Strictly speaking, representationId doesn't exist during IFC parsing. It's a localId the importer allocates at write time when it builds the .frag (see representationsLocalIds.unshift(nextId++) in packages/fragments/src/Importers/IfcImporter/src/geometry/index.ts). The .frag's meshes.representationIds vector is what surfaces it at runtime.

What does exist during parsing is the geometry's express ID, and that's what fragments uses internally as the equality key while building. In packages/fragments/src/Importers/IfcImporter/src/geometry/ifc-file-reader.ts, look at processFlatGeometry (and the parallel methods):

const geometryRef = mesh.geometries.get(geometryIndex);
// ...
const geometryData: IfcGeometryInstance = {
  id: geometryRef.geometryExpressID,
  // ...
};

That geometryExpressID is web-ifc's stable id for the underlying IFC representation. Two products that share a representation get the same geometryExpressID, which is exactly the "instances of the same design" key you want. The importer dedupes on it (_previousGeometriesIDs) and emits one entry per unique express id, then samples reference it.

If you're walking IFC raw (no web-ifc), the equivalent is the express id of IFCSHAPEREPRESENTATION for direct geometry, and IFCREPRESENTATIONMAP (referenced through IFCMAPPEDITEM.MappingSource) for true instanced geometry. Those are the IFC schema's own "shared geometry" handles.

So the recipe for your importer: when you see an IFCMAPPEDITEM, read its MappingSource (an IFCREPRESENTATIONMAP express id) and use that as your equality key. For non-mapped items, use the IFCSHAPEREPRESENTATION express id of the product's Representation. That gives you the same grouping fragments uses, available before any conversion runs.

Originally posted by @agviegas in #709

[agviegas]

Two threads. Taking them separately.

Numerical instability

Want to make sure I'm reading the problem right before I propose anything. Could you share:

1. A small example IFC where this is happening (or even a snippet of a few IFCSHAPEREPRESENTATION lines that should match but don't).
2. What you're getting (presumably N distinct representationIds).
3. What you expect (one representationId shared across the N items).

Tekla's IFC export has a few quirks and the right fix depends on which one you're hitting. With an example in front of me I can point at the actual mismatch instead of guessing.

Splitting

Spec-wise I agree it'd be nice if same-design items lived in the same split. The problem is that "same design" isn't a property the IFC reliably exposes. Plenty of IFC files have entities whose geometry is virtually identical but that the source CAD emitted as separate entities (this is the same thing biting you in the Tekla case above). The only way to know two entities are deduplicable is to actually compute their meshes and compare, and that will tank the splitter's throughput. The splitter today is fast precisely because it's working at the line / id level, not the geometry level.

So the natural place to deduplicate isn't the splitter, it's a post-pass on the generated .frag files. Idea:

• Splitter runs on the IFC at line speed, produces N .frags, doesn't care about geometry equality across them.
• A second tool walks the generated .frags, hashes each representation's vertex/index buffers, and merges duplicate representations across files (rewriting the representationId references to point at the canonical one).

That's much faster than doing the same work upstream because:

• The geometry is already meshed; no parser to run.
• It works on .frag pairs even when the source isn't an IFC.
• It can be incremental: hash once, cache, run the dedup whenever a new split lands.

Worth its own issue and a separate PR. The splitter stays line-level fast; the dedup is opt-in and runs when you care about cross-file grouping.

[ShaMan123]

The following is a hash I generated using the code above

  "1962263_DIA/1": [
    831338,
    1381964,
    1557176
  ],
  "1962262_DIA/1": [
    831623,
    1557383
  ],

The files in the zip are the result of extracting the ids for readability.
The ifc files exported from Tekla have a lot of duplication.

extracts.zip

[ShaMan123]

This is the ifc containing all

[831338, 1381964, 1557176, 831623, 1557383]

extracted.ifc.zip

[ShaMan123]

Worth its own issue and a separate PR. The splitter stays line-level fast; the dedup is opt-in and runs when you care about cross-file grouping.

I will want to be able to add indexes at the dedup level if we are handling geometry there