IDS support in IfcOpenShell

[Moult]

4e889b7 implements a very basic BIMTester integration. Ping @aothms as well as @rbertucat who this might impact, if IfcRail is testing pset applicability too.

To run, just point to the IDS XML instead of a feature file.

$ bimtester -f ids01.xml -i Duplex_A_20110505.ifc -r report.html

HTML report looks like this:

Because it just uses the same argument to specify a feature file, this means that the Blender UI to BIMTester, the FreeCAD UI to BIMTester, as well as the standalone UIs to BIMTester built by @berndhahnebach and @rbertucat all can just work out of the box.

I really like this new IDS approach. I can read the XML and it just makes sense.

BIMTester can provide additional value on top of this IDS XML spec by providing a test running interface, as well as a HTML report generator, and an xUnit compatible output (therefore, will integrated into many continuous integration apps). BIMTester also provides a nice interface for bSI standards development as you can specify a custom IFC schema prior to running tests.

As a separate concept to test running, there is the whole Gherkin approach. The biggest benefit of Gherkin is the ability for humans to understand it, and that it can be included in contracts, and read by non-technical people. At the end of the day, it still needs code to check. Right now, this is the biggest disadvantage of BIMTester. If you want to write your own check, you need to know a bit of Python + the IfcOpenShell API. I see this IDS XML as replacing the Python coding for simple checks that anybody can write who doesn't need / want to know Python or the IfcOpenShell API. And whilst IDS XML is great for general and common specifications, it will never cover everything, so allowing both approaches is great.

Possible Gherkin sentences:

 * IFC data must comply with the IDS provided in "ids01.xml"

... this is easy, but doesn't communicate well to non-technical people - defeats the purpose of Gherkin. Maybe it should exist as a template simply for techies who are prototyping new bSI standards, but definitely deserves the blank looks it'll encounter on real projects :)

 * All entities belonging to "IfcSpace" must have a property set called "PSet_Revit_Identity Data" with a "Name" property filled with one of the following values:
    | Living Room |
    | Kitchen     |

... this can autoconvert XML<->Gherkin, and is highly specific. Kinda midway - readable but contains jargon for non-techies, and extra effort for techies, since it'll probably end up being its own DSL and would need to be translated into different languages ...

 * All spaces must be either a living room or kitchen.

... this is perhaps the best from a Gherkin point of view. This looks like exactly the type of sentence that already exists in contracts. From a UX perspective, there can be a simple UI that a spec writer can use to link this arbitrary user sentence to an XML definition - or even better, reuse <specification name="">. Easy for the manager, easy for the spec writer (can use any tool, IfcOpenShell Python API optional). I personally think this would be the way to go.

[Moult]

A couple of minor implementation comments - one of the really valuable things about the way most BIMTester checks are implemented is that it doesn't just say "this failed" or "this passed". It formats it in this way:

 1. We expect foo to be bar. <-- this is the exchange requirement we want. This should always read exactly the same thing.
(Note: if foo is bar, then we move on. No details are shown. Otherwise... read on...)
The element 3621=IfcSpace('2gRXFgjRn2HPE$YoDLX3FC',#33,'B205','',$,#3611,#3620,'Utility',.ELEMENT.,.INTERNAL.,$) has a property foo with the value "baz" instead of "bar".

This way:

1. There is a consistent, unchanged sentence that clearly explains the expected exchange requirement (right now, the screenshot in the previous comment has a changing sentence, which can create confusion)
2. Only if there is an error, is complex technical stuff shown (i.e. the STEP code). The concept is that managers are interested in green, whereas techies are interested in red.
3. The error should not just say "failed", it should give useful information like "it was baz, not bar". This really, really helps with users debugging.

[aothms]

@Moult very cool! I formatted the messages, feel free to adapt to your wishes. The logger verbosity indicates error or success so you don't need to test the string content.

Given an instance with an entity name 'IfcSpace'
We expect a property 'Name' in 'PSet_Revit_Identity Data' with value 'Living Room or Kitchen'
#349=IfcSpace('0pNy6pOyf7JPmXRLgxs3sW',#1,'R301','',$,#17700,#16934,'Roof',.ELEMENT.,.INTERNAL.,$) has a property 'Name' in 'PSet_Revit_Identity Data' with value 'Roof' so is not compliant

'Living Room or Kitchen' should be 'Living Room' or 'Kitchen' ideally, but that's a bit of work that I'd like to postpone.

[Moult]

By the way, I think there would be value in building a UI to generate the IDS XML as not everybody can write XML - this can also help test out the usability and full workflow for the average BIM user. Happy to do this, but probably need some guidance on the rules (e.g. an XSD, or an explanation of what it allows).

[aothms]

Good point. The xsd is still in progress at bSI. That's when things get precise. So I don't think it'll be ready in a short time frame Sent from a mobile device. Excuse my brevity. Kind regards, Thomas Op ma 1 mrt. 2021 09:24 schreef Dion Moult <notifications@github.com>:

…

By the way, I think there would be value in building a UI to generate the IDS XML as not everybody can write XML - this can also help test out the usability and full workflow for the average BIM user. Happy to do this, but probably need some guidance on the rules (e.g. an XSD, or an explanation of what it allows). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1349 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAILWV33UIHSNNOREUXK3WDTBNFM3ANCNFSM4YKUUKNQ> .

[berndhahnebach]

guys, thats great stuff :) How do you create ids files? Are there examples around?

[Moult]

@berndhahnebach https://github.com/buildingSMART/IDS/blob/master/Development/First%20production%20release/Examples.md

Ping @aothms and @berlotti - I will potentially be digging into the guts of the new FM Handover View MVDs as well as the spreadsheetML conversions. I think this makes for an excellent usecase for writing an IDS that checks whether or not all the appropriate fields are filled out. If this goes ahead, perhaps I can use this usecase to battle test the IDS capabilities?

[atomczak]

Great feature @Moult!

Following our conversation, I would like to further develop this during GSoC'21 by:

• developing and making sure IDS checking works against new XSD (is it this one?)
• creating some IDS samples and wiki to make it easier to start for new people from AEC like myself :)
• add the BCF output of such report (isn't it linked to #1244?)
• add IDS authoring and integrate with BIMTester web UI - part of another GSoC project.

My GSoC part in blue:

I'm submitting the GSoC proposal, also available here: https://docs.google.com/document/d/15oQDhYrApuHsI1Zee5bDvSqRoUz67xHMr79hdFJUoZ8/edit?usp=sharing

[Moult]

Great proposal! I believe the XSD link you have is correct. I have also attached an old version of the XML here.

ids01.xml.txt

Comparing the old XML with the new XML examples may provide some insight on the changes made formal in the XSD.

We should also organise a weekly catchup.

BTW @aothms the BCF lib uses xmlschema to load the XML content and validate against an XSD in one go, and ships with the XSDs. I noticed with minidom validation is done manually. Any thoughts on whether it is worth switching xml libs?

[Moult]

@atomczak by the way, the webapp seems to be emerging with a Python backend, so when you get around to developing the webapp to author IDSes, if the XML generation is done server-side, then it would be ideal for it to be a Python thing too. If it is done client-side, no worries.

[Moult]

Oh, and one more thing, It would be good to investigate the ability to integrate IDS with MicroMVD's Gherkin syntax, where one would simply "call" the other. This way, it provides a nice human readable description that ties into business objectives, but the implementation still follows strict standards rather than bespoke test definitions.

[aothms]

BTW @aothms the BCF lib uses xmlschema

This looks really good. By the time I started there wasn't an XSD, I think it might be worth it. At the same time though there might be different dialects and versions (milestones) of IDS and the minInclusive etc aren't really part of the XSD schema, so perhaps the flexibility of using a schemaless approach might be beneficial for the time the IDS schema and implementations between vendors are still maturing.

[aothms]

IDS and the minInclusive etc aren't really part of the XSD schema

I was just informed that the minInclusive etc, things are going to be part of the XSD schema (or maybe a second imported schema namespace). I think it's worth an experiment to see if xmlschema makes life easier.