Refactor the BlenderBIM Add-on to perform incremental editing

[Moult]

No new IFC export functionality is planned until this task is complete. Also, no new release will be made until this is fixed.

This is a tracking issue for the general goal to migrate the BlenderBIM Add-on code to become an incremental editor. It'll take a while until this is closed.

In short, the BlenderBIM Add-on currently functions primarily as an importer and exporter. In other words, it parses the entire IFC dataset atomically, then imports it into Blender. During export, it does the inverse, it parses the entire Blender scene atomically, then exports it into IFC. This creates problems:

1. Lots of data loss. The Blender data model will never fully match the IFC data model. We do a damn good job already (i.e. visibly better to a user compared to proprietary authoring apps), but we're still missing 30% of the spec.
2. Long, very inefficient import / export times. So long as this is the case, it becomes a pain to use IFC as a source of truth. If this issue is not solved, it also impacts drawing generation efficiency (see Make construction drawing generation like, really, really awesome #1153). Drawings are IFC based, and if we cannot very rapidly and continuously export and access the latest IFC data, that slows down our ability to then generate a drawing from it. It also impacts parametric geometry support (half solved, now). Drawings is one example, but the same applies for clash detection, COBie, and everything else.
3. Lack of code modularity. This creates an unscalable maintenance nightmare, as the entire BIM model is treated atomically.
4. Very difficult to scale to collaborative editing paradigms - with this new approach, users can only partially load what's interesting, Blender can more easily handle huuuuuuge models, and theoretically work across many apps, and changes even versioned logically without expensive reverse engineered diffs ... this new approach will make it possible in the future to implement multi user editing, not just within Blender, but across other applications too.
5. Lots of operations cannot be made headless or authoring application agnostic - more and more experience with IfcPatch, IfcSverchok, and seeing what's happening in FreeCAD demonstrate that there is a huge amount of double-up. With an incremental editor, editing is done via ... well, IFC editing! Not translation, so the code can be reused across FreeCAD, NewCloudBIMApp, and so on.
6. Difficult to write test cases for, as import / export is atomic and depends on Blender. If you can't test it, you can't trust it.
7. Difficult to onboard new potential developers. Due to the atomic nature of the import / export process, developers will never be able to be sure of the scope and dependency flow of their changes. I've already tried my best to decouple the export code as much as possible, and those familiar with the existing code will realise that I've already decoupled parsing the Blender scene from the actual IFC authoring, so although it looks monolithic and scary, it should be able to be broken down quite easily and also make it easier for future devs.
8. Errors are only discovered at export time. This can lead to a very slow, ambiguous, and annoying cleanup process, and requires the user to have previous knowledge of what the exporter expects.
9. It discourages using dedicated UX/UI for BIM aspects that don't translate well to a Blender interface. For example, spreadsheeting, mass data manipulation, resolving change conflicts, paperspace... this new approach will allow many dedicated tools to augment Blender's capabilities, rather than trying to force everything into the Blender paradigm. This will allow other applications to do what they do best, and Blender to do what it does best, and they can both work together.

For the BlenderBIM Add-on to become the best IFC authoring application, it has to only not only operate using the IFC data model, it has to do it live, decoupled from the full Blender scene. This is what is meant by "incremental editing", or partial editing. In short, the IFC dataset, whether it is atomic or a subset of a greater central repository, shall be held in memory (in a file scenario), or if not, accessible via a standardised API. The import process therefore does not need to theoretically translate the entire model into the Blender environment. It can only partially import / visualise / display what's needed. Then, actions in Blender will edit the IFC dataset on the fly. The export process then does no IFC processing whatsoever, and just turns into a disk write operation (in a file scenario - in a central database scenario, there are options too, but let's keep this simple for now). This means faster imports, ultra-fast exports, no data loss, and the Blender editing functions naturally then segment themselves into modules - a module which only edits geometry, a module which only edits materials, a module which only edits documents, and so on.

There are disadvantages to this approach:

1. It's a lot of work, and a lot of rework. Basically all the current functionality needs to be rebuilt. (Well, theoretically, mostly refactored, but still a lot of work)
2. Some UI aspects in Blender may need rethinking. E.g. the simple action of duplicating an object now may create complications. Another example is when the user presses undo. The UX needs to make it clear that Blender is not the BIM model ... Blender is simply a client through which it can view and modify portions of the BIM model.

The good news is that the initial attempts to move into this direction are already positive. The new full native roundtripping feature effectively uses this approach already very successfully. With that option enabled, there is zero geometry data loss. Cool, right? What if that was everything? Also, after commit c844076 with that option enabled, you can edit parametric geometry, and export blazing fast.

Modules which are basically done (can always be improved, but enough that it works)

• Aggregates (medium)
• Attributes (medium)
• BCF (easy)
• BIMTester (easy) (thanks @ihabelaghoury !)
• Clash (easy)
• Classification (hard)
• COBie (easy) (thanks @ihabelaghoury !)
• Contexts (medium)
• Constraint (medium)
• Covetool (easy)
• CSV (easy) (thanks @ihabelaghoury !)
• Debug (easy)
• Diff (easy) (thanks @ihabelaghoury !)
• Document (medium)
• Drawing (easy, apparently? Note: does not address Make construction drawing generation like, really, really awesome #1153 at all - only prevents regressions)
• Geometry (very hard)
• Georeferencing (medium)
• IfcSverchok (easy)
• Material (hard)
• Model (medium) (known regression for drawing support - will reimplement in future release)
• Owner History (medium)
• Patch (easy)
• Presentation Layers (medium)
• Project (medium)
• Pset (hard)
• PSet Template Definitions (easy)
• Qto utilities (easy)
• Root (medium)
• Search (easy)
• Spatial (easy)
• Style (medium)
• Type (easy)
• Unit (easy)
• Void (medium)

Modules / functionality that we will delay implementation until a future release

• BIM Server Library (medium) (minimal existing implementation, likely never used by a single user)
• Group (medium) (minimal existing implementation, so user won't miss anything)
• Structural (very hard) (minimal existing implementation, so user won't miss anything) (ping @ihabelaghoury )

Watch this space.

[PlumpMath]

Moult cheer up. We're gonna be on this planet.
From the unknown cave hive to the hacked free space.
yes~
Happy new year~~~ everybody thanks. :))