Updating readme on package indices

[Andrej730]

Noticed that PyPI description looks kind of empty and unofficial. I've found that there is already fix for that for PyPI (f0e4397) but PyPI wasn't updated since then.

Similar thing with docker, conda and conda-forge

[Moult]

Ping @Krande @aothms for docker/conda

[Krande]

@Moult The official conda-forge package of Ifcopenshell looks like this:

Do you want us to add the Readme file to the description for the conda package as well (assuming its possible)?

[Moult]

Yes I guess if possible all official packages on channels should share the official README :)

[Moult]

Huh, I've tried a few times and no idea what I'm missing to get PyPI happy.

[Moult]

OK I think I've figured this out. It turns out in the case of multiple download packages (e.g. many wheels), PyPI only considers the FIRST uploaded wheel for metadata like project readme... NOT the last / most recent wheel which completely caught me off guard.

This is a problem because historically we only have one "version" of 0.7.0 on PyPI, with a "rolling release" of wheels. (haha get it)

In any case I've adjusted it to now release 0.7.0.YYMMDD suffixes so now we can have proper versions and end-users can also lock to a particular YYMMDD release if they want.

However then I ran into this incredible bug with PyPI pypa/packaging-problems#74 where if you delete a file on their web dashboard it creates a situation where you can never use that filename again (unless you do some manual packaging hacks or something supposedly). So let's wait until UTC 18th Feb where we'll get a fresh release and hopefully everything will be fixed tomorrow.

[Moult]

Fixed. Finally.

[Andrej730]

Shouldn't we keep it open for docker and conda readme? 👀

[aothms]

I just copied our readme and desc to the docker hub page.

[Moult]

Ping @Krande

[Krande]

I don't have access to directly modify the conda-forge package of IfcOpenShell, so we'll have to fix that somehow in the next release on conda forge. I did however update the daily IfcOpenShell conda package description. Unfortunately the conda description field does not support markdown tables, so I had to convert the tables from markdown to html (with a little help from chatgpt :) ) from the README.md

[Moult]

Bump @aothms :)

[aothms]

I don't think there's anything I can do here?

[Moult]

I don't have access to directly modify the conda-forge package of IfcOpenShell, so we'll have to fix that somehow in the next release on conda forge.

^ @aothms is this not you? Or is it someone else?

[Krande]

@Moult it's just the way conda forge operates I believe. No one has admin access to conda forge packages on anaconda.org except the conda forge maintainers. Package descriptions are meant to be updated by users using their CI interface on github.

But I can issue a new conda forge release later today and try to see if I can update the readme!

[Moult]

Bump @Krande :)

[Krande]

@Moult ! Hey, thanks for reminding me :)

I got the readme all ready to go months ago, but I've had a really hard time with getting conda-forge to compile the latest releases of IfcOpenShell on linux. From what I can tell It seems the ci linux workers being used by azure pipelines (which is what conda-forge uses) AND github actions for that matter seems to regularly fail due to insufficient memory during the compilation.

I've started another build on conda forge hoping the workers might have made some stability improvements :)

[Krande]

@Moult and @aothms

Unfortunately, it seems that memory limits on the linux CI runners on both github actions and conda-forge (azure pipelines) are a limiting factor on our conda package deployments. Failing linux conda compilation jobs has for some reason become the normal.

Are there any things we could consider for the v0.8.0 release to reduce the memory footprint of our compilation? Say, for example if it is possible to split up the cmake compilation steps?

One thing that has crossed my mind is if it would make sense to split up the different operations in our current CMakeLists.txt into multiple *.cmake files. That way we have the option to create a separate CMakeLists.txt for conda that can produce separate builds or split the compilation somehow.

Would something like this make any sense to you?

- root/
   - cmake/
       - CMakeLists.txt
       - usd.cmake
       - gltf.cmake
       - wasm.cmake
       - opencascade.cmake
       - collada.cmake
       - hdf5.cmake
       - msvc.cmake
       - utilties.cmake  # various supporting functions 
       - etc.. 
   - conda/
       - CMakeLists.txt
       - meta.yaml

In our main cmake/CMakeLists.txt file we would then only need to include all the different *.cmake files (using the include("utilities.cmake") command) to compile everything as we do today. And in the conda/CMakeLists.txt we are free to either split up the conda ifcopenshell package into ifcopenshell-schema, ifcopenshell-geom etc by simply including the functions in the order we want to include them.

Any thoughts?

[aothms]

Are there any things we could consider for the v0.8.0 release to reduce the memory footprint of our compilation?

We should look at how the schema is compiled. Most other C++ libs do not have these monolithic schema includes, e.g https://github.com/ifcquery/ifcplusplus/blob/master/IfcPlusPlus/src/ifcpp/IFC4X3/include/IfcAdvancedFace.h

Breaking it up into small parts greatly reduces memory footprint, but at the expense of compilation time. It'll be much slower, because all the shared headers need to be constantly reevaluated. We should look into precompiled headers.

We should also revisit the latebound schema information https://github.com/IfcOpenShell/IfcOpenShell/blob/v0.7.0/src/ifcparse/Ifc4x3-schema.cpp Maybe this shouldn't be supplied as code, but as data (either a payload, or an external json file). Compilation adds no performance gain for this kind of data, but compilers don't like it.

[Krande]

Hey @Moult: I've successfully updated the conda-forge package for IfcOpenShell, meaning the README file is updated at https://anaconda.org/conda-forge/ifcopenshell.

@aothms I think I've also fixed the reoccurring failed github actions workers for the daily conda builds by reducing the number of IFC SCHEMA we compile for.

Basically I reduced the number of schema in our conda builds from:

"2x3" "4" "4x1" "4x2" "4x3_rc1" "4x3_rc2" "4x3_rc3" "4x3_rc4" "4x3" "4x3_tc1" "4x3_add1"
(which is the default)

to

"2x3" "4" "4x1" "4x3" "4x3_add1"

Now the build times are significantly reduced and it seems the memory usage is no longer breaking the ci runner caps.

Then we might be able to close this issue?