Web-based UI integration with Blender

[Moult]

Outline

The BlenderBIM Add-on extends the Blender interface with a lot of buttons, dropdowns, and more to interact with 3D models. However, in many scenarios, the Blender interface is not suitable for the type of interface we want. For example, we might want to show a gantt chart, or spreadsheet interface, and it's really difficult to do this with Blender's UI capabilities.

This project will aim to create a web UI that will run alongside the active Blender session. The web UI needs to be able to grab data from the current Blender session, and ideally even run operators in Blender from the web UI.

Details

Running a web interface alongside Blender could be tricky. It may involve setting up a local web server (e.g. with something minimal like flask or werkzeug) and mapping calls to that server to executing Blender operators.

We'll start by doing a simple proof of concept where the web UI displays data that Blender knows about (e.g. an objects name), and then pressing a button in the web UI which then causes Blender to run an operator.

Then, we'll progress to showing a simple HTML table of tabular data. You will be provided with the tabular data and how to access or generate this tabular data in Python in Blender.

If time permits, a gantt chart may also be generated. There is existing code for a gantt chart that is statically generated, so this needs to be modified to update dynamically depending on Blender activities.

Expected Outcome

1. A web UI that shows data from Blender
2. A web UI that lets you trigger operators in Blender
3. A prototype that shows tabular data in a web UI

Future Possibilities

Project Properties

Skills

• Python, including how to setup a basic local web server (as minimal as possible - no Django!)
• Blender Python API (may be learned on the job, it isn't too tricky)
• Javascript (only basic Javascript, we will not be building complex web applications)

Difficulty

Medium

Size

Medium or Long

Additional Information

• Potential mentor(s): Dion Moult
• Organization website: https://blenderbim.org http://ifcopenshell.org
• Communication channels: https://osarch.org/chat

[bdamay]

Hi Dion,
i would be glad to give a try on this one, if i manage to free enough time for it. 1(75h might be too much to dream of)

That said i never coded on blender nor blenderbim yet, and can't really appreciate the "trickyness" of hosting a webserver within blenderbim without crashing the whole app.

But first things first, I have a few questions to check if my understanding of what you aim for is correct.

• I understand a webserver is initiated from the BlenderBim Add on, am i right ? (button that launch a local server and open a local browser) ? (is fastapi an option ? ).
• I understand browser is feeded with blenderbim data with (sort of) api http requests (am i right ?)
• shall it be synced real time with websockets for example or refreshed from time to time or on user action?
• I guess server should be an API rest server with JSON requests ans response for transferring data and posting requests from client to blenderbim ? right ?
• Client part: Is using a third party javascript library like Vue.js is an option ? (may be not at first but later ?)

Have a good day, (night?)
Benoit

[Moult]

1. One option is to launch the webserver from Blender. An alternative approach is to create a separate app that runs alongside Blender.
2. Yes, this is for you to help figure out.
3. The appropriate sync method is for you to discover / test during this project :)
4. That sounds like a good starting point!
5. Later, sure. To begin with, this is far more low level than Vue.JS and should be agnostic of Javascript framework.

[Manideep-Maddileti]

Hello @Moult This project seems to perfect to me for GSOC'24.I have all potential skills for this project.Can you please guide where to start and please assign some good first issues

[Moult]

@Manideep-Maddileti g'day! GSoC 2024 has not yet started and we don't know yet if the organisation is approved (ping @brlcad ) - we're just collecting ideas. If you're keen anyway a good starting point would be to fire up Blender and start experimenting with options and get familiar with how their Python is bundled. Very little Javascript is expected.

[Moult]

Hey @Manideep-Maddileti ! The organisation has been approved so now is an excellent time to make a start. The checklist is here: https://opencax.github.io/gsoc_checklist.html and you can reach out at https://osarch.org/chat

[prashant226]

Hi @Moult,

I'm Prashanth D, a frontend developer with skills in HTML, CSS, JS, React, Python, and C++. I'm particularly interested in web-based UI integration with Blender projects. Coming from a mechanical background, I also have substantial experience working with CAD software, including SolidWorks and AutoCAD.

Excited about the prospect of discussing potential collaborations or projects in this space. Looking forward to connecting!

Best,
Prashanth D

[Moult]

@prashant226 g'day! The next best step is to start getting familiar with Blender, write a draft application, say hi in the chat, and follow the checklist :)

[prashant226]

@Moult thank you, can please provide me the chat group link.

[Moult]

https://osarch.org/chat as linked above

[marwiss]

Ping @Moult
@bdamay wrote:

But first things first, I have a few questions to check if my understanding of what you aim for is correct.

• I understand a webserver is initiated from the BlenderBim Add on, am i right ? (button that launch a local server and open a local browser) ? (is fastapi an option ? ).

I can also recommend FastAPI for the project, because it is the same API framework as used in this project: #93 There could be synergies from using the same framework in both projects.

FastAPI is fast, modern, lightweight, stable python API framework, quickly gaining popularity for NoSQL and microservice API projects.

[Vanuan]

I've done something similar for FreeCAD:
https://github.com/Vanuan/FreeCADjs
https://forum.freecad.org/viewtopic.php?t=48611

I've used Flask and GraphQL to display a simple settings form. It didn't get much interest from the community as FreeCAD is already spread thin between C++ and Python.

The challenge was to start/stop the server as a part of a desktop application. I didn't solve it, I just used FreeCAD as a library.
Another challenge is to generate REST/GraphQL API from python API. I guess now AI tools could help with that. But if you just building a POC, some simple action/data would be enough.

I would approach it this way:

1. Investigate start a websocket server from Blender. I think you can build an addon. Quick search suggests this has been already attempted: https://github.com/KoltesDigital/websocket-server-for-blender/blob/develop/websocket_server.py (uses ws4py package, there are multiple other packages, e.g. websockets and Flask-SocketsIO)
2. Investigate starting another server as a part of that addon, this time http that would serve a React app.
3. Learn how to connect to websocket from a React app (or ditch React and just do plain DOM API).
4. Create a build pipeline that would package and install a Blender addon, build the html app and run Blender with addon started.
5. Now, that you created a boilerplate, you can experiment with executing python scripts in Blender context.
6. Implement a message to request a simple piece of data, e.g. BlenderBIM version.
7. Implement a message to execute a simple action, e.g. open or create a file. At first it could be a simple local file path, later could be extended to upload files from web interface to some temp directory. Potentially, could be stored inside the web browser on the client side.

But this is just a technological side, a demo of possibilities. First, you should think about a real useful use case. Otherwise this project would be stillborn. What can be done using web interface more effectively than from a desktop application? Just one thing comes to mind: collaboration and sharing. Gantt chart sounds like a good idea, but it already exists. I would try something in the exporting space. E.g., save IFC files to cloud storage, display drawings, export PDFs. To truly leverage the power of websockets and do something useful, one could implement a remote backup server that would backup IFC files at regular intervals and display the backed up file using IFCjs/web-ifc.
This way you could stream your desktop edits, making an impressive demo.