Complete the Bullet C-API (was googlecode Issue 43)

[erwincoumans]

See https://code.google.com/p/bullet/issues/detail?id=43&colspec=Modified%20ID%20Type%20Stars%20Status%20Owner%20Summary

[kyle-emmerich]

Hey Erwin, any updates on the C API? I'd like to use Bullet in my pure LuaJIT 3D game engine and this means we need to bind with the C API with FFI to avoid the slower actual Lua API bindings.

Thanks!

[Tal500]

Hey Erwin! (and every one)

First, I think that Bullet is great!
I don't have experience with Bullet so sorry for my ignorance.

I would like to support Bullet C API, since as said before, it is the most important feature if we want portability with other language (I do :-) ).

Since you tagged this in milestone Bullet 3.00, I test the latest C-API(of ngb...@gmail.com) in the Bullet3 repo.

First I found this tree possible bugs:

• Bug in Bullet3? : function btCollisionWorld::getCollisionConfiguration() doesn't exist in Bullet3
• Bug in C binding: field btGjkPairDetector::ClosestPointInput::m_stackAlloc doesn't exist, used in plNearestPoints. I've checked this and this applies both for Bullet2&3.
• A placement new() operator is called after btAlignedAlloc(), but no destructor gets called before btAlignedFree(). It's not the correct behavior according to C++ rules[1]

Question that I have:

• Does Bullet ever "steal" pointer ownership? I.e. is Bullet ever delete pointer it took from user? It has a huge impact for the Create&Destroy functions implementation.

After those global points, I would like to describe the current status, why it's not good and my alternative.

== Current Status - OpenPL-like library ==

1. The C wrapper is different than what a C++ API.
   The description in header for function "plNewBulletSdk()" says:
   //this could be also another sdk, like ODE, PhysX etc.

It sound nice, OpenGL-ES-like idea, every one could implement the library differently.
Why I think it's bad? Two reasons which is actually the same:

• This is not a C wrapper, as the same way the OpenGL isn't a C wrapper.
• It's give C developers far less access to Bullet API than C++ developers. When initializing Bullet, there are many options in the constructor and in member functions. This is only one example for many features in this wrapper.

1. In my opinion, "pl" prefix isn't good for Bullet wrapper, unless we choose the OpenPL idea.
2. It's not trivial to match between the C function to the Bullet C++ function. Sometime this function call more than one function.

If someone want to make an OpenPL-like idea, it should be separated from Bullet C wrapper.

== Alternative - "real" C wrapper ==

I think that bullet should look like the "standard" style of API. I took ODE for example, but actually it's the same as most wrappers(for every language).

Notice that the numbers matches to previous numbers.

1. The C API should be as much as powerful as C++ API.
   The idea is that should be a one to one match between those two, except to cases when it's impossible or not beneficial on the other language(C). In my opinion, it's a definition to a good C wrapper.
   Each function in the C wrapper should either call the matched C++ function, or do the same(e.g. optimized mathematics functions/macros).
2. In my opinion, a good prefix is "cbt" or "btc"(not to be confused with Bitcoin) but this is the least important issue.
3. As same as ODE, function and structs should have the following match:
   C++:
   class btSomeClass {
   void foo();
   }
   C:
   btSomeClassFoo(btSomeClassHandle someClass);

Believe me, it's easier for everyone:

• C wrappers which maintain this
• Wrappers for other languages
• The simple, end-user non-C++ developer.

== Summery ==

I know I might put a bomb after people hard work, but I only wish the best for Bullet.
I really want to hear your opinion about this, before I start something unofficially.

BTW: Sorry for my English.

• Tal

[1] See the example for placement new in: http://en.cppreference.com/w/cpp/language/new

[resttime]

I'm interested in seeing this C API come together (as someone who might write a wrapper) and I think @Tal500 makes good points. I'd rather see the C API designed like that.

[Tal500]

@resttime thanks for your comment!

Lately, I was very short in my free time, so my current result are not very huge.
However, I would like to post it here to hear your opinion, if I'm going at the right direction at all.

Here is my work repository:
https://bitbucket.org/tal500/bullet-c-proposal/src

The first goal is to have a fast and convince C API as possible, for the Hello World example (work in progress).

Notice that macro helpers are meant to be used widely in the CPP files, the header should be easy to understand.

I know I need to fix alignment to be 16-byte aligned, and the Vector3 is probably not the smartest thing, should change it to Vector(4 components).

Any suggestion would be appreciated.

[erwincoumans]

I am still figuring the new Bullet 3.x API out, let alone a C-API. Note that Bullet 2.x is moving all into Bullet 3. I did one experiment for a C-API file in Bullet 3 here: bullet3\src\Bullet3OpenCL\Initialize\b3OpenCLUtils.h
Please have a look at that file, it supports both C++ and C.

[resttime]

@Tal500
Nice work on it so far. I can't really give any suggestions without any experience writing a C API for C++, but a cursory glance at the code generates a positive view for the direction the C API is going in while looking at BulletCTest.c

@erwincoumans
Take as long as you need! I'm glad that the C API will be worked on and hopefully done before/when bullet3 comes out. This will be very important to opening avenues for this excellent physics library to enter other programming languages. Many will jump at the chance to use bullet3 as an option when the C API is available.

[Tal500]

First, I want to thank you for your review and comments! It's really motivate me.

I'm glad to announce I've complete to wrap a very minor API in order to complete Hello World test/example. it's working! (tested with bullet-2.82-r2704)
(It's located in the same repository link I posted before)

I have tried to push down restricted pointers, macro, and inlining in purpose, to see how far it could work, and I think it worked well. Some could be dropped if you decide it too complex.
The code is not organized nor documented well yet however.

There are many places in the code that I thought to myself - why we need structure duplication? It's hard to maintain, bad in converting performance, and a source for bugs!
Therefore, I thought to myself, why not go to the direction Erwin suggest?

Bullet could support the entire C API directly, in its headers, everything with the "bt"/"b3" prefix, no need for "cbt", like at my header.
There are 3 level of structures:

• Basic structures (e.g. vector, matrix, transform, ...)
• Simple structures (e.g. btDefaultCollisionConstructionInfo)
• High-level object (all "classes")

As same as Erwin suggest in b3OpenCLUtils.h, for example, the header could look like this:

// Example header

#include <btVector.h>
...

// C&C++ shared structures
typedef struct {
    btScalar num;
}
AConstructionInfo;

// C++ special defs
#ifdef __cplusplus
class btA {
    A(const AConstructionInfo& info);
    ...
    void foo(const btVector& some_vector, const btOtherCommonStruct& something);
    ...
};
#else
BT_DECLARE_HANDLE(btA)
#endif

// C wrapper
#ifdef __cplusplus
extern "C" {
#endif
// ~~~ All btA creation&destruction funcs ~~~
// ...
void btA_foo(btA* handle, const btVector* some_vector, const btOtherCommonStruct* something);
// ...
#ifdef __cplusplus
}
#endif

Finally, I have two suggestion about changing API that would fit both C and C++:

• Omit the "m_" prefix from basic and simple structures, for convenience, but it's a matter of opinion.
• Mathematical structures should support plain C, including all alignment&SIMD optimization. If not in C++, the member function wouldn't be available.
• The should be a structure, btVector, which would be the base class for btVector3 and btVector4. It's not convenient to use btVector3 with 4 fields in C, just because there is no inheritance in C.

Future thinking - custom C sub-classing and overriding with vtables?

What do you think? What C features from here do you want in Bullet3?

[erwincoumans]

I don't have much time to digg into this further at this moment, but I'll get to it. For the math C-API, it would be helpful to stay compatible with OpenCL when it makes sense to do so. In OpenCL there is a float4, and there is some b3Float4 for compatibility, see src\Bullet3Common\shared\b3Float4.h. The idea is to store common structures can be used in C++, C and OpenCL in 'shared' folders. Again, I need to think more about this and your feedback/thoughts are welcome.

[Tal500]

Fair enough.

I see that B3 API is far from being stable, so I won't try making them C-compatible yet.
Since you're working on higher priority tasks which necessary for Bullet3, I think to fork Bullet3 and make the lagecy BT API compatible with C, without touching(yet) the B3 API.

This would allow to users to use this fork of Bullet if they want to use it in the lagecy API.
When the time is right I think it should be merged with master branch.

What do you think?

[Tal500]

Erwin, as I said before, I've forked bullet3 for embedded C binding:
https://github.com/Tal500/bullet3

Right now, I'm focused on the old Bullet2 API, since the B3 API isn't stable yet.

I'm currently working on LinearMath, making mathematical structures the same between C&C++.
I did thought about src\Bullet3Common\shared\b3Float4.h, and I think that a b3Float4 adapter isn't the best approach. I don't claim the structure is useless, I even suppose it's more natural for OpenCL code, but shouldn't be treated as C adapter.

The idea is that ALL mathematical structure is shared between C&C++. If the user is in C++, she can call function member and language features. See my src/LinearMath/btVector3.h for example.

And about the simplified structures, such as b3Float4, it can be easily retrieved from btVector:

btVector4 some1;
b3Float4 some2 = some1.m_floats;

And the opposite in a similar way.

The code is far from being finished, but would you consider such approach in b3?

[chino]

I really hope bullet 3 api will have a supported c api.

Lack of a stable supported fully featured wrapper has been biggest slow down with using Bullet in my non c++ projects.

Is the bullet 3 api going to be able to downgrade back to the cpu if the video card card isn't capable enough?

Thanks for all the great work.

[AndresTraks]

I've also been working on a C API as part of a P/Invoke .NET wrapper. It's pretty much complete for bullet 2 and I've planned to start with bullet 3 this summer. https://github.com/AndresTraks/BulletSharpPInvoke/tree/master/libbulletc

It's a bit different though as it's first and foremost designed to be an intermediate layer between bullet and other higher level wrappers instead of being a natural C API. The code is completely independent from bullet, so nothing needs to be modified in bullet itself.

Here's the idea of how it currently works:

//btCollisionShape_wrap.h
#ifndef BT_COLLISION_SHAPE_H
#define btCollisionShape void
#endif
EXPORT void btCollisionShape_delete(btCollisionShape* obj);

//btSphereShape_wrap.h
#ifndef BT_SPHERE_MINKOWSKI_H
#define btSphereShape void
#endif
EXPORT btSphereShape* btSphereShape_new(btScalar radius);
EXPORT btScalar btSphereShape_getRadius(btSphereShape* obj);

//btCollisionShape_wrap.cpp
#include <BulletCollision/CollisionShapes/btCollisionShape.h>
#include "btCollisionShape_wrap.h"

void btCollisionShape_delete(btCollisionShape* obj)
{
    delete obj;
}

//btSphereShape_wrap.cpp
#include <BulletCollision/CollisionShapes/btSphereShape.h>
#include "btSphereShape_wrap.h"

btSphereShape* btSphereShape_new(btScalar radius)
{
    return new btSphereShape(radius);
}

btScalar btSphereShape_getRadius(btSphereShape* obj)
{
    return obj->getRadius();
}

//bulletc.h
#include "btCollisionShape_wrap.h"
#include "btSphereShape_wrap.h"

//test.cpp
#include <bulletc.h>

btSphereShape* shape = btSphereShape_new(1.0f);
btScalar radius = btSphereShape_getRadius(shape);
btCollisionShape_delete(shape);

[Tal500]

AndresTraks,
As you might see, I aim to have C API embedded in Bullet 3.
For doing this, the big challenge is to make LinearMath library of Bullet compatible with C, with no loose of optimizations. (Actually, I theoretically improved optimization since I use a generic "restricted" pointer specifier when possible

After this, the C API might be easily written in full Bullet 3 API.

Do you think we should combine our efforts?

[AndresTraks]

Yes, I can surely help. My interest is that Bullet could be compiled into a .dll/.so library with the C symbols exported. This is a standard way of writing wrappers for other languages like C# and Lua.

[ebkalderon]

Bump. Any progress on this since the last update? I am very interested in writing up-to-date Bullet bindings for Rust, which would require a C FFI.

[Tal500]

Sadly I'm very busy these days, but I didn't forget it, just need some time and help.

As said above, I focused on first supporting LinearMath before the rest of Bullet.
I have completed the API for Bullet vectors, working on the rest.

You might use this, and extend this binding to the Bullet API function in your project, and send your implementation for updating the C API.

I would like however some help on finishing LinearMath implementation, contact me if you're interested.

[Tal500]

Hi all,

I've completed basic Bullet 2 LinearMath C API.

Now I want to write embedded C API for Bullet 2 Core(collision&dynamics). By embedded I mean that the function decelerations are in the original header, and their implementation are in the original source.

(AndresTraks especially, I want you to answer this :)
How do you recommend me to merge AndresTraks work there? Where to start?
Does any one want to join?

[bmcorser]

@ebkalderon ☝️

[Tal500]

After some serious thinking, I decided to pause my effort on the Bullet fork with embedded support, until some merging or decision from official Bullet.

As I described many before, I finished making LinearMath compatible with C. I thought it was crucial if we want C API inside the official Bullet API. If ODE did only C API, Bullet can do both C and C++ API.

Now, after this LinearMath C API infrastructure, writing a C API in the same headers should be not difficult. I'm not going to do so if it won't be merged, or a strategic decision will be made.

Even known it's only for Bullet 2 right now I still think it's important, since it's the widely used version right now.

Wait for your response.

[CapsAdmin]

@Tal500 Maybe you should make a pull request and propose a first step towards a C api.

[Tal500]

@CapsAdmin, as you suggested I created this pull request: #636

[rweichler]

@kyle-emmerich @erwincoumans just FYI, im working on a C wrapper, with the end goal to be used with the LuaJIT ffi:

https://github.com/rweichler/bullet_luajit/blob/master/bullet_wrapper.cpp

[erwincoumans]

Thanks for sharing the various projects and thoughts about a Bullet 2.x specific C-API. We have pybullet python bindings for bullet3/examples/SharedMemory/PhysicsClientC_API.h and it would be straight-forward to add Lua bindings the that C-API as well. The move to Bullet 3.x and 4.x is going to happen, and don't want to extend the life of the 2.x API, even if that is the current most active version. Let's keep this issue open for discussion, some people may like to use the Bullet 2.x specific C-APIs.

[EvilPudding]

How is this going along? Is there any viable C API nowadays?

[Tal500]

EvilPudding, I stopped the work since Erwin didn't accept it. He said(as much as I understand) that the new C API will be part of Bullet3 API.

[EvilPudding]

that the new C API will be part of Bullet3 API

What does that mean? And is it available right now?

[erwincoumans]

Yes, the new C API is here:
https://github.com/bulletphysics/bullet3/blob/master/examples/SharedMemory/PhysicsClientC_API.h

It is used in PyBullet, see the quickstart guide here.
You can also use the C++ API on top of this C-API. As you can see, it is more high-level.
This is also available as 'BulletRobotics' using cmake (in Extras).

[EvilPudding]

@erwincoumans Thanks a lot!

[Tal500]

Hi Erwin
It seems like this C API has two major disadvantages:

• Is it following Singelton design pattern? I.e. only one global world.
• User pass handlers, but no user_data approach.
  I mean, you can have a callback foo(args...,void* user_data), so handlers can get data locally, not by global memory.

[EvilPudding]

@Tal500 The good thing about that it's backwards compatible, meaning you can pass a function callback whose signature does not receive the user_data, libs like gtk even encourage this.

[erwincoumans]

@Tal500 There is no global world: you can create multiple physics clients/servers and each server has its own world. I highly recommend trying to prototype in Python/PyBullet and try out the API.
You can set user-data if desired, for each link. This API doesn't use callbacks: you create one or more physics clients/servers, and you create objects, step the simulation, query for transforms, all using unique ids for each client/server.

[Tal500]

@erwincoumans I didn't understand the API at first, but now it's look great!

[EvilPudding]

@erwincoumans Why is the C API you linked in an example and is not installed by default? Do we have to copy the example API for each project and update it manually every time there's an update?

[EvilPudding]

I don't even know how to compile this... I tried to use the cmake file of the SharedMemory example, and I get the following error:

... bullet3/examples/SharedMemory/plugins/pdControlPlugin/pdControlPlugin.cpp:8:10: fatal error: LinearMath/btScalar.h: No such file or directory
 #include "LinearMath/btScalar.h"

Why does this file assume it is in the src folder? Am I building it wrong?

[EvilPudding]

I would thank someone who would walk me through how to get a compiled library so I can use the C API? Why is this so tangled?

[erwincoumans]

@EvilPudding
cmake should build the BulletRobotics library out-of-the-box, which includes the C and C++ API. It will be installed using the INSTALL_EXTRA_LIBS option.

App_RobotSimulator and App_HelloBulletRobotics are two example of using the C and C++ API. Both should build out-of-the box as well, using cmake.

For Linux and Mac, use the build_cmake_pybullet_double.sh as an example. For Windows, see the build_visual_studio_vr_pybullet_double_cmake.bat in the root of Bullet. Alternatively, premake also builds BulletRobotics library, with C-API.

[EvilPudding]

Thanks for the help, I'm sorry if this was obvious stuff, it's just it's a bit different from other libraries I've used before. Thanks for all the hard work.

[EvilPudding]

Ok, here's where I got it wrong, my package manager has an older version of the library which does not contain the C api... Sorry about that. Now I see the C API does get installed in the default folder.

[EvilPudding]

@erwincoumans I understand if you don't have the time to answer, but I'm trying to compile a C file using the API, and I can't seem to link it with libBulletRobotics.a

If I just link it, cc outputs a bunch of undefined reference to vtable ..., and if I try to include it as a shared library using -shared -fPIC I get:

/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../lib64/libBulletRobotics.a(LoadMeshFromCollada.o): warning: relocation against `_Z17readNodeHierarchyPN8tinyxml210XMLElementER9btHashMapI12btHashStringiER20btAlignedObjectArrayI23ColladaGraphicsInstanceERK11btMatrix4x4' in read-only section `.text'
/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: /usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../lib64/libBulletRobotics.a(PhysicsServerCommandProcessor.o): relocation R_X86_64_PC32 against symbol `gVRTeleportPos1' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-pc-linux-gnu/7.3.0/../../../../x86_64-pc-linux-gnu/bin/ld: final link failed: Bad value

Again, I don't want to be a bother, so if you don't have the time, I understand if you don't answer.

[EvilPudding]

Turns out I needed to add -DBUILD_SHARED_LIBS=ON to the cmake options. Though the following symbols are still missing:

b3CreateInProcessPhysicsServerAndConnect
b3CreateInProcessPhysicsServerAndConnectMainThread
b3CreateInProcessPhysicsServerAndConnectSharedMemory
b3CreateInProcessPhysicsServerFromExistingExampleBrowserAndConnect3

I tried to search for them in the elf information in the installed libs and can't find where they ended up. If I don't use those functions it looks like it works fine.

This is a great library, and I'm very thankful for all the work put in here, thanks @erwincoumans

[erwincoumans]

cmake App_RobotSimulator_NoGUI and App_RobotSimulator link against BulletRobotics without a problem. Let's close this old issue: Bullet/examples/SharedMemory/PhysicsClientC_API.h is the API.