Exposing C-style functions for DLL usage via PInvoke within C#, ...

[JaXt0r]

Hi lmichaelis,

thank you for your wonderful G1/G2 parser library! I'm currently developing a rebuild of Gothic in VR with Unity heavily relying on your parser library. The thing is: As Unity leverages C#, we can't just consume your C++ library, but instead need to leverage PInvoke with C-style via shared libraries (DLLs). Therefore we need to bridge your project.

I wanted to check with you if:

1. you have better ideas to expose phoenix functionality to other languages
2. you're interested to backport the bridge I'm building into the official code
3. [whatever your ideas are 😉]

A few hints about what needs to be changed if exposing your API as C functions:

• std::string / std::string_view - There is no way of using C++ features. We therefore need to transform strings into char* (e.g. via someString.c_str())
• To return char* we need to send the pre-allocated heap heap space as buffer and strcpy() it's data into the char* parameter.
• DaedalusVM uses std:shared_ptr. To exchange it with managed code we need to work with the underlying pointers. (e.g. someSharedPtr.data())

For sure, there will be more. Nevertheless, you're solution is really neat and I overcame every C++->C demands as of now.

Feel free to have a look at my implementation for further discussions:
https://github.com/GothicVRProject/phoenix-csharp-bridge

Best
JaXt0r

[lmichaelis]

Hi @JaXt0r,
as I've already mentioned in #61, I am currently considering implementing a full C-style interface for phoenix. I have already started work on a proof-of-concept implementation which you can look at in the trial/c-api branch. Make sure to set PHOENIX_BUILD_CAPI=ON during CMake configure [⚠️ This API is untested and might not even work at the moment]. Only some functionality is implemented right now though. Take a look into include/phoenix/capi/functions.h to get an idea of the feature set.

With this, I am especially looking for feedback on the getter-style functions (like px_man_name) and general layout of the API since this is the first time I am implementing a C-style interface for accessing C++-managed data. Since you have already implemented part of the API your feedback would be really valuable!

As for the VM, I have some ideas on how to properly integrate it with C. There are some problems to work out though still so here are my ideas. Parsing scripts and creating a VM from them as well as accessing its symbols and performing code execution will be very simple (even though a lot of work) to implement. The tricky part comes with all the template magic I've created to provide a type-safe implementation including runtime checks for creating instances, calling functions and registering externals. There are two ways I can think of to make this work in C.

1. Requiring manual stack push/pop for return values and function parameters

This implementation would not be very ergonomic to use since any registered external would have to manually pop its parameters off the VM's stack and push its return value onto it. Function calls could probably be implemented using a printf-like API where the return and parameters types are provided as a string and the parameter values are taken as variadics. Initializing instances would not be fully type-safe any more and might be tricky to implement correctly (currently relies on instances inheriting from phoenix::instance which can't easily be replicated in C).
Here's an example implementation:

typedef void(*PxVMExternal)(PxVM*);

PxVM* px_vm_create(PxScript*);
void px_vm_destroy(PxVM* vm);

void px_vm_register_external(PxVM* vm, PxVMExternal* ext);
void px_vm_call_function(PxVM* vm, char const* func, ...);
void px_vm_call_function_by_index(PxVM* vm, uint32_t index, char const* signature, ...);

// I am unsure about this:
void px_vm_init_instance(PxVM* vm, char const* name, void* instance);

One would use this API like so:

void my_external(PxVM* vm) {
    char const* name = px_vm_pop_string(vm);
    // ...
}

int main() {
    PxScript* script;
    PxVM* vm = px_vm_create(script);

    // Registering an external:
    px_vm_register_external(vm, "PlayVideo", my_external);

    // Calling a function by name:
    void* my_instance /* from somewhere*/;
    int rv /* will be the return valuie of MyScriptFunc */;
    px_vm_call_function(vm, "MyScriptFunc(Iis)i", my_instance, 100, "my string", &rv);
    
    // Calling a function by index:
    void* my_instance /* from somewhere*/;
    int rv /* will be the return valuie of the function */;
    px_vm_call_function_by_index(vm, 0x1234, "(Iis)i", my_instance, 100, "my string", &rv);

    // ... and I'm not sure about initializing instances yet.

    px_vm_destroy(vm);
    return 0;
}

2. Determining external parameters and return values from a signature parameter

This works basically the same as 1. but here we also use a printf-style string for registering externals. I don't even know if this idea is possible in C so I have to do some more research on it.

typedef void(*PxVMExternal)(PxVM*, ...);

void px_vm_register_external(PxVM* vm, char const* signature, PxVMExternal* ext);

and you would use it like this:

void my_external(PxVM* vm, char const* par0) {
    // ...
}

int main() {
    PxScript* script;
    PxVM* vm = px_vm_create(script);

    // Registering an external:
    px_vm_register_external(vm, "PlayVideo(s)v", my_external);

    // And the rest like in 1.

    px_vm_destroy(vm);
    return 0;
}

---

As you can probably tell, the second way of registering externals is a lot simpler to implement for a potential API-consumer since the function parameters don't have to be popped off the VM's stack manually. It also allows for better detection of invalid function signatures at runtime since the expected signature is already provided. Again, I am not sure if something like this is even possible in C.

The first way of writing this API is much easier to implement for me though since I already know how to do it. It also has the advantage of being very flexible with how externals can be implemented but it comes at the cost of ergonomics and less straightforward function signature checks at runtime, making it less safe as well.

Do you have any other ideas on how to approach this which would be helpful to you specifically, maybe? I'm happy about any feedback or ideas you might have :)

[lmichaelis]

Hey all,
a C-Interface for phoenix, made in collaboration with @JaXt0r, is now available publicly in the new GitHub organization: https://github.com/GothicKit/phoenix-shared-interface! Alongside this, as I've been testing things, I've also ended up writing a very small wrapper around this API for Java.

This might be helpful to you, @Elgcahlxukuth. You can find the sources here and I've attached a JAR you can use here. I'd be happy about any feedback you might have and feel free to open pull requests if you end up needing additional features or just want to help improve the project :) You will need to reference the C header files still though, since some APIs return untyped enums or pointers.

[Elgcahlxukuth]

Thank you so much @lmichaelis ! :)

I'm still struggling with GraalVM, waiting for a support with some internal issue, so I still haven't managed to mix C++ with Java yet (it should be possible in theory, but doesn't fully work for me at the moment). Hence, I will probably take a closer look at the Java Native Interface approach that you suggested earlier. The C-interface you prepared makes it easier I guess, so I'm grateful. Cheers!

[lmichaelis]

@Elgcahlxukuth The JAR I attached should work in any JVM, even GraalVM, to my knowledge :)