Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem when try to build a DLL for the API #31

Open
whirwind opened this issue Sep 25, 2020 · 18 comments
Open

Problem when try to build a DLL for the API #31

whirwind opened this issue Sep 25, 2020 · 18 comments

Comments

@whirwind
Copy link

Hello!

I want to get the access to the sensor data in Unity. Now what I am doing is trying to build a Universal Windows DLL file. And I got this error:
image

I am new in this field and I guess I might have made some basic mistakes about this project.

  1. Will it work if I use DLL(Universal Windows) to export the ResearchMode APIs Theoretically?
  2. If 1) is right, what should I do to creatre a project environment where the APIs can work and clear the error above?
  3. If my start point is totally wrong. Is there another possible way to do this?

Best,
whirwind

@aal-rvr
Copy link

aal-rvr commented Sep 26, 2020

Hi!

I am having a similar problem. I had the same error trying to build a DLL for my Unity app.
I followed the sample code - making an explicit call to load the DLL, using LoadLibraryA("ResearchModeAPI") and then doing the following:

typedef HRESULT(__cdecl* PFN_CREATEPROVIDER) (IResearchModeSensorDevice** ppSensorDevice);
        PFN_CREATEPROVIDER pfnCreate = reinterpret_cast<PFN_CREATEPROVIDER>(GetProcAddress(hrResearchMode, "CreateResearchModeSensorDevice"));
        if (pfnCreate)
        {
            winrt::check_hresult(pfnCreate(&m_pSensorDevice));
        }
        else
        {
            winrt::check_hresult(E_INVALIDARG);
        }

I got the DLL to build and was able to call it through my C# program but the line: HMODULE hrResearchMode = LoadLibraryA("ResearchModeAPI"); returns NULL.

Is there a way to make an implicit call like suggested in the documentation.
What would be the correct way to load the ResearchModeAPI Library?
Does the plugin DLL being called from the C# process cause this problem? Do I need to do this from a separate process? and if so, how do we share m_pSensorDevice pointer across processes?

Thanks.

@dorinung
Copy link
Contributor

I would not try to return research mode interfaces into unity. Just export entry points that call into the rm apis. Basically initialize open, close and getbuffer exports from your dll, Also if your Unity app is not arm64 the RM dll will not load. We are working on adding arm support.

@whirwind
Copy link
Author

Hi @aal-rvr

It seems that you are using windows runtime component template to build the project? My project template in VS2019 is DLL(Universal Windows). I just followed sample in the API Doc, so my code is extremely simple .

CreateResearchModeSensorDevice(&m_pSensorDevice); // Factory function
m_pSensorDevice->GetSensorCount(&sensorCount);
m_pSensorDevice->GetSensorDescriptors(m_sensorDescriptors.data(), m_sensorDescriptors.size(), &sensorCount);

for (const auto& sensorDescriptor : m_sensorDescriptors)
{
    m_pSensorDevice->GetSensor(sensorDescriptor.sensorType, &pSensor);
    pSensor->GetSampleBufferSize(&sampleBufferSize);
    pSensor->OpenStream();
    for (UINT i = 0; i < 4; i++)
    {
        pSensor->GetNextBuffer(&pSensorFrame);
        ProcessFrame(pSensor, pSensorFrame, i);
    }
    pSensor->CloseStream();
    pSensor->Release();
}

Hi @dorinung
Yes, exactly. I don't return any research mode interfaces. I just plan to return some general parameters such as resolution, data buffer, or status. So I think this is not the reason for that error. The error I guess is something about my project environment. I am still looking for my mistake.

Best,
whirwind

@dakesse
Copy link

dakesse commented Sep 28, 2020

@aal-rvr Dll and Unity build have to be ARM64. I am loading the the rm API with LoadLibraryA("ResearchModeApi") as well and #include "researchmode/ResearchModeApi.h" it in my header file. I'm also using a VS 2019 DLL (Universal Windows) Project template.
I defined m_pSensorDevice as global and static:

extern "C"
{
	class DllExports
	{
	public:
		DllExport static void InitSensors();
		...
	};

	//##### Global Vars #####
	// static vars for research mode init
	static IResearchModeSensorDevice* s_SensorDevice;
	static std::vector<ResearchModeSensorDescriptor> s_SensorDescriptorsList;

	// static vars for sensor access
	static IResearchModeSensor* s_AccSensor;
	static IResearchModeSensor* s_GyroSensor;
	static IResearchModeSensor* s_MagnetSensor;
	static size_t s_AccSampleBufferSize;
	static size_t s_GyroSampleBufferSize;
	static size_t s_MagnetSampleBufferSize;
}

And use similar code as @whirwind to init the sensors.
I tested my DLL outside of Unity first, with a normal C++ VS UWP Project. There you have to set 'Content' to 'Yes' (in the DLL's properties) otherwise the DLL will not be copied to the HL2 and therefore cannot be found. In Unity this should not be a problem. But you have to modify the .appxmanifest of the Unity build before you deploy it to your HL2.
If you can't see any probems in your project environment, maybe you can use GetLastError() inside your DLL to get some more infos about your error.

@whirwind
Copy link
Author

@dakesse Thanks for your advice, I successfully get a resolution parameter from my DLL in Unity. Although more work need to do to get the raw sensor data, initial problem has been solved. Here is what I learn:

  1. About the initial problem, just follow sample code and use LoadLibraryA("ResearchModeAPI") to call 'CreateResearchModeSensorDevice' function
  2. Once the Unity build is done, follow sample projects and modify 'Package' and 'Capability' in Package.appxmanifest
  3. If the implementation of the code in the document is different from the sample, try to follow the sample first.

@dakesse
Copy link

dakesse commented Oct 6, 2020

@whirwind did you manage to marshal the data buffer of
sensorAccelFrame->GetCalibratedAccelarationSamples(&accelBuffer, &bufferOutLength);
to use it in C#?

@agroenenberg
Copy link

agroenenberg commented Oct 6, 2020

@whirwind , I get the exact same error as you did in the beginning. How did you call the 'CreateResearchModeSensorDevice' function via LoadLibraryA("ResearchModeApi") ? I can't seem to get it working. Thanks 👍

@whirwind
Copy link
Author

whirwind commented Oct 11, 2020

@dakesse Exactly. I was working for this past few days. I return a Byte* pointer in C++ and receive it in C# using IntPtr, then use marshal.Copy. But today I found that app will have crashing problem. I guess it is because I did something stupid while passing the pointer and lead to some kinds of memory leak.

  1. If I get the 'IntPtr Ptr=DLL.GetImage()', and do 'marshal.copy', then call a 'ReleaseMemory(Ptr)'. The app will crash immediately at beginning.
  2. If I just get the IntPtr and do marshal.copy without 'ReleaseMemory(Ptr)', the app will run for a short time and crash eventually.

Do you have any advice about this? Thanks.

@agroenenberg Sorry for the late reply. It is clear in the init() function in samples. It is some thing like this:
`

HMODULE hrResearchMode = LoadLibraryA("ResearchModeAPI");
if (hrResearchMode)
{
    typedef HRESULT(__cdecl* PFN_CREATEPROVIDER) (IResearchModeSensorDevice** ppSensorDevice);
    PFN_CREATEPROVIDER pfnCreate = reinterpret_cast<PFN_CREATEPROVIDER>(GetProcAddress(hrResearchMode, "CreateResearchModeSensorDevice"));
    if (pfnCreate)
    {
        hr = pfnCreate(&m_pSensorDevice);
    }
    else
    {
        hr = E_INVALIDARG;
    }
}


m_pSensorDevice->QueryInterface(IID_PPV_ARGS(&m_pSensorDeviceConsent));



m_pSensorDevice->DisableEyeSelection();

m_pSensorDevice->GetSensorCount(&sensorCount);
m_sensorDescriptors.resize(sensorCount);

m_pSensorDevice->GetSensorDescriptors(m_sensorDescriptors.data(), m_sensorDescriptors.size(), &sensorCount);`

@dakesse
Copy link

dakesse commented Oct 14, 2020

@whirwind
I first tried it with marshalling but due to multiple assembly errors on C#- and (memory) access errors on C++-side I tried using 'unsafe'. It's surely not the best way to do it but it's working for accelero and gyro values. But somehow accessing the magnetometer causes a crash. I have to take a deeper look on that. But here is how I did it:

On the C++ side I added a new struct because I had problems to access the array of the sensor values inside the respective struct array. So I just splitted them up into 3 single values.

struct SensorDataStruct {
		uint64_t VinylHupTicks;
		uint64_t SocTicks;
		float SensorValueX;
		float SensorValueY;
		float SensorValueZ;
		float temperature; };

My functions then return a pointer to such a struct array.
DllExport static SensorDataStruct* GetAccDataBatched();
Inside these functions I did the following.

	// init the return array
	SensorDataStruct* data = new SensorDataStruct[ACCBATCHSIZE];

        // get the sensor Frame and next buffer
        ...

	if (SUCCEEDED(hr)) {
		// read sensor values ...
		hr = sensorAccelFrame->GetCalibratedAccelarationSamples(&accelBuffer, &bufferOutLength);
		…

		// … and copy them to return array
		for (UINT i = 0; i < bufferOutLength; i++) {
			data[i].VinylHupTicks = accelBuffer[i].VinylHupTicks;
			data[i].SocTicks = accelBuffer[i].SocTicks;
			data[i].SensorValueX = accelBuffer[i].AccelValues[0];
			data[i].SensorValueY = accelBuffer[i].AccelValues[1];
			data[i].SensorValueZ = accelBuffer[i].AccelValues[2];
			data[i].temperature = accelBuffer[i].temperature;
		}
		// release resources
	        ...

        // and return the data
	return data;

On the C# side I have the same struct with corresponding C# types that I just use inside a 'unsafe' block to get the data from my C++ DLL.

    public unsafe struct SensorDataStruct {
        public ulong VinylHupTicks;
        public ulong SocTicks;
        public float SensorValueX;
        public float SensorValueY;
        public float SensorValueZ;
        public float temperature; };

By declaring the imported function as 'unsafe' you can just use pointers as if you where working with C++.

[DllImport("newDll.dll", EntryPoint = "?GetAccDataBatched@DllExports@@SAPEAUSensorDataStruct@@XZ", CallingConvention = CallingConvention.Cdecl)]
public unsafe static extern SensorDataStruct* getAccDataBatched();
 // init sensor
 …

unsafe {
	// get data from dll ...
	SensorDataStruct* dataPointer = getAccDataBatched(); ;

	for (int i = 0; i < ACCBATCHSIZE; i++) {
	// … and save it in a safe struct
	}
}

To work with the retreived data everywhere else in Unity i copy them into another safe struct.

@agroenenberg
Copy link

agroenenberg commented Oct 19, 2020

Okay I've been trying to get this DLL working on the HoloLens 2 for days now and it's driving me to insanity. I tried every way I could find, the testfunction works perfectly on Unity but I keep getting this error on the HL2:

DLLTestRun.exe' (Win32): Loaded 'U:\USERS\DefaultAccount\AppData\Local\DevelopmentFiles\Template3DVS.Debug_ARM64.Annabel\DLLResearchMode.dll'. Symbols loaded.
'DLLTestRun.exe' (Win32): Unloaded 'U:\USERS\DefaultAccount\AppData\Local\DevelopmentFiles\Template3DVS.Debug_ARM64.Annabel\DLLResearchMode.dll'
Exception thrown at 0x00007FFD50CD39EC in DLLTestRun.exe: Microsoft C++ exception: Il2CppExceptionWrapper at memory location 0x000000CFB72FCA50.
DllNotFoundException: Unable to load DLL 'DLLResearchMode': The specified module could not be found.
  at CallTest.TestFunction (System.Int32 a, System.Int32 b) [0x00000] in <00000000000000000000000000000000>:0

I did everything as described, built the DLL for x64 and ARM64, put them in the right folders in Unity(ARM64 for UWP and WSAplayer only), x64 for every platform except WSAplayer. Added DLL to project and set content to "Yes". I tried everything with each time a different component. (only ARM64, only x64, build via Unity, not adding to project). Everthing built and configured in Debug Mode. It should just give me (int a + int b) in the VS terminal after uploading to HL2. Please can anyone help me? @dakesse @whirwind How did you guys get it working on the HL2?

@aal-rvr
Copy link

aal-rvr commented Oct 20, 2020

@aal-rvr Dll and Unity build have to be ARM64. I am loading the the rm API with LoadLibraryA("ResearchModeApi") as well and #include "researchmode/ResearchModeApi.h" it in my header file. I'm also using a VS 2019 DLL (Universal Windows) Project template.
I defined m_pSensorDevice as global and static:

extern "C"
{
	class DllExports
	{
	public:
		DllExport static void InitSensors();
		...
	};

	//##### Global Vars #####
	// static vars for research mode init
	static IResearchModeSensorDevice* s_SensorDevice;
	static std::vector<ResearchModeSensorDescriptor> s_SensorDescriptorsList;

	// static vars for sensor access
	static IResearchModeSensor* s_AccSensor;
	static IResearchModeSensor* s_GyroSensor;
	static IResearchModeSensor* s_MagnetSensor;
	static size_t s_AccSampleBufferSize;
	static size_t s_GyroSampleBufferSize;
	static size_t s_MagnetSampleBufferSize;
}

And use similar code as @whirwind to init the sensors.
I tested my DLL outside of Unity first, with a normal C++ VS UWP Project. There you have to set 'Content' to 'Yes' (in the DLL's properties) otherwise the DLL will not be copied to the HL2 and therefore cannot be found. In Unity this should not be a problem. But you have to modify the .appxmanifest of the Unity build before you deploy it to your HL2.
If you can't see any probems in your project environment, maybe you can use GetLastError() inside your DLL to get some more infos about your error.

Hi @dakesse ,

While trying to make the DLL work for a C++ UWP app, I am running into an error: 'A dependent dll was not found'.
I tried adding the .dll file to the Project and set the 'Content' to 'Yes', but that still gives the same error. I have tried other things like, adding the .lib file to references and including the header file (which is what I have always done to add a DLL to a project).

Do you know what else needs to be changed?

Thanks!

@agroenenberg
Copy link

I found the solution to my problem and maybe for yours too @aal-rvr . I installed Dependency Walker to check if my DLL was dependent on any other dlls that were not included. After some checks and research this is what solved it: When building the DLL itself, I changed C/C++> Code Generation> Runtime Library from MDd to MTd and now it works! Even if yours doesn't work, try https://www.dependencywalker.com/ to check your dll on dependencies.

@dakesse
Copy link

dakesse commented Oct 21, 2020

@aal-rvr Dll and Unity build have to be ARM64. I am loading the the rm API with LoadLibraryA("ResearchModeApi") as well and #include "researchmode/ResearchModeApi.h" it in my header file. I'm also using a VS 2019 DLL (Universal Windows) Project template.
I defined m_pSensorDevice as global and static:

extern "C"
{
	class DllExports
	{
	public:
		DllExport static void InitSensors();
		...
	};

	//##### Global Vars #####
	// static vars for research mode init
	static IResearchModeSensorDevice* s_SensorDevice;
	static std::vector<ResearchModeSensorDescriptor> s_SensorDescriptorsList;

	// static vars for sensor access
	static IResearchModeSensor* s_AccSensor;
	static IResearchModeSensor* s_GyroSensor;
	static IResearchModeSensor* s_MagnetSensor;
	static size_t s_AccSampleBufferSize;
	static size_t s_GyroSampleBufferSize;
	static size_t s_MagnetSampleBufferSize;
}

And use similar code as @whirwind to init the sensors.
I tested my DLL outside of Unity first, with a normal C++ VS UWP Project. There you have to set 'Content' to 'Yes' (in the DLL's properties) otherwise the DLL will not be copied to the HL2 and therefore cannot be found. In Unity this should not be a problem. But you have to modify the .appxmanifest of the Unity build before you deploy it to your HL2.
If you can't see any probems in your project environment, maybe you can use GetLastError() inside your DLL to get some more infos about your error.

Hi @dakesse ,

While trying to make the DLL work for a C++ UWP app, I am running into an error: 'A dependent dll was not found'.
I tried adding the .dll file to the Project and set the 'Content' to 'Yes', but that still gives the same error. I have tried other things like, adding the .lib file to references and including the header file (which is what I have always done to add a DLL to a project).

Do you know what else needs to be changed?

Thanks!

Hi @aal-rvr ,
I'm not sure if I had the exact same error. But I remember that at some point I had to add the .lib file as well and set 'Content' to 'Yes' in its settings.
It seemes a bit random to me because I just tested it again with my C++ UWP app and it still worked after excluding the .lib file. I'm sorry not being a big help with that.

@dakesse
Copy link

dakesse commented Oct 21, 2020

I found the solution to my problem and maybe for yours too @aal-rvr . I installed Dependency Walker to check if my DLL was dependent on any other dlls that were not included. After some checks and research this is what solved it: When building the DLL itself, I changed C/C++> Code Generation> Runtime Library from MDd to MTd and now it works! Even if yours doesn't work, try https://www.dependencywalker.com/ to check your dll on dependencies.

Nice that you found a solution.
My setting for this is '/MD'. But perhaps there are some other differences in our project setting that caused the errors.
Dependency Walker is always worth a try.

@aal-rvr
Copy link

aal-rvr commented Oct 21, 2020

I found the solution to my problem and maybe for yours too @aal-rvr . I installed Dependency Walker to check if my DLL was dependent on any other dlls that were not included. After some checks and research this is what solved it: When building the DLL itself, I changed C/C++> Code Generation> Runtime Library from MDd to MTd and now it works! Even if yours doesn't work, try https://www.dependencywalker.com/ to check your dll on dependencies.

Thanks for the advice. My setting is /MDd by default. My original DLL might have had some differences as it was created in VS2017 (although the properties match my new DLL exactly), After quickly making a UWP DLL in VS 2019 it works as expected for C++ UWP Blank App. For this I only added the .lib file and didn't have to add the DLL.
I don't know what was wrong with my previous DLL. I could have missed a property somewhere.
Thanks!

@agroenenberg
Copy link

Did anyone actually use a Windows Runtime Component instead of a DLL? I'm trying to record/ save 1 pointcloud from the AHAT sensor so I can process that on my desktop and send some data back to the HL2. I don't want/need any frames of the data on my Unity app, I just want to save a pcloud via my app that I can then access.
Unfortunately, the DLL format doesn't allow for using winrt::Windows::Something so the StartRecording & StopRecording functions can't be incorporated in the DLL... Would building it in a Windows Runtime Component be an option?

@zhangyk18
Copy link

@dakesse Hi there, thanks for your detailed instructions.
I see only ACCEL, GYRO, MAG sensor types on your code. Have you ever tested camera sensors (VLC or depth) with the way of extern "C"?

@dakesse
Copy link

dakesse commented Nov 10, 2020

@dakesse Hi there, thanks for your detailed instructions.
I see only ACCEL, GYRO, MAG sensor types on your code. Have you ever tested camera sensors (VLC or depth) with the way of extern "C"?

Sorry, unfortunately not. I was only interested in the IMU data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants