-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature request: HDF5 for .NET Standard #57
Comments
I've just had a play around with this. One of the main differences with the .NET Standard is how DLL's are loaded. I think the biggest challenge here will be resolving the DLL's. I managed to convert the project and get the unit tests to pass but that was only after manually copying the HDF5 DLL's into the bin folder. I will continue to look into this and try and learn how DLL loading is done by the .NET Standard. |
For reference, a FAQ on .NET Standard: https://github.com/dotnet/standard/blob/master/docs/faq.md Unfortunately my knowledge of HDF5 is restricted to very high-level interfacing using h5py in Python.. But I do believe that a .NET Standard HDF5 API would be a good long-term investment. |
I think it might be a completely new project. This one is P/Invoking to a windows dll which is extremely platform dependent. In order to properly migrate to .NET Standard (so that works on phones, windows, Linux, mac etc.) you'll have to rewrite the library to natively use HDF5 format. |
If I#m not mistaken, P/Invoke can also be done on UWP. Would it be possible to build targeting UWP and add this to the NuGet package? |
At the moment it is, true. But as you can see in HDF5/H5DLLImporter.cs there was at least an attempt to support Linux and OSX too. |
I just threw that code into a fresh UWP project. Here's the thing: the |
HDF5 is build with CMake. CMake is able to generate a Visual Studio solution-file. When you open the solution in VS you can build the project "libraries/hdf5-shared" which will generate the desired .dlls. Should work out of the box. If you need help with CMake tell me ;) |
@JanWosnitza oh, I didn't notice that there was a reply! |
So I'm trying to build it right now. My progress so far:
So far so good. I'm actually surprised I made it that far. But what's next? I suppose these should end up in What would it take to get them distributed with the HDF.PInvoke package? thanks! |
Do the tests pass? I'd be happy to include them, but I don't have an ARM system for testing. Links to the source code for SZIP and ZILB can be found near the bottom of this page. I think we decided against distributing |
I was able to build SZIP and ZLIB with Release/ARM, but no .dll output was generated. The PInvoke code is not entirely compatible with UWP since it does use some APIs that were depreceated. I created a new project for both PInvoke and the UnitTests and right now I'm trying to make it work for 86x UWP. |
Which APIs were deprecated? (Just curious.) SZIP and ZLIB aren't an issue as long as you don't try to read or write compressed (w/ deflate or szip) datasets. |
No success yet, but some insights: #if X86
const string Platform = "x86";
#elif X64
const string Platform = "x64";
#elif ARM
const string Platform = "ARM";
#endif
const string DLLFilePath = "\\HDF.PInvoke.UWP\\native_1.8\\" + Platform + "\\hdf5.dll";
const string HLDLLFilePath = "\\HDF.PInvoke.UWP\\native_1.8\\" + Platform + "\\hdf5_hl.dll"; I think the conditional compilation symbols are easier to maintain than the current implementation based on the PATH variable. Or is there another reason why the PATH variable has to be set? |
I think overall these are no big deal as long as one can get the |
Another update: #if X86
const string DLLFolder = "bin32";
#elif X64
const string DLLFolder = "bin64";
#elif ARM
const string DLLFolder = "ARM";
#endif
public const string DLLFileName = DLLFolder + "\\" + "hdf5.dll";
public const string HLDLLFileName = DLLFolder + "\\" + "hdf5_hl.dll"; Several Attributes had to be slightly modified too. Within the solution, I can reference it and all tests pass! (tested x86/x64) A NuGet package with embedded PowerShell script to copy the right dlls to the output directory seems to be the best solution. In short: It was relatively easy to make it into a .NET Standard 1.1 library, but creating the right NuGet package is tricky. Here's what I have: https://1drv.ms/u/s!Ao_tF5GvDxQdv45-snwBuoh5qQhnMg |
Actually DllImport SHOULD work fine in any implementation that supports .Net Standard, always. It's a required interface, and should work with anything like a shared object or a DLL; in fact, .Net Core handles this pretty well. Just remove the There's a good example dealing with cross-platform P/Invoke: ManagedBass. It targets at AnyCPU and simply refers to bass, works well except that its user has to get bass.dll by themselves. Assuming you'd prefer ship h5 DLLs via NuGet, maybe the best choice for you is to publish a series of packages, each containing h5 DLLs for one single platform, like windows, uwp, Linux, and so on. But this is to be a hard work; I'm not so familiar with this, but if you want, I think I can help. As a .Net Core heavy user, I'm eager to see more and more projects (and also NuGet packages) become platform independent. |
Sorry but I've forget the global variables.. It seems that being totally platform independent is impossible due to this design of original HDF5 library. However, if possible, distribute runtime packages separately is still a good practice, just like many Microsoft's own packages (I mean those packages' names start with |
Any progress here? |
See my work at #127. I have NuGet packages at https://www.myget.org/feed/Packages/coreports . |
Thank you, any chance to get it published on the official nuget repository? |
It sounds compelling to target .NET Standard 1.?, indeed. However, the statement "this library targets .NET Standard" implies that any user of the library can reference the library and be sure that is works on any runtime / platform implementing the same version of .NET Standard - even on any platform not known so far! Now, since HDF/PInvoke is basically a thin wrapper around the HDF5 binaries this statement will not hold. HDF/PInvoke derives the platform dependence from the unmanaged binaries it wraps. According to the approach followed above in this thread, a user - while being able to reference the library as .NET Standard lib from any project - will still have to deal with the unmanaged dependencies for the specific target platform. So there will be no full platform compatibility anyways. There is no (feasible) way to create an assembly for Windows, take the .exe and run it on iOS and expect it to work when unmanaged binaries are involved (and leaving too exotic solutions aside). At development time a conscious decision about the target platform is required. A feasible solution for now might be to |
As already stated in the HDF mailing list I would also love to see a solution to be able to use a (in my case) .NET Core project which references the HDF.PInvoke library without giving warnings about possible incompatibility. If I am not wrong, the upcoming .NET Core 2.1 will be the last version that allows loading NET Framework assemblies. I am not sure if targetting specific runtimes would solve the problem. In my opinion this leads to more overhead than required because multiple projects with marginal differences must be maintained then. It is not clear to me how one would ensure that the right native library is loaded (and distributed). Manually copying the correct library would at least for me no realistic option. I am planning to use the NuGet.PackageManagement package to let the end user install and uninstall plugins (where one of the plugins depends on HDF). This requires that all packages (or plugins) and their dependencies are fully functional when they are downloaded. This process should not require the end user to manually download appropriate native libaries from a web page. If instead say for example the following .NET Standard packages would exist (like @Blealtan suggested):
Then they could deliver the right native dependencies and it is clear to the user that the Windows package does not work on Linux. I am sure that using msbuild and a modified .targets file the whole build and publish process could be automated. Microsoft itself also publishes platform specific packages that target (among others) .NET Standard. See for example System.Diagnostics.EventLog. Unfortunately this package does not make clear it is only working on Windows. But with proper naming as suggested above, it should be unambiguous. |
Please see my comment for new developments regarding HDF.PInvoke for .NET Standard. |
For broad compatibility and less fragmentation it would be really great to have a .NET Standard implementation of HDF5. This table gives a good overview which versions of Mono/UWP/.NET Framework work with .NET Standard: https://github.com/dotnet/corefx/blob/master/Documentation/architecture/net-platform-standard.md#mapping-the-net-platform-standard-to-platforms
As I understand it, it would require to build the HDF5 source with VS into a PCL.
The text was updated successfully, but these errors were encountered: