Help Needed: Non-Windows Platform Support #21
Comments
|
GetSQL apparently includes SQLite binaries that support Linux: |
|
Worth noting that it doesn't support ARM architecture, though. Also, I pointed jhoneill over to this issue in the hopes he can provide more context on how he acquired/compiled the non-Windows binaries. |
|
I will take a look when I get home from vacation. |
|
Hi there, I Stumbled across this thread... You can extract the assemblies from there. And with a few switches (Assembly load logic), this could do the trick for you. Or you use the sqlite-netFx-source-1.0.XXX.0.zip file to build the libSQLite.Interop.so / SQLite.Interop.dll yourself ;-) Let me know if you need support/help. |
That's the one I'm using. I include a script in GetSQL to download and set them up :-) As @ImportTaste says though, that one doesn't have ARM support. Really the ideal would be for .NET to support SQLite directly on all platforms, but I'm not holding out a lot of hope of that happening soon. |
|
Sorry @jhoneill I don't get the point! You can create the required SQLite.Interop.dll for yourself, everything is open source! This was created on a Microsoft Surface Pro 9 5G (SQ3 based ARM64)! In my case: I use the latest preview on this device! But it will work fine with the regular Visual Studio 2022 community edition. And a regular cross compile will work fine as well: And I did some tests (again) today with 1.0.118.0, like this: The rest is up to your usage, in my case it's a highly customized implementation (with parts a reuse from the MySQLite module). My Build script generates a SQLite.Interop.dll for all my platforms, like Windows on AMD64 and ARM, MacOS ARM64 (x64 is no longer my scope), or Linux on AMD64 and ARM64. Nothing fancy, but it works fine. And again: I can provide the generic binaries described above and/or the plain source code (I might need to remove some internals only, but that will have no impact). |
|
And by the way: The Build on Linux and macOS will not require Visual Studio. The GCC Tooling will be used in this case. |
|
@jhochwald that's helpful., thanks. There's a bit of a mindset of "I don't build binaries, I deliver PowerShell code", and "People should not need to trust binaries I've compiled" (Plus there are problems running tests on a platform I don't have). And in my case I've kept adding things to GetSql because I need them not as a community effort. So the world gets what I wanted rather than a product for the greatest number of people - that's how we got here, not a defence or saying that's how it should be. Jeff and I can both look at building and distributing the binary (@jdhitsolutions it makes sense for only one of us to do the work), but I'd prefer to deliver it with instruction "Get the source from here, and follow these steps to build for your platform". These days because I can compile the few things I need with a DotNet.exe command and I do all my editing with VSCode I no longer have Visual Studio installed (I still have a licence for the full version just didn't put it on this laptop when I got it in Feb 2022). |
|
@jdhitsolutions you might want to take a look at these repositories: If you want to upgrade from sqlite-netFx 1.0.117.0 to 1.0.118.0, you need to tweak the SHA256 checksum. no further changes are required. On Windows I did a lot of tests with the Cross-Compile, and it seems to work fine (building the ARM/ARM64 binaries on an i7/X64). I also did many builds on native ARM64 just to ensure that it works the same way :) For Linux, you can use a simple WSL or Docker image. On my Dev Box and the Surface Pro 9 5G I had to use the WSL, because docker never really worked on these two! As I mentioned before: I never created the x64 binaries on a Mac! I should work the same way, but I never tested it (mostly because I don't need it anymore). What do you think @jhoneill? Is this a kind of an "Get the source from here, and follow these steps to build for your platform" instruction you mentioned above? |
|
And I published my changes here: Hope this helps. |
|
It would probably be best to come up with a Github Actions workflow to build all the different binaries, honestly. |
How would you do that for Windows? You need a full blown Visual Studio 2022 (at least the community edition), with some special workloads installed. |
Here's the runner they use for Windows, it has that and more: https://github.com/actions/runner-images/blob/main/images/win/Windows2022-Readme.md |
Awesome! I never saw all the details (workloads), but I’m not really use GitHub for builds. |
And b the way: https://github.com/actions/runner-images/blob/main/images/macos/macos-13-arm64-Readme.md |
|
I have updated my repositories to build 1.0.118.0. SQLite.Interop and SQLite.Interop-win SQLiteConnection is also published on the PowerShell Gallery. This demonstrates how to package all the native shared libraries in a PowerShell module. sqlite-tools has also been updated to provide native ARM compiled tools on Windows. I use a Windows Dev Kit 2023 to validate the Windows ARM64 builds. The macOS versions are built on a mini and the Linux builds in dockers. I have also simplified the code to determine the Runtime Identifier by getting it directly from dotnet rather than predicting it. |
|
I think I finally got this working on Linux. I have not done anything with MacOS because I have no way of testing anything. I am open to PRs with MacOS support that have been thoroughly tested and verified. |
|
Given that I have SQLiteConnection already published in the gallery with compiled native dlls I was wondering if it was worth refactoring to create the database connection with New-SQLiteConnection, then you could get rid of all of the native dll and interop code and your solution would be pure PowerShell functions and platform agnostic. I do have a different one for PowerShell Desktop because that only has the Windows dlls and the native module loading is handled differently. I have a separate one for Alpine but believe that I can put the alpine shared libraries under "linux-musl-x64" etc in the core module. |
|
@rhubarb-geek-nz In a perfect world with unlimited time and resources, that would probably make the most sense. But that is a bigger project that I can't tackle now. I have a working module that supports Windows and Linux. If someone wants to submit a PR with validated MacOS support that would be great. Even if I refactored, I still have no way to test MacOS code and I am leery of publishing something I can't run. Now, if someone wants to buy me a new Macbook Pro, I wouldn't say no. 😄 |
|
Glad you have something working. On a source-code management front, rather than having the DLLs in the git repository it is better to have either the project, makefile or script to fetch it in the repository. Having the straight DLLs stops this being an open source project because you can't see the source used to create the dlls, you can't recreate them and you don't know their origin. The user can't see the version of the dll, the compiler or compiler options used or any patches applied to the code. I suggest that in git you should have the scripts then attach the built binaries to the releases and of course publish in the gallery. For example in my releases for 1.0.118.0 the built binaries are attached along with a SHA256 hash to verify the integrity of the dll. It would also allow somebody to then build an ARM or ARM64 version. Is the DLL at assembly\SQLite.Interop.dll required given your mySQLite.psm1 module file calls Add-Type with the platform specific versions? |
|
I get your point about the open source question. At the very least I should include file hashes. I'll have to think about this. This module is a dependency on other projects and I don't want to force the user to have to manually build files before they can use the other module. I suppose I could modify this module to include code to build the assemblies and ship the module with the most current build. But I have to find a balance between true open source and ease of use. I don't want to bury the user with requirements. |
Hopefully most users are just installing from the powershell gallery |
|
I am absolutely expecting that. |
I would love to be able to support non-Windows platforms using PowerShell 7. Unfortunately, I have not had any luck finding compatible libraries, i.e.,
System.Data.SqLite.dllandSQLite.Interop.dll. I am more than happy to accept PRs or even pointers on where to look.The text was updated successfully, but these errors were encountered: