Installer for WASM Module

[dinther]

I made some progress with the SimCom library. One of the demo commandline utilities is SimComMon which depends on the presence of the WASim WASM Module in the community folder. So when I write an installer I can drop any old version of the WASM module into the community folder.

I would like guidelines on how to correctly install the WASM module in the correct location ensuring newer versions are not overwritten. Ideally the WaSim WASM module is packaged in a installer. It is important that the WASimCommander support files that are managed properly otherwise we all end up throwing our own identical copies in the community folder or am I missing something?

[mpaperno]

Hi Paul,

The module is versioned, and the version is available in the manifest.json file which would be in the module's Community folder location if it is already installed.

So if you're going to be "throwing" (: anything in there then it's your responsibility to check for presence of the module and it's current version, if any, and take "appropriate" action.

I'm not assuming you'd do this anyway, but just personally I think it would be presumptuous of an installer to just overwrite any already-installed version of anything w/out at least a user prompt, regardless of if it's older or not.

HTH,
-Max

[dinther]

Thanks Max, maybe I see problems that are not really there. If WASimCommander is to be the interface for many third party developers, then each of these developers will be forced to roll their own installation solution. Doesn't that mean that my working installation can be broken by a faulty installer from someone else? I know, anyone can format the hard-drive but WASimCommander is to be a shared resource.

Would it not be wize to at least give WASimCommander users guidelines on how to ensure a safe installation. This problem is rather common is it not?

From your last paragraph, are you suggesting that we might end up with multiple versions of WASimCommander in the community folder?

[mpaperno]

I hope there won't be "version fragmentation" and I certainly aim to keep the server-side module backwards-compatible in terms of the protocol. Perhaps there might be some break in a new major version, but even then I'd be reluctant to do that.

Beyond that... I'm not really sure what else to do. The "installation" itself is pretty trivial, and it's not high on my personal list to make an installer for the module. If someone else were to make one, that could be a good option for users who prefer such methods, of course.

The only other project I know of that uses the module right now is my own Touch Portal plugin. I just tell those users when it's time to update the module, and include it (zipped) separately in the release assets (it's actually optional in this case... the whole plugin could use an installer, but it hasn't been an issue so far).

So if I'm releasing a new module version it will definitely need to be backwards-compatible, even if the plugin users don't need to update their version (eg. because plugin doesn't use new module features).

If someone were to fork the module and release their own version which isn't compatible or doesn't keep up feature-wise with the source, then yea, that could become an issue. In that case I'd expect them to at least change the name and relevant internal IDs of their version so that both could be installed in parallel.

Does that help? And if anyone else is using it in a project, they could just speak up... ;-)

-Max

[mpaperno]

PS. To at least partially answer my own (implicit) question.... I could publish a version of what I just wrote about backwards-compatibility in the README to make it "official." Will do!

[dinther]

OK, clearly not a big deal. Maybe I can write the application in such way that it can detect if the WASM module is present or incompatible and then take action.

On further thought, I recall your motivation to write WASimCommander is to keep bloat and resource use in the community folder to a minimum, but maybe this get's optimized internally and the overhead of multiple similar WASM modules isn't too bad.

I'll start to write some installer and if it's ok I'll ask you to review it.

[HansBilliet]

And if anyone else is using it in a project, they could just speak up... ;-)

I guess you mean me? 😊 I’m indeed forking the WASimCommander in my own GitHub account (it’s even private, nobody can see it), and have now added some features with relation to Custom Events. Simply because this wasn’t available, and I needed it for my project. Although, I have no intention to release a new version of WASimCommander “in my own name”. I’m in touch with Max on this, and if I’m not mistaken he might include my changes. Once that is done, I will also revert back to the “official” WASimCommander. I’m only using it for my own project now. If I ever would decide to make that project visible to others, and allow them to use it, then of course they will need the adapted version of WASimCommender. But maybe at that time, Max has already included my changes in his version. If not… well, I haven’t thought about it yet. One thing is for sure – if I even have to include my own version, then I will have to give it a different name. I can also not make my own car and call it BMW. I am also not intending to create an installer. Only a few nerds like m make use of my application, so it’s not for the “common public”. In that case, I would add a readme that links to the WASimCommander GitHub, and with some simple instructions how to deal with the installation. I hope that this email reaches you both. I see some very strange email addresses in my TO and CC. From: Max Paperno ***@***.***> Sent: Saturday, 16 December 2023 12:21 To: mpaperno/WASimCommander ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [mpaperno/WASimCommander] Installer for WASM Module (Discussion #19) I hope there won't be "version fragmentation" and I certainly aim to keep the server-side module backwards-compatible in terms of the protocol. Perhaps there might be some break in a new major version, but even then I'd be reluctant to do that. Beyond that... I'm not really sure what else to do. The "installation" itself is pretty trivial, and it's not high on my personal list to make an installer for the module. If someone else were to make one, that could be a good option for users who prefer such methods, of course. The only other project I know of that uses the module right now is my own Touch Portal plugin. I just tell those users when it's time to update the module, and include it (zipped) separately in the release assets (it's actually optional in this case... the whole plugin could use an installer, but it hasn't been an issue so far). So if I'm releasing a new module version it will definitely need to be backwards-compatible, even if the plugin users don't need to update their version (eg. because plugin doesn't use new module features). If someone were to fork the module and release their own version which isn't compatible or doesn't keep up feature-wise with the source, then yea, that could become an issue. In that case I'd expect them to at least change the name and relevant internal IDs of their version so that both could be installed in parallel. Does that help? And if anyone else is using it in a project, they could just speak up... ;-)

…

-Max — Reply to this email directly, view it on GitHub <#19 (reply in thread)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNTKJSRJA3VKF5Z2XLNFWDYJV723AVCNFSM6AAAAABAXOQLR2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQNZQGQ4TQ> . You are receiving this because you are subscribed to this thread. <https://github.com/notifications/beacon/ABNTKJUSHTOK65YRYWSEM6LYJV723A5CNFSM6AAAAABAXOQLR2WGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAPAMCE.gif> Message ID: ***@***.*** ***@***.***> >

[mpaperno]

Hi Hans :)

Well I meant anyone I didn't know about... but I do appreciate your input! And I haven't forgotten about your changes... no worries (sorry, 'been distracted). At any rate there is nothing in them which would break BC, or even require a new module version (I think... at least so far :) ).

Cheers,
-Max

(The e-mails get posted back to the (public) GitHub Discussion from whence they originate.)

[dinther]

No Hans! I really thought there be tens if not hundreds of users of WASimCommander.

[HansBilliet]

No worries Max. I’m working on my own application now, because I took the final decision to use WASimCommander. So now I have to make it work (not talking anymore, but doing it 😊). So far, it looks very promising. Didn’t do any performance tests yet, for that it’s still to early. But I will certainly do. Although, I already could control my SPD button that is an encoder, and it feels very responsive and fast. And I even didn’t optimize stuff yet. From: Max Paperno ***@***.***> Sent: Saturday, 16 December 2023 12:43 To: mpaperno/WASimCommander ***@***.***> Cc: HBilliet ***@***.***>; Comment ***@***.***> Subject: Re: [mpaperno/WASimCommander] Installer for WASM Module (Discussion #19) Hi Hans :) Well I meant anyone I didn't know about... but I do appreciate your input! And I haven't forgotten about your changes... no worries (sorry, 'been distracted). At any rate there is nothing in them which would break BC, or even require a new module version (I think... at least so far :) ). Cheers, -Max (The e-mails get posted back to the (public) GitHub Discussion from whence they originate.) — Reply to this email directly, view it on GitHub <#19 (reply in thread)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNTKJTAYJFJFVLZ5YDITZLYJWCKPAVCNFSM6AAAAABAXOQLR2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQNZQGU3TG> . You are receiving this because you commented. <https://github.com/notifications/beacon/ABNTKJVXNDLW23MOM35JIC3YJWCKPA5CNFSM6AAAAABAXOQLR2WGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAPAMG2.gif> Message ID: ***@***.*** ***@***.***> >

[HansBilliet]

Just small question. Your code is re-entrant, right? I have seen you make use of “unique_lock lock{mtxRequests};” in some critical sections, but I didn’t found any in “setVariable”. Reason is that in my software, my RxPumps of my devices all run as different tasks, hence they can fire a “setVariable” at any time. Not 100% if that will work. Anyway, if it isn’t, then I use my favorite method by including a TxPump that is fed with a ConcurrentQueue, which insures that commands are processes in sequence, and not in parallel. From: Max Paperno ***@***.***> Sent: Saturday, 16 December 2023 12:43 To: mpaperno/WASimCommander ***@***.***> Cc: HBilliet ***@***.***>; Comment ***@***.***> Subject: Re: [mpaperno/WASimCommander] Installer for WASM Module (Discussion #19) Hi Hans :) Well I meant anyone I didn't know about... but I do appreciate your input! And I haven't forgotten about your changes... no worries (sorry, 'been distracted). At any rate there is nothing in them which would break BC, or even require a new module version (I think... at least so far :) ). Cheers, -Max (The e-mails get posted back to the (public) GitHub Discussion from whence they originate.) — Reply to this email directly, view it on GitHub <#19 (reply in thread)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNTKJTAYJFJFVLZ5YDITZLYJWCKPAVCNFSM6AAAAABAXOQLR2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQNZQGU3TG> . You are receiving this because you commented. <https://github.com/notifications/beacon/ABNTKJVXNDLW23MOM35JIC3YJWCKPA5CNFSM6AAAAABAXOQLR2WGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAPAMG2.gif> Message ID: ***@***.*** ***@***.***> >

[mpaperno]

Yes, and thread-safe unless otherwise mentioned (could have sworn I documented that somewhere but can't find it now... 🤷🏼 ). All SimConnect access is locked with a single mutex (in that "magical" macro/helper), and in the case of setVariable() that's the only aspect that is possibly not thread-safe. In the other cases you're seeing there are separate mutexes used which are locking things like stored data access (eg. the maps) or waiting on events (eg. responses from server). But with setVariable() there's no access to such resources, so you're good to go!

-Max

[mpaperno]

PS. On the module side any concurrent access/code execution at all is completely impossible (as far as I can tell... and I tried... :-P ), so no worries there.

[HansBilliet]

I didn’t say that. I was meaning my own application (CockpitHardwareHUB). From: Paul van Dinther ***@***.***> Sent: Sunday, 17 December 2023 09:54 To: mpaperno/WASimCommander ***@***.***> Cc: HBilliet ***@***.***>; Comment ***@***.***> Subject: Re: [mpaperno/WASimCommander] Installer for WASM Module (Discussion #19) No Hans! I really thought there be tens if not hundreds of users of WASimCommander. — Reply to this email directly, view it on GitHub <#19 (reply in thread)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABNTKJRV7T5P4YT3Y2HONG3YJ2XLVAVCNFSM6AAAAABAXOQLR2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TQNZVGQ4DE> . You are receiving this because you commented. <https://github.com/notifications/beacon/ABNTKJQO4O7XMNV377GI623YJ2XLVA5CNFSM6AAAAABAXOQLR2WGG33NNVSW45C7OR4XAZNRIRUXGY3VONZWS33OINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAPAVZU.gif> Message ID: ***@***.*** ***@***.***> >

[dinther]

Hi Max, I added a tonne of stuff to SimCom

This is a commandline demo. It installs WASim module if required, it can get,set and monitor all kinds of variables
https://github.com/dinther/SimCom/tree/master/SimComMon

EXE demo zip files here https://github.com/dinther/SimCom/tree/master/Demo_exe

Also an installer class that figures out where the community folder is, the MSFS install folder and if MSFS is running.
Based on that info it goes ahead and installs WASimCommander module if files are newer

https://github.com/dinther/SimCom/blob/master/SimCom/FlightSimInstallInfo.cs

Finally a question. Let's say I monitor A:GEAR LEFT POSITION with an interval of 50 milli seconds and a delta of 0.1
Position up is 0 and down is 1.

The data updates stop coming in when the last step is smaller than 0.1 and thus I never reach 1.
I noted that this does work in your WASimUI program. Did you do anything special for that?

[mpaperno]

Hi Paul,

Congrats on "shipping" the project!

I'm not sure what you're seeing with the gear position/delta issue... and no, nothing "special" that the UI does, though I guess I'm not sure what that would be (do you mean specifically for that variable? Using the delta feature in general?). The only requirement for the delta feature to work is using one of the predefined SimConnect data types, which seems like your client is already doing (and BTW SimConnect is the one doing the delta comparisons).

50ms refresh on a gear position? That's some lively gears you got :)

Your installer should only update an already-installed module if the actual version is newer. File timestamps are meaningless. Also it should prompt before overwriting anything... but maybe it does already, I can't tell.

Cheers,
-Max

[mpaperno]

Paul, you also need to include the proper licensing information and the original license files for the WASimModule distribution, and the WASimClient licenses if you're distributing the library as part of your project (eg. in your executable/packages). You may license your part of the project under MIT (or however you want), but that doesn't remove the [L]GPL agreement requirement from the WASimCommander bits. Your users need to be aware of that (and including the original license is required by just about every license out there).

Thanks!
-Max

[dinther]

Thank you Max, I really appreciate your Pro suggestions.

Licenses

Sorry about the license issue. Obviously I have no intention to break the rules on licensing. I am very inexperienced with software licensing so I might need some guidance to at least respect yours.

I found your licenses all over the place in the WASimCommander folder:

C:\MSFS SDK\WASimCommander
C:\MSFS SDK\WASimCommander\module
C:\MSFS SDK\WASimCommander\include
C:\MSFS SDK\WASimCommander\lib\dynamic\msvc142
C:\MSFS SDK\WASimCommander\lib\static\msvc142
C:\MSFS SDK\WASimCommander\lib\managed

In most cases there are three license related files:

LICENSE.GPL.txt
LICENSE.LGPL.txt
LICENSE.txt

No idea which ones are relevant. They all look the same to me. I guess my application zip file will contain a licenses folder where I put all the licenses from software and libraries I have used?

But how can that work? I would have file name clashes. None of the license files bear any reference in the file name to the product it is supposedly about so that causes clashes. Should I rename them? That is probably tampering and not allowed.

Solution for gear pos reporting.

I think I have a solution for the problem I was having with:

(A:GEAR LEFT POSITION,number) (A:GEAR RIGHT POSITION,number) + (A:GEAR CENTER POSITION,number) + 3 /

When I set deltaEpsilon to zero and a reasonable interval of say 300 ms the problem goes away.
Another way I found to do this is to just handle integers.

(A:GEAR LEFT POSITION,percent) (A:GEAR RIGHT POSITION,percent) + (A:GEAR CENTER POSITION,percent) + 3 /

Output ranges between 0 and 100 using an interval of 300ms and a deltaEpsilon of 1.

Module installing

So rather than the file date stamp you want me to check the manifest?

manifest.json

{
  "dependencies": [],
  "content_type": "MISC",
  "title": "WASimCommander WASM Module",
  "manufacturer": "",
  "creator": "Copyright Maxim Paperno; All rights reserved.",
  "package_version": "1.2.0",
  "minimum_game_version": "1.34.16",
  "release_notes": {
    "neutral": {
      "LastUpdate": "",
      "OlderHistory": ""
    }
  }
}

I presume package_version is the info I am after? This does raise a question. Is there a way for me to confirm the integrity of the module installation? How do I know that the other files have not been tampered with or replaced by other files or even deleted?

I can definetely offer the user the option to cancel installation, overwrite if sensible or install in a different folder. But that negates the whole point of WASimCommander, which is to minimize the amount for modules taking up valuable PC resources.

[mpaperno]

Hi Paul,

No worries, I know you're not out to hide anything. And you mention the WASim project enough times in your docs to certainly qualify as "giving credit" (thanks!).

All the license files of the same name have the same content. They're just placed separately in the SDK distro to clarify how each component is licensed and/or if a person is only looking at a particular part of the SDK package. Probably overkill, really, but it does hopefully make it clear. The only "actual" license file is LICENSE.txt. The other two are simply copies of the actual standard licenses... they don't technically have to be included (the main license explicitly states "as published by the [FSF]").

If you're distributing WASimModule, include LICENSE.txt alongside wherever you're storing your copy in your repo, just like in the SDK distro. The copyright is in the actual module's manifest already.

I'm not sure how you're keeping WASimClient lib (don't see it in your repo, just a hard-coded path in the project file... maybe you include it in the distributed zip?). But anyway however that is distributed, it should have the relevant license file(s) with it. You can rename them if you want (eg. WASimClient.LICENSE.txt), or place in separate folders, or whatever. For example here's how I keep my own libs with my own project: https://github.com/mpaperno/MSFSTouchPortalPlugin/tree/next/MSFSTouchPortalPlugin/lib/WASimCommander

Here's an example of how I do attribution and copyright. There are many styles of course... the key point is just to be clear about what is being used and how. Especially if any of the sub-licensed components use more restrictive licenses than the parent project. In my case none do, but [L]GPL is much more restrictive than MIT, so you may need to take that into account.
https://github.com/mpaperno/MSFSTouchPortalPlugin/tree/next#credits

And yes, package_version from manifest.json is the version number of the included WASModule. It doesn't/can't include the final build number that the actual project includes, but it doesn't matter in this case.

I'm not sure what you mean in terms of "integrity." Of what exactly? Who's tampering with what?
At any rate, file integrity is usually tracked by keeping checksums/hashes of the contents of the original files, then comparing them to the same hash algorithm run on whatever file you want to verify. I do not publish hashes... though it's probably not a bad idea (which I guarantee no one will ever check).

Honestly I'm confused by the whole issue with distributing or installing the module part. Why not just point users to the official releases and have them drop it in their Community folder? Surely anyone even remotely interested in a developer tool like this needs to at least know where that is located (and may even prefer to specify it vs. some tool trying to guess it... from a quick look at your code I don't think it would even work on my sim installation at all, for example). And the user can handle any version conflicts however they see best.

Running multiple unmodified versions of WASimModule simultaneously in the sim will not work. Consider that it listens for connections using a pre-set event name which must be unique. This can be changed at the top of WASimCommander.h, but there's currently no way to configure that at runtime. Like you said, that would kinda defeat part of the point, but OTOH there's no rule against it either.

Cheers,
-Max

[dinther]

Honestly I'm confused by the whole issue with distributing or installing the module part. Why not just point users to the official releases and have them drop it in their Community folder? Surely anyone even remotely interested in a developer tool like this needs to at least know where that is located (and may even prefer to specify it vs. some tool trying to guess it... from a quick look at your code I don't think it would even work on my sim installation at all, for example). And the user can handle any version conflicts however they see best.

I think there is a misunderstanding. You seem to think that I want my SimCom library to install your WASimCommander module in the community folder.

That is not at all my intention. I am developing a windows application for a range of USB input devices that I designed. This application is aimed at the end user but it does rely on SimCom. It is this application that needs to ensure the required resources are present in the community folder. It's intent is similar to Mobiflight but it has an onscreen presence. Click install for the progam, plug in the USB device and run MSFS and the whole lot just works. No config required.

Normally I would use a tool like Inno to build the installer but because WASimCommander is potentialy shared with other software packages I can not rely on those modules to be always present and unchanged. Therefore my user application will make that check every single time it starts.

Right now I am implementing your recommendations with regard to versions checking. For the integrety check I intend to use the layout.json

[HansBilliet]

Hello, Should I take a lawyer? 😊😊 Just to be sure that I’m ok from my side as well, although I discussed shortly some details with Max already a while ago. In my readme, I show the following (as agreed with you Max). @paul: I just realize that I have not mentioned you (although in the source code I do), which I should because I’m using your ‘SerialPortManager.cs’ – really not intentional – I will add it in the list today I have the default LICENSE.TXT in the root of my repository (is done by default by GitHub if you select your license if I remember well). But if I understand correctly, I should put the license.txt in my distribution files? I have 2 folders: 1. Executables – this contains the ‘WASimCommander.WASimClient.dll’ 2. WASM Module - Which is an exact copy of your WASimCommander WASM module (V1.2.0.0) – I do mention it in my explanations that people can copy it from your repository (and you are of course mentioned there), but I put a copy in mine for convenience. As all is licensed under the same GNU license (same as GPLv3? – shoot me – I’m really not knowing anything about this stuff), I assume I should just add the same LICENCE.TXT that is in my root, into my 2 distribution folders? Hans

[mpaperno]

Hi Hans,

Your situation is a bit different since you chose GPL (GNU Public License) for your whole project. Since these are the same terms as my WASim licensing (and even more restrictive than the LGPL), there's no real need to further inform users. In Paul's case his project is using the (much less restrictive) MIT license... so users may reasonably assume that covers the whole project unless specified otherwise.

Personally I'd include the license and copyright info with my software distributions, but of course that's up to you. Certainly no one reads them, it's more of a legalese question for the highly unlikely scenario that something actually ends up in a court. Personally I also add the copyright and license terms to all my source files -- this is actually the biggest issue (both legally and practically) to protect one's work when the source is public... even someone just having a single file cannot claim ignorance of the terms.

And yes, I've had people "unintentionally" take my GPL-covered work and try to sell it as their own... one was a naive attempt and they got "cancelled" in court of public opinon after I found out and posted proof (the person tried to claim they didn't understand the licensing... then turned out his main gig was as a property lawyer!). So it's not entirely theoretical. Of course what someone in China does with ripped-off code is a lot harder to deal with... :-( (PS. Also, I'd like to ban ChatGPT or any other "AI" bot that doesn't quote its sources, but apparently this is tricky or impossible).)

Cheers!
-Max