Sound libraries abstraction for NodeJS (or other runtimes)

[Brian-McBride]

Feature Request
The work here is fantastic. I would love to contribute to the UI aspect of the software and as I was thinking that if it was possible to use something like Electron (the same route that VS Code took) we could develop a multi-platform UI that fronts the excellent work on the C++ library.

The separation of concerns might allow for easier porting to other platforms as well. The audio code could be updated and improved alongside an evolving UI with an internal update process.

I suppose it is almost like a Jamulus codec that many apps could utilize.

Possible helpful reference on "how"
https://medium.com/jspoint/a-simple-guide-to-load-c-c-code-into-node-js-javascript-applications-3fcccf54fd32

[corrados]

develop a multi-platform UI

Jamulus is already multi-platform. Which additional platforms do you have in mind?

The audio code could be updated

There is already an Issue for that: #116

[Brian-McBride]

I specifically mean a UI. Or many UIs perhaps.

In my case, I am working with a school to, perhaps, have some sort of distance singing with the class. The latency via Zoom or similar apps just don't work. However, we are going to try Jamulus.

Although the app is very simple, asking dozens to hundreds of families to set things up right is... challenging. I'd love to automate everything for them. Which is why I was thinking if the Jamulus had a plugin surface that I could get NodeJS to talk to for settings and initializing a connection as well as some status events - Then I could wrap the whole thing in an electron app along with a few other school related items. Like a Chrome browser with viewing limits set by the school and allowing the teacher to pull up a website on everyone's screen.

Anyway. what is being done with Jamulus is very cool. Another way of thinking about it - if you can expose it and allow people to build electron apps with the client code - other's might expand the use and add more side features as well, which won't have to be maintained in Jamulus directly.

[gilgongo]

Hi Brian - thanks for thinking about this. It's always good to know about different Jamulus users' situations and support for non-technical users is a big issue.

asking dozens to hundreds of families to set things up right is... challenging

Compared to having an electron app or similar, how would simply using remote desktop sharing (eg Windows Quick Assist tool) compare in terms of being able to set people up? This would of course give you full visibility of things like device settings and names etc.. A lot of the issues that occur in setting up Jamulus happen outside of the app.

[Brian-McBride]

To answer your question, from an IT standpoint - it just doesn't work trying to do remote desktop for a class, much less a whole school. Imagine trying to do remote support with 100 kids of various ages and parents. Assume that most of them are technically inept. Also, you show them once and they will forgot the next day.

The reason I am suggesting a client NodeJS C Module is that I could a TON of tools to generate custom clients for educational facilities. They could stand up a server and then I could make a generator that would compile a custom client with all the settings, a branded skin, etc.. that the school could distribute.

I don't suggest trying to add all that to the Jamulus project. What is being done here is awesome. Again, you give a developer like me the ability to create custom UIs leveraging the DOM and we will do some amazing stuff.

Node’s N-API is basically a bridge communication layer. What I am thinking is a bridge from Node to send settings. Send commands (connect, disconnect, etc...). But all that is similar to settings - sending commands to the Jamulus client code. Then out of the Jamulus code would be events of interest to a UI. Finally, a few response calls to fetch the current Jamulus state - in the case the UI lost connection to the events or needed to get a fresh snapshot of the Jamulus state.

[gilgongo]

I'm not an engineer, so I'm struggling to understand this. Are you describing a remote management UI for Jamulus clients and a sort of meta-package as well?

[Brian-McBride]

Let's think about it this way.
Jamulus has a server and client. That code is responsible for an audio connection and the settings around making it work.

On top of all that is a user interface. The buttons, sliders, text boxes - those are all just UI elements that send data to and from the server and client libraries.

Currently, they are tightly coupled. Meaning, the UI is part of the client. What I am suggesting is that a client that has no UI is made to support other runtimes as if it was a plugin or module. Typically, the ask would be, "Can I run Jamulus in my audio mixing software".

In this case, my ask is (because my C++ is very rusty and I have little experience working with audio libs) that the client is exposed as a NodeJS C Module so that I can run wild with some very slick UI/UX. Offering lots of automation for teachers and students. Class management. I'd probably spin up a server for each class and integrate the whole thing into Google Classrooms / GSuite. I have the possibility of having a GCP project donated if the platform is open for educational facilities to leverage for free this fall.

Maybe another analogy is bluetooth. There are apps that have a UI and use Bluetooth as the transport. But, you can use the BLE library and build your own UI. I firmly believe that providing a Jamulus transport lib to other developers who build UI centric apps would only benefit the platform as a whole. And, the reason I suggest NodeJS is that Electron makes it very easy to develop software for multi-platform from one codebase. (Just look at VSCode or Slack for instance).

[henryaz]

I like the idea of separating functionality from the UI to enable all sorts of possibilities.

@Brian-McBride - would deep linking URL support be a quick fix? I could imagine having a URL that accepts parameters to set up and connect the client the way you would want.

[corrados]

Currently, they are tightly coupled. Meaning, the UI is part of the client.

That is not exactly true. You can run the client in headless mode without even any Qt GUI code compiled and used at all. See, e.g., https://github.com/corrados/jamulus/blob/master/distributions/raspijamulus.sh#L102 and https://github.com/corrados/jamulus/blob/master/distributions/raspijamulus.sh#L163

edit: note that the raspijamulus.sh file has changed so the above links do not point to the correct place anymore

[corrados]

Ok, so it sounds to me that if somebody wants to support this NodeJS request, a wrapper has to be build around the CClient class.

[ann0see]

@Brian-McBride Currently I am working/thinking of a comparable project (without integration in other platforms). See https://sourceforge.net/p/llcon/discussion/developerforum/thread/bcf6cc196d/
The headless build of Jamulus sounds promising.
I have just coded a small bash script (not yet finished/tested) which could setup a raspberry pi.
Also see https://github.com/ann0see/rasjam/
There's no license yet, but I probably license it under MIT/LGPL if that's possible.

[ann0see]

Does jamulus support some kind of other outside interaction/API or can a midi controller be emulated?
The headless build is quite difficult to manage.

[WolfganP]

Does jamulus support some kind of other outside interaction/API or can a midi controller be emulated?
The headless build is quite difficult to manage.

Look at https://github.com/corrados/jamulus/wiki/Tips,-Tricks-&-More , a whole section dedicated to midi control

[uthd-dev]

Hi!
I am also a NodeJS developer with very little C++ background, is there any update on potential APIs / outside interaction with a Jamulus server? I've tried using the WireShark extension to understand the "Jamulus" protocol but haven't gotten anywhere in being able to replicate it in something like node / js.

I'm currently a music student but am a web dev in my spare time and would like to look into the possibility (however difficult / impossible to pull off) of creating a Jamulus web client. This would be something that would still use the Jamulus backend server and would allow regular Jamulus clients to connect but could potentially open the possibility for connecting through a browser.

eg. Our biggest issue keeping us from using Jamulus for rehearsals is equipment. School IT won't let us install 3rd-party, much-less unsigned software on school provided hardware and some students may not be tech knowledgeable or may not be able to use personal computers. Using a web-based solution feels like the logical no-download solution to me but I know that latency will be a difficult issue to conquer with web audio APIs. I'm just trying to see my first steps to building something like this without having to do it from scratch.

Anyways, thanks in advance for any information you may have.
Cheers!

[WolfganP]

Hi!
I am also a NodeJS developer with very little C++ background, is there any update on potential APIs / outside interaction with a Jamulus server? I've tried using the WireShark extension to understand the "Jamulus" protocol but haven't gotten anywhere in being able to replicate it in something like node / js.

I'm currently a music student but am a web dev in my spare time and would like to look into the possibility (however difficult / impossible to pull off) of creating a Jamulus web client. This would be something that would still use the Jamulus backend server and would allow regular Jamulus clients to connect but could potentially open the possibility for connecting through a browser.

eg. Our biggest issue keeping us from using Jamulus for rehearsals is equipment. School IT won't let us install 3rd-party, much-less unsigned software on school provided hardware and some students may not be tech knowledgeable or may not be able to use personal computers. Using a web-based solution feels like the logical no-download solution to me but I know that latency will be a difficult issue to conquer with web audio APIs. I'm just trying to see my first steps to building something like this without having to do it from scratch.

Anyways, thanks in advance for any information you may have.
Cheers!

The clients and the server are totally decoupled, and all interactions (control+data) go thru UDP packets, no special API needed. So the way I see it is a matter of what you are allowed to run in your school environment that supports raw UDP packet processing (and obviously close to the metal/low level audio libraries to interact with the audio devices to keep low overall latency).

[gene96817]

@uthd-dev Perhaps we need a special interest group to address these issues. I currently have an evaluation server installed on a school campus. The education system IT dept has made a security scan and I have a few items to resolve (minor configuration items). My current plan is to give the music department a customized installation package that is signed. The customization is probably just standardized parameters so all the future servers follow an approved template. This would allow the IT department to add servers with no additional work from me.

In a month I will have stronger opinions about supporting the students. One ad hoc (non-solution) solution is to train student volunteers. They have a system of peer mentors in other subjects. I will also be exploring JamulusOS. It would be good to have a way for the teachers to standardized client configurations. Some schools have standardized on Chromebooks and that is a big problem.

I don't see providing real-time support as a viable option. I believe we need to onboard students in a test session and then "lock" the configuration for their classes or practice sessions.

[gilgongo]

Hi all - I'm moving this off the Issues list and into a discussion until we can firm up one or more work items for it.