[TAUDIO]: A new plugin library is being launched

[Larpoux]

Taudio Streams : A new plugin library is being launched

We want to launch a new plugin library completely independent of Flutter Sound.
The purpose of this library will be to handle Audio Streams, and specifically PCM Audio Streams.
This plugin will replace the current code inside Flutter Sound which does PCM Streams, but is not completely implemented.

I will write a paper on that later.

But now, we must decide a few things for the launch :

The name of the library

I suggest something like :

• Taudio Streams (plurial)
• Taudio Stream (singular)
• Taudio (when abbreviated)
• Tau (when completely abbreviated)

Where to setup our git repo

It can be :

• github (we will have then Taudio and Flutter Sound side by side on github canardoux orga)
• gitlab (which is free and open source)
• bitbucket
• ... other ...

I suggest to use gitlab because it is open source and github is close source.

The tool for the documentation

It can be :

• jekyll (same as Flutter Sound)
• Notion.so (which is close source, I think)
• Hugo
• Gatsby
• Docusaurus
• Git Book (which is close source)
• ... other ...

I suggest to use jekyll because it was the only one static site generator from markdown which was compatible with dartdoc.

The license

It can be :

• GPL
• LGPL
• MPL (like Flutter Sound)
• MIT
• Apache
• ... other ...

I really insist for GPL which is the only license which has a very strong copyleft clause

PLEASE VOTE !!!!

[alexda8999]

@Larpoux

Name of Library

• TAudio - however , I would like to propose another name iAudio, iAudioMax, iAudioComplete

Where to host

• I like gitHub, must I'm open to gitLab.

Documentation

• I strongly recommend Notion.so and would have no problem whatsoever contributing to the annual cost (First year is very cheap) - the reason why I recommend Notion is because it opens up documentation to ANYBODY who has a browser and the tool is graphical in nature so anybody could easily use this.

The license

• In order of preference MIT, Apache, MPL. I dont like licenses which purport to be free but with strings attached.

[Alvarocda]

@Larpoux

Name of Library

for me, whatever

Where to host

for me, whatever

Documentation

I don't know of any tool to make documentation available.
Here at the place where I work, we are considering using Wordpress to perform this function.

The license

In order of preference MIT, Apache, MPL. I dont like licenses which purport to be free but with strings attached.

[Larpoux]

• I strongly recommend Notion.so and would have no problem whatsoever contributing to the annual cost (First year is very cheap)

The problem for me is not the price. It is that it is not open source.

[Larpoux]

• In order of preference MIT, Apache, MPL. I dont like licenses which purport to be free but with strings attached

For me, MIT is a very bad license :

• Chromium has an MIT license. Thanks to that, Google was allowed to produce Chrome, which is a close software and is the most used internet browser.

• Android AOSP has an MIT license. Thanks to that, Google was allowed to produce a proprietary Google API and the OS most used is actually a close source OS

• Webkit was released under MIT (I think). Thanks to that, the internet browser most used on the Apple word is Safari : a close source browser

• GNU-Linux was produced under GPL. Thanks to that, GNU-Linux is still free, 40 years after.

I will not fight with you. I perfectly know that I am minority and I will loose.
I am very unhappy to have to work during several years on a project not correctly protected.
At least MPL will forbid to modify my code without publishing the modifications.

[Larpoux]

Project management

We can decide of a Project Management tool. My thinking is that just a Kanban Table is enough.
I think that we are not enough contributors for using a very heavy Project management tool, with Pert or Gantt diagrams and things like that.

The problem on Flutter Sound is not the lack of a Project management tool, but that we have no project manager nor real active contributors.

[Larpoux]

I have just finished my investigations about the Web Audio implementation that already exist OFF-THE-SHELF.
Here are my conclusions :

Now, my little example that play 2 seconds of white noise runs correctly on :

• on Chrome with a simple JS script (of course)
• on Flutter Web
• on iOS, using Cordova
• on Android, using Cordova
• on MacOS, using electron
• Windows, using electron
• Linux, using electron
• Blazor

The Web Audio API is not supported on all the native frameworks :

• React Native
• Flutter on Mobiles
• Microsoft MAUI
• Native Script

Now, my questions are :
The Web Audio API being implemented in so many better alternatives than Flutter on Mobiles : is it really a good idea to do a GPL port of this API on Flutter ?
Will we have more than a handful of developers interested about a GPL library doing that ?
Can we spend the many months necessary to do this port on projects more useful?

You are welcome on discord if you want to have impact on the Flutter Sound future.

[Larpoux]

A huge API

The W3C Web Audio API is huge. Really huge.
The W3C defined 36 classes. Each of those 36 classes can have a dozen of methods.

More, some of those methods are about Audio Stream processing.
Things like Audio Volume Compression, Audio mixer, Equalizer, pass low and pass high filters, echo chamber, etc...

We are not musicians nor Audio Sound Processing experts. If we want to build an Web Audio Implementation on iOS and Android, we will have to borrow the algorithms from someone else. The Chromium sources and the Firefox sources will be the main sources for us.

But better if we can use an already existing portable library doing that..

Web Audio Libraries

I tried to find on ethernet the Web Audio libraries already existing.
I found those projects :

• Audiojs Web-Audio-API
• Mohayonao Web-Audio-Engine
• LabSound
• Node Audio
• Descriptinc Web-Audio-JS
• WAAPISim
• Orottier Web-Audio-API-rs

Audiojs Web-Audio-API

This is a partial Javascript implementation of the API.
This project is dead : the last commit was five years ago.
This project never reached V1.0

The only nodes implemented are :

• AudioContext (partially)
• AudioParam (almost there)
• AudioBufferSourceNode
• ScriptProcessorNode
• GainNode

One of the contributors suggest to switch to another library here.
My position is that this Project is not good for us.

Mohayonao Web-Audio-Engine

This is another port of the Audio Web API under Javascript
This project is dead. The last commit was four years ago.

This project implements most of the Nodes defined by the W3C. The nodes implemented are :

• AnalyserNode
• AudioBuffer
• AudioBufferSourceNode
• AudioContext
• AudioDestinationNode
• AudioNode
• AudioParam
• BiquadFilterNode (audio rate parameter is not supported)
• ChannelMergerNode
• ChannelSplitterNode
• DelayNode (noisy..)
• GainNode
• IIRFIlterNode
• OscillatorNode (use wave-table synthesis, not use periodic wave)
• PeriodicWave
• ScriptProcessorNode
• StereoPannerNode
• WaveShaperNode

My position is that this project is more or less a possible candidate for us.
The major problems are that this port is dead and using a Javascript library from Flutter on Mobiles looks like a hack.
The good point is that a port to React Native should be simple.

LabSound

This is a port of the Web Audio API under C++
The last commit was 5 months ago.
511 stars on Github

LabSound originated as a fork of WebKit's WebAudio implementation, as used in Google's Chrome and Apple's Safari.

LabSound is derived from one of the original WebAudio implementations, but does not maintain 
full compatibility with the [spec](http://www.w3.org/TR/webaudio/). 
In many cases, LabSound has deliberately deviated from the spec for performance or API usability reasons. 
This is expected to continue into the future as new functionality is added to the engine.

My position is that it is one of the two best candidates for us.

Node Audio

This is a Javascript wrapper above LabSound.

node-audio is a graph-based audio processing library for Node.Js. It's very similar to the web-audio api.
In fact in its current iteration it is little more than a binding for [LabSound] with some sugar on top!
...
Currently only a small subset of the web-audio-api is available.

The last version was published four years ago.

My position is that I do not see any benefit to use a Javascript wrapper instead of LabSound which is a C++ library.

Descriptinc Web-Audio-JS

This is a Javascript implementation of the W3C Web Audio API.

The class implemented are more or less the same than web-audio-engine.

This is a fork of mohayonao/web-audio-engine
with following changes:

• Add new RawDataAudioContext (#5)
• Add support for DynamicsCompressorNode (#1)
• Fixes for BiquadFilterNode and DelayNode (#8)
• Fix WAVE file decoding (#5)
• Fix AudioNode.disconnect(x) not disconnecting (#7[]

My position is same as for mohayonao/web-audio-engine.
It is more or less a possible candidate for us if we want a Javascript port of WAAPI

WAAPISim

This is again a Javascript port of the W3C Web Audio API.
This project is dead. The last commit was six years ago.

The classes implemented are more or less the same than for Audiojs Web-Audio-API

My position is same as for mohayonao/web-audio-engine.
It is more or less a possible candidate for us if we want a Javascript port of WAAPI

Orottier Web-Audio-API-rs

This is a post of the W3C WAAPI under Rust.

This project is really alive. The last commit was 13 days ago !
Github has only 27 stars. I guess that this library is not mainly used by many developers.

The library depends on the Rust library CPAL, which is an abstraction layer above all the platforms we want to support.
Specifically the OBOE library which is the reference to do streams on Android.

This seems to be a very good design.

My position is that this port can be one of the two best candidates for us.

Conclusion

I do not know any one of these libraries.
My actual thought is the ports under Javascript is not what we need :

• All those ports are dead (last commit more than four years ago)
• The use of a Javascript library looks like a kind of hack when used for Flutter or C# .DOTNET

I suggest to try to port my little example that plays 2 seconds of white noise under :

• The C++ library : LabSound
• The rust library : Orottier Web-Audio-API-rs

Then we will have a better idea of what library we can depend on.

You are welcome on discord if you want to have an impact on the Flutter Sound future.

[alexda8999]

@Larpoux This is really good what you have put here !

My gut feel here - I dont like JS as a solution for using natively, this definitely isn't the right approach to adopt here - I have not heard of (or used) any of those libraries you have mentioned but on the face of it - If our initial focus is Flutter then I would look at extending the example to C++/C so that we can easily access via FFI.

So my vote is a YES for :

I suggest to try to port my little example that plays 2 seconds of white noise under :

The C++ library : LabSound
The rust library : Orottier Web-Audio-API-rs

[Larpoux]

Hi Alex. I am glad to have news from you. I was afraid that you got the Corona Virus a second time. We really need you.

I have tested the latest version of Flutter Sound 9.1.2 now - This includes integrating with Just_audio, Just Audio service, Flutter TTS, Audio Session etc ...
I tested on iOS and Android and pleased to say that this works now in harmony with no issues

This is great :)

I only have 1 issue that I didnt have before in this new version which relates to how we display progress when we record ....

I think I have never seen during my life a so simple thing giving so many problems. This is unbelievable.
Probably more than one hundred issues on this simple functionality.
When things are like that, I think that the only good choice is to erase the functionality and re-code it completely.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Leave a comment or this will be closed in 7 days.