html midi API bindings are broken

[robertmuth]

Dart VM version: 2.0.0-dev.58.0 (Unknown timestamp) on "linux_x64"

(tried ddc but I believe dart2js does no work either)

The problem seems to go beyond promises:

void Ready(succ) {
    for (dynamic k in succ.inputs.keys) {
      print("$k ${succ.inputs[k]}");
    }

}

void main() async {
  ...
  dynamic a = HTML.window.navigator.requestMidiAccess();
  print(">>>> $a");
  a.then(allowInterop(Ready));
  ...
}

succ.inputs is supposed to be a map from device-id to an object but
the object returned seems to always be the empty dict.
In JS I get a real object.

[terrylucas]

Robert looks like the requestMidiAccess should be:

Future requestMidiAccess([Map options]) {
if (options != null) {
var options_1 = convertDartToNative_Dictionary(options);
return promiseToFuture(JS("", "#.requestMIDIAccess(#)", this, options_1));
}
return promiseToFuture(JS("", "#.requestMIDIAccess()", this));
}

However, when I now call requestMIDIAccess() I get a DOMException of

"Platform dependent initialization failed."

I was running this on a Linux box which might be the problem too. I'll make the change when I'm back in the office on Monday and try on a Mac.

[terrylucas]

Tested on a Mac and I get back a MIDIAccess object. Looks right to me. I'll make the change and check it in.

[robertmuth]

Just tested this with
dart --version
Dart VM version: 2.1.0-dev.0.0 (Unknown timestamp) on "linux_x64"

I do get a MidiAccess Object but that object is not quite right.

E.g this fragment

void Ready(HTML.MidiAccess succ) {
print("INPUTS ${succ.inputs}");
print("OUTPUTS ${succ.outputs}");
}

void main() async {
StatsFps fps =
StatsFps(HTML.document.getElementById("stats"), "blue", "gray");

dynamic a = HTML.window.navigator.requestMidiAccess();
print("@@ PROMISE >>>> $a");

a.then(Ready);
}

produces
@@ PROMISE >>>> [object Promise]
INPUTS {1A0CE1F47C19C43DAEB93B9595F6E4BA822ED318FE8D8BF2750B66286B5BEC38: {}, 4D64D764A8E7ACC999F36DCA46353327DA51FA03A1E1720E637B4AE659CB51BD: {}}
OUTPUTS {6FF5590044F4859ED50C5167BCFE9700A1798E39AA55A628E86D39011FAECD5D: {}, 63D30F7D6DEFA1323CFF0A6E791F0189F4B09818B1723F6FC918949E7D957CE0: {}}

note that the map values for INPUTS and OUTPUTS are empty maps

MidiAccess.inputs and MidiAccess.outputs do not seem to be properly initialized

[robertmuth]

This has regressed even more. Running the code from my previous comment now yields

Uncaught (in promise) TypeError: this.requestMidiAccess is not a function
at Navigator.[dartx.requestMidiAccess] (dart_sdk.js:82597)
at main (midi_input.dart:20)

where midi_input.dart:20 is the line:
dynamic a = HTML.window.navigator.requestMidiAccess();

dart --version
Dart VM version: 2.3.0 (Unknown timestamp) on "linux_x64"

[mandersson1024]

Still happening, half a year later.

Uncaught TypeError: this.requestMidiAccess is not a function

Dart VM version: 2.5.2 (Tue Oct 8 16:17:11 2019 +0200) on "macos_x64"

[robertmuth]

@vsmenon

any update? This bug is soon celebrating its 2 year anniversary.

[mortenboye]

Still seeing this. Is there a known workaround? Using JS maybe?

[clevertree]

This would be yet another example of corporate sabotage. They are holding back the PWA..

[stuartwk]

Any update on getting MIDI to work for Flutter web?

[xhevahir]

Curious, and unfortunate, that this has remained unaddressed for so long. I understand that are some security concerns to do with Web MIDI; could that be the reason?

[robertmuth]

No, it is just not a priority for them.

[DominikStarke]

At least for flutter one could work around the limitation like so:

import 'dart:js' as js;
import 'dart:html';

// ...
final script = window.document.createElement("script");
final head = window.document.getElementsByTagName("head").first;
script.text='var MidiShim={requestMIDIAccess:(s,e)=>{navigator.requestMIDIAccess().then(s,e)}}';
head.append(script);
final module = js.context['MidiShim'];

module.callMethod('requestMIDIAccess', [(midiAccess) {
  midiAccess['addEventListener'].apply(['statechange', (Event event) => print(event)]);
  midiAccess['inputs'].callMethod('forEach', [(input) {
    input['onmidimessage'] = (MidiMessageEvent msg) {
      print(msg.data);
    };
  }]);
}, (error) {
  print(error);
}]);

I don't know how reliable this is though.

[clevertree]

These days I always consider the possibility of corporate sabotage. I tend to get banned for suggesting this, but I've seen it so many times now that it has become the rule, not the exception. This is the only way the old industry can stay competitive with the emerging.

[maks]

Thanks to @DominikStarke comment, I'm using a variation of his work around successfully at the moment in a Flutter web app.

Which shows that its just a case of needing to fix whatever the dart:html WebMidi code is doing to do the initialisation so I would like to have a go at submiting a proper fix for this but I can't for the life of me even find where in the dart sdk repo that code even lives?

@terrylucas could you point me to where the source for the WebMidi classes for dart:html is?

[zejji]

+1 I would also love to see a fix for this issue

[maks]

Ok after a lot of trial and error, probably more than any sane person would do (but this 4 year old bug really annoyed me) I seem to have discovered the cause of the problem, or at least the main problem.
What led me to this discovery was noticing that switching to the excellent new js_bindings package from @jodinathan had exactly the same problem, even though it uses the new package:js.
So after much tinkering what I've discovered is the issue seems to be due to WebMidi's use of "maplike" objects objects for MidiInput and MidiOutputMap.

eg. setting up some JS to pass thru the inputs as an actual array works fine:

var getMidi =
  function (callback) {
    navigator.requestMIDIAccess({ sysex: true }).then((midiAccess) => {
      console.log('access', midiAccess);
      let inputs = [];
      midiAccess.inputs.forEach((input, name) => {
        console.log(`input name: ${name} input:${input.name}`);
        inputs.push(input);
      });
      callback(inputs)
    });
  }

And with "receiving" Dart I get:

void success(a) {
      print('SUCCESS IN DART got: ${a}');
      print('inputs IN DART got: ${a}');
      for (var inp in a) {
        final name = js_util.getProperty(inp, 'name');
        print('input name in Dart: $name');
      }
    }
    void failure() {
      print('FAILURE IN DART');
    }
    getMidi(allowInterop(success));

I get output as follows:

input name: 1A0CE1F47C19C43DAEB93B9595F6E4BA822ED318FE8D8BF2750B66286B5BEC38 input:Midi Through Port-0
midi.js:16 input name: F888200556D3C1DA1130EB1F10DA98E9A757602ADA9EDD38E618467214E79F9C input:FL STUDIO FIRE Jack 1
SUCCESS IN DART got: [[object MIDIInput], [object MIDIInput]]
inputs IN DART got: [[object MIDIInput], [object MIDIInput]]
input name in Dart: Midi Through Port-0
input name in Dart: FL STUDIO FIRE Jack 1

BUT switch the JS to return midiAccess.inputs directly and bam! I get the same failure mode as had been documented in this issue all the way back to 2018 (and there are actually older bugs that suggest this bad behaviour was happening well before then :-( )

Unfortunately I have no idea how to go about addressing this issue as I think it could be down to how the Dart->JS compiler (I'm testing with DDC) is generating code to map across "maplike" objects from JS to Dart?

The only references I could find to "maplike" here in the sdk repo that looked like it may have something to do with this was here but I've honestly no idea if thats really relevant to this.

Perhaps someone like @srujzs or @jmesserly or some other Dart team member who works on the Dart->JS compiler would know more about this? It would be nice to close a very long standing bug like this, even if WebMidi is a bit niche, as Firefox has finally now also just actually released support for WebMidi too in their nightlies.

[jodinathan]

@maks What I would do is use a build.yaml in release mode and check the main.dart.js in .dart_tool/path/main.dart.js and check the transpilled JS to try to understand what is wrong there.

I guess you know how to do this but nevertheless here is an example of build.yaml to use dart2js with webdev serve:

targets:
  $default:
    sources:
      exclude: []
    builders:
      build_web_compilers|entrypoint:
        options:
          compiler: dart2js
          dart2js_args:
            - -O4
            - --no-minify

[jodinathan]

@maks how are you returning the inputs?

I have no MIDI here so my inputs are empty be it JS or Dart.

What happens with this code?

import 'package:js_bindings/js_bindings.dart';

Future<JsMap> foo() async {
  final a = await window.navigator.requestMIDIAccess();
  print('got MIdiAccess: ${a.inputs}');

  return a.inputs;
}

void main() async {
  document.title = 'JS Bindings example';

  final inps = await foo();

  window.console.log(inps);
  for (var i in inps.values) {
    print('midi input:$i');
  }
}

[maks]

@jodinathan Thanks for looking into this so quickly! And apologies for me not providing a better demo of the bug and what I thought to be the issue.
I've now done a minimal dart webapp that reproduces both the issue with using maplikes and the work around of using isntead a plain JS array to pass the same MidiInput objects: https://github.com/maks/webmidi_bug_demo

I'm not sure why you were not seeing any midi inputs even with your plain JS code, for me on Chrome on Linux even with no midi devices plugged in I always see the "Midi Through Port - 0" input (pls see screenshot linked in the Readme of the above demo repo)

I'm not sure why you were not seeing the through port at least, perhaps you had some setting in your Chrome browser that was overriding the Midi permission request dialog from being shown? It seems for instance that DartPad does this via a HTTP header.

Hopefully my demo repo works for you to help demonstrate the issue here.

[maks]

@jodinathan also thank you so much for the instructions on how to look at the transpiled JS 👍🏻 I was actually searching for exactly that yesterday and had been able to find out how to do it.
I'll have a go later today and using that setting to see the code for my minimal demo app and see if I can spot what the problem is with the generated JS.

[maks]

so I now really looked at the prev comment from @DominikStarke with the JS script workaround and finally noticed the use of forEach() and sure enough this works in my demo app:

import 'package:js/js.dart';
import 'package:js/js_util.dart' as jsutil;
import 'package:js_bindings/js_bindings.dart' as html;


void showInputs(dynamic a, dynamic b, dynamic c) {
  print('got args from forEach: [$a] [$b] [$c]');
  final inp = a as html.MIDIInput;
  print('input name: ${inp.name}');
}

void main() async {
  final access = await html.window.navigator
      .requestMIDIAccess(html.MIDIOptions(sysex: true, software: false));

  final inputs = access.inputs;

  jsutil.callMethod(inputs, 'forEach', [allowInterop(showInputs)]);
}

and I get the following output:

got args from forEach: [[object MIDIInput]] [1A0CE1F47C19C43DAEB93B9595F6E4BA822ED318FE8D8BF2750B66286B5BEC38] [{1A0CE1F47C19C43DAEB93B9595F6E4BA822ED318FE8D8BF2750B66286B5BEC38: {}}]
js_primitives.dart:30 input name: Midi Through Port-0

I had to do a quick head scratch about why its 3 args to the forEach callback function, but looking at the docs for Map.forEach() thats actually correct and its just that the 3rd arg here which should be the "map being traversed" happens to be empty which kind of makes sense given that its not a "real" Map, just "map-like".

So @jodinathan I'm using callMethod to invoke forEach but I guess if you could have an extension in your MidiInputMap instead of extending JSMap extend a different class, something like JSMapLike which then exposed a forEach() that invoked the underlying JS forEach() I think this whole thing could work without my needing to resort to using js_util - does that make sense?

[jodinathan]

@maks regarding the empty map I have nothing about MIDI for what I know.

The JsMap already have a forEach, however, it is manual.

I will change it to use the underlying Js forEach. However, would you mind testing with the current version just to see if it works?

[DominikStarke]

@jodinathan if you need virtual midi devices and are on windows you can use https://www.tobias-erichsen.de/software/virtualmidi.html
On linux you can create virtual midi inputs with ALSA and on MacOS you can use the system settings dialog "Audio MIDI Setup"

[maks]

@jodinathan First off thanks again for all your work on js_bindings! It makes things so much nicer/easier than using "package js" directly!
Unfortunately the existing forEach() on JSMap does work here. If I modify my previous code to:

import 'package:js/js.dart';
import 'package:js/js_util.dart' as jsutil;
import 'package:js_bindings/js_bindings.dart' as html;

void showInputs(dynamic a, dynamic b, dynamic c) {
  print('got args from forEach: |$a| |$b| |$c|');
  final inp = a as html.MIDIInput;
  print('input name: ${inp.name}');
}

void main() async {
  //querySelector('#output')?.text = 'Your Dart app is running.';

  final access = await html.window.navigator
      .requestMIDIAccess(html.MIDIOptions(sysex: true, software: false));

  final inputs = access.inputs;

  jsutil.callMethod(inputs, 'forEach', [allowInterop(showInputs)]);
  inputs.forEach((p0, p1) {
    print('got args from DART forEach: |$p0| |$p1|');
  });
}

The output I get is:

got args from forEach: |[object MIDIInput]| |1A0CE1F47C19C43DAEB93B9595F6E4BA822ED318FE8D8BF2750B66286B5BEC38| |{1A0CE1F47C19C43DAEB93B9595F6E4BA822ED318FE8D8BF2750B66286B5BEC38: {}}|
js_primitives.dart:30 input name: Midi Through Port-0
js_primitives.dart:30 got args from DART forEach: |new| |function() {
    _interceptors.JavaScriptObject.__proto__.new.call(this);
    ;
  }|
js_primitives.dart:30 got args from DART forEach: |is| |function is_C(obj) {
      return obj != null && (obj[isClass] || dart.is(obj, this));
    }|
js_primitives.dart:30 got args from DART forEach: |as| |function as_C(obj) {
      if (obj != null && obj[isClass]) return obj;
      return dart.as(obj, this);
    }|

---

On the topic of testing WebMidi, it looks like the default "Midi Through Port - 0" is a Linux thing, I just checked on a Mac and it doesn't have it so I think if you are on MacOS or Windows you'll need to create at least one virtual midi port per @DominikStarke latest comment here to test out the Webmidi api in Chrome, even with plain JS.

[jodinathan]

@maks I've published a 0.0.6-dev version with a fix for Map-like structures.
Could you test please?

[maks]

@jodinathan thank you again! Just tried 0.0.6-dev and your new forEach() works perfectly for me 👍🏻
That has allowed me to remove the last direct usage I had of dart:js_util in my code

[jodinathan]

@maks I've tried to make the JsMap extension to fulfill the JS Map features, so you can also use .values and .keys =]

In fact, would be nice if you could test them.

I will publish a stable version rn

[jodinathan]

@sigmundch I guess issues like this can be directed to use js_bindings and closed?

[maks]

@sigmundch I would agree, given it's now possible to use requestMidiAccess(), with a "modern" equivalent to the dart:html api being provided by js:bindings without any JS workarounds needed, I think this issue can (finally 🎉 ) be closed.

[sigmundch]

Thank you both! This very exciting!

js_bindings really highlights why we've been investing heavily in making the low-level JSInterop building blocks more robust. It was precisely with this goal in mind: to enable a future where most DOM APIs are available through interop and where they can be adjusted more easily over time. Exactly like you have achieved with js_binding today!

/cc @srujzs @rileyporter let's discuss how we want to approach this and other similar bugs on our next sync up when we are all back online. I'll keep this open in the meantime.