New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Midi-Profiles #28

Closed
mheidt opened this Issue Nov 30, 2017 · 67 comments

Comments

Projects
2 participants
@mheidt
Copy link
Contributor

mheidt commented Nov 30, 2017

What do we need to do, to let zynthian become a midi-host?

  • Listing all midi devices?
  • aconnect

Do you want to have at least a midi-host-on/off switch in the zynthian-ui?

Maybe we should use this task to get a zynthian-ui / webconf communication working using the websocket.

@mheidt mheidt added the enhancement label Nov 30, 2017

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Nov 30, 2017

Hi @mheidt !

Currently Zynthian is a MIDI-Host, it take all the MIDI input coming from MIDI-IN and USB-MIDI, applies the filter rules and sends the result to the MIDI-OUT port and the internal Synth-Engines.

I suposse you would like to also send the output to the USB-MIDI devices. It would be really nice to do that "auto-magically", but some USB-MIDI devices have input and output ports and most of times we would want to avoid creating self-device loops.

My proposal is:

  • When a USB-MIDI device have ONLY input OR output ports, then we can connect it auto-magically.
  • When a USB-MIDI device have both kind of ports, we should use it as an input port by default, but allowing to change this behaviour from the webconf, giving 3 options:
    • input
    • output
    • input/ouput

What do you think?

Regarding the "tools", i've removed the use of ALSA tools (i.e. aconnect). Only jack should be used. Take a look to the autoconnector class in the zynthian-ui ;-)

Regards,

@jofemodo jofemodo added this to Analyzing in Aruk Nov 30, 2017

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Nov 30, 2017

Don't we have those 4 Options?
USB-IN / MIDI-OUT (does it mean THROUGH as well?)
USB-OUT / MIDI-IN
USB-IN / MIDI-IN (for playing 2 keyboards with different midi channels)
USB-OUT / MIDI-OUT (in case zynthian becomes a playback machine in the future)

And depending on the supported USB options, we can disable accordingly.
Can we switch MIDI-IN/OUT per software without reboot?

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Nov 30, 2017

And what if we have two Midi jacks? Do we want an USB-IN/OUT-switch or do we want to be able to configure both real MIDI jacks freely?

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 1, 2017

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Dec 1, 2017

It's what i'm using for the auto-connect system in zynthian-ui. It's already installed in Zynthian ;-)

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 1, 2017

This means, we need a common code base :)
My first steps would be listing all MIDI Ports:
client.get_ports(is_midi=True, is_physical=True)

once with is_input=True and separately is_output=True

If a port is in both list, I will enable a select box (IN / OUT), so that the user can determine the behaviour?

But maybe I just watch and learn first and then we decide about a common code base.

And no matter what, I won't change the behaviour directly. That has to be done by -sys.
The webconf should only save config descriptions, that can be backed up and restored.

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 5, 2017

I checked in a new branch which only contains a simple list of the midiports

image

I wonder, why "in" and "out" don't match the is_input or is_output of the get_ports call.

I think of adding a new modal "+"-dialog similar to the midifilter.

Guess, I should only work with a single dialog, where one can select the midi ports. I have to figure out the disclosure rules. In my case, if I select midi_playback_1, i can't select midi_capture_1. If there is a second real midi-port, it is midi_xyz_2?
How would a second USB-Midi interface look like?
ttymidi:MIDI_out_2 ?
ttymidi2:MIDI_out ?

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 9, 2017

latest checkin provides the selection of midi_ports and stores it into config['ZYNTHIAN_MIDI_PORTS'].
Now @jofemodo could integrate it into zynthian_sys?

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 10, 2017

@jofemodo
I would like to introduce "midi profiles".
Maybe the whole midi section could be tab-enabled and each tab is a profile.
In the top a select box which defines the currently enabled profile.
The profiles should be selectable in the zynthian-ui as well.
And it should be stored in the snapshot.
What do you think?

@jofemodo jofemodo moved this from Analyzing to Work in Progress in Aruk Dec 11, 2017

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Dec 11, 2017

Hi @meidt!

I like the idea of MIDI profiles. It could be really useful. If we do so, we should use a configuration file/directory structure for saving the different profiles. Only the "active" profile would be exported as a environment variable.

Some ideas that come to my brain when reviewing the #gh28:

  • It should be nice to get the "ALSA device" name, as it's more explicit about the device. MOD-UI does that ;-) "Jackd" names (midi_capture_1, etc.) could change depending the order of connection, so we should avoid it.

  • Currently Zynthian routes like that:

    • FROM ALL physical MIDI inputs (ttymidi:MIDI_in, system:*) => (Zyncoder:input)
    • APPLY MIDI FILTER RULES (Zyncoder:output) => (ttymidi:MIDI_out)
    • APPLY MIDI FILTER RULES (Zyncoder:output) => (engine[i]:input)
      _
  • I think that MIDI-IN and MIDI-OUT should be ALWAYS active as input and output. If you don't want to use it, don't connect devices to it ;-) So, it should be removed from the list.

  • Same for an USB-MIDI device that has only an INPUT or OUPUT port. If you connect the device, then you want to use it, and if it only can be used as INPUT or OUTPUT, then there is no choice to make.

  • If an USB-MIDI device has INPUT & OUTPUT ports, then it has sense to give the option of configuring how we want to use the device. I think that we should give the 3 options:

    • INPUT
    • OUTPUT
    • INPUT/OUTPUT => this could be useful in some cases ...why not?

Regards,

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 11, 2017

Ok, i will only list USB-MIDI ports and change radio to checkbox.
And we should integrate this MIDI over Ethernet feature.
Maybe you should compile it and let me know, if I have to integrate those ports as well.

Midi profile file: YAML or JSON?

Ah. We should extend the MIDI-Filter rules, so that a rule triggers certain scripts (with a defined parameter structure to pass the midi parameters)
AND... to trigger encoder changes.
It would be possible to handle the zynthian with a midi keyboard and no encoders, if the midi commands map to the dummy/virtual encoders.
We need virtual encoders, so that any midi cc can be defined. and it should be possible to define if you have a real encoder wired up. A peaceful harmony between real encoders and virtual ones that are steered by web-ui or midi. Midi commands change the encoder and vice versa, visualized in a web-ui.

Maybe this should be discussed in a different gh-issue :)

@mheidt mheidt changed the title Midi-Host Midi-Profiles Dec 11, 2017

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 12, 2017

Midi Profile Files: Maybe we split it and have it the envar-style? A file you can "run" and which sets the envars?
In the zynthian_envar we store the filename.
A default /zynthian/zynthian-data/midi-profiles/default-profile.sh will be provided.
User profiles in my-data...
New profile takes the vars of the currently selected values.

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 12, 2017

Is this one used?
ZYNTHIAN_MASTER_MIDI_BANK_CHANGE_CCNUM
It doesn't appear in the zynthian_envar.sh
But it should have a value...0 or 32...

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 13, 2017

I want to let the parameters begin with ZYNTHIAN_MIDI.
This means, that we have to rename
ZYNTHIAN_PRESET_PRELOAD_NOTEON

All those ZYNTHIAN_MASTER_MIDI... parameters aren't not used yet? Maybe some of them could be implemented with an enhancement of MIDI_FILTER_RULE?

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 13, 2017

You can have a look now.
I wasn't quite sure, what you meant by Alsa name. I took the shortname now.
And I didn't filter the ports yet.
I couldn't get this qmidinet working on my linux machine. My hope lies on you and a zynthian integration.

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Dec 18, 2017

Hi @mheidt !

I see you are using "midi-profiles" directory in zynthian-data. Please, better move this directory to "zynthian-my-data". "zynthian-data" is a repository with the "factory" data, and no user data should be wrote there.

Regards,

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 18, 2017

@jofemodo
The plan is, reading from both locations, but only saving and deleting from my-data.
zynthian-data for the default factory settings...
Isn't it implemented like that?

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Dec 18, 2017

OK. Sorry. I just saw that you define system midi profiles and user midi profiles. Don't change nothing ;-)

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Dec 18, 2017

I have created both directories, but it gives an error:

ERROR:tornado.application:Uncaught exception GET /api/ui-midi (192.168.1.198) HTTPServerRequest(protocol='http', host='zynthian.local', method='GET', uri='/api/ui-midi', version='HTTP/1.1', remote_ip='192.168.1.198', headers={'Connection': 'keep-alive', 'Cookie': 'user="2|1:0|10:1513018320|4:user|8:cm9vdA==|be08e602f8b959c17560ef3b499a635ce78052edefcfbbeaa8510230a524d3fb"', 'Accept-Encoding': 'gzip, deflate', 'User-Agent': 'Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:57.0) Gecko/20100101 Firefox/57.0', 'Host': 'zynthian.local', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.5', 'Upgrade-Insecure-Requests': '1'}) Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/tornado/web.py", line 1323, in _execute result = self.prepare() File "/zynthian/zynthian-webconf/lib/midi_config_handler.py", line 135, in prepare self.load_midi_profile_directories() File "/zynthian/zynthian-webconf/lib/midi_config_handler.py", line 332, in load_midi_profile_directories self.current_midi_profile_script = os.getenv('ZYNTHIAN_SCRIPT_MIDI_PROFILE',self.midi_profile_scripts[0]) IndexError: list index out of range ERROR:tornado.access:500 GET /api/ui-midi (192.168.1.198) 10.94ms ^CTraceback (most recent call last): File "./zynthian_webconf.py", line 103, in <module> tornado.ioloop.IOLoop.current().start() File "/usr/local/lib/python3.4/dist-packages/tornado/ioloop.py", line 815, in start event_pairs = self._impl.poll(poll_timeout)

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 18, 2017

You didn't put a default file in the zynthian-data folder?
Containing all ZYNTHIAN_MIDI entries of zynthian_envvar?

#!/bin/bash
export ZYNTHIAN_MIDI_NETWORK_ENABLED="0"
export ZYNTHIAN_MIDI_MASTER_PROGRAM_CHANGE_UP=""
export ZYNTHIAN_MIDI_FILTER_RULES=""
export ZYNTHIAN_MIDI_MASTER_PROGRAM_CHANGE_DOWN=""
export ZYNTHIAN_MIDI_MASTER_PROGRAM_CHANGE_TYPE="Custom"
export ZYNTHIAN_MIDI_MASTER_CHANNEL="16"
export ZYNTHIAN_MIDI_MASTER_BANK_CHANGE_UP=""
export ZYNTHIAN_MIDI_MASTER_BANK_CHANGE_CCNUM="32"
export ZYNTHIAN_MIDI_PRESET_PRELOAD_NOTEON="0"
export ZYNTHIAN_MIDI_FINE_TUNING="440"
export ZYNTHIAN_MIDI_PORTS=""
export ZYNTHIAN_MIDI_MASTER_BANK_CHANGE_DOWN=""

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 18, 2017

But as i said, I think, we won't need those Program_change entries. Do you use them anyways? They smell like an enhancement for the midi filter rules.

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Dec 19, 2017

This "Program Change" entries are used for configuring snapshot loading, do you remember? ;-)
Of course, this feature could be re-defined as new filter rules ... let me think about it ;-)

Regards,

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Dec 19, 2017

Ahhh! The right names are: ZYNTHIAN_MASTER_MIDI_XXX ;-)

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 19, 2017

Not anymore...The right ones are now ZYNTHIAN_MIDI_MASTER_XXX
If a field does not startwith ZYNTHIAN_MIDI, it is not stored in the midi profile.
Other upper-case ZYNTHIAN_ vars are stored in zynthian_envvar. AND if it is lowercase, it is ignored.

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Dec 19, 2017

but I don't see those PROGRAM/BANK Change fields as separate parameters anymore but as rule.

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Dec 19, 2017

OK! I will change that in the UI ;-)
And i will think about moving the "Master Program Change" feature to the MIDI filter system ... ;-)

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 15, 2018

Can you check the "git status" in your zynthian-sys directory? It seems like not being updated (perhaps you add some new file by hand?)

Anyway, the trick is done in the "update_zynthian_sys.sh" script, look at lines 146 and 188.

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Jan 15, 2018

I see the part for my-data.
But I expect
/zynthian/zynthian-data/midi-profiles/default.sh that holds the former values of zynthian_envvars

And I don't see, where the ZYNTHIAN_SCRIPT_MIDI_PROFILE is called.

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 16, 2018

Can you check the "git status" in your zynthian-sys directory? It seems like not being updated (perhaps you add some new file by hand?)

Anyway, the trick is done in the "update_zynthian_sys.sh" script, look at lines 146 and 188.

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 16, 2018

OK! I have it!
You have to run "update_zynthian_data.sh" for pulling zynthian-data repositoy. This will create the midi-profiles directory in zynthian-data ;-)

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 16, 2018

Anyway, it can't be like that. Files in zynthian-data can't be changed in anyway locally, as it comes from repository and it would be overwriten in the next update, so "midi-profiles/default.sh" should be in "my-data". It should be only one "midi-profiles" directory and it should be in "zynthian-my-data".

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 16, 2018

I just commited some changes ...

  • modified the code for using only the my-data directory
  • fixed the "update_profile" function, escaping the "\n" in strings
  • fixed some encoding/decoding problems when changing the MIDI profile from the select. MIDI filter rules didn't get correctly escaped.

I think we are close to close now .... ;-)

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Jan 17, 2018

Are these items addressed?

  • Network Midi
  • Midi Port List
  • Zynthian-Ui selection of a midi profile.

What do you need from me for the midi port list?
Only USB ports?

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 17, 2018

I'm working on it ;-)

Regarding the MIDI port list, keep it like this, but:

  • Explicit (ALSA) names should be used
  • All input ports should be active by default
  • MIDI-OUT should be active by default
  • In the popup: Input port column should be at left and output port column at right. It's more intuitive and follow the convention used in MOD-UI.

Perhaps the best approach is thinking in terms of black/white list:

  • Input Port Black List: ports explicitly disabled. Empty by default.
  • Output Port White List: ports implicitly enabled. By default only "MIDI-OUT".

Thanks!

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Jan 17, 2018

I changed the name already to something else.
How do I get the ALSA name?

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 17, 2018

I see midi_playback_1, midi_capture_1 and so on.
I called "ALSA" name, but i think you can get it from the jack device/port object in python.
Currently you are using "midi_port.shortname". What about the "midi_port.name"?

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 17, 2018

OK! port.name doesn't work. I'm looking for a solution in the MOD-UI source code ;-)

@jofemodo jofemodo closed this Jan 17, 2018

Aruk automation moved this from Analyzing to Done Jan 17, 2018

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 17, 2018

OK! We need this jack function:

int jack_port_get_aliases (const jack_port_t *port, char *const aliases[2])

but it's not implemented in the python library. MOD-UI is using its own "utils" library. I would try to load the MOD-UI library and will see ...

Regards

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 17, 2018

I've pushed some changes. Don't forget to pull it before modifying!

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 17, 2018

Hi @mheidt !

I've modified the code for using port aliases instead of jack names. I'm using the utils library from mod-ui, but it should be changed because when MOD-UI is enabled it doesn't work due to the hardcoded jack client name (mod-ui), that collides. So, i've to implement a smaller zynthian "utils" library using the mod-ui library as "inspiration" ;-)

Anyway, the "port management" feature is almost done and i've uplaoded the changes to the repository, so you can check and test if you like ;-)

Of course, i've to implement the "auto-connector" side of the thing, but i will do when the new "utils" library is ready, because i will use there too.

Regards,

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 18, 2018

Hi @mheidt !

I forked the JackClient python library and implemented the "alias" property. I've created a pull request for including this changes in the official version.

Until then, we can install the modified version from the Zynthian repository:

pip3 uninstall JACK-Client
git clone https://github.com/zynthian/jackclient-python.git
cd jackclient-python
python3 ./setup.py install

Then, i've changed the webconf "MIDI ports" code for using this new "alias" property instead of calling the mod-ui utils library. The changes are pushed and it works OK ;-)

Enjoy!

@jofemodo jofemodo reopened this Jan 24, 2018

Aruk automation moved this from Done to Analyzing Jan 24, 2018

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Jan 24, 2018

Ups! It seems like i closed the issue by accident. I just re-opened and i hope it's really close to the end.

These sub-tasks have been completed/implemented/integrated and tested =>

  • MIDI ports: Alias-names are used for the devices/ports, not the generic jack names that could change depending on the order of connection/detection of USB devices. Input Ports are implicitly enabled (must be explicitly disabled) and Output Ports are implicitly disabled (must be explicitly enabled). The default profile explicitly enable the legacy MIDI-OUT ;-)

  • Zynthian-UI: There is a new option in the admin menu for selecting the midi-profile. It will re-load the midi profile without re-starting the GUI and maintaining the layers and current status.

  • Auto-Connector: The auto-connector now use the ports configuration (lists of disabled input ports and enabled output ports) for dynamically connecting everything. If the port configuration changes, the auto-connector will reflect the changes in the next refresh (about 2 seconds).

By the way, i have re-factorized some code, and created some new module/library (zynconf) to avoid redundant code in Zynthian-UI and Webconf.

Only the "MIDI over network" subtask is pending. I will do it ASAP.

Please, if you have time, i would love you could test everything by yourself.
Anyway, i'm quite confident and it's in master now ;-)

Kind Regards

mheidt added a commit that referenced this issue Feb 5, 2018

@jofemodo jofemodo moved this from Analyzing to Work in Progress in Aruk Feb 6, 2018

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Feb 26, 2018

The midi settings like tuning is not displayed correctly in the dashboard

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Feb 26, 2018

Ups!! My fault! It's solved now. You also need to pull zynthian-ui because of "zynconf" library modifications. "update_zynthian.sh" will do the task for normal @zynthianers ;-)

@mheidt

This comment has been minimized.

Copy link
Contributor Author

mheidt commented Mar 15, 2018

can we close this one?

@jofemodo

This comment has been minimized.

Copy link
Member

jofemodo commented Mar 16, 2018

It's pending of integrating the "MIDI over network" feature ... but perhaps i can open another task for that and close this one ...

@jofemodo jofemodo closed this Mar 16, 2018

Aruk automation moved this from Work in Progress to Done Mar 16, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment