Fixture definition rework

[mcallegari]

The plan is documented here
The rework ~~~is happening in the newdef branch~~~ has been merged to master
When completed, this should address #696 and #989

Items on the list:

• 1. Channel presets
• 2. Channel default value
• 3. Capability presets
• 4. Global/per-mode physical information
• 5. Channel alias
• 6. Update the current definitions to the new format
  • 6.1 Single capability channels
  • 6.2 Capability presets
  • 6.3 Aliases
  • 6.4 fix map update script
  • 6.5 update XML schema
• 7. Update the QLC+ 4 UI with the new features
• 8. Update the QLC+ 5 UI with the new features

[mcallegari]

@janosvitok @siegmund42 @fxedel for reference

[fxedel]

Will you change the channel alias syntax as well to support multiple modes?

[mcallegari]

Will you change the channel alias syntax as well to support multiple modes?

I don't think so. I believe you're referring to the example Lukas gave about the Eurolite bar.
When assigning DMX addresses to a Fixture, you need to avoid conflicts with other addresses.
So, the Eurolite bar will have always 12 channels reserved, where by default when CH1 = 0, the other 11 channels will be set to "Nothing".
When changing CH1 value, aliases will kick in, and CH2 to CH12 will appear/disappear on the fly.

In general, guys, please let's not complicate things too much. Everything could be done, indeed, but ask yourself: is it worth it ?
Since this rework is affecting the engine, I don't want to end up handling super complicated cases used maybe by 2-3 fixtures over 1000.
Please let's be practical.

[mcallegari]

Feature 4 addressed by a18c124

[fxedel]

I don't refer to Lukas' example with different modes depending on the first channel. I refer to the reuse of such channels in several (normal, traditional) modes.

My problem is that in your example, the channel number 7 is hardcoded:

<Capability Min="200" Max="255" Preset=”Alias”>Sound active
  <Alias name=”Sound sensitivity” number=”7” />
</Capability>

This may work in mode 1, where Sound Sensitivity is at position 7, but not in mode 2, where Sound Sensitivity is at position 9 (e.g. because Pan fine and Tilt fine are added in this mode). This syntax simply doesn't allow the reuse in different modes – and I thought you were aiming to reduce redundancy?

My proposal was to not reference a channel number (7), but a channel placeholder string, like Program Speed / Sound Sensitivity: The capability sets the value of this placeholder to an existent channel (like Sound Sensitivity) and the mode only references the placeholder.

Possible implementation:

 <Channel Name="Program Speed / Sound Sensitivity">
  <Group Byte="0">Placeholder</Group>
  <Default>Program Speed</Default>
 </Channel>
 <!-- ... -->
 <Channel Name="Auto Programs">
  <Group Byte="0">Effect</Group>
  <Capability Min="0" Max="10">No function</Capability>
  <Capability Min="11" Max="50">Program 1</Capability>
  <Capability Min="51" Max="100">Program 2</Capability>
  <Capability Min="101" Max="255" Preset=”Alias”>Sound active
   <Alias set="Program Speed / Sound Sensitivity" to=”Sound sensitivity” />
  </Capability>
 </Channel>
 <!-- ... -->
 <Mode Name="8bit">
  <Physical><!-- ... --></Physical>
  <Channel Number="0">Pan</Channel>
  <Channel Number="1">Tilt</Channel>
  <Channel Number="2">Auto Programs</Channel>
  <Channel Number="3">Program Speed / Sound Sensitivity</Channel>
 </Mode>
 <Mode Name="16bit">
  <Physical><!-- ... --></Physical>
  <Channel Number="0">Pan</Channel>
  <Channel Number="1">Pan fine</Channel>
  <Channel Number="2">Tilt</Channel>
  <Channel Number="3">Tilt fine</Channel>
  <Channel Number="4">Auto Programs</Channel>
  <Channel Number="5">Program Speed / Sound Sensitivity</Channel>
 </Mode>

That's the way how we handle it at OFL, and it allows us to reuse such switching channels amongst different modes.

Here's a list of example fixtures – they all use the same switching channel in multiple modes at different positions, which means that a hardcoded channel number wouldn't work:

• American DJ Revo Burst
• cameo Hydrabeam 100
• cameo Hydrabeam 300 RGBW
• Futurelight DMH-75.i LED
• Martin MAC Viper AirFX
• Martin MAC Viper Performance
• Robe ColorSpot 2500E AT
• Showtec LED Light Bar RGB V3
• Stairville Stage TRI LED
• Venue TriStrip3Z

As of practicalness: Of 109 OFL fixtures, 28 (25.6%) use switching channels and 10 (9.2%) of them use them in different modes in different positions – Lukas' fixture is an edge case, but this isn't.

[mcallegari]

@fxedel I see your point, but I'm not convinced about placeholders. It's too difficult for users.
I need to think some more about it.
Maybe my idea just lacks of a "mode" attribute, like this:

<Capability Min="200" Max="255" Preset=”Alias”>Sound active
  <Alias name=”Sound sensitivity” mode="10 Channel" number=”7” />
  <Alias name=”Sound sensitivity” mode="16 Channel" number=”5” />
</Capability>

So basically the workflow is:

• define all the possible channels supported by a fixture
• add modes
• define aliases

[mcallegari]

Feature 2 addressed by 72aa602

[fxedel]

@mcallegari I haven't thought about this solution, but this fixes the mode-issue as well. Nice! Looking forward to exporting our switching channels to QLC+ aliases without data loss 😃

Could you give an example how defaultValue would be implemented in XML? We already store it in OFL, so it might be pretty easy to export this on our side.

[mcallegari]

Could you give an example how defaultValue would be implemented in XML? We already store it in OFL, so it might be pretty easy to export this on our side.

Pretty straightforward actually:

<Channel Name="Tilt" Default="128" Preset="PositionTilt"/>

The Default attribute is present only if value != 0

[siegmund42]

Lukas' fixture is an edge case

At least it is part of a group of "edge cases". I did some research and it turns out at least the following fixtures already included in QLC+ will most probably have the same problem:

• Stairville LED BAR 252 RGB (no. 234564)
• Chauvet 4Bar and 4Bar Tri
• Eurolite LED Bar RGB 252/10 40° Indoor (as already mentioned)

Some fixtures not yet included in QLC+ with possible same problem:

• Chauvet ColorBar
• Eurolite LED Bar-18 TCL

I don't exactly know the implications of this, but I guess the number of fixtures is quite low and nobody has ever complained about a problem with it so it should be fine (let's hope no manufacturer will built such fixtures with a silly DMX chart). Just was curious how much of the fixtures will be affected.

Anyway, keep up the good work - looks good to me so far!

[fxedel]

We also have such a fixture in our database:

• JB Systems Twin Effect Laser

We basically use aliases for each channel to be either a working or a Nothing channel, and we added different modes for each mode that can be set in the first channel. It was quite difficult to add this fixture, but it kind of works. A solution to make this easier may be useful, but we also rate this as a low-priority thing.

[mcallegari]

Alright, took me an awful amount of time and patience, but I went through all the 9926 single capability channels and migrated them to the new format.
The space saved is significant (1.5MB on 943 definitions) and the advantages are clear.
Now the update-fixture-map.rb won't work anymore, and I know nothing about Ruby.
To be honest I'd switch to Python, which is more popular than Ruby, so the chances of maintenance are higher.

[janosvitok]

To be honest, I'd migrate the code to C++ (most of the code is already there) and include it in fixture editor and/or a command line tool that would be run as part of build process.

[fxedel]

Updating the schema should be another task in the TODO list, shouldn't it? This could also prove that your immense work on updating the fixtures was right and didn't cause any errors.

As of fixture scripts: As it's important to use maintainable code, Python would probably be a good choice. (Even if I don't like it that much, but I don't know Ruby and therefore can't help updating the existing script.) I think Python is also the most commonly used scripting language in this project, isn't it?

[mcallegari]

@janosvitok updated the definition schema. Thanks.

I've tweaked the Python script I started to generate also the Fixture map. It's now called 'fixtures-tool.py'
The idea is to have a unified tool to do everything with several command line options.
For now --convert and --map are there. --validate could be next.
I have also stripped the map to remove as much as possible redundant information (like the fixture file name extension). The file went from 76k to 62k.
While this doesn't make any difference on a PC, it might help less capable devices (e.g. Raspberry Pi) to start QLC+ faster. We're moving toward 1000 fixture defs, but in a long run they'll be thousands.

[mcallegari]

For who is still following this thread, I've started to add channel aliases.
As I suspected, this is a very complicated change. I already changed my mind like 3 times while writing the code. I have updated the shared document with the (hopefully) final syntax for aliases, which looks like:

<Capability Min="200" Max="255" Preset=”Alias”>Sound active
  <Alias Mode="10 Channel" Channel="Dimmer" With="Sound sensitivity" />
  <Alias Mode="16 Channel" Channel="Dimmer" With="Sound sensitivity" />
</Capability>

That you can read like this: in Mode="10 Channel", replace Channel="Dimmer" With="Sound sensitivity"
It's very verbose, but there are reasons behind this choice:

• aliases are stored into capabilities, but capabilities know nothing about modes and channels. So no indices can be used
• I realized that once a channel is replaced at runtime, its original name is lost, unless the whole channel name (or pointer) is stored somewhere. Therefore the "Channel" attribute is now a string and not a number, to allow "reverting" to the original channel when the DMX value exits an alias.

Changes to the Fixture Editor
For the above reasons, I had to move aliases at fixture level, which has knowledge of modes and all channels. To experiment, I am working on @siegmund42 Eurolite LED Bar RGB 252/10 case, which is massive in terms of alias usage.
The base mode definition looks like this:

And in the new "Alias" tab the situation is this:

Obviously this is an extreme case. Normally the Alias tab will contain just 4-5 entries.
I also attach how the Eurolite bar definition looks like with all the latest changes included.
Eurolite-LED-Bar-RGB-252_10_40__Indoor.zip

Now I "only" need to handle the channels replacement at runtime and update the UI accordingly.
God knows how many implications this carries along 😢

[mcallegari]

There you go

[fxedel]

@mcallegari The preview looks awesome! 🎉

Is it also possible to make a scene that only affects "Sound Sensitivity" and not "Program Speed", if "Sound Sensitivity" is an alias of "Program Speed"? (without setting the channel that activates this alias)

[mcallegari]

Is it also possible to make a scene that only affects "Sound Sensitivity" and not "Program Speed", if "Sound Sensitivity" is an alias of "Program Speed"? (without setting the channel that activates this alias)

Not sure I understand the question, but if I got it right, then I think the answer is no. A Scene is just a snapshot of fixture/channel index/value triples.
In the end, aliases are just a "visual effect" (*). The channels meaning and their usage is up to the user.

(*) with one exception: if an alias replaces a channel from HTP to LTP, then the engine acts accordingly, and that happens internally, without any UI involvement

[fxedel]

Have a look at the Chauvet DJ SlimPAR Pro QZ12 (11-channel mode):

There's a "Programs" channel that includes some Auto modes and a Sound mode and therefore switches another channel between "Program Speed" and "Sound Sensitivity".

So it makes sense to have faders for each "channel behavior", of which only one applies at a time, like so:

Having the ability to only set one channel's value (even if it may be inactive) in scenes, sequences, and so on extends this use case.

Currently, it seems like it's only possible to have a single fader for both "Program Speed" and "Sound Sensitivity", which are completely different effects (e.g. when I have loud and fast music, I want fast speed and low sensitivity).

However, I can guess the huge effort to make changing the channel names and icons depending on an other channel's value working ;). Implementing switching channels in OFL had also been very complicated and has still some limitations.

[mcallegari]

I see the usage case, but I'd say you can resolve it by placing "Program Speed" and "Sound Sensitivity" sliders in a multi page frame. This way both sliders control the same channel, but the value used is the one of the active frame page. Probably you can even automatically switch the frame page when clicking on VC buttons.
Makes sense ?

[fxedel]

Interesting idea! However, I wasn't able to make it working in my example show from above:

I've put each slider into a different frame page, both affect the same channel using level mode. But when I switch the page, the newly shown fader's value isn't applied – only when I move the handle a little bit with the mouse (which is not much better than having a single fader and adjusting it each time).

I thought automatic switching was easy with Loopback, but as a page is shown when the attached input channel changes, switching from "Sound" to "Auto X" didn't work: Loopback channel 1 ("Show Program Speed") and Loopback channel 2 ("Show Sound Sensitivity") have changed, and the latter takes precedence.

Considering the effort it took me to implement this in a show, I think it'll be worth saving my proposal as an idea for future additions. (Maybe at first only for faders)

[fxedel]

@mcallegari
Great news! Is there a rough plan when the new version will be released? We'll create a new OFL plugin for QLC+ besides the existing one soon to make the new fixture format downloadable at https://open-fixture-library.org.

As of fixtures: I'll have a closer look at the list later. Some of them are also included in OFL, so we can replace them as soon as the new plugin is finished.

[mcallegari]

@fxedel
the rough plan really is a QLC+ 4 release every 6 months, since it's in maintenance mode.
However, I can think of releasing a 4.12.0 one to highlight that something big has changed.

[fxedel]

@mcallegari What exactly is the difference between "PulseInSlowToFast" and "PulseOutSlowToFast"? In my understanding, pulse is a sinus or ziczac curve (of Intensity), so what's "in" and what's "out"? Maybe something like ramp up / down? If so, which would be the correct preset for a sinus pulse?

Here's a visualization of my understanding of Shutter effects:
https://jsfiddle.net/0p97phms/22/

[janosvitok]

XML schema is outdated. Fixture definitions do not validate at current git master.

Aliases do not validate yet. I don't like the way they are implemented. I think that a tag should contain either direct content or other tags, but not both.

[mcallegari]

What exactly is the difference between "PulseInSlowToFast" and "PulseOutSlowToFast"? In my understanding, pulse is a sinus or ziczac curve (of Intensity), so what's "in" and what's "out"? Maybe something like ramp up / down? If so, which would be the correct preset for a sinus pulse?

@fxedel Pulse In and Pulse Out are what you call Ramp Up / Ramp Down. I tried to watch youtube videos showing this feature but I haven't found much material. And to be honest I've never seen "ramp" into definitions, always "pulse". As for Sine Pulse, I can add the enum to capability presets.

Aliases do not validate yet. I don't like the way they are implemented. I think that a tag should contain either direct content or other tags, but not both.

@janosvitok the idea is there since February. Maybe you could come out with this statement before ?
Anyway, it's your opinion. Before chosing that way I checked, and in XML it's perfectly legit to have both values and sub tags.

[janosvitok]

And to be honest I've never seen "ramp" into definitions, always "pulse".

• MA2
• Hog4

[mcallegari]

Doesn't make any difference to me to rename stuff. The difficult part is to simulate them in the UI.

PulseInSlowToFast -> RampUpSlowToFast
PulseInFastToSlow -> RampUpFastToSlow
PulseOutSlowToFast -> RampDownSlowToFast
PulseOutFastToSlow -> RampDownFastToSlow
PulseInFrequency -> RampUpFrequency
PulseOutFrequency -> RampDownFrequency
PulseInFreqRange -> RampUpFreqRange
PulseOutFreqRange -> RampDownFreqRange
SinePulseSlowToFast
SinePulseFastToSlow
SinePulseFrequency
SinePulseFreqRange

[fxedel]

I've also seen "ramp up" in some manuals and I think renaming the presets makes the meaning clearer.

Maybe simply "Pulse" instead of "SinePulse" to also allow ziczac and other up/down pulsing?

[janosvitok]

@janosvitok the idea is there since February. Maybe you could come out with this statement before?

I know. I didn't find the time sooner.
I've updated the PR to accept current state.

Before chosing that way I checked, and in XML it's perfectly legit to have both values and sub tags.

The problem is that you can't specify text position between tags once you allow mixed content (mixed="true"). Most probably (I'm not expert on XSD) you can't write a schema that allows <A>text<B/></A> but doesn't allow <A><B/>text</A> or even <A>text1<B/>text2</A>.

[mcallegari]

So, we've got schemas for several QLC+ XML formats, but they're not used in any automation.
So, since the fixture definition validation is still missing in the current python script, I intend to implement 2 methods: one with schema validation (http://lxml.de/validation.html#xmlschema) and one like what I've done in PHP.
Sounds good ? Ideas ?

[fxedel]

I strongly support your proposed idea. We also have schema and logical tests in OFL and are glad everytime they find an error we wouldn't have seen.

What about running the tests by Travis or a similar CI?

[mcallegari]

What about running the tests by Travis or a similar CI?

Well, validation makes sense only when a new fixture is pushed. Doing it on every commit seems like too much to me :)
Since I need to regenerate the fixture map on every new definition, I can hook the validation to it, so I don't forget to do it.

[janosvitok]

https://github.com/mcallegari/qlcplus/blob/master/resources/fixtures/check contains call to xmllint to verify the fixtures and fixturemap.xml. With little reformatting, it can be run from travis. It takes just few seconds to run. So there are two issues:

1. properly report exit codes
2. there are some failures due to missing dimensions - we need to either exclude the fixtures from testing or fill in some bogus dimensions. I guess the latter is better since the dimensions are needed for proper display.

[janosvitok]

I've implemented the above in #1065.

I propose that we split the fixtures directory by manufacturer. 1000 files in a directory is a bit much.
We may still deploy them to a single directory.

[mcallegari]

I propose that we split the fixtures directory by manufacturer. 1000 files in a directory is a bit much.
We may still deploy them to a single directory.

I noticed that too. Mainly a GitHub limitation though...
I'd say not only to split by manufacturer (and deploy by manufacturer), but also optimize the fixture map even further, like:

<M n="American DJ">
  <F n="American-DJ-12P-Hex" d="12P Hex Pearl"/>
 ...
</M>

We can even rename all the files since manufacturer is redundant. Like: 12P-Hex.qxf
I'll surely do the split next days. Still thinking about renaming.

[mcallegari]

Done. Fixtures are now committed and installed in folders, by manufacturer.
However, I struggled to find a way to preserve spaces on folder names (e.g. "American DJ"), but qmake doesn't help. So for now, spaces are replaced by '_' and back to spaces in the engine.
If someone is able to figure out how to properly handle spaces (in a cross platform way), please send a PR.
FixtureMap.xml has decreased in size by a further 13%.
@janosvitok ~~~Fixture map schema needs to be updated, please~~~ Nevermind. Fixed it myself.

I took the chance also to improve gobos installation. The only drawback is that ANY file in folders will be installed now. But since we have control over it, only PNG files will be there.

[FloEdelmann]

Hi Massimo,

our big capability schema redesign in the Open Fixture Library is finally over, too! I was able to write a new export plugin for the new QLC+ fixture format syntax quite easily with our new syntax; see OFL PR #547.

Please see my comment in the PR to see how the output changed from QLC+ 4 plugin to QLC+ 5 plugin for our test fixtures.

The exported .qxf files are automatically checked with both your XSD schema and your fixture tool; most fixtures pass both. There are just some where the physical data is missing or incomplete, thus both tests fail there.

If you have any remarks, please let us know!

[mcallegari]

@FloEdelmann well done with the export plugin ! Even though "QLC+ 5" is not exactly the correct name. New definitions can be used also on 4.12.0, when I release it. The main difference is that QLC+ 5 considers much more definition details than QLC+ 4.

So, I've dedicated the whole day to fixtures, since nobody picked up the txt file I shared almost 3 months ago. I realized some more presets were missing, so I added them.
I also added the possibility to specify the number of faces of a prism (if present, otherwise leave 0)
There is a new -optional- attribute in the Lens tag named PrismFaces.
I'm pretty confident I can simulate prisms in 3D which is super cool.

Also I moved Iris presets to the shutter category, since it seems an iris can do also strobe/pulse effects.

The reference definition that has pretty much everything is: Briteq-BTX-180LS.qxf

In this regard, can anyone point me to some documentation or give an explanation about the differences and usages of shutter, iris, zoom and focus ? I'm interested also in the mechanical aspects of them.
For example, why should a fixture have both a shutter and an iris when the iris can do shutter effects too ?
Or why iris and zoom together ?

So, except for the need of manpower to review existing definitions capabilities and aliases, this task is pretty much over. I'll leave it open for a few more days and then close.
I will slowly review defs when I have nothing better to do.

[fxedel]

Specifying the prism faces (and especially visualizing them) is a good idea! However, several fixtures have multiple prisms (see Martin MAC Axiom Hybrid), e.g. a 3-facet and an 8-facet one. How to solve this? Maybe by using the capability's "Res1" attribute instead?

Zoom and Focus are lenses. Zoom affects the beam size (the beam keeps "sharp"), Focus the focus point (blurred -> sharp).

In my understanding, Iris is a circular shutter. If set to 50%, the beam diameter is half of the full size (while keeping the beam edges "sharp").
A shutter set to 50% dims the light; it doesn't decrease the beam size.

This all refers to the question where an effect (Focus, Iris, ...) is applied inside the moving head: Before or after the focus point -> "blurred or sharp" beam transformation.

[mcallegari]

@fxedel thanks for the info. Actually there is also the "frost" effect which is still kind of a mystery to me...
I guess it's another lens effect like a stained glass wheel or something.

Your proposal for prism faces via capability preset is good! I haven't thought about it. It might be a special case overriding the global prism faces number.
In general though, I have already understood it is almost impossible to simulate everything in 3D. Or better, very very difficult. There are too many variables on the table.
So to start I'll stick with the most common features that can be found on the market.

[janosvitok]

Frost is a (gradual) diffusion filter. It allows to change light from hard edge (spot) to soft edge (wash). Together with beam gobos (full gobo with small hole in the middle) allows to create hybrid fixtures like Robe Pointe. One fixture that is able to act as beam, spot and wash. It a spot that has frost filter to make it wash and pinhole gobo to make it a beam (although with smaller light output, since part of the light is blocked by the gobo).

(optical) Prism is a what it says: a piece of glass that splits light in several beams on its faces (think of Dark side of the moon cover). See http://www.movinghead.net/Uploads/201410/543b7337e2bf5.jpg

The prism is either put in the light path or not, and can be rotated.

Sometimes there are more prisms (for example 3-face circular + 6-face linear), that are exclusive to each other (if they are on the same wheel/flap) or they can be selected both (if they are separate).

[mcallegari]

@fxedel @FloEdelmann I reverted my "global prism faces" setting in favour to the solution proposed by Felix. It's better cause it doesn't impact at all those fixture that have nothing to do with prisms. (and creates less confusion to definition creators)

[mcallegari]

Documentation updated. Closing this then.