Headless Orange (and a discussion about functions/mega widgets)

[borondics]

Is your feature request related to a problem? Please describe.
We would like to minimize the chance of user errors with data selection/loading.

Here, write a clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

We would like to start Orange from command line or another program, load a pre-verified workflow and change the selected file in the File (or other data loader) widget.

Describe the solution you'd like

$ orange-canvas path1/workflow.ows path2/file to open workflow.ows and load file

Add a clear and concise description of what you want to happen here.

The two parameters (workflow and file path) would run an automated data analysis scheme so that the user needs only to interact with the widgets already placed on the canvas or simply the visualizations if they are not proficient enough for parameter changes.

Describe alternatives you've considered
Chaning the file by hand, but this prevents the possibility for full automatization during/after data collection.

[gengyabc]

I think this is a very useful feature.

And what is more, could we have a widget that could wrap the other workflow into it, so that we call the other workflow in this workflow.

[janezd]

We discussed this at today's meeting.

This would in principle be possible, but too complicated: to do it properly, we would need to allow setting widget's options from command line. For this, widgets would need (unique) id's and settings would have to be documented (currently they are treated as "internal" and we don't care about keeping them across versions; backward compatibility is solved through "migrations", which probably wouldn't be useful for this). And I can't even imagine how to reach context settings in this way. So, a general solution is not practically feasible.

One could say that it the workflow contains one and only one file widget, then the second argument should be understood as the file name for this widget. This could be implemented, but it would be an exception that would hack through the code ... it is too ugly, we can't do it.

We decided we could have "headless" Orange, that is, an option to load the canvas, run it and save any results (through Save widgets), but without showing anything. #4966 changes the behaviour of Save Widget so that auto_save is no longer disabled.

[gengyabc]

Then could we have a widget that wraps the other workflow into it, as a "big widegt"? All the works are still in canvas. So some complicated workflows can be cleaner and more DRY

[janezd]

No. This idea is proposed now and then on github but has not stirred any enthusiasm among developers - the so-called "mega widgets" are rather frowned upon. I believe that reasons were described in corresponding old issues, probably already multiple times. :)

In short: mega widgets are needed in tools where you, for instance, need to connect a several widgets just to define cross validation procedure. There, "cross validation" would be a "mega widget". Orange's workflows don't tend to be modular in such fashion, so mega widgets would just introduce a lot of complex code and concepts without, in our opinion, being worth it. We haven't even placed mega widgets to #4090, which contains features that we would like to (and we hope to) implement someday, because it would be dishonest. :)

[borondics]

This seems to be two different issues here. One is the CLI (would be great to do automated processing) and the other is the mega widget (would be great to simplify workflow readability). I have something to show for the second one.

Having a complicated problem sometimes generates workflows like this.

@janezd, what do you think? :)

[janezd]

I think it's impressive. :)

I always considered mega widgets as "functions" that appear in different places in your workflow. It was thus about reusability - like grouping recurring lines of code into functions that can be called from different places. In your case, I guess you're looking more into "collapsing" parts of the workflow.

This looks simpler, because mega widgets would need to have defined inputs and outputs, so other widgets can be connected to the mega widget. For collapsing, the workflow would already be fully connected and mega widget's connections with the outside world wouldn't be changed without uncolappsing, that is dissolving the mega widget.

We may discuss this on Friday or, better, the next time you come to visit us. But even if we decide we like it, this is a huge undertaking and I doubt it will ever climb to the top of @ales-erjavec's priorities. :|

[borondics]

I think a collapse (black hole widget :)) would be great and exactly what we would need.

On one hand, the aim is to make massive workflows like the example easier to handle and on the other hand, some users are not interested / skilled to actually change too many parameters, so in an extreme case we could give them workflows, collapse all processing and expose a only a plotting widget at the end.

[ajdapretnar]

Very much in favour of a black hole widget which would just suck all the widgets into itself. ⚫

[borondics]

It could have a cool icon too...

[janezd]

I split this into two issues.

Mega widgets are added to #4090. This is a big step. We have so far refused to have them as an open issue. :) We agreed we'd think about how to implement them, but it's a big project.

Headless Orange belongs to orange-canvas-core repository and was moved there (biolab/orange-canvas-core#144).

[PetrDlouhy]

I have started work on --batch-mode, which would detect that initial workflow run has finished and stop the exit the whole Orange application. I also tried to report widget errors to stdout and set the exit status code non-zero if there are errors in some widgets.
The experimental code can be found here: https://github.com/algops-com/orange-canvas-core/tree/batch_mode and here: https://github.com/algops-com/orange3/tree/batch_mode

I also find, that the headlessness can be achiveved by xvfb:

sudo apt-get install xvfb
xvfb-run python -m Orange.canvas

(taken from https://stackoverflow.com/questions/13215120/how-do-i-make-python-qt-and-webkit-work-on-a-headless-server)

[PetrDlouhy]

Even simpler solution to headless mode:

QT_QPA_PLATFORM=offscreen python3 -m Orange.canvas workflow.ows

[PetrDlouhy]

According to @ales-erjavec (on biolab/orange-canvas-core#209) there is one more alternative to this. If the Orange is executed

python -m Orange.canvas.run

it runs without GUI and the console return status is 0 if the workflow runs OK.