PRU-to-Xenomai interrupt

[giuliomoro]

This seemed like a promising way of getting more out of the CPU. @apmcpherson where did we get stuck with this? Does it require a recompiled kernel?

[apmcpherson]

This already exists. It can be enabled by #define BELA_USE_XENOMAI_INTERRUPTS in PRU.cpp. It does require user-space interrupt support enabled in the kernel, which I think ought to be in the latest kernel (at least it was in the ones I built). That's a Xenomai 2.x feature only which was removed in Xenomai 3.0 -- one of the reasons I stuck with Xenomai 2.x for now.

The performance of this is better insofar as you can get even more oscillators running on the oscillator bank before it breaks up. The problem was that it used the CPU so efficiently that under some circumstances, you could lock the board solid because the audio task would never return enough CPU to the rest of Linux to interact with it. That's why I ultimately turned it back off.

An alternative might be to put a single short delay at the end of each block to force a certain amount of non-Bela time, then listen for an interrupt after that. But I haven't tested it.

[giuliomoro]

I am fairly sure that that kernel never made its way to the board.
Is this the flag that needs to be enabled?

# CONFIG_XENO_OPT_NATIVE_INTR is not set

If it requires more than this, could you please provide your kernel config file ?

current XENO-related config options are these:

root@bela /boot$ grep XENO config-3.8.13xenomai-bone41
CONFIG_XENOMAI=y
CONFIG_XENO_GENERIC_STACKPOOL=y
CONFIG_XENO_FASTSYNCH_DEP=y
CONFIG_XENO_FASTSYNCH=y
CONFIG_XENO_OPT_NUCLEUS=y
CONFIG_XENO_OPT_PERVASIVE=y
# CONFIG_XENO_OPT_PRIOCPL is not set
CONFIG_XENO_OPT_PIPELINE_HEAD=y
# CONFIG_XENO_OPT_SCHED_CLASSES is not set
CONFIG_XENO_OPT_PIPE=y
CONFIG_XENO_OPT_MAP=y
CONFIG_XENO_OPT_VFILE=y
CONFIG_XENO_OPT_PIPE_NRDEV=32
CONFIG_XENO_OPT_REGISTRY_NRSLOTS=512
CONFIG_XENO_OPT_SYS_HEAPSZ=256
CONFIG_XENO_OPT_SYS_STACKPOOLSZ=128
CONFIG_XENO_OPT_SEM_HEAPSZ=12
CONFIG_XENO_OPT_GLOBAL_SEM_HEAPSZ=12
CONFIG_XENO_OPT_STATS=y
CONFIG_XENO_OPT_DEBUG=y
# CONFIG_XENO_OPT_DEBUG_NUCLEUS is not set
CONFIG_XENO_OPT_DEBUG_XNLOCK=y
# CONFIG_XENO_OPT_DEBUG_QUEUES is not set
# CONFIG_XENO_OPT_DEBUG_REGISTRY is not set
# CONFIG_XENO_OPT_DEBUG_TIMERS is not set
CONFIG_XENO_OPT_DEBUG_SYNCH_RELAX=y
CONFIG_XENO_OPT_WATCHDOG=y
CONFIG_XENO_OPT_WATCHDOG_TIMEOUT=4
# CONFIG_XENO_OPT_SHIRQ is not set
CONFIG_XENO_OPT_SELECT=y
CONFIG_XENO_OPT_HOSTRT=y
# CONFIG_XENO_OPT_TIMING_PERIODIC is not set
CONFIG_XENO_OPT_TIMING_VIRTICK=1000
CONFIG_XENO_OPT_TIMING_SCHEDLAT=0
# CONFIG_XENO_OPT_SCALABLE_SCHED is not set
CONFIG_XENO_OPT_TIMER_LIST=y
# CONFIG_XENO_OPT_TIMER_HEAP is not set
# CONFIG_XENO_OPT_TIMER_WHEEL is not set
CONFIG_XENO_HW_FPU=y
# CONFIG_XENO_HW_UNLOCKED_SWITCH is not set
CONFIG_XENO_SKIN_NATIVE=y
CONFIG_XENO_OPT_NATIVE_PERIOD=0
CONFIG_XENO_OPT_NATIVE_PIPE=y
CONFIG_XENO_OPT_NATIVE_PIPE_BUFSZ=1024
CONFIG_XENO_OPT_NATIVE_SEM=y
CONFIG_XENO_OPT_NATIVE_EVENT=y
CONFIG_XENO_OPT_NATIVE_MUTEX=y
CONFIG_XENO_OPT_NATIVE_COND=y
CONFIG_XENO_OPT_NATIVE_QUEUE=y
CONFIG_XENO_OPT_NATIVE_BUFFER=y
CONFIG_XENO_OPT_NATIVE_HEAP=y
CONFIG_XENO_OPT_NATIVE_ALARM=y
CONFIG_XENO_OPT_NATIVE_MPS=y
# CONFIG_XENO_OPT_NATIVE_INTR is not set
CONFIG_XENO_OPT_DEBUG_NATIVE=y
CONFIG_XENO_SKIN_POSIX=y
CONFIG_XENO_OPT_POSIX_PERIOD=0
# CONFIG_XENO_OPT_POSIX_SHM is not set
# CONFIG_XENO_OPT_POSIX_INTR is not set
CONFIG_XENO_OPT_POSIX_SELECT=y
CONFIG_XENO_OPT_DEBUG_POSIX=y
# CONFIG_XENO_SKIN_PSOS is not set
# CONFIG_XENO_SKIN_UITRON is not set
# CONFIG_XENO_SKIN_VRTX is not set
# CONFIG_XENO_SKIN_VXWORKS is not set
# CONFIG_XENO_OPT_NOWARN_DEPRECATED is not set
CONFIG_XENO_SKIN_RTDM=y
CONFIG_XENO_OPT_RTDM_PERIOD=0
CONFIG_XENO_OPT_RTDM_FILDES=128
CONFIG_XENO_OPT_RTDM_SELECT=y
# CONFIG_XENO_OPT_DEBUG_RTDM is not set
CONFIG_XENO_OPT_DEBUG_RTDM_APPL=y
CONFIG_XENO_DRIVERS_16550A=m
# CONFIG_XENO_DRIVERS_16550A_PIO is not set
# CONFIG_XENO_DRIVERS_16550A_MMIO is not set
CONFIG_XENO_DRIVERS_16550A_ANY=y
CONFIG_XENO_DRIVERS_TIMERBENCH=y
CONFIG_XENO_DRIVERS_KLATENCY=m
CONFIG_XENO_DRIVERS_IRQBENCH=y
CONFIG_XENO_DRIVERS_SWITCHTEST=y
CONFIG_XENO_DRIVERS_RTDMTEST=m
# CONFIG_XENO_DRIVERS_CAN is not set
# CONFIG_XENO_DRIVERS_ANALOGY is not set
CONFIG_XENO_DRIVERS_RTIPC=y
CONFIG_XENO_DRIVERS_RTIPC_XDDP=y
CONFIG_XENO_DRIVERS_RTIPC_IDDP=y
CONFIG_XENO_OPT_IDDP_NRPORT=32
CONFIG_XENO_DRIVERS_RTIPC_BUFP=y
CONFIG_XENO_OPT_BUFP_NRPORT=32

[apmcpherson]

I enabled CONFIG_XENO_OPT_NATIVE_INTR and CONFIG_XENO_OPT_POSIX_INTR though the actual calls I'm using are part of the Native API so probably the former would be sufficient.

This was on the board for a while; it probably dropped off sometime before the first Bela release when the kernel got rebuilt.

[giuliomoro]

Good I'll try to rebuild with these.
I think it is important to add the interrupt because it is cheaper than polling, allowing to save some CPU percent and more consistent performance. Sure we need to add a compulsory sleep in there (possibly user-selectable between 20% (default) and 10% (hardcore)) to prevent the board from becoming unresponsive.

One more thing on the topic of Xenomai eating my board's CPU: what is more difficult to account for is that the audio thread is not everything. If anyone has an expensive AuxiliaryTask running, which still runs at higher priority than the kernel, then that would potentially fill all the CPU time left available from the audio thread.
It is always potentially dangerous if you have a program scheduled to run at startup and it eats up all of your CPU so you cannot even disable it.
A way to prevent this from happening could perhaps be a Xenomai timer which suspends all Xenomai processes every 95% blocksize?

[apmcpherson]

Agreed about the sleep times.

Another feature that would address the AuxiliaryTask situation while also being useful in other contexts: a short push (say 0.5s) of the button on the cape could stop the program. A long hold (say 3s) could shut down the board. If the core Bela software monitors the button, no user interaction would be needed to stop it. Xenomai is already savvy enough to kill a task that takes up all the CPU; the problem I had was that it left some CPU, but not enough to effectively run the USB-network link.

[giuliomoro]

Performance comparison at blocksize 2 for the bypass program:
with interrupts

CPU  PID    MSW        CSW        PF    STAT       %CPU  NAME
  0  0      0          3112771    0     00500080   64.6  ROOT
  0  2620   2          2          0     00b00380    0.0  exampleTempProj
  0  2622   0          357302     0     00300186   29.5  bela-audio
  0  0      0          357302     0     00000000    4.6  IRQ21: bela-pru-irq
  0  0      0          1822591    0     00000000    0.0  IRQ67: [timer]

with polling

CPU  PID    MSW        CSW        PF    STAT       %CPU  NAME
  0  0      0          3122211    0     00500080   64.6  ROOT
  0  3020   2          2          0     00b00380    0.0  exampleTempProj
  0  3022   0          321721     0     00300186   40.5  bela-audio
  0  0      0          1765481    0     00000000   11.7  IRQ67: [timer]

Not sure how to interpret the IRQ's CPU time: in the second case, ROOT and bela-audio add up to 100%, and IRQ67's CPU time would take the total above 100%. In the first case, 100% is obtained by summing bela-audio, IRQ21 and ROOT

[giuliomoro]

Here is the advice for Xenomai 3
https://github.com/siemens/xenomai-forge/blob/master/doc/asciidoc/MIGRATION.adoc#userirqtip

[giuliomoro]

done a while ago