Best effort execution mode for RX, DO and TX phases of ETH boards

[marcoaccame]

This PR adds configurability for the best effort mode for the execution of the RX, Do and TX phases for all ETH boards and for improved logging of the execution cycle.

The new parameters, if not present, are considered by icub-main with the default values.

Associated PRs

• Best effort execution mode for RX, DO and TX phases of ETH boards icub-firmware#617
• Best effort execution mode for RX, DO and TX phases of ETH boards icub-firmware-shared#115
• Best effort execution mode for RX, DO and TX phases of ETH boards icub-main#1030
• Best effort execution mode for RX, DO and TX phases of ETH boards #746
• ems v3.105, mc4plus v3.105, mc2plus v3.84, amc v2.24 icub-firmware-build#215

Description

The full description of the new best effort mode and of how the ETH board computes measures about the execution cycles is in here:

• Best effort execution mode for RX, DO and TX phases of ETH boards icub-firmware#617

The above measures are used for both execution overflow immediate signaling or for periodic emission of execution statistics.

In here we show how to fully configure and enable / disable the emission of such diagnostics messages.

See the following.

<group name="RUNNINGMODE">

    <!-- execution can be [synchronized, besteffort]. 
    
         - synchronized
          
           In such a mode each phase RX, DO, TX executes inside exact time slots that are
           [0, maxTimeOfRXactivity), [maxTimeOfRXactivity, maxTimeOfRXactivity+maxTimeOfDOactivity) and [maxTimeOfRXactivity+maxTimeOfDOactivity, period)
           If any phase seldom overflows outside its time slot, the mode ensures re synchronization to the slots by means of immediate execution
           of late phases.                       
           You should use this mode if you want synchronous execution so that some actuations to motors do not drift inside the period 

              -------     --------          ------      --------   ---------         ------       --
             | RX    |   | DO     |        | TX   |    | RX      | | DO      |       | TX   |    | RX
              -------     --------          ------      ---------   ---------         ------      ---
             ^           ^                 ^           ^           ^                 ^           ^ 
             o-----------d-----------------t-----------o-----------d-----------------t-----------o----------
             | period n                                | period n+1                              | period n+2
             |-rx budget-|-do budget-------|-tx budget-|-rx budget-|-do budget-------|-tx budget-| ... 

             Figure. Typical synchronized execution w/out overflow. Note that teh values of rx/do/tx budget are onfigurable w/ parameters
                     maxTimeOfRXactivity, maxTimeOfDoactivity and maxTimeOfTXactivity                        

              --------------  --------      ------      --------   ---------         ------       --
             | RX           || DO     |    | TX   |    | RX      | | DO      |       | TX   |    | RX
              --------------  --------      ------      ---------   ---------         ------      ---
             ^           ^^^^^             ^           ^           ^                 ^           ^ 
             o-----------d-----------------t-----------o-----------d-----------------t-----------o----------
             | period n                                | period n+1                              | period n+2
             |-rx budget-|-do budget-------|-tx budget-|-rx budget-|-do budget-------|-tx budget-| ...  
               
             Figure. Typical synchronized execution w/ overflow of RX

           In section LOGGING.IMMEDIATE, the param emitRXDOTXoverflow enables or disables warnings emitted by the board when any phase 
           overflows beyond its max time of activity.
           Similarly, the param emitPERIODoverflow enables or disables warnings emitted by the board when the complete period overflows.
           Advised logging settings are:
           - LOGGING.IMMEDIATE.emitRXDOTXoverflow = true and LOGGING.IMMEDIATE.emitPERIODoverflow = false 
           - or even LOGGING.IMMEDIATE.emitRXDOTXoverflow = false if you want to get rid of some annoying overflow warnings (w/ some risks
             to lose visibility of the wanted motor control rate).
           

         - besteffort 
         
           In such a mode the activation is done exactly at start of the period and then phases RX, DO and TX executes as fast as they can.
           If the three of them seldom overflow outside the period, the mode ensures re synchronization to the start of period by means of 
           immediate execution of a new burst.                       
           You should use this mode if you have difficulties to determine correct slots for the synchronized execution mode and you can accept
           some drifts of execution inside the period.

              -------  --------  ------                 ------------------  ---------  ------     --
             | RX    || DO     || TX   |               | RX               || DO      || TX   |   | RX
              -------  --------  ------                 ------------------  ---------  ------     ---
             ^                                         ^                                         ^
             o-----------------------------------------o-----------------------------------------o----------
             | period n                                | period n+1                              | period n+2                                                                             | 

             Figure. Typical best effort execution w/out overflow

              -----------------------------  -------------  ------  -------  ------  ------       --
             | RX                          || DO          || TX   || RX    || DO   || TX   |     | RX
              -----------------------------  -------------  ------  -------  ------  ------       ---
             ^                                         ^^^^^^^^^^^^^ (activation remains valid)  ^
             o-----------------------------------------o-----------------------------------------o----------
             | period n                                | period n+1                              | period n+2   
             
             Figure. Typical best effort execution w/ overflow to next period (shortened for sake of displaying)

           In section LOGGING.IMMEDIATE, the param emitRXDOTXoverflow enables or disables warnings emitted by the board when any phase 
           overflows beyond its max time of activity.
           Similarly, the param emitPERIODoverflow enables or disables warnings emitted by the board when the complete period overflows.
           Advised logging settings are:
           - LOGGING.IMMEDIATE.emitRXDOTXoverflow = false and LOGGING.IMMEDIATE.emitPERIODoverflow = true 
           It is dangerous to set LOGGING.IMMEDIATE.emitPERIODoverflow = false because it may hide the failure to maintain
           the target motor control rate. 
           
           It's synchronized by default if the param is not specified.

     -->  

    <param name="execution">                synchronized        </param>


    <!-- The following parameters express the time in us that is dedicated to the processing activities.
         They are: period, maxTimeOfRXactivity, maxTimeOfDOactivity, maxTimeOfTXactivity, safetygap     
      -->
      
    <!-- The duration [in us] of the execution cycle that includes the three activities RX, DO and TX. 
         So far, it is locked to be only 1000.  
         Very important: it must be period = maxTimeOfRXactivity + maxTimeOfDOactivity + maxTimeOfTXactivity
      -->
    <param name="period">                   1000                </param>               

    <!-- The max time [in us] of the execution cycle dedicated to the RX activity. Default is 400, range is [5, 990].
         In case LOGGING.IMMEDIATE.emitRXDOTXoverflow = true, if the execution time of the RX exceeds maxTimeOfRXactivity then a warning message is emitted.                
         In case of execution = synchronized the value maxTimeOfRXactivity is used to start the execution of the DO phase so that it is inside the slot
         [maxTimeOfRXactivity, maxTimeOfRXactivity+maxTimeOfDOactivity).
      -->
    <param name="maxTimeOfRXactivity">      400                 </param>

    <!-- The max time [in us] of the execution cycle dedicated to the DO activity. Default is 400, range is [5, 990]. 
         In case LOGGING.IMMEDIATE.emitRXDOTXoverflow = true, if the execution time of the DO exceeds maxTimeOfDOactivity then a warning message is emitted.  
         In case of execution = synchronized the value maxTimeOfDOactivity is used to start the execution of the TX phase so that it is inside the slot
         [maxTimeOfRXactivity+maxTimeOfDOactivity, period).
      -->
    <param name="maxTimeOfDOactivity">      300                 </param>  

    <!-- The max time [in us] of the execution cycle dedicated to the TX activity. Default is 400, range is [5, 990]. 
         In case LOGGING.IMMEDIATE.emitRXDOTXoverflow = true, if the execution time of the TX exceeds maxTimeOfTXactivity then a warning message is emitted. 
      -->
    <param name="maxTimeOfTXactivity">      300                 </param>  
    
    <!-- Amount of time [in us] of the execution cycle that should be left available for system processing.
         It is used just for diagnostics purposes and it does not count in triggering of activities. 
         For instance if LOGGING.IMMEDIATE.emitPERIODoverflow is true and the execution time is > (period-safetygap) a warning message is emitted
         It's 0 by default if the param is not specified.
      -->
    <param name="safetygap">                100                 </param>                   

    <!-- The period in multiples of RUNNINGMODE::period with which the ETH boards transmits the regular ROPs. Its range is [1, 20] and if out of bound it is clipped to limits. 
         The recommended value is 5 (3 for boards which support skin). IMPORTANT: reply ROPs and diagnostics ROPs are transmitted ASAP with 1 ms granularity.
                      
         This number is related to many other system parameters: (1) the ETH_BOARD::ETH_BOARD_SETTINGS::regularsTXrate of other boards; (2) the PC104::PC104RXrate decoding 
         rate inside PC104; (3) rate of the wrappers which typically is 10 ms. 
         A value of PC104::PC104RXrate = 1 for every board is possible, but it adds extra useless effort in the CPU of PC104 to decode many UDP packets. 
         The effort is useless because the information carried by regular ROPs is read by wrappers typically at 10 ms rate.
         hence, any value lower than the rate of wrappers is enough. The only exception is for presence of skin, where it is required a lower value so that the ETH 
         board can consume the CAN packets sent by the CAN boards without losing info (if it happens, diagnostics messages are sent which complain about loss of skin data).  
      -->              
    <param name="TXrateOfRegularROPs">      5                   </param> 

    
    <group name="LOGGING">
    
        <!-- Contains enabling / disabling of logging that must be emitted immediately as soon as the condition appears
             If the group or a param is not present, the default values are always false or 0.0, except for LOGGING.IMMEDIATE.emitRXDOTXoverflow that is true.
          --> 
        
        <group name="IMMEDIATE">

            <!-- Enables / disables the emission of warning diagnostics related to the measured execution time of RX higher than the configured
                 maxTimeOfRXactivity and similarly for DO and TX. 
                 It's true by default if the param is not specified. 
              -->
            <param name="emitRXDOTXoverflow">       true        </param>
            
            <!-- enables / disables the emission of warning diagnostics that alert that the measured execution time of the complete period (RX, DO, TX) 
                 is higher than the configured period - safetygap
                 It's false by default if the param is not specified.
              -->
            <param name="emitPERIODoverflow">       true        </param>
        
        </group>
        
        <!-- Contains enabling / disabling of logging that must be emitted periodically -->
        
        <group name="PERIODIC">
        
            <!-- The period of the emission of periodic diagnostics expressed in seconds in the interval [1.0, 600.0]. 
                 If 0.0 every periodic transmission is disabled 
                 It's 0.0 by default if the param is not specified.
              -->
            <param name="period">                   10.0          </param>

            <!-- enables / disables the periodic emission of info diagnostics about the statistics of the execution time of the RX, DO and TX phase
                 in the form of minimum, average and maximum values. 
                 The legacy param emitRXDOTXstatistics is just an alias. When both are present, emitRXDOTXminavgmax wins.
                 It's false by default if the param is not specified.
              -->                        
            <param name="emitRXDOTXminavgmax">      true        </param>
            
            <!-- legacy name of emitRXDOTXminavgmax. When both are present emitRXDOTXminavgmax wins.
                 It's false by default if the param is not specified.
              -->                        
            <param name="emitRXDOTXstatistics">     true        </param>
        
            <!-- enables / disables the periodic emission of info diagnostics about the statistics of the execution time of the complete period (RX, DO, TX)
                 in the form of minimum, average and maximum values.
                 It's false by default if the LOGGING group is not present or if the param is not specified.
              -->                        
            <param name="emitPERIODminavgmax">      true        </param>

            <!-- enables / disables the periodic emission of info diagnostics about the statistics of the execution time of the complete period (RX, DO, TX)
                 in the form of a probability density function in 8 bins of period/4 us each. 
                 For period = 1000: [0, 250), [250, 500), [500, 750), [750, 1000), [1000, 1250), [1250, 1500), [1500, 1750), [1750, +oo)
                 So, just for example, if over PERIODIC.period seconds the RX-DO-TX cycles lasts 70% of the times exactly 400 us, 20% of the times exactly 600 us 
                 and only 10% of the times exactly 800 us, then the reported values will be [0, 70, 20, 10, 0, 0, 0, 0].
                 It's false by default if the param is not specified.
              -->                        
            <param name="emitPERIODhistogram">     true        </param>
        
        </group>

    </group>

</group> 

Code Listing. Configurability of the ETH_BOARD.RUNNINGMODE group.

Mergeability

This PR is useful only when all the associated PRs are merged but does not harm if icub-main is not able to parse the new params. Moreover, so far the change of xml is applied only to the template file and to the lego setup used for the tests.

So, the PR can be safely merged as the first of the list of associated PRs.

[gemini-code-assist]

Summary of Changes

Hello @marcoaccame, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new 'best effort' execution mode for the RX, DO, and TX phases of ETH boards, offering an alternative to the existing 'synchronized' mode. It also enhances logging and diagnostics capabilities to provide better insights into the execution times of these phases and the overall period, allowing for more flexible configuration and monitoring.

Highlights

• Execution Mode: Introduces a 'best effort' execution mode for RX, DO, and TX phases of ETH boards, alongside the existing 'synchronized' mode.
• Configuration Parameter: Adds a new parameter 'execution' within the 'RUNNINGMODE' group to select between 'synchronized' and 'besteffort' execution modes.
• Logging Improvements: Enhances logging capabilities with a new 'emitPERIODoverflow' parameter to detect period overflows in 'besteffort' mode and provides more detailed statistics.
• Diagnostics: Adds diagnostics for execution time of RX, DO and TX phases, including min/avg/max values and a histogram for period execution time.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a 'best effort' execution mode for ETH boards, configurable via a new execution parameter in the XML template. It also adds several new logging and diagnostic parameters to monitor execution timing, including period overflows and statistics. The changes are well-documented with extensive comments explaining the new functionality. My review focuses on improving the clarity and correctness of these comments, as they serve as important documentation for users configuring the system.

[marcoaccame]

@martinaxgloria: here is the improved xml I told you about.