Skip to content

Run Time Buffers Configuration Update design

Andriy Moroz edited this page Feb 1, 2018 · 4 revisions

1. Overview

Run-time Buffers Configuration feature allows to adjust buffer parameters on port when port speed changes. Current buffers configuration implementation performs only initial configuration on switch start.
An update suggested in this design document enables dynamic buffer profile selection based on current port parameters (speed and attached cable length) which allows to guarantee lossless traffic flows after port speed change.

2. General feature design

PG profile update on port speed change includes:

  • Get cable length for this port
  • Get speed for this port (speed )
  • Using speed&cable-to-pg_profile look-up table determine new profile.
    Port speed and cable length are the two parameters which uniquely define the profile to be used on the port.
  • If profile is different from the old one – replace profile configuration on port
    This also includes:
    • Check whether old profile is still used on other ports – if not – delete it
    • Check whether new profile is already used. If not – create it first

2.1 Databases

Three Databases are involved:

  • CONFIG_DB: Stores buffers information expected to be applied on the switch and some information used by Buffer manager Daemon.
  • ASIC_DB: Stores information actually applied on the switch. Can be used in tests for validation.

2.2 Buffers Template

The template used for buffers config generation can implement static or variable configuration.
Static part contains information about cables length, buffer pools description and port profiles which should not change on port speed change.
Optionally there could be variable part of buffers configuration. This assumes having a look-up table for pg buffer profiles.

2.3 Buffer Configuration Daemon

Buffer Configuration Daemon(BCD) reacts to port speed change and updates PG profiles on port. This includes the following:

  • subscribe to CONFIG_DB:* to access buffers related tables and catch port update event
  • create/delete PG profiles in CONFIG_DB
  • update PG profile for port which speed was changed

2.4 Buffer configuration in orchagent

Existing PG creation code needs to be updated to to remove unused PG profiles.

3. Detailed feature design

3.1 Buffers Configuration Data Flow

3.2 Databases

3.2.1 CONFIG_DB

The following tables to be added to the

Static part (example)
{
    "BUFFER_POOL": {
        "egress_lossless_pool0": {
            "size": "3637248",
            "type": "egress",
            "mode": "static"
        },
        "egress_lossless_pool1": {
            "size": "3637248",
            "type": "egress",
            "mode": "static"
        },
        "egress_lossy_pool": {
            "size": "5491712",
            "type": "egress",
            "mode": "dynamic"
        }
    },
    "BUFFER_PROFILE": {
        "ingress_lossless_profile0": {
            "pool": "[BUFFER_POOL:ingress_lossless_pool0]",
            "xon": "18432",
            "xoff": "76800",
            "size": "113664",
            "static_th": "0"
        },
        "ingress_lossless_profile1": {
            "pool":"[BUFFER_POOL:ingress_lossless_pool1]",
            "xon":"18432",
            "xoff":"76800",
            "size":"113664",
            "static_th": "0"
        }
    }
}
Variable part (example)
  • cables length table

    { "CABLE_LENGTH": { "AZURE": { "Ethernet0": "5m", "Ethernet4": "40m", "Ethernet8": "300m", ... } } }

  • look-up table for mapping speed and cable length to pg buffer profiles

      # PG lossless profiles.
      # speed cable sizexon  xoff threshold
      10000  5m   34816  18432 16384  0
        25000  5m   34816  18432 16384  0
        40000  5m   34816  18432 16384  0
        50000  5m   34816  18432 16384  0
       100000  5m   36864  18432 18432  0
        10000  40m  36864  18432 18432  0
        25000  40m  39936  18432 21504  0
        40000  40m  41984  18432 23552  0
        50000  40m  41984  18432 23552  0
       100000  40m  54272  18432 35840  0
        10000  300m 49152  18432 30720  0
        25000  300m 71680  18432 53248  0
        40000  300m 94208  18432 75776  0
        50000  300m 94208  18432 75776  0
       100000  300m 184320 18432 165888 0
    
  • Profile and port to profile mapping for each port (created by Buffer Manager)

      "BUFFER_PG": {
          "Ethernet0:3-4": {
          	"profile" : "[BUFFER_PROFILE:pg_lossless_100G_300m_profile]"
          }
      }
    
      "BUFFER_PROFILE": {
          "pg_lossless_100G_300m_profile": {
              "pool":"[BUFFER_POOL:ingress_lossless_pool]",
              "xon":"18432",
              "xoff":"165888",
              "size":"184320",
              "dynamic_th":"1"
          }
      }
    

3.3 Buffers Template

To make the buffers configuration more flexible and adjustable to different configurations it will be partially implemented as a template. The following parts is implemented in the template:

  • Cable length table
    Generated from template to support different number of ports and distinguish cable length from the switch and neighbor roles.

  • port names
    Some static configurations (like queue profiles, egress or lossy PG profiles) should be applied for all ports. To avoid hard coding interface names they can be generated in the template.

3.4 Buffer Configuration Daemon

3.4.1 Input data

Besides the table declaring ports cable length, Buffer Configuration Daemon needs a lookup table described in 3.2.1 (variable part of the config). It will be passed to the utility as a command-line argument.

3.4.2 Start up

Buffer Configuration Daemon will be implemented as a stand alone utility. It will load inside swss docker container. So the appropriate record to the sonic-buildimage/dockers/docker-orchagent/supervisord.conf need to be added:

[program:buffermgrd]
command=/usr/bin/buffermgrd -l /usr/share/sonic/hwsku/pg_profile_lookup.ini
priority=10
autostart=false
autorestart=false
stdout_logfile=syslog
stderr_logfile=syslog

3.4.3 Logic

Every time when speed change event arrives, Buffer Configuration Daemon determines which profile need to be used for the port and updates appropriate BUFFER_PG table. Before updating the BUFFER_PG the utility checks whether needed profile is already created (it maybe used by some other port). If not - it needs to be created first (table BUFFER_PROFILE).

4. Testing and validation

4.1 Unit testing

Initial buffers configuration is validated on compile stage by the unit test.
Validations to be implemented:

  • no unused pools defined
  • all profiles referenced in the look-up table are declared
  • all pools referenced by profiles are declared

4.2 Run time validations

Test for VS.

5. Open questions

Clone this wiki locally
You can’t perform that action at this time.