Skip to content

Commit 8545b75

Browse files
davem330davem330
committed
Merge branch 'octeontx2-rvu-rep'
Geetha sowjanya says: ==================== Introduce RVU representors This series adds representor support for each rvu devices. When switchdev mode is enabled, representor netdev is registered for each rvu device. In implementation of representor model, one NIX HW LF with multiple SQ and RQ is reserved, where each RQ and SQ of the LF are mapped to a representor. A loopback channel is reserved to support packet path between representors and VFs. CN10K silicon supports 2 types of MACs, RPM and SDP. This patch set adds representor support for both RPM and SDP MAC interfaces. - Patch 1: Implements basic representor driver. - Patch 2: Add devlink support to create representor netdevs that can be used to manage VFs. - Patch 3: Implements basec netdev_ndo_ops. - Patch 4: Installs tcam rules to route packets between representor and VFs. - Patch 5: Enables fetching VF stats via representor interface - Patch 6: Adds support to sync link state between representors and VFs . - Patch 7: Enables configuring VF MTU via representor netdevs. - Patch 8: Adds representors for sdp MAC. - Patch 9: Adds devlink port support. - Patch 10: Implements offload stats. - Patch 11: Implements tc offload support. - patch 12: Adds documentation for rvu port representor. pci/0002:1c:00.0 Command to create PF/VF representor Rpf1vf0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether f6:43:83:ee:26:21 brd ff:ff:ff:ff:ff:ff Rpf1vf1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 12:b2:54:0e:24:54 brd ff:ff:ff:ff:ff:ff Rpf1vf2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 4a:12:c4:4c:32:62 brd ff:ff:ff:ff:ff:ff Rpf1vf3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether ca:cb:68:0e:e2:6e brd ff:ff:ff:ff:ff:ff Rpf2vf0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 06:cc:ad:b4:f0:93 brd ff:ff:ff:ff:ff:ff ~# devlink port pci/0002:1c:00.0/0: type eth netdev Rpf1vf0 flavour physical port 0 splittable false pci/0002:1c:00.0/1: type eth netdev Rpf1vf1 flavour pcivf controller 0 pfnum 1 vfnum 1 external false splittable false pci/0002:1c:00.0/2: type eth netdev Rpf1vf2 flavour pcivf controller 0 pfnum 1 vfnum 2 external false splittable false pci/0002:1c:00.0/3: type eth netdev Rpf1vf3 flavour pcivf controller 0 pfnum 1 vfnum 3 external false splittable false ----------- v11:v1: - Submitted refactoring changes as a separate patch set. https://lore.kernel.org/netdev/20241023161843.15543-1-gakula@marvell.com/T/ - Moved documentation to a separate patch. - patch 9: Added code changes to forward updated mac address to VF. - Implemented TC offload support. v10-v11: - As suggested by "Jiri Pirko" adjusted the documentation. - Added more commit description to patch1. v9-v10: - Fixed build warning w.r.t documentation. v8-v9: - Updated the documentation. v7-v8: - Implemented offload stats ndo. - Added documentation. v6-v7: - Rebased on top net-next branch. v5-v6: - Addressed review comments provided by "Simon Horman". - Added review tag. v4-v5: - Patch 3: Removed devm_* usage in rvu_rep_create() - Patch 3: Fixed build warnings. v3-v4: - Patch 2 & 3: Fixed coccinelle reported warnings. - Patch 10: Added devlink port support. v2-v3: - Used extack for error messages. - As suggested reworked commit messages. - Fixed sparse warning. v1-v2: -Fixed build warnings. -Address review comments provided by "Kalesh Anakkur Purayil". ==================== Signed-off-by: David S. Miller <davem@davemloft.net>
2 parents ef04d29 + 6050b04 commit 8545b75

File tree

28 files changed

+1942
-131
lines changed

28 files changed

+1942
-131
lines changed

‎Documentation/networking/device_drivers/ethernet/marvell/octeontx2.rst‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ Contents
1414
- `Basic packet flow`_
1515
- `Devlink health reporters`_
1616
- `Quality of service`_
17+
- `RVU representors`_
1718

1819
Overview
1920
========
@@ -340,3 +341,93 @@ Setup HTB offload
340341
# tc class add dev <interface> parent 1: classid 1:2 htb rate 10Gbit prio 2 quantum 188416
341342

342343
# tc class add dev <interface> parent 1: classid 1:3 htb rate 10Gbit prio 2 quantum 32768
344+
345+
346+
RVU Representors
347+
================
348+
349+
RVU representor driver adds support for creation of representor devices for
350+
RVU PFs' VFs in the system. Representor devices are created when user enables
351+
the switchdev mode.
352+
Switchdev mode can be enabled either before or after setting up SRIOV numVFs.
353+
All representor devices share a single NIXLF but each has a dedicated Rx/Tx
354+
queues. RVU PF representor driver registers a separate netdev for each
355+
Rx/Tx queue pair.
356+
357+
Current HW does not support built-in switch which can do L2 learning and
358+
forwarding packets between representee and representor. Hence, packet path
359+
between representee and it's representor is achieved by setting up appropriate
360+
NPC MCAM filters.
361+
Transmit packets matching these filters will be loopbacked through hardware
362+
loopback channel/interface (i.e, instead of sending them out of MAC interface).
363+
Which will again match the installed filters and will be forwarded.
364+
This way representee => representor and representor => representee packet
365+
path is achieved. These rules get installed when representors are created
366+
and gets active/deactivate based on the representor/representee interface state.
367+
368+
Usage example:
369+
370+
- Change device to switchdev mode::
371+
372+
# devlink dev eswitch set pci/0002:1c:00.0 mode switchdev
373+
374+
- List of representor devices on the system::
375+
376+
# ip link show
377+
Rpf1vf0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether f6:43:83:ee:26:21 brd ff:ff:ff:ff:ff:ff
378+
Rpf1vf1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 12:b2:54:0e:24:54 brd ff:ff:ff:ff:ff:ff
379+
Rpf1vf2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 4a:12:c4:4c:32:62 brd ff:ff:ff:ff:ff:ff
380+
Rpf1vf3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether ca:cb:68:0e:e2:6e brd ff:ff:ff:ff:ff:ff
381+
Rpf2vf0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT group default qlen 1000 link/ether 06:cc:ad:b4:f0:93 brd ff:ff:ff:ff:ff:ff
382+
383+
384+
To delete the representors devices from the system. Change the device to legacy mode.
385+
386+
- Change device to legacy mode::
387+
388+
# devlink dev eswitch set pci/0002:1c:00.0 mode legacy
389+
390+
RVU representors can be managed using devlink ports
391+
(see :ref:`Documentation/networking/devlink/devlink-port.rst <devlink_port>`) interface.
392+
393+
- Show devlink ports of representors::
394+
395+
# devlink port
396+
pci/0002:1c:00.0/0: type eth netdev Rpf1vf0 flavour physical port 0 splittable false
397+
pci/0002:1c:00.0/1: type eth netdev Rpf1vf1 flavour pcivf controller 0 pfnum 1 vfnum 1 external false splittable false
398+
pci/0002:1c:00.0/2: type eth netdev Rpf1vf2 flavour pcivf controller 0 pfnum 1 vfnum 2 external false splittable false
399+
pci/0002:1c:00.0/3: type eth netdev Rpf1vf3 flavour pcivf controller 0 pfnum 1 vfnum 3 external false splittable false
400+
401+
Function attributes
402+
===================
403+
404+
The RVU representor support function attributes for representors.
405+
Port function configuration of the representors are supported through devlink eswitch port.
406+
407+
MAC address setup
408+
-----------------
409+
410+
RVU representor driver support devlink port function attr mechanism to setup MAC
411+
address. (refer to Documentation/networking/devlink/devlink-port.rst)
412+
413+
- To setup MAC address for port 2::
414+
415+
# devlink port function set pci/0002:1c:00.0/2 hw_addr 5c:a1:1b:5e:43:11
416+
# devlink port show pci/0002:1c:00.0/2
417+
pci/0002:1c:00.0/2: type eth netdev Rpf1vf2 flavour pcivf controller 0 pfnum 1 vfnum 2 external false splittable false
418+
function:
419+
hw_addr 5c:a1:1b:5e:43:11
420+
421+
422+
TC offload
423+
==========
424+
425+
The rvu representor driver implements support for offloading tc rules using port representors.
426+
427+
- Drop packets with vlan id 3::
428+
429+
# tc filter add dev Rpf1vf0 protocol 802.1Q parent ffff: flower vlan_id 3 vlan_ethtype ipv4 skip_sw action drop
430+
431+
- Redirect packets with vlan id 5 and IPv4 packets to eth1, after stripping vlan header.::
432+
433+
# tc filter add dev Rpf1vf0 ingress protocol 802.1Q flower vlan_id 5 vlan_ethtype ipv4 skip_sw action vlan pop action mirred ingress redirect dev eth1

‎drivers/net/ethernet/marvell/octeontx2/Kconfig‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,11 @@ config OCTEONTX2_VF
4646
depends on OCTEONTX2_PF
4747
help
4848
This driver supports Marvell's OcteonTX2 NIC virtual function.
49+
50+
config RVU_ESWITCH
51+
tristate "Marvell RVU E-Switch support"
52+
depends on OCTEONTX2_PF
53+
default m
54+
help
55+
This driver supports Marvell's RVU E-Switch that
56+
provides internal SRIOV packet steering and switching.

‎drivers/net/ethernet/marvell/octeontx2/af/Makefile‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,5 @@ rvu_mbox-y := mbox.o rvu_trace.o
1111
rvu_af-y := cgx.o rvu.o rvu_cgx.o rvu_npa.o rvu_nix.o \
1212
rvu_reg.o rvu_npc.o rvu_debugfs.o ptp.o rvu_npc_fs.o \
1313
rvu_cpt.o rvu_devlink.o rpm.o rvu_cn10k.o rvu_switch.o \
14-
rvu_sdp.o rvu_npc_hash.o mcs.o mcs_rvu_if.o mcs_cnf10kb.o
14+
rvu_sdp.o rvu_npc_hash.o mcs.o mcs_rvu_if.o mcs_cnf10kb.o \
15+
rvu_rep.o

‎drivers/net/ethernet/marvell/octeontx2/af/common.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,7 @@ enum nix_scheduler {
156156
#define NIC_HW_MIN_FRS 40
157157
#define NIC_HW_MAX_FRS 9212
158158
#define SDP_HW_MAX_FRS 65535
159+
#define SDP_HW_MIN_FRS 16
159160
#define CN10K_LMAC_LINK_MAX_FRS 16380 /* 16k - FCS */
160161
#define CN10K_LBK_LINK_MAX_FRS 65535 /* 64k */
161162

‎drivers/net/ethernet/marvell/octeontx2/af/mbox.h‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,9 @@ M(LMTST_TBL_SETUP, 0x00a, lmtst_tbl_setup, lmtst_tbl_setup_req, \
144144
msg_rsp) \
145145
M(SET_VF_PERM, 0x00b, set_vf_perm, set_vf_perm, msg_rsp) \
146146
M(PTP_GET_CAP, 0x00c, ptp_get_cap, msg_req, ptp_get_cap_rsp) \
147+
M(GET_REP_CNT, 0x00d, get_rep_cnt, msg_req, get_rep_cnt_rsp) \
148+
M(ESW_CFG, 0x00e, esw_cfg, esw_cfg_req, msg_rsp) \
149+
M(REP_EVENT_NOTIFY, 0x00f, rep_event_notify, rep_event, msg_rsp) \
147150
/* CGX mbox IDs (range 0x200 - 0x3FF) */ \
148151
M(CGX_START_RXTX, 0x200, cgx_start_rxtx, msg_req, msg_rsp) \
149152
M(CGX_STOP_RXTX, 0x201, cgx_stop_rxtx, msg_req, msg_rsp) \
@@ -319,6 +322,7 @@ M(NIX_MCAST_GRP_DESTROY, 0x802c, nix_mcast_grp_destroy, nix_mcast_grp_destroy_re
319322
M(NIX_MCAST_GRP_UPDATE, 0x802d, nix_mcast_grp_update, \
320323
nix_mcast_grp_update_req, \
321324
nix_mcast_grp_update_rsp) \
325+
M(NIX_LF_STATS, 0x802e, nix_lf_stats, nix_stats_req, nix_stats_rsp) \
322326
/* MCS mbox IDs (range 0xA000 - 0xBFFF) */ \
323327
M(MCS_ALLOC_RESOURCES, 0xa000, mcs_alloc_resources, mcs_alloc_rsrc_req, \
324328
mcs_alloc_rsrc_rsp) \
@@ -380,12 +384,16 @@ M(CPT_INST_LMTST, 0xD00, cpt_inst_lmtst, cpt_inst_lmtst_req, msg_rsp)
380384
#define MBOX_UP_MCS_MESSAGES \
381385
M(MCS_INTR_NOTIFY, 0xE00, mcs_intr_notify, mcs_intr_info, msg_rsp)
382386

387+
#define MBOX_UP_REP_MESSAGES \
388+
M(REP_EVENT_UP_NOTIFY, 0xEF0, rep_event_up_notify, rep_event, msg_rsp) \
389+
383390
enum {
384391
#define M(_name, _id, _1, _2, _3) MBOX_MSG_ ## _name = _id,
385392
MBOX_MESSAGES
386393
MBOX_UP_CGX_MESSAGES
387394
MBOX_UP_CPT_MESSAGES
388395
MBOX_UP_MCS_MESSAGES
396+
MBOX_UP_REP_MESSAGES
389397
#undef M
390398
};
391399

@@ -1364,6 +1372,37 @@ struct nix_bandprof_get_hwinfo_rsp {
13641372
u32 policer_timeunit;
13651373
};
13661374

1375+
struct nix_stats_req {
1376+
struct mbox_msghdr hdr;
1377+
u8 reset;
1378+
u16 pcifunc;
1379+
u64 rsvd;
1380+
};
1381+
1382+
struct nix_stats_rsp {
1383+
struct mbox_msghdr hdr;
1384+
u16 pcifunc;
1385+
struct {
1386+
u64 octs;
1387+
u64 ucast;
1388+
u64 bcast;
1389+
u64 mcast;
1390+
u64 drop;
1391+
u64 drop_octs;
1392+
u64 drop_mcast;
1393+
u64 drop_bcast;
1394+
u64 err;
1395+
u64 rsvd[5];
1396+
} rx;
1397+
struct {
1398+
u64 ucast;
1399+
u64 bcast;
1400+
u64 mcast;
1401+
u64 drop;
1402+
u64 octs;
1403+
} tx;
1404+
};
1405+
13671406
/* NPC mbox message structs */
13681407

13691408
#define NPC_MCAM_ENTRY_INVALID 0xFFFF
@@ -1525,6 +1564,41 @@ struct ptp_get_cap_rsp {
15251564
u64 cap;
15261565
};
15271566

1567+
struct get_rep_cnt_rsp {
1568+
struct mbox_msghdr hdr;
1569+
u16 rep_cnt;
1570+
u16 rep_pf_map[64];
1571+
u64 rsvd;
1572+
};
1573+
1574+
struct esw_cfg_req {
1575+
struct mbox_msghdr hdr;
1576+
u8 ena;
1577+
u64 rsvd;
1578+
};
1579+
1580+
struct rep_evt_data {
1581+
u8 port_state;
1582+
u8 vf_state;
1583+
u16 rx_mode;
1584+
u16 rx_flags;
1585+
u16 mtu;
1586+
u8 mac[ETH_ALEN];
1587+
u64 rsvd[5];
1588+
};
1589+
1590+
struct rep_event {
1591+
struct mbox_msghdr hdr;
1592+
u16 pcifunc;
1593+
#define RVU_EVENT_PORT_STATE BIT_ULL(0)
1594+
#define RVU_EVENT_PFVF_STATE BIT_ULL(1)
1595+
#define RVU_EVENT_MTU_CHANGE BIT_ULL(2)
1596+
#define RVU_EVENT_RX_MODE_CHANGE BIT_ULL(3)
1597+
#define RVU_EVENT_MAC_ADDR_CHANGE BIT_ULL(4)
1598+
u16 event;
1599+
struct rep_evt_data evt_data;
1600+
};
1601+
15281602
struct flow_msg {
15291603
unsigned char dmac[6];
15301604
unsigned char smac[6];
@@ -1563,6 +1637,7 @@ struct flow_msg {
15631637
u8 icmp_type;
15641638
u8 icmp_code;
15651639
__be16 tcp_flags;
1640+
u16 sq_id;
15661641
};
15671642

15681643
struct npc_install_flow_req {

‎drivers/net/ethernet/marvell/octeontx2/af/rvu.h‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -513,6 +513,11 @@ struct rvu_switch {
513513
u16 start_entry;
514514
};
515515

516+
struct rep_evtq_ent {
517+
struct list_head node;
518+
struct rep_event event;
519+
};
520+
516521
struct rvu {
517522
void __iomem *afreg_base;
518523
void __iomem *pfreg_base;
@@ -595,6 +600,15 @@ struct rvu {
595600
spinlock_t cpt_intr_lock;
596601

597602
struct mutex mbox_lock; /* Serialize mbox up and down msgs */
603+
u16 rep_pcifunc;
604+
int rep_cnt;
605+
u16 *rep2pfvf_map;
606+
u8 rep_mode;
607+
struct work_struct rep_evt_work;
608+
struct workqueue_struct *rep_evt_wq;
609+
struct list_head rep_evtq_head;
610+
/* Representor event lock */
611+
spinlock_t rep_evtq_lock;
598612
};
599613

600614
static inline void rvu_write64(struct rvu *rvu, u64 block, u64 offset, u64 val)
@@ -853,6 +867,14 @@ bool is_sdp_pfvf(u16 pcifunc);
853867
bool is_sdp_pf(u16 pcifunc);
854868
bool is_sdp_vf(struct rvu *rvu, u16 pcifunc);
855869

870+
static inline bool is_rep_dev(struct rvu *rvu, u16 pcifunc)
871+
{
872+
if (rvu->rep_pcifunc && rvu->rep_pcifunc == pcifunc)
873+
return true;
874+
875+
return false;
876+
}
877+
856878
/* CGX APIs */
857879
static inline bool is_pf_cgxmapped(struct rvu *rvu, u8 pf)
858880
{
@@ -1051,7 +1073,8 @@ int rvu_ndc_fix_locked_cacheline(struct rvu *rvu, int blkaddr);
10511073
/* RVU Switch */
10521074
void rvu_switch_enable(struct rvu *rvu);
10531075
void rvu_switch_disable(struct rvu *rvu);
1054-
void rvu_switch_update_rules(struct rvu *rvu, u16 pcifunc);
1076+
void rvu_switch_update_rules(struct rvu *rvu, u16 pcifunc, bool ena);
1077+
void rvu_switch_enable_lbk_link(struct rvu *rvu, u16 pcifunc, bool ena);
10551078

10561079
int rvu_npc_set_parse_mode(struct rvu *rvu, u16 pcifunc, u64 mode, u8 dir,
10571080
u64 pkind, u8 var_len_off, u8 var_len_off_mask,
@@ -1064,4 +1087,9 @@ int rvu_mcs_flr_handler(struct rvu *rvu, u16 pcifunc);
10641087
void rvu_mcs_ptp_cfg(struct rvu *rvu, u8 rpm_id, u8 lmac_id, bool ena);
10651088
void rvu_mcs_exit(struct rvu *rvu);
10661089

1090+
/* Representor APIs */
1091+
int rvu_rep_pf_init(struct rvu *rvu);
1092+
int rvu_rep_install_mcam_rules(struct rvu *rvu);
1093+
void rvu_rep_update_rules(struct rvu *rvu, u16 pcifunc, bool ena);
1094+
int rvu_rep_notify_pfvf_state(struct rvu *rvu, u16 pcifunc, bool enable);
10671095
#endif /* RVU_H */

‎drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c‎

Lines changed: 0 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -45,33 +45,6 @@ enum {
4545
CGX_STAT18,
4646
};
4747

48-
/* NIX TX stats */
49-
enum nix_stat_lf_tx {
50-
TX_UCAST = 0x0,
51-
TX_BCAST = 0x1,
52-
TX_MCAST = 0x2,
53-
TX_DROP = 0x3,
54-
TX_OCTS = 0x4,
55-
TX_STATS_ENUM_LAST,
56-
};
57-
58-
/* NIX RX stats */
59-
enum nix_stat_lf_rx {
60-
RX_OCTS = 0x0,
61-
RX_UCAST = 0x1,
62-
RX_BCAST = 0x2,
63-
RX_MCAST = 0x3,
64-
RX_DROP = 0x4,
65-
RX_DROP_OCTS = 0x5,
66-
RX_FCS = 0x6,
67-
RX_ERR = 0x7,
68-
RX_DRP_BCAST = 0x8,
69-
RX_DRP_MCAST = 0x9,
70-
RX_DRP_L3BCAST = 0xa,
71-
RX_DRP_L3MCAST = 0xb,
72-
RX_STATS_ENUM_LAST,
73-
};
74-
7548
static char *cgx_rx_stats_fields[] = {
7649
[CGX_STAT0] = "Received packets",
7750
[CGX_STAT1] = "Octets of received packets",

‎drivers/net/ethernet/marvell/octeontx2/af/rvu_devlink.c‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1500,6 +1500,9 @@ static int rvu_devlink_eswitch_mode_get(struct devlink *devlink, u16 *mode)
15001500
struct rvu *rvu = rvu_dl->rvu;
15011501
struct rvu_switch *rswitch;
15021502

1503+
if (rvu->rep_mode)
1504+
return -EOPNOTSUPP;
1505+
15031506
rswitch = &rvu->rswitch;
15041507
*mode = rswitch->mode;
15051508

0 commit comments

Comments
 (0)