Skip to content

Commit 7ad18ff

Browse files
solbjorndavem330
authored andcommitted
gro: fix napi_gro_frags() Fast GRO breakage due to IP alignment check
Commit 38ec494 ("gro: ensure frag0 meets IP header alignment") did the right thing, but missed the fact that napi_gro_frags() logics calls for skb_gro_reset_offset() *before* pulling Ethernet header to the skb linear space. That said, the introduced check for frag0 address being aligned to 4 always fails for it as Ethernet header is obviously 14 bytes long, and in case with NET_IP_ALIGN its start is not aligned to 4. Fix this by adding @nhoff argument to skb_gro_reset_offset() which tells if an IP header is placed right at the start of frag0 or not. This restores Fast GRO for napi_gro_frags() that became very slow after the mentioned commit, and preserves the introduced check to avoid silent unaligned accesses. From v1 [0]: - inline tiny skb_gro_reset_offset() to let the code be optimized more efficively (esp. for the !NET_IP_ALIGN case) (Eric); - pull in Reviewed-by from Eric. [0] https://lore.kernel.org/netdev/20210418114200.5839-1-alobakin@pm.me Fixes: 38ec494 ("gro: ensure frag0 meets IP header alignment") Reviewed-by: Eric Dumazet <edumazet@google.com> Signed-off-by: Alexander Lobakin <alobakin@pm.me> Signed-off-by: David S. Miller <davem@davemloft.net>
1 parent 8d892d6 commit 7ad18ff

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

net/core/dev.c

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5914,7 +5914,7 @@ static struct list_head *gro_list_prepare(struct napi_struct *napi,
59145914
return head;
59155915
}
59165916

5917-
static void skb_gro_reset_offset(struct sk_buff *skb)
5917+
static inline void skb_gro_reset_offset(struct sk_buff *skb, u32 nhoff)
59185918
{
59195919
const struct skb_shared_info *pinfo = skb_shinfo(skb);
59205920
const skb_frag_t *frag0 = &pinfo->frags[0];
@@ -5925,7 +5925,7 @@ static void skb_gro_reset_offset(struct sk_buff *skb)
59255925

59265926
if (!skb_headlen(skb) && pinfo->nr_frags &&
59275927
!PageHighMem(skb_frag_page(frag0)) &&
5928-
(!NET_IP_ALIGN || !(skb_frag_off(frag0) & 3))) {
5928+
(!NET_IP_ALIGN || !((skb_frag_off(frag0) + nhoff) & 3))) {
59295929
NAPI_GRO_CB(skb)->frag0 = skb_frag_address(frag0);
59305930
NAPI_GRO_CB(skb)->frag0_len = min_t(unsigned int,
59315931
skb_frag_size(frag0),
@@ -6143,7 +6143,7 @@ gro_result_t napi_gro_receive(struct napi_struct *napi, struct sk_buff *skb)
61436143
skb_mark_napi_id(skb, napi);
61446144
trace_napi_gro_receive_entry(skb);
61456145

6146-
skb_gro_reset_offset(skb);
6146+
skb_gro_reset_offset(skb, 0);
61476147

61486148
ret = napi_skb_finish(napi, skb, dev_gro_receive(napi, skb));
61496149
trace_napi_gro_receive_exit(ret);
@@ -6232,7 +6232,7 @@ static struct sk_buff *napi_frags_skb(struct napi_struct *napi)
62326232
napi->skb = NULL;
62336233

62346234
skb_reset_mac_header(skb);
6235-
skb_gro_reset_offset(skb);
6235+
skb_gro_reset_offset(skb, hlen);
62366236

62376237
if (unlikely(skb_gro_header_hard(skb, hlen))) {
62386238
eth = skb_gro_header_slow(skb, hlen, 0);

0 commit comments

Comments
 (0)