Skip to content
Permalink
Browse files

Merge next (v2019.11 release) into master

  • Loading branch information...
eugeneia committed Nov 8, 2019
2 parents 24c9a67 + f6dc402 commit 4b0c18beb86223414fcdbf0dc9e9f5fa5aaa2179
Showing with 8,376 additions and 2,553 deletions.
  1. +1 −1 .version
  2. +1 −1 README.md
  3. +12 −15 lib/ljsyscall/syscall/linux/syscalls.lua
  4. +1 −1 src/Makefile
  5. +6 −2 src/README.md
  6. +4 −2 src/apps/intel/intel10g.lua
  7. +44 −0 src/apps/intel_avf/README.md
  8. +852 −0 src/apps/intel_avf/intel_avf.lua
  9. +20 −0 src/apps/intel_avf/selftest.sh
  10. +102 −0 src/apps/intel_avf/tests/back2back/test.snabb
  11. +11 −0 src/apps/intel_avf/tests/info/test.snabb
  12. +49 −0 src/apps/intel_avf/tests/setup.sh
  13. +14 −6 src/apps/intel_mp/README.md
  14. BIN src/apps/intel_mp/broadcast.pcap
  15. +589 −327 src/apps/intel_mp/intel_mp.lua
  16. +51 −0 src/apps/intel_mp/test_10g_2q_loopback.snabb
  17. +1 −1 src/apps/intel_mp/test_10g_counters.sh
  18. +31 −0 src/apps/intel_mp/test_10g_loopback.snabb
  19. +50 −0 src/apps/intel_mp/test_10g_vmdq_loopback.snabb
  20. +35 −0 src/apps/intel_mp/test_10g_vmdq_loopback_self.snabb
  21. +46 −0 src/apps/intel_mp/test_10g_vmdq_mcast.snabb
  22. +9 −0 src/apps/intel_mp/test_1g_1q_blast_vmdq.sh
  23. +1 −1 src/apps/intel_mp/test_1g_2q_blast.sh
  24. +19 −0 src/apps/intel_mp/test_1g_2q_blast_vlan.sh
  25. +20 −0 src/apps/intel_mp/test_1g_2q_blast_vmdq.sh
  26. +24 −0 src/apps/intel_mp/test_1g_2q_blast_vmdq_auto.sh
  27. +46 −0 src/apps/intel_mp/test_1g_vmdq_mcast.snabb
  28. +83 −0 src/apps/intel_mp/test_1g_vmdq_mirror.snabb
  29. +99 −0 src/apps/intel_mp/test_1g_vmdq_pool_sel.snabb
  30. +68 −0 src/apps/intel_mp/test_1g_vmdq_race.snabb
  31. +80 −0 src/apps/intel_mp/test_1g_vmdq_reconfig_mac.snabb
  32. +5 −0 src/apps/intel_mp/test_1g_vmdq_tx.sh
  33. +3 −2 src/apps/intel_mp/testsend.snabb
  34. +32 −18 src/apps/ipsec/README.md
  35. +140 −9 src/apps/ipsec/esp.lua
  36. +10 −75 src/apps/ipsec/selftest.sh
  37. +76 −0 src/apps/ipsec/test-linux-compat.sh
  38. +41 −40 src/apps/ipsec/{selftest.lua → test-linux-compat.snabb}
  39. +4 −2 src/apps/keyed_ipv6_tunnel/tunnel.lua
  40. +0 −1 src/apps/lwaftr/binding_table.lua
  41. +2 −2 src/apps/lwaftr/lwutil.lua
  42. +9 −1 src/apps/lwaftr/rangemap.lua
  43. +28 −0 src/apps/test/README.md
  44. +41 −0 src/apps/test/npackets.lua
  45. +16 −2 src/apps/test/synth.lua
  46. +14 −2 src/core/app.lua
  47. +5 −2 src/core/lib.lua
  48. +6 −5 src/core/main.lua
  49. +1 −1 src/core/memory.c
  50. +36 −4 src/core/packet.lua
  51. +93 −3 src/dasm_x86.lua
  52. +2 −0 src/doc/genbook.sh
  53. +22 −1 src/lib/README.numa.md
  54. +99 −0 src/lib/README.poptrie.md
  55. +14 −3 src/lib/binary_search.dasl
  56. +3 −60 src/lib/cpuset.lua
  57. +8 −23 src/lib/ctable.lua
  58. +2 −2 src/lib/fibers/timer.lua
  59. +48 −0 src/lib/gauge.lua
  60. +27 −13 src/lib/hardware/README.md
  61. +18 −12 src/lib/hardware/pci.lua
  62. +10 −9 src/lib/hardware/register.lua
  63. +64 −30 src/lib/hash/siphash.dasl
  64. +1 −1 src/lib/interlink.lua
  65. +9 −6 src/lib/ipsec/README.md
  66. +0 −301 src/lib/ipsec/aes_128_gcm.lua
  67. +472 −0 src/lib/ipsec/aes_gcm.lua
  68. +239 −80 src/lib/ipsec/{aes_128_gcm_avx.dasl → aes_gcm_avx.dasl}
  69. +75 −59 src/lib/ipsec/esp.lua
  70. +4 −1 src/lib/ipsec/seq_no_t.lua
  71. +2 −0 src/lib/ipsec/track_seq_no.c
  72. +2 −0 src/lib/ipsec/track_seq_no.h
  73. +5 −0 src/lib/macaddress.lua
  74. +11 −0 src/lib/maxpc.lua
  75. +8 −4 src/lib/multi_copy.dasl
  76. +111 −74 src/lib/numa.lua
  77. +601 −0 src/lib/poptrie.lua
  78. +182 −0 src/lib/poptrie_lookup.dasl
  79. +4 −1 src/lib/ptree/action_codec.lua
  80. +60 −26 src/lib/ptree/ptree.lua
  81. +38 −4 src/lib/ptree/support.lua
  82. +373 −306 src/lib/ptree/support/snabb-softwire-v2.lua
  83. +5 −0 src/lib/ptree/worker.lua
  84. +3 −2 src/lib/rrd.lua
  85. +3 −3 src/lib/timers/ingress_drop_monitor.lua
  86. +48 −0 src/lib/timers/memory_info.lua
  87. +84 −20 src/lib/yang/binary.lua
  88. +156 −36 src/lib/yang/data.lua
  89. +397 −203 src/lib/yang/ietf-softwire-br.yang
  90. +321 −219 src/lib/yang/ietf-softwire-common.yang
  91. +322 −12 src/lib/yang/path_data.lua
  92. +169 −14 src/lib/yang/schema.lua
  93. +19 −0 src/lib/yang/snabb-softwire-v2.yang
  94. +11 −5 src/lib/yang/yang.lua
  95. +1 −1 src/program/ipfix/probe/probe.lua
  96. +2 −2 src/program/lisper/lisper.lua
  97. +6 −3 src/program/loadtest/find-limit/find-limit.lua
  98. +80 −1 src/program/lwaftr/doc/CHANGELOG.md
  99. +61 −34 src/program/lwaftr/run/README
  100. +6 −10 src/program/lwaftr/run/run.lua
  101. +6 −6 src/program/lwaftr/setup.lua
  102. BIN src/program/lwaftr/tests/benchdata/ipv4-0054.pcap
  103. +1 −1 src/program/lwaftr/tests/propbased/genyang.lua
  104. +13 −20 src/program/lwaftr/tests/release-benchmarks/release-benchmarks.sh
  105. +230 −4 src/program/lwaftr/tests/subcommands/config_test.py
  106. +2 −2 src/program/lwaftr/tests/test_env.py
  107. +24 −17 src/program/packetblaster/lwaftr/README
  108. +326 −273 src/program/packetblaster/lwaftr/lib.lua
  109. +203 −49 src/program/packetblaster/lwaftr/lwaftr.lua
  110. BIN src/program/packetblaster/lwaftr/test_lwaftr_1.pcap
  111. BIN src/program/packetblaster/lwaftr/test_lwaftr_2.pcap
  112. +12 −4 src/program/packetblaster/selftest.sh
  113. +39 −0 src/program/rrdcat/README
  114. +1 −0 src/program/rrdcat/README.inc
  115. +278 −0 src/program/rrdcat/rrdcat.lua
  116. +6 −1 src/program/snabbmark/README
  117. +35 −19 src/program/snabbmark/snabbmark.lua
  118. +1 −1 src/program/snabbnfv/README.md
  119. +2 −2 src/program/snabbnfv/nfvconfig.lua
  120. +1 −1 src/program/snabbnfv/traffic/README
  121. +17 −17 src/scripts/dock.sh
  122. +29 −23 src/scripts/snabb_bot.sh
  123. +11 −6 src/scripts/snabb_doc.sh
@@ -1 +1 @@
2019.01
2019.11
@@ -142,5 +142,5 @@ Here are the ways you can get involved:
- Use the Snabb applications in your network.
- Create your very own application: [Getting Started](src/doc/getting-started.md).
- Create Github Issues with your ideas and questions and problems.
- [Join](https://join.slack.com/t/snabb/shared_invite/enQtMzIyOTIwMTg5ODYyLTMzY2FjMGEzM2QzNDlhMDYxNzU0M2UyNjQ1MDc4MDRjY2Q3MWMwY2Q4YWQ1NDllY2E3NTZkZGUyZTQxNzgyNjc) the [Snabb Slack chat](https://snabb.slack.com/) to hang out and shoot the breeze.
- [Join](https://join.slack.com/t/snabb/shared_invite/enQtMzIyOTIwMTg5ODYyLWUwYzg2MTIwMjAxZTM1N2RlNGFjMWY1YzRkZGZiN2U5ZGU1NTYwNWJiMmQ3MWQ1MDFjYTg0MjNjNTZjMTFlZGQ) the [Snabb Slack chat](https://snabb.slack.com/) to hang out and shoot the breeze.

@@ -483,33 +483,30 @@ local function get_maxnumnodes()
return math.floor(((#line+1)/9)*32)
end
end
-- If we don't know, guess that the system has a max of 1024 nodes.
return 1024
end

local function ensure_bitmask(mask, size)
if ffi.istype(t.bitmask, mask) then return mask end
return t.bitmask(mask, size or get_maxnumnodes())
end

function S.get_mempolicy(mode, mask, addr, flags)
mode = mode or t.int1()
local size
if ffi.istype(t.bitmask, mask) then
-- if mask was provided by the caller, then use its size
-- and let the syscall error if it's too small
size = ffi.cast("uint64_t", tonumber(mask.size))
else
local mask_for_size = t.bitmask(mask)
-- Size should be at least equals to maxnumnodes.
size = ffi.cast("uint64_t", math.max(tonumber(mask_for_size.size), get_maxnumnodes()))
mask = t.bitmask(mask, tonumber(size))
end
local ret, err = C.get_mempolicy(mode, mask.mask, size, addr or 0, c.MPOL_FLAG[flags])
mask = ensure_bitmask(mask);
local ret, err = C.get_mempolicy(mode, mask.mask, mask.size, addr or 0, c.MPOL_FLAG[flags])
if ret == -1 then return nil, t.error(err or errno()) end
return { mode=mode[0], mask=mask }
end
function S.set_mempolicy(mode, mask)
mask = mktype(t.bitmask, mask)
mask = ensure_bitmask(mask);
return retbool(C.set_mempolicy(c.MPOL_MODE[mode], mask.mask, mask.size))
end

function S.migrate_pages(pid, from, to)
from = mktype(t.bitmask, from)
to = mktype(t.bitmask, to)
from = ensure_bitmask(from);
to = ensure_bitmask(to, from.size)
assert(from.size == to.size, "incompatible nodemask sizes")
return retbool(C.migrate_pages(pid or 0, from.size, from.mask, to.mask))
end
@@ -45,7 +45,7 @@ LUAJIT_A := ../lib/luajit/src/raptorjit.a
# for each module that has a top-level selftest () function.
TESTMODS = $(shell find . -regex '[^\#]*\.\(lua\|dasl\)' -printf '%P ' | \
xargs grep -s -l '^function selftest *[[:punct:]]' | \
sed -e 's_\.lua__' -e 's_\.dasl__' -e 's_/_._g')
sed -e 's_\.lua__' -e 's_\.dasl__' -e 's_/_._g' -e 's/-/_/g')

# TESTSCRIPTS expands to:
# lib/watchdog/selftest.sh ...
@@ -816,9 +816,12 @@ end
Returns hexadecimal string for bytes in *string*.
— Function **lib.hexundump** *hexstring*
— Function **lib.hexundump** *hexstring*, *n*, *error*
Returns byte string for *hexstring*.
Returns string of *n* bytes for *hexstring*. Throws an error if less than *n*
hex-encoded bytes could be parsed unless *error* is `false`.
*Error* is optional and can be the error message to throw.
— Function **lib.comma_value** *n*
@@ -1016,6 +1019,7 @@ Groups of Snabb processes each have the following special properties:
mastering (DMA) is disabled upon termination before any DMA memory
is returned to the kernel. This prevents "dangling" DMA requests
from corrupting memory that has been freed and reused.
See [lib.hardware.pci](#pci-lib.hardware.pci) for details.
The `core.worker` API functions are available in the main process only:
@@ -91,9 +91,10 @@ function new_sf (conf)
end

function M_sf:open ()
self.fd = pci.open_pci_resource_locked(self.pciaddress, 0)
pci.unbind_device_from_linux(self.pciaddress)
pci.set_bus_master(self.pciaddress, true)
self.base, self.fd = pci.map_pci_memory_locked(self.pciaddress, 0)
self.base = pci.map_pci_memory(self.fd)
register.define(config_registers_desc, self.r, self.base)
register.define(transmit_registers_desc, self.r, self.base)
register.define(receive_registers_desc, self.r, self.base)
@@ -497,9 +498,10 @@ function new_pf (conf)
end

function M_pf:open ()
self.fd = pci.open_pci_resource_locked(self.pciaddress, 0)
pci.unbind_device_from_linux(self.pciaddress)
pci.set_bus_master(self.pciaddress, true)
self.base, self.fd = pci.map_pci_memory_locked(self.pciaddress, 0)
self.base = pci.map_pci_memory(self.fd)
register.define(config_registers_desc, self.r, self.base)
register.define_array(switch_config_registers_desc, self.r, self.base)
register.define_array(packet_filter_desc, self.r, self.base)
@@ -0,0 +1,44 @@
# Intel AVF (adaptive virtual function) app (apps.intel_avf.intel_avf)

The `intel_avf.Intel_avf` app provides drivers for the Virtual Functions exported
by recent generations of Intel network cards.

The links are named `input` and `output`.

DIAGRAM: Intel_avf
+-----------+
| |
input ---->* Intel_avf *----> output
| |
+-----------+
## Configuration
— Key **pciaddr**

*Required*. The PCI address of the NIC as a string.

— Key **ring_buffer_size**

*Optional*. Number of DMA descriptors to use i.e. size of the DMA
transmit and receive queues. Must be a multiple of 128. Default is not
specified but assumed to be broadly applicable.

## Supported Hardware
Ethernet controller [0200]: Intel Corporation Ethernet Virtual Function 700 Series [8086:154c] (rev 02)

## Unsupported features
* Multiple queues per VF. This driver supports a single queue. The spec allows for up to 4 queues.
* RSS with only 1 queue RSS doesn't make sense.
* Multiple vlans are unsupported, `ip link` can be used to map all traffic to a single vlan.
* Multiple MAC addresses are unsupported, `ip link` can be used to set the mac before snabb startup.
* All of the advanced offload features are unsupported.
* 16 byte RX descriptors are unsupported.

## Setting up VFs under linux
echo 2 > /sys/bus/pci/devices/0000\:03\:00.1/sriov_numvfs
ip link set up dev enp3s0f1
ip link set enp3s0f1 vf 0 mac 02:00:00:00:00:01
ip link set enp3s0f1 vf 0 vlan 10

A more complete example can be found in apps/intel_avf/tests/setup.sh

0 comments on commit 4b0c18b

Please sign in to comment.
You can’t perform that action at this time.