Snabb 2016.11 “Babaco”

@eugeneia eugeneia released this Nov 7, 2016

This release introduces a new library for managing NUMA pinning from within Snabb, a new design for more efficient shifting and appending to packets, and re-synchronization functionality for our ESP implementation, meaning its conceptually complete. Babaco also synchronizes the lwAFTR application with Igalia, and Snabb NFV has received some love as well. As usual this release contains many miscellaneous bug fixes and improvements.

Major Changes

  • #1046 #1047 Synchronized lwAFTR program with Igalia
  • #1038 Introduced lib.numa for managing NUMA pinning from within Snabb
  • #1032 Introduced new design for shifts and appends in core.packet: instead of moving the packet data, it now re-positions the pointer
  • #1000 Implemented re-synchronization in lib.ipsec.esp

Minor Changes

  • #911 Added YANG model for Snabb NFV configuration
  • #1051 Fixed bug in checksum method of lib.protocol.ipv4
  • #1054 Polished and fixed critical bug in ESP integration for Snabb NFV
  • #1039 Fixed a bug in Snabb NFV where redundant reconfiguration occurred
  • #1034 Virtio: use packet shifting to tag packets
  • #1031 Included various missing sections in the Snabb manual


Andy Wingo (85):

  • 4540e60: Add missing lwaftr end-to-end-vlan conf file
  • c91d3a9: RateLimitedRepeater operates real line rate
  • e114ec9: loadtest prints out bitrates with overhead
  • 8260923: PodHashMap: Be more like CTable
  • 36780f5: Add PodHashMap:iterate
  • 7f5cca8: RangeMap: Add :iterate method
  • 231c59e: Add BindingTable:iterate_psid_map
  • 697c560: Add BindingTable:iterate_br_addresses
  • 6d97550: Add BindingTable:iterate_softwires
  • 033e746: Add documentation.
  • 9278eb0: Add BindingTable:dump
  • a051092: Hook up binding table dumping to the lwAFTR
  • dd5e5b3: Fix promise bug with more than one argument.
  • 2c118cd: Loadtest supports different programs
  • 3b1dfb4: Forward-port CHANGELOG.
  • 163e611: Disable JIT flush/drop monitor on loadtest; print ingress drops.
  • e382b30: Update documentation
  • f0d75a3: Add get_mempolicy, set_mempolicy NUMA syscalls
  • af8f015: Add beginnings of NUMA module for Snabb
  • fc0bb6d: Add cpu_get_numa_node
  • 0cb8bd8: Add more NUMA methods
  • a3d8fa4: Add choose_numa_node_for_pci_addresses
  • 449490d: pci_get_numa_node can accept canonical addresses
  • 85f0319: Add numa.check_affinity_for_pci_addresses
  • 806746d: Work around a broken `getcpu'
  • e062328: Use lib.numa in lwaftr run
  • d9390f3: ljsyscall: Fix getcpu()
  • fabb961: ljsyscall: Add migrate_pages.
  • 50a920b: numa: Use ljsyscall's fixed "getcpu()" implementation
  • d802b76: numa: Migrate existing pages when binding to NUMA node.
  • 976b5df: lwaftr loadtest uses lib.numa
  • 0f2fa19: More performance documentation updates
  • b792e57: v2.7 changelog
  • e45397a: More helpful jit.flush message
  • d87de59: Fix time printout.
  • eb4aac4: Print troubleshooting link at every jit.flush.
  • 841266d: Change --cpu mentions to taskset/numactl
  • 0c89a4d: v2.8 release notes
  • c9676f6: virtio-net driver: Use C header files
  • e25a6bb: Add some headroom before a packet's data.
  • 0650c04: Disable backpressure on intel driver
  • d6087e5: virtio-net driver: Use direct descriptors
  • da583ac: Fix assumptions that packet ==
  • 30ddd81: Indirect access to packet data
  • fbec950: Bring length back to beginning of struct packet
  • ae121ca: lwaftr is resilient to changes in
  • dfe46a2: Refactor packet headroom mechanism
  • 8c15b78: Add assertions in packet.lua
  • 0e619fe: Adapt virtio to use packet.shift for its headers
  • 57177b2: Bump packet headroom to 64 bytes.
  • 1b401ba: Default headroom in Lua, not C
  • 814b208: v2.9 changelog
  • 780e46a: Refactor ingress drop monitor to have configurable actions
  • c249604: Enable warning ingress drop monitor on the NFV
  • 5540648: snabbnfv traffic: Only re-start engine when configuration changes
  • a2a6713: Add v2.10 changelog.
  • 938594d: Fix assumptions that packet ==
  • 5416f9c: lwaftr is resilient to changes in
  • 71679a5: Implement shiftleft/shiftright by moving packet around
  • c2097d2: Add minimum packet alignment.
  • 998427e: virtio-net driver: Use C header files
  • a5541e5: virtio-net driver: Use direct descriptors
  • 3e61642: Convert lwaftr V4V6, generator, nh_fwd apps to new config
  • c09e62d: Fix name of Join output link.
  • 3f8f337: Supply nh_fwd app with minimal configuration.
  • 93629e0: Add tests.
  • 12f77a3: Fix allocate/new_packet mixup.
  • 7bfbba4: Add get_mempolicy, set_mempolicy NUMA syscalls
  • 9a0abb2: ljsyscall: Fix getcpu()
  • c2f2d3d: ljsyscall: Add migrate_pages.
  • 3b5fe60: Add NUMA module for Snabb
  • 243480d: Remove stale timers code
  • 4b8930d: Revert message change relative to uptream ctable
  • 2567ddc: snabbnfv traffic: Only re-start engine when configuration changes
  • ade4b3e: Reformat No change.
  • b5a2cde: Merge lwAFTR performance notes to
  • eef390a: Tweaks
  • 6c1f585: Refactor ingress drop monitor to have configurable actions
  • de93573: Enable warning ingress drop monitor on the NFV
  • 3489e4b: Address feedback
  • 6bb456f: Address feedback bis
  • 21759e2: Add
  • e115542: Update for packet.shiftleft et al
  • 4b9becf: Fix typos in
  • 876addc: Fix shm.exists() check in ingress drop monitor

Diego Pino Garcia (136):

  • 11ff29f: Update and clean up documentation
  • 7ecb418: Monitor ingress packet drops and jit flush if threshold exceeded
  • f9d6289: Remove invalid NDP NS test
  • 6846a3f: NDP: Match Neighbour Solicitation target address to lwAFTR IPv6 address
  • 740f29b: Rename arguments v4-pci and v6-pci to v4 and v6
  • cf46993: Update documentation regarding v4 and v6 arguments
  • c1758a2: Remove short options for arguments v4 and v6
  • 377b121: Add warning message about deprecated arguments '--v4-pci' and '--v6-pci'
  • 018b086: Several improvements in virt/lwaftrctl
  • 9e86b23: Handle NS requests and retries without timers
  • f04e111: Implement next-hop forwarder
  • 7ca5e52: Add nh_fwd unit tests
  • f5eaada: Dump packet in 'od' format
  • 000b99e: Return source ethernet if Target Link is not present
  • 08525dd: Assert destination ethernet address in selftest
  • abe6584: Add Neighbor Advertisement unit test
  • e2bf05c: Clear out Flags in ICMPv4 echo-reply
  • 0b5c313: Add method timer.deactivate(name)
  • 0cae549: Retry Neighbour Solicitations during time limit
  • 5edd4a2: Send NS packet inmediately and retry after every second
  • 21d6145: Refactor dump lwAFTR configuration
  • 481f2b5: Dump binding table to text file
  • 4bbe3a9: Change dump file names format
  • 164dc62: Redo lwAFTR's conf dump selftest
  • 2f829bd: Use conf parser to read conf file in dump.lua's selftest
  • c6788af: Update
  • a461a48: Reintroduce v4-pci and v6-pci for backwards compatibility
  • cc677cc: Implement ARP requests retries
  • 223da17: Cleanup of lwaftrctl script
  • 4ad3379: Fix IPSec's esp selftest
  • 06b3483: Adapt lwAFTR counters to new counters API
  • c1ed420: Name parameters in 'snabb_run_and_cmp'
  • 5b8895d: Fix test 'from-to-b4 IPv6 packet NOT found in the binding table'
  • 85299c2: Print error if couldn't find file
  • 7e03587: Add test 'Testing sending non-IPv6 traffic to the IPv6 interface'
  • b727704: Add test 'Testing sending non-IPv4 traffic to the IPv4 interface'
  • dc381f6: Add on-a-stick mode
  • 10331e2: Add lwAFTR monitor program
  • 2b35a95: Add V4V6 selftest
  • c1471a0: Refactor load_check
  • d74c72c: Tag ARP packets as IPv4 packets
  • f8250d9: Add on-a-stick support for 'lwAFTR check'
  • 8aca7e6: Run end-to-end tests in on-a-stick mode too
  • 2959333: Run end-to-end-vlan tests in on-a-stick mode too
  • 719b3ab: Rename 'v4v6' app to 'V4V6'
  • 4fe25a3: Add support for icmpv4_rate_limiter_n_packets and icmpv4_rate_limiter_n_seconds configuration parameters
  • 45a66d4: Refactor transmit_icmpv6_with_rate_limit
  • ad87150: Rate-limiting of ICMPv4 packets
  • 98519bf: Remove 'ingress_drop_monitor' dead code
  • 1119dfb: Import 'lwaftr query' program to lwaftr branch
  • a8353a7: Format numbers with lib.comma_value
  • f20229a: Filter counters to print
  • 30a83b2: Sort counters
  • c880ee8: Align counter values
  • 6de1d4a: Add option to print out all available counter names
  • f915586: Add a flag to allow monitor all packets in on-a-stick mode
  • 696c33e: Add lwaftr nexthop program
  • f4c48d7: Import ipv6_equals
  • a26698d: Fix self.debug comparison
  • 4182e16: Add method 'exists' to core/shm
  • 83dd337: Print out multiple snabb instance pids
  • 32f81df: Move parsing of arguments to parse_args
  • 58691a2: Fetch PID by lwAFTR ID
  • ec6223f: Remove unused variables
  • b69925f: Update lwAFTR query documentation
  • 60aaa9c: Fix counters path
  • 6e5f416: Assert counters were initialized
  • 4132426: Fix initialization of Reassembly/Fragmentation apps
  • b023195: Remove empty counter files
  • ed3d6a1: Rename V4V6 input and output links
  • 6c29222: Import Snabbvmx
  • 4d404dc: Initialize counters and reassembly/fragmentation apps
  • e1e0de4: Add hairpinning queue
  • 29156d0: Add parameter --mirror to Snabbvmx
  • 379b33d: Store lwAFTR ID in shared memory
  • 5ee7afb: Default MTU value 9500 in snabbvmx
  • 4d86ace: Create lwAFTR counters in apps/lwaftr/counters
  • 236cffe: Add documentation for SnabbVMX and fix code issues
  • de87bce: Fix fetch lwAFTR instance by name
  • 0af0f68: Add support of VLAN tagging in on-a-stick mode
  • 41f300f: Pass load_on_a_stick arguments as hash
  • adf5060: Load two virtual NICs if using two different VLAN tags
  • 6b01e0d: Flush ingress-packet-drops to a counter
  • b65a362: Monitor ingress-packet-drops in SnabbVMX
  • 318116e: Use discard attributes to setup ingress-drop-monitor
  • e03efe4: Read ingress-packet-monitor values from SnabbVMX conf file
  • dac8774: Add script tcpreplay
  • 53ba35f: Add Snabbvmx test
  • a2b1fb4: Increase time interval
  • 0f2824e: Clean up tcpreplay.lua script
  • 482a196: Fix vlan assignment
  • 331b40e: Refactor SnabbVMX selftest
  • f23359e: Add VLAN tag testing to SnabbVMX selftest
  • 8bf708f: Add nexthop test datasets and configuration files
  • 9c29680: Add nexthop selftest
  • 24e41f7: Remove unused app 'Repeater'
  • 07426fe: Append .counter suffix if necessary
  • fb2fc13: Import snabbvmx top
  • b2969df: Adapt snabbvmx top
  • e0ce047: Update README
  • 1f62c93: Remove top/util.lua
  • 36cd34b: Select snabb instance per id
  • b0ebb75: Adjust columns width
  • 12a942d: Update README
  • bd5456c: Tidy up top.lua
  • ea2f572: Remove ipv6_address parameter in snabbvmx config file
  • f75c7cb: Add missing file
  • 1b2611b: Fix v4-only and v6-only options
  • 7c695c5: Define LWAFTR_DEBUG env var
  • 6395d10: Centralize packet deallocation in transmit_icmpv4_reply
  • eae3759: Separate test-data from
  • 2cd9a92: Document test data array entry
  • bd29d04: Call flush_haipin if link is PKT_HAIRPINNED
  • 5aec24e: Add test hairpin queue gets flushed
  • aa70cd5: Add VLAN test
  • 43f4c33: Add program soaktest
  • f9f8639: Add soaktest script for CI testing
  • 65804a8: Move soaktest to own folder
  • a971f6d: Refactored lwaftr check on-a-stick code
  • 4d74ca3: Update README
  • 05674f7: Create counters file if it doesn't exist
  • 96cbe55: Check counters file is a table
  • 0a4bb2e: Append path to conf file
  • 0b2afc4: Fix nexthop test
  • cb527ac: Fix update of ingress-packet-drops counter
  • 7fc9533: Allow empty binding table
  • e421d19: Use lwAFTR inet MAC address as ARP source
  • a6c60ae: Move lwaftr/nexthop to snabbvmx/nexthop
  • c211921: Move apps/nh_fwd/nh_fwd.lua to apps/lwaftr/nh_fwd.lua
  • aba846c: fix nexthop test
  • 21bc5bb: lwAFTR check on-a-stick mode was ignored
  • 7c77ac6: Remove skipped test
  • 111a6b4: Extract common functions from lwaftr check
  • f044d9e: Add SnabbVMX check
  • 5518db2: Add SnabbVMX check test case
  • 08ad8ae: Move fatal, file_exists, dir_exists and nic_exists to lwutil
  • f007d1f: Import missing function fatal
  • d100475: Allow to set monitor by id.
  • d92abf4: Fix print error message
  • 2729c6b: Exit snabbvmx top if selected non-lwaftr instance
  • 4f88d34: Fix reporting of incoming ipv6 packets in packetblaster lwaftr mode
  • 0fb961a: Calculate effective VLAN
  • e0424bd: Use intel10g.ring_buffer_size setter
  • b4a383d: Remove tcpreplay script and use packetblaster --no-loop instead
  • c0ee520: Remove old benchmarking tools
  • b759464: Use ethernet:ntop and ipv6:ntop
  • 4c395fc: Define SKIPPED_CODE in nexthop/
  • 3a36e95: Reset IPv4 checksum before recalculating it

Katerina Barone-Adesi (36):

  • 04b3d8f: Added ARP support to the lwaftr.
  • 7cdd3a5: Added documentation of filter performance
  • cb79190: Fix ICMP error encapsulation to hosts behind B4s
  • 8692f21: Remove phantom IPv4 packets while hairpinning
  • 9e525a6: Added random ejection to ctables
  • 0db5e19: Hardened IPv6 reassembly
  • ee00591: Addressed nits
  • e67046b: Normalize the vlan and non-vlan test suite
  • 6aa512f: Addressed nits
  • 6b522ae: Fixed an erroneous comment (bytes -> bits).
  • a25cfd0: Removed unused global
  • 1ae5334: Hardened IPv4 reassembly. All tests pass.
  • eb31f54: Nits, halved default packet reassembly table size
  • 7334ad8: Remove obsolete l2_size approach
  • 16102b3: Fragmentation counters, for IPv4 and IPv6
  • 1bf66c0: Use lwutil.keys
  • 2d25d80: Move the line-oriented counters towards the edges of the app network
  • 65fc13b: Move ctable changes to a wrapper
  • dbc715a: Document 'check -r'
  • e2d0cba: Got rid of packet.clone_to_memory
  • 168e27a: Removed unused parameter
  • 3461a48: Documented fragment counters and troubleshooting
  • 01eb36c: Rename: needsreassembly -> needs-reassembly
  • 9fe6b79: Rename: randomevicted -> random-evicted
  • 4a0047e: Reverted moving the wire-oriented counters to the edges
  • ec0b434: Dashes strike again
  • 3149a39: Update counter path in docs
  • 3ff3f6d: If there is a br address for a bad softwire, use it
  • fceff6a: Make ntohl/htonl unsigned
  • 6f5a2dc: Renamed var, removed comment about it, per review
  • ccb1222: Added a regression test for issue 453
  • af2ed2e: lwutil refactorings
  • 5f74716: Make the core lwAFTR drop fragments
  • 37bc4f0: Add -D: duration option to check
  • 8448ccd: Fix variable name: self->lwaftr
  • b847569: Addressed review comments

Marcel Wiget (17):

  • 3c3c41b: check and use linux interface via rawSocket
  • 96aeaee: add snabbvmx query (to xml)
  • de05632: snabbvmx query to xml
  • aea673f: fixed path to counters_dir in lwcounter
  • a3d0dc4: no more special treatment for lwaftr app required
  • a6cd1ec: remove unused function read_lwaftr_counters
  • 8a4d9d6: remove reference to unused lwcounter
  • d26fa9e: remove unused variables
  • 592f0cc: fix ipv4 checksum in cache refresh packet
  • f984b76: increment only after all packet sizes are sent
  • 1194c25: ok to have no v4/v6 input/output ports in passthru mode
  • 91c5bdc: sanitize xml tags and include links pci and engine stats
  • 2fbf247: use the correct readme
  • bb521e8: use integer for instance id and add name tag for the nic id
  • 5fb1ad4: using :match() for readability and check if name exists
  • b8953d9: set interface mtu based on lwaftr v4/v6 mtu
  • 025fe10: use lwaftr.constants and adjust mtu in case of vlan, cover interface.mtu

Max Rottenkolber (19):

  • 5c4d5dc: snabbnfv: initial YANG model for configuration.
  • adde897: snabb-nfvconfig.yang: s/gigabytes/gigabits.
  • 3909d47: snabb-nfvconfig.yang: Fix port.tunnel.type description.
  • 4b275a4: snabb-nfvconfig.yang: Add initial revision.
  • 1b08428: lib.ipsec.esp: minor formatting/style.
  • f35f4e9: lib.ipsec.esp: refactor and fixes - move all resync logic into resync method - use self.aes_128_gcm.auth_buf for packet regeneration - remember to free temporary packet used in resync - move auditing code into audit method - make auditing configurable as specified in RFC 4303 - only log audits for packets that were actually rejected - update docs and comments
  • dcd1a45: doc/ add missing apps to documentation.
  • 7cc37b4: documentation: minor formatting fixes.
  • 1bdd98d: link to documentation.
  • 16fba64: snabbnfv: do not include gbps unit in QoS config options.
  • 9c4a4d1: Rename snabb-nfvconfig.yang to snabb-nfvconfig-v1.yang (version model).
  • 62d5a6f: apps.ipsec.esp: do not keymat for both streams, accepts two key/salt pairs instead. lib.ipsec: get rid of confusing “keymat” term.
  • 8e1461b: lib.ipsec.esp: automatically pad window_size to implementation req.
  • d1012d4: apps.ipsec.esp: refuse to operate with transmit_salt == receive_salt. See
  • 7598462: …ipsec: fix documentation typos.
  • baabc08: ipsec documentation: use $mdroot, include AES128gcm app docs.
  • 9772042: snabb-nfvconfig-v1.yang: add crypto node.
  • b1b60d2: neutron2snabb: update test fixtures (ignoring the sillyness for now).
  • 1ada400: snabbnfv: support deprecated rx_police_gbps and tx_police_gbps.

Nicola Larosa (30):

  • 4dd9553: Cleanup of the lwaftr app code
  • 2902ff5: Several changes reverted after review
  • dbf5a83: Add loading pf filter definitions from file (#317)
  • 34c401b: Add ingress, egress and hairpin counters, with preliminary testing support
  • 0de9d59: Add documentation for counters, including diagrams and descriptions
  • 6dcac11: Updated text and images
  • a1abc8f: Add drop counters and update all end-to-end tests to check them too
  • ce9a564: Update all end-to-end tests, vlan ones included, so that they check the counters' values too
  • c1d09a5: Moved counters' declaration to a single table, used dashes in all names, no underscores anymore
  • 50a0330: Fix test data filenames, start adding drop counters' tests
  • 8b5260f: Add drop counters' data to the tests, complete counters' renaming
  • 7d39e1d: Implement drop-all counters
  • a78d0c9: Remove TODOs for drop-all counters
  • 1c90f57: Shorten long lines
  • aafa4d9: Remove obsolete tests
  • fd0bfe1: Count hairpinned packets in out-ipv4 too
  • f64f07e: Do not count internally generated ICMPv4 packets in out-ipv4
  • 39cc6e0: Extract the lookup queue from the binding table
  • f78bd20: Add drop counters, with tests
  • fec3c27: Reenable disabled test
  • 92f8886: Add missing drop policy code section, fix one test and add two new ones
  • bbad242: Add docs for the counters
  • ce4edd1: Add a 'snabb lwaftr query' subcommand
  • 92d6f52: Remove conflicts from the counters doc. (#404)
  • cda6ca8: Add Hydra mode and filename support to bench CSV output (#468)
  • 69825fe: Update the 'encapsulate queue to b4' diagram (#474)
  • 7277bec: Make the run, run_nowh and transient lwAftr subcommands output CSV data to a file (#472)
  • 3e4fef1: Add docs for packet reassembly config settings (#478)
  • 0994e88: Remove useless column from benchmark CSV output in Hydra mode (#498)
  • fe28653: Misc. cleanup of the program/lwaftr/ subtree (#501)

Timo Buhrmester (5):

  • ef2b402: lib.ipsec.esp: Anti-replay - resynchronization (RFC 4303 App. A3), more sophisticated approach without duplicating packets/copying data in the normal (non-resync) code paths
  • a6b7714: Eliminate a variable
  • b0ac831: lib.ipsec.esp: Get rid of awkwardly named dual-use variable, slightly improve clarity
  • 38ac409: lib.ipsec.esp: Add another test to verify we do not resync with very old, replayed packets
  • f140d1c: lib.ipsec.esp: Make the resync threshold and depth configurable, add preliminary defaults; document it.


Snabb 2016.10 “Ananas”

@eugeneia eugeneia released this Oct 7, 2016 · 499 commits to master since this release

This release introduces a new and unified way to declare what configuration parameters your apps accept, and verifies app arguments uniformly. If you are a user of apps.vpn, do note that it has been removed in Ananas, and that you should upgrade to use the new and improved implementation in the l2vpn branch. Also noteworthy, packetblaster has gained a new switch for when you just want to transmit a finite burst of packets, instead of transmitting continuously. As usual this release also contains many small bug fixes and improvements.

Major Changes

  • #1008 Removed apps.vpn (obsoleted by the l2vpn program on the l2vpn branch)
  • #1010 Added --no-loop switch to packetplaster replay for emitting finite bursts of packets
  • #1019 Unified the way apps specify their configuration options, and how they are verified

Minor Changes

  • #1014 Optimized Tap app
  • #1018 Replaced built-in assert with a version that synergizes with ljsyscall
  • #1023 #1024 #1027 Fixed bug where htonl performed a signed operation
  • #1005 #1015 #1017 Minor bug fixes and improvements


Alexander Gall (1):

Andy Wingo (3):

  • 8d58c03: Add lib.parameters
  • 823fcb9: Convert rate limiter to use parameters.parse
  • 6d9c784: Add missing include

Diego Pino Garcia (13):

  • a1af15e: Getter/setter ring_buffer_size
  • c304027: Format packetblaster.lua and removed unused variables
  • f3cc41c: Check 'packetblaster replay' arguments should be at least 2
  • fb32e41: Refactor 'packetblaster synth'
  • e58e124: Pass array of options instead of duration
  • 4de74b1: Add switch 'no-loop' to 'packetblaster replay'
  • 7429a78: Initialize loop flag in subprogram
  • 0b035ae: Add --no-loop to 'packetblaster synth'
  • bbdd182: Update 'replay' and 'synth' README
  • 81d1d6a: Remove --no-loop flag in synth
  • c547d90: Properly configure link for Intel82599
  • b73f488: Report LoadGen results in synth.lua
  • 65cdc1c: Cast VLAN tags as int32

Domen Kožar (1):

  • da860a6: manual: support NixOS 16.03 and 16.09 texlive package sets

Katerina Barone-Adesi (2):

  • 83a4216: Make ntohl/htonl unsigned
  • 2194fa5: Added ntohl selftest sanity checks

Max Rottenkolber (15):

  • 598c5d0: core.main: check the return value of fork.
  • eabe7bf: Provide global assert implementation that plays nicely with ljsyscall.
  • 9daa25c: Make use of new assert implementation.
  • 8c5164c: Integrate app arg parsing.
  • 4411ad3: Move lib.assert to core.main, fixing global name space breakage.
  • a3faf0c: set zone. core.main: print zone in handler.
  • 8593f8e: apps.keyed_ipv6_tunnel: fix typo.
  • 14656c8: intel_app: do not call parse_app_arg in reconfig.
  • 0c1f3b4: apps.test.lwaftr: add missing options to config.
  • def857d: Revert " set zone. core.main: print zone in handler."
  • 82dbe04: core.config: print a helpful error message on failure to configure an app.
  • 52a0bd7: core.lib.parse: improve error messages.
  • 53bc9ac: snabbnfv: do not pass bogus parameter to AES128gcm app.
  • 9236f15: apps.pcap.PcapWriter: add back mode parameter.
  • f0a9bb0: Revert "Cast VLAN tags as int32" in favor of #1024, keep test case

Pete Bristow (3):

  • d65e9dc: avoid allocating a packet on every breath
  • f06c88f: Use output rather than out as Joins only output
  • 82eecef: The last fix for allocating on every breath actually broke under load, this one does not


Snabb 2016.09 “Zapote”

@eugeneia eugeneia released this Sep 2, 2016 · 559 commits to master since this release

This release changes the way Snabb apps utilize links to avoid back pressure, and the default ring size was quadrupled. Most apps no longer avoid overflowing their output links, and therefore excessive packets will be dropped at the link level instead of being buffered ad-hoc by apps. The result is that link statistics now reflect actual congestion, and the performance of the iperf benchmark improved slightly. Zapote also introduces anti-replay protection as well as logging of auditable events for lib.ipsec.esp as per RFC 4303, meaning Snabb’s ESP implementation is almost complete. Our vhost_user implementation has seen some major improvements in both performance and stability. A bug on our side of the protocol implementation was fixed that was responsible for crashes with some QEMU versions, and by tweaking how the JIT compiles the vhost_user code the performance of the Snabb-to-DPDK benchmark was improved substantially. Last but not least, this release contains miscellaneous bug fixes and improvements.

Major Changes

  • #950 Links are now used asynchronously throughout Snabb, and the new variable engine.pull_npackets governs how many packets apps pull into the app network on each breath
  • #988 lib.ipsec.esp now implements anti-replay protection and logs “auditable” events
  • #1001 The vhost_user implementation was optimized, and its performance improved substantially

Minor Changes

  • #984 Fixed a bug in our vhost_user implementation that caused crashes with some QEMU versions
  • #990 Added exists method to core.shm to probe for SHM objects
  • #991 snabb top now lists the PIDs of Snabb instances if multiple are detected and no target is specified
  • #985 #986 #996 Minor bug fixes and improvements
  • #966 Various SnabbBot/CI improvements


Diego Pino (3):

  • 0b62e29: Support private repos
  • 1e59dc7: Remove temporary directories if they exist
  • f8443f5: Append CLIENT_ID and CLIENT_SECRET to urls

Diego Pino Garcia (4):

  • 3a254ac: Revert "Remove temporary directories if they exist"
  • 64491b9: Skip SnabbNFV test if cannot find QEMU image
  • 3ea2348: Add method 'exists' to core/shm
  • bc8c2ab: Print out multiple snabb instance pids

Katerina Barone-Adesi (1):

  • bb9b3c2: Packet.lua: add clone_to_memory

Luke Gorrie (5):

  • d77e43d: vhost_user: Enable/disable vring processing on kick/get_base
  • 2a98aa5: snabbnfv: Update filter.ports config to allow iperf
  • d4b71e6: net_device: Avoid unnecessary checksum calculation
  • 7f1d205: virtio net_device: JIT each connection separately
  • 54ac86d: Revert "Packet.lua: add clone_to_memory"

Max Rottenkolber (19):

  • 0f654f7: snabb_bot: Dirty fix for GitHub API pagination to get runway.
  • 8ae18fa: snabb_doc: same as previous commit but for SnabbDoc.
  • 1dfa664: lib.ipsec.esp: Test minimum packet sizes.
  • 54e7b90: remove link.full, increase link buffer size, add engine.pull_npackets.
  • a1f60a2: Replace usage of link.full/nwritable with engine.pull_npackets.
  • cca676e: scripts/ push via https.
  • 325ddca: lib.ipsec.seq_no_t: remove unused `full' method.
  • b4a7968: lib.ipsec.esp: refactor track_seq_no.
  • 13a12e9: snabbmark esp: work around anti-replay window.
  • b6bed30: snabb_bot: fix bug where bogus performance regressions where reported.
  • 55b1572: Fix-up Lwaftrgen:pull and basic_apps.Tee.
  • c3c3413: must be a power of two.
  • f52861f: Revert "remove link.full", document link.nwritable, link.nreadable.
  • 0d40aad: core.shm.exists: document return value type.
  • b11191a: apps.virtio_net: Fix README to correctly reflect role of VirtioNet app.
  • 0214574: core.packet: make packet_t public accoding to documentation.
  • ee57dae: core.packet: document clone_to_memory.
  • 1d2b27e: Revert "core.packet: make packet_t public accoding to documentation."
  • cc829d2: core.packet: remove mention of packet_t in favor of “struct packet”.

Timo Buhrmester (7):

  • 4c311c9: lib.ipsec.esp: Anti-replay (RFC 4303 App. A)
  • 363b1bf: Refactoring broke advance_window; fix that
  • 72532f6: lib.ipsec.esp: More thourough testing of the anti-replay facilities
  • 29de34c: lib.ipsec.esp: Reinsert a few comments that got refactored out
  • 54899c0: lib.ipsec.esp: Use core.lib.logger for the auditable events "integrity error" and "replayed"
  • c37e01b: lib.ipsec.esp: One logger per module rather than per decryption context
  • 2e75c11: lib.ipsec.esp: Fix typo


  • Aug 24, 2016


    Augment stats counters for apps.tap.tap
    Add counters for interface type, MAC address, MTU and operational
    status to the apps.tap.tap app.  Also accept the MTU as a
    configuration parameter to check whether the MTU configured on
    the Snabb side matches the one configured on the TAP device to
    avoid MTU mismatches.
  • Aug 22, 2016


    Version 4
    Fetch physical MAC address from interface
      This obsoletes the <mac> option for
      L3 interfaces
    Generalize interface SNMP configuration
      The interface-specific SNMP code has been
      factored out of the drivers and is now
      a global configuration option
    Add SNMP support for sub-interfaces
  • Aug 10, 2016


    Version 3
    Add VLAN support for interfaces
      Enable trunking mode
      Add VLAN-based sub-interfaces
    Add VLAN support for l2vpn
      Let uplink/ACs reference sub-interfaces
    Add static ND app
      An ND app is present on a L3 port in all cases
    Sort out ifDescr/ifName/ifAlias in SNMP

Snabb 2016.08 “Yolanda”

@eugeneia eugeneia released this Aug 3, 2016 · 629 commits to master since this release

This release adds a handy utility to bind and unbind PCI devices to and from Linux, overhauls Snabb’s shared memory subsystem, and adds software I/O mode to SnabbNFV. The deprecated “data” and “length” functions from core.packet were removed. Statistics counters in the vhost_user app were removed as well, because they caused a performance regression for which not solution was found yet. As usual, this release also contains many bug fixes and minor improvements.

Major Changes

  • #971 Added new utility snabb pci_bind to help with PCI device management
  • #969 Added software I/O mode to SnabbNFV allowing it to be run without dedicated network hardware
  • #972 Overhauled shared memory subsystem

Minor Changes

  • #967 Removed deprecated and packet.length
  • #968 Fixed bug in apps.bridge.blooding
  • #975 Skip Intel1g selftest if required environment variable is empty
  • #979 Added SNABB_SHM_ROOT and SNABB_SHM_KEEP environment variables
  • #983 Reverted statistics for apps.vhost.vhost_user because of performance regression


Diego Pino Garcia (2):

  • 00f6ff6: Fix intel1g selftest
  • bf09217: Replace os.getenv for lib.getenv

Katerina Barone-Adesi (1):

  • dcaf4f9: pci_bind: A utility to bind/unbind PCI cards

Luke Gorrie (2):

  • 78c6764: core.shm: $SNABB_SHM_ROOT overrides /var/run/snabb
  • 5c317aa: main: $SNABB_SHM_KEEP prevents removal of shm dir

Max Rottenkolber (10):

  • a95751f: core.packet: Remove data/length reader functions.
  • 0be75ab: SnabbNFV: Add support for virtual/software I/O mode.
  • 38cf53c: SnabbNFV test reorganization and parameterization.
  • f6ef311: SnabbNFV: make virtio mrg_rxbuf option configurable.
  • 0814fd9: apps.bridge.flooding: Fix regression introduced by 4b4b8ad / #638.
  • 2552468: SnabbNFV: add port option no_indirect_desc.
  • 28a3de1: statistics: introduce shm frame abstraction to eliminate boilerplate code...
  • 70ad231: snabb top: remove obsolete unlinking of runtime files (supervisor).
  • a37441f: snabbnfv: Added symlink to for Hydra compatibility.
  • 77c0d57: vhost_user: revert statistics.


Snabb 2016.07 “Xigua”

@eugeneia eugeneia released this Jul 13, 2016 · 659 commits to master since this release

This release introduces a new link callback for apps to configure themselves after their links are set up, and a new supervisor process is started alongside the usual Snabb process to clean up after it. SnabbNFV got support for encrypted tunnels, and per-app shared memory counters for statistics and diagnostics were added to many apps. The “length” and “data” functions from core.packet were deprecated in favor of exposing the packet ctype and its ABI. As usual, this release also contains many bug fixes and minor improvements.

Major Changes

  • #905 Introduced link callback for apps
  • #930 Added supervisor process to manage Snabb worker process
  • #949 Added ESP (encrypted tunnel) support to SnabbNFV
  • #931 #947 #953 #963 Added shared memory statistics counters for Apps

Minor Changes


Alexander Gall (1):

Andy Wingo (1):

  • 995f9d8: vhost-user: Support operation without MRG_RXBUF.

Diego Pino (1):

  • c86ec3e: Call app:start only if defined

Diego Pino Garcia (4):

  • c3c69d2: Add 'start' method to apps
  • 9dabdd2: Call app.start instead of app_table[name].start
  • 0a5bfac: Remove unassigned initialization
  • 80a7ee7: Trigger start event for each app

Domen Kožar (8):

  • d0cc3c6: default.nix: enable parallel builds
  • 738ef31: Snabb version is now statically stored in .version
  • a119aeb: release.nix: don't rely on SNABB_PCI* defaults
  • 1feb2c9: release.nix: by default use lugano servers
  • 307b1e1: default.nix: remove .git from source since it's not deterministic
  • 1d4e766: src/Makefile: silence grep
  • 694b1c4: output kernel console to stdout
  • 1b60645: default.nix: don't hardcode the version

Luke Gorrie (13):

  • b34c3ee: engine: Set shm path to "app/$name"
  • 80614e1: doc/ Rewritten based on new experience
  • 12d0009: doc/ Added section on upstreaming subsystems
  • af8ae46: doc/ Partly rewritten draft
  • eccb614: doc/ Created
  • 113b607: Rewrote section on becoming a maintainer
  • 4a37b98: Wrote about upstreaming subsystem branches
  • 6c46b5d: doc: Remove stale .images/Branches.png file
  • 4eb21b6: virtio/net_device: Fix bug with mergeable rx buffers
  • e5f23a9: vhost_user: Fix & comment "feature cache"
  • b20cb17: core.memory: Add 'align' argument to dma_alloc()
  • 398dd16: core.main: Fix bug in SNABB_DEBUG env var handling
  • e81a192: core.timer: Inhibit debug messages

Max Rottenkolber (73):

  • 94ff234: Amendments to #766: - Use "apps/" instead of "app/" for uniformity - Set shm path to "apps/$name" when calling app:stop' too - Unlink "apps/$name" afterapp:stop' using `shm.unlink' - Add a test case to selftest
  • fad0f43: core.counter: Qualify counter names using `shm.resolve'.
  • 7ed4ed0: snabb top: add `--app' option to print app counters.
  • eb9005b: snabb top: unlink own shm tree to avoid clutter.
  • 5fbe0d6: vhost_user: Add RFC 7223 app counters.
  • 8bb3215: Intel_app: Add RFC 7223 app counters.
  • 7a55478: snabb top: Add --link parameter to list link counters.
  • dde5da2: Put app counters under "counters/", update snabb top.
  • 924ff4e: lib.json: Import JSON4Lua 1.0.0, include encode functionality.
  • 8e34093: lib.macaddress: Support numeric initialization; add method to get numeric representation.
  • 5f9efd2: Create “discontinuity-time” counters.
  • 7b39148: snabb top: add `--yang' option to print YANG model as JSON.
  • e457300: Update
  • 8984741: snabb top --yang: Represent uint64_t as decimal string.
  • ee00d16: [core.lib] Generalize `timer' to optionally accept 'repeating' option and support injecting a function to determine the current time.
  • 45490b8: Revert "Intel_app: Add RFC 7223 app counters."
  • f0ed10b: intel_app: expose per-pciaddress statistics in `counters/'.
  • b853268: core.packet: document ctype and deprecate data and length functions.
  • b11e375: core.packet: properly document shiftleft and shiftight.
  • c186591: lib.protocol.ethernet: Add n_mcast, branch-free Multicast predicate.
  • b09e843: Fix for f0ed10b: require macaddress module.
  • 62e2fbf: Revert "lib.json: Import JSON4Lua 1.0.0, include encode functionality."
  • f4834a5: snabb top: revert --yang dump.
  • 793c394: snabbify statistics model.
  • 2b9ef2c: intel_app: add ifTable MIB related counters.
  • e9f5dda: lib.macaddress: add bytes method.
  • 6b12e9c: spin out SNMP code from to lib.ipc.shmem.iftable_mib.
  • 7c697b6: main: fork into worker and supervisor; perform clean up.
  • be09fe5: main: send HUP instead of KILL.
  • 3db174b: main: return correct exit code.
  • ce2f220: main: supervise xpcall(main, hanlder).
  • 4950cbb: main: check status of S.waitpid.
  • 28c568c: main: cleanup after worker_pid.
  • 4af7091: Remove obsolete `snabb gc'.
  • 1c9f585: Move model.txt to src/doc/
  • 339f8f1: apps.intel_app: Revert rxerrors calculation, remove broken txerrors.
  • 7ec7f80: Revert "lib.macaddress: add bytes method." and remove int method.
  • 327804f: doc/ remove duplicates.
  • 90fc893: fix bug in #766 introduced while merging 6cac870.
  • ceb080f: fix bug in #766, use app.shmpath during reconfig.
  • d82729a: fix typo in previous commit.
  • 70b8967: intel1g: canonicalize env var name, document it in, pass it through
  • 9f48b14: use pci.map_pci_memory_locked.
  • ed687f1: main: properly handle unsupported worker states (SIGSTOP/SIGCONT)-
  • db9e343: main: comment syscall usage.
  • c3b8672: Revert 'ed687f19f8459413145fabc56a462d7b38513ba8'.
  • 5eb0f69: core.main: handle SIGSTOP and SIGCONT to worker.
  • 76a5d6e: main: allow setting developer_debug/debug_on_error through environment.
  • 127d21f: main: do not unlink runtime directory in developer_debug mode.
  • 1befce4: remove packet.length/data.
  • 751cc28: apps.bridge.base: use `app:start()' to run post configuration.
  • 11890e6: edit documentation of app:start().
  • 5488c68: lib.protocol.ethernet: add is_bcast and n_bcast.
  • 9044c6f: apps.vhost.vhost_user: count rxbcast/txbcast.
  • 5a01612: apps.tap.tap: add statistics counters.
  • 6233d15: core.packet: make max_payload public.
  • 788438e: apps.socket.raw: add statistics counters.
  • 7d5b973: Rename app:start from #905 to app:configure.
  • 4dec991: scripts/ also pass through SNABB_PCI_INTEL1G1.
  • a9f7011: lib.ipsec.esp: fix regression introduces by previous commit.
  • e60226c: Rename app:configure to app:link.
  • 0f38505: lib.ipsec.esp: Test minimum packet sizes.
  • b8cbd65: SnabbNFV: Add “crypto” tunnel option using lib.ipsec.esp.
  • ab5d1af: scripts/ use eugeneia/snabb-nfv-test-vanilla by default.
  • 626c6fc: apps.keyed_ipv6_tunnel: add diagnostics counters.
  • c0f7076: apps.ipsec.esp: add statistics counters.
  • fc7363c: apps.ipv6.nd_light: add statistics counters.
  • 272a0d1: apps.packet_filter: add statistics counters.
  • 506c66b: apps.rate_limiter: add statistics counters.
  • e3fcbbf: Revert "lib.protocol.ethernet: Add n_mcast, branch-free Multicast predicate." Reason: its actually slower than the initial naive version.
  • b418f8f: intel_app: fix wrong “speed” counter value.
  • d52d89c: apps.ipv6.nd_light: revise counters, south = rx / north = tx.
  • 905bf8d: vhost/vhost_user: avoid callbacks.

Pete Bristow (2):

  • 7a6296f: Add a test demonstrating pci device ids, with capital A-F
  • 5d6baa4: remove superfluous syscall.sysctl