Filter for packetcapture service (#273)

Added tcpdump-like filtering for packetcapture service

.gitignore

@@ -8,3 +8,5 @@ swagger-codegen*
 cmake-build-debug/*
 Vagrantfile
 libyang/*
+
+cmake-build-debug/

AUTHORS

@@ -12,6 +12,7 @@ off on commits in the Polycube repository:
   Mauricio Vásquez Bernal                 mauriciovasquezbernal@gmail.com
   Nico Caprioli                           nico.caprioli@gmail.com
   Riccardo Marchi                         riccardo.marchi4@gmail.com
+  Riccardo Rusca                          riccardo.rusca11@gmail.com
   Sebastiano Miano                        mianosebastiano@gmail.com
   Simone Magnani                          simonemagnani.96@gmail.com
   Yunsong Lu                              roamer@yunsong.lu

Documentation/installation.rst

@@ -99,7 +99,7 @@ Install dependencies
 
     # Install polycube dependencies
     sudo apt-get -y install git build-essential cmake bison flex \
-        libelf-dev libllvm5.0 llvm-5.0-dev libclang-5.0-dev \
+        libelf-dev libllvm5.0 llvm-5.0-dev libclang-5.0-dev libpcap-dev \
         libnl-route-3-dev libnl-genl-3-dev uuid-dev pkg-config \
         autoconf libtool m4 automake libssl-dev kmod jq bash-completion \
         gnupg2

Documentation/polycubectl/polycubectl.rst

@@ -41,6 +41,7 @@ Refer to :doc:`Quick Start <../quickstart>`.
         pbforwarder       service   Policy-Based Forwarder Service
         bridge            service   Bridge Service
         nat               service   NAT Service
+        packetcapture     service   Packetcapture Service
 
         connect           command   Connect ports
         disconnect        command   Disconnect ports

Documentation/services/pcn-packetcapture/packetcapture.rst

@@ -1,22 +1,24 @@
 Packetcapture service
 =====================
 
-Packetcapture is a transparent service that allows to capture packets flowing through the interface it is attached to, apply (simple) filters and obtain capture in *.pcap* format. In particular, the service supports either saving captured packets in the local filesystem (e.g., useful in case of high network traffic) or it can interact and deliver packets to a remote client that stores them in the remote filesystem.
+Packetcapture is a transparent service that allows to capture packets flowing through the interface it is attached to, apply filters and obtain capture in *.pcap* format. In particular, the service supports either saving captured packets in the local filesystem (e.g., useful in case of high network traffic) or it can interact and deliver packets to a remote client that stores them in the remote filesystem.
 
 An example of a client that uses the REST api of the packetcapture service is available in '*Packetcapture_Client*' directory.
 
 Features
 --------
+
 - Transparent service, can be attached to any interface of any Polycube service
-- Support for (simple) IPv4 filters: source prefix, destination prefix, source port, destination port and layer 4 protocol.
+- Support for filters (i.e., source prefix, destination prefix, source port, destination port, layer 4 protocol, etc.).
 - Support partial capture of packets (i.e., snaplen)
 - Support localmode (store data locally) or network mode (send packets to a remote client) operations
 
 Limitations
 -----------
-- No IPv6 filtering
+
 - Traffic is returned as is, without any anonimization primitive.
 
+
 How to use
 ----------
 The packetcapture service is a transparent service, it can be attached to a cube port.
@@ -37,13 +39,14 @@ This service can operate in four working modes (actually, the forth mode is just
 - turn packet capture off: **capture=off**
 
 *capture* option indicates the direction of the packets that the service must capture.
-The direction of the captured packets is independent of the operation in "nework mode" or "non network mode".
+The direction of the captured packets is independent of the operation in "network mode" or "non network mode".
 
 In this example the service named '*mysniffer*' will work in bidirectional mode.
 
 
 Attach to a cube port
 ^^^^^^^^^^^^^^^^^^^^^
+
 ::
 
     # Attach the service to a cube port
@@ -56,70 +59,76 @@ Now the packetcapture service is attached to the port *toveth1* of the bridge *b
 
 
 
-Filters
+Filter
 -------
-Traffic can be selected by means of the following filters:
+Traffic can be selected by adding filters with syntax (tcpdump like):
+
+    polycubectl <service name> set filter=<string value>
+
+-if the filter contains only one word you can put it normally
+-if the filter contains more than a word you have to put the string inside the ""
+
+-if you want to capture all the traffic you can put as filter: all
+
+    polycubectl <service name> set filter=all
+
+- default filter captures no packets (the eBPF datapath simply returns ok)
+
+Filter can be viewed using the command **polycubectl mysniffer filter show**
+Snaplen can be viewed using the command **polycubectl mysniffer snaplen show**
+
+For further details of the implementation of the filter see :doc:`Packetcapture filter <packetcapture-filter>`
 
-- source prefix
-- destination prefix
-- source port
-- destination port
-- later 4 protocol
+For more details about the filters supported by libpcap (hence, the syntax allowed to specify filters) see `pcap-filter <https://linux.die.net/man/7/pcap-filter>`__
+
+
+Examples of possible filters
+----------------------------
 
-Source prefix filter
-^^^^^^^^^^^^^^^^^^^^
 ::
 
     # Example of the source prefix filter
-    polycubectl mysniffer filters set src=10.10.10.10/24
+    polycubectl mysniffer set filter="ip src 10.0.2.11"
 
-Destination prefix filter
-^^^^^^^^^^^^^^^^^^^^^^^^^
 ::
-    
-    # Example of the destination prefix filter
-    polycubectl mysniffer filters set dst=10.10.10.10/24
 
-Source port filter
-^^^^^^^^^^^^^^^^^^
-::
-    
     # Example of the source port filter
-    polycubectl mysniffer filters set sport=80
+    polycubectl mysniffer set filter="src port 80"
 
-Destination port filter
-^^^^^^^^^^^^^^^^^^^^^^^
-::
-    
-    # Example of the destination port filter
-    polycubectl mysniffer filters set dport=80
 
-Layer 4 protocol filter
-^^^^^^^^^^^^^^^^^^^^^^^
 ::
     
     # Example of the layer 4 protocol filter
-    polycubectl mysniffer filters set l4proto=tcp
+    polycubectl mysniffer set filter=tcp
 
-Snaplen filter
-^^^^^^^^^^^^^^
 ::
     
     # Example of the snaplen filter
-    #  In this case we capture only the first 80 bytes of each packet
-    polycubectl mysniffer filters set snaplen=80
+    # In this case we capture only the first 80 bytes of each packet
+    polycubectl mysniffer set snaplen=80
 
 
-Filters can be viewed using the command **polycubectl mysniffer filters show**
-
 Get the capture dump
 --------------------
-When the service is not set in *networkmode*, the dump is automatically written in a resilient way in the temporary user folder.
+When the service is not set in *networkmode*, the dump is by default written in a resilient way in the temporary user folder.
+The folder where the dump is written can be changed by using the syntax:
+
+    polycubectl <service name> set dump="<string value>"
+
+::
+
+    # Example of new dump folder
+    polycubectl mysniffer set dump="/home/user_name/Desktop/capture"
+
+At the end of the file name will be added the file extension ".pcap"
+
+If a file with the same name already exists it will be overwritten otherwise it will be created
 
 The path of the capture file can be shown using the command: **polycubectl mysniffer show dump**
 
 Otherwise, if the service is set in network mode, the capture file can be requested through the use of the provided Python client, or queried simply through the service API.
 
+
 How to use the demo client
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
 ::
@@ -137,3 +146,51 @@ Set network mode
 
     # Start sniffer in local model
     polycubectl mysniffer set networkmode=false
+
+
+Implementation details
+----------------------
+The pipeline to convert into C code the filtering string entered in the packetcapture service is the following:
+
+**pcap filter** → *libpcap* → **cBPF** → *cbpf2c* → **C code**
+
+More in details, the first step is to obtain the cBPF (assembly) code from the filtering string, using the ``libpcap``/``tcpdump`` format. The filtering string is read from ``polycubed`` REST interface, then it is compiled in cBPF using the ``pcap_compile_nopcap()`` function that returns a ``bpf_program`` structure containing a list of ``bpf_insn``.
+
+Then, the code creates a ``sock_fprog`` structure called ``cbpf`` that contains all the required filter blocks.
+
+The second step (traslation from cBPF to C) starts with the validation of the cBPF code.
+Function ``_cbpf_dump()`` is called for each filtering block and it returns a string containing the equivalent C code for that block.
+
+Inside ``_cbpf_dump()``, a switch statement creates two variables, ``op`` (operation) and ``fmt`` (operand) depending on the type of instruction of the block (e.g.,return, load, store, alu op. etc.); the above variables will be used to generate the final C code.
+
+This ASM-to-C traslator is ispired to a similar project proposed by `Cloudflare <https://blog.cloudflare.com/xdpcap/>`_; however, in Polycube the translator is written in C/C++ (the CLoudfare one is in Go); furthermore, in Polycube the final output of the translator is a C equivalent of the packet filter, while in the latest version of the Cloudfare project, the final outcome of the translation are eBPF assembly instructions.
+
+The C output facilitates any further modification of the code, e.g., with when additional processing steps are needed, although it impacts on the overall filter conversion time as it requires one additional processing pass involving CLANG/LLVM to convert the C code into eBPF assembly.
+
+
+Example of C code generated
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+As a example, we list here is the generated C code for the filter ``icmp``:
+
+::
+
+    L0:	 if ((data + 14) > data_end) {
+           return RX_DROP;
+         }
+         a = ntohs(* ((uint16_t *) &data[12]));
+    L1:	 if (a == 0x0800) {
+           goto L2;
+         } else {
+           goto L5;
+         }
+    L2:	 if ((data + 24) > data_end) {
+           return RX_DROP;
+         }
+         a = * ((uint8_t *) &data[23]);
+    L3:	 if (a == 0x01) {
+           goto L4;
+         } else {
+           goto L5;
+         }
+    L4:	 return pcn_pkt_controller(ctx, md, reason);
+    L5:	 return RX_OK;

scripts/pre-requirements.sh

@@ -30,6 +30,8 @@ PACKAGES+=" sudo" # needed for pcn-iptables, when building docker image
 PACKAGES+=" kmod" # needed for pcn-iptables, when using lsmod to unload conntrack if not needed
 PACKAGES+=" jq bash-completion" # needed for polycubectl bash autocompletion
 PACKAGES+=" libpcre3-dev" # needed for libyang
+PACKAGES+=" libpcap-dev" # needed for packetcapture filter
+
 
 if [ "$MODE" == "pcn-k8s" ]; then
   PACKAGES+=" curl" # needed for pcn-k8s to download a binary

src/services/pcn-packetcapture/datamodel/packetcapture.yang

@@ -30,7 +30,7 @@ module packetcapture {
     description "Packet capture status";
   }
 
-  leaf anomimize {
+  leaf anonimize {
      type boolean;
      default false;
   }
@@ -43,40 +43,18 @@ module packetcapture {
   leaf networkmode {
     type boolean;
     default false;
-    description "operative mode";
+    description "Operating mode";
   }
 
-  container filters {
-    leaf snaplen {
-      type uint32;
-      default 262144;
-      description "Snapshot length";
-    }
-
-    leaf src {
-      type inet:ipv4-prefix;
-      description "IP source filter";
-    }
-
-    leaf dst {
-      type inet:ipv4-prefix;
-      description "IP destination filter";
-    }
-
-    leaf l4proto {
-      type string;
-      description "Level 4 protocol filter";
-    }
-
-    leaf sport {
-      type uint16;
-      description "Source port filter";
-    }
-
-    leaf dport {
-      type uint16;
-      description "Destination port filter";
-    }    
+  leaf snaplen {
+    type uint32;
+    default 262144;
+    description "Snapshot length";
+  }
+
+  leaf filter {
+    type string;
+    description "filtering string (e.g., 'host 1.2.3.4 and src port 80')";
   }
 
   container globalheader {

src/services/pcn-packetcapture/src/CMakeLists.txt

@@ -6,6 +6,9 @@ aux_source_directory(base BASE_SOURCES)
 
 include_directories(serializer)
 
+find_library(PCAP_LIBRARY
+        NAMES pcap)
+
 if (NOT DEFINED POLYCUBE_STANDALONE_SERVICE OR POLYCUBE_STANDALONE_SERVICE)
   find_package(PkgConfig REQUIRED)
   pkg_check_modules(POLYCUBE libpolycube)
@@ -19,11 +22,11 @@ add_library(pcn-packetcapture SHARED
   ${SERIALIZER_SOURCES}
   ${API_SOURCES}
   ${BASE_SOURCES}
-  Filters.cpp
   Globalheader.cpp
   Packet.cpp
   Packetcapture.cpp
-  Packetcapture-lib.cpp)
+  Packetcapture-lib.cpp
+  cbpf2c.cpp)
 
 # load ebpf datapath code a variable
 load_file_as_variable(pcn-packetcapture
@@ -39,10 +42,12 @@ load_file_as_variable(pcn-packetcapture
   packetcapture_datamodel)
 
 target_link_libraries(pcn-packetcapture ${POLYCUBE_LIBRARIES})
+target_link_libraries(pcn-packetcapture ${PCAP_LIBRARY})
 
 # Specify shared library install directory
 
 set(CMAKE_INSTALL_LIBDIR /usr/lib)
+set(CMAKE_REQUIRED_LIBRARIES ${PCAP_LIBRARY})
 
 install(
     TARGETS