Extend sharkd to load data from URLs

[lmangani]

Currently sharkd only supports loading from files it can directly access through function sharkd_session_process_load in turn calling load_cap_file

In order to support a broader scope of scenarios, it should be extended to support loading data from web APIs, ad-hoc interfaces and/or through piping other system commands.

This issue is just an intention pointer to track design elements and input from other devs.

[lmangani]

rebooting attempt!

A potential lightweight route could be adding URL path detection (^http.*) in filenames passed to the cf_open function by the application, and using a sidekick curl function to fetch the file before passing it to the load_cap_file function again.

• load_cap_file( 'http://localhost/some.pcap', ... )
• cf_open detect ^http, call curl sidekick
• curl fetch and store file (http://localhost/some.pcap > /capture/some,pcap)
• cf_open('/capture/some.pcap')

#include <stdio.h>
#include <curl/curl.h>
#include <curl/types.h>
#include <curl/easy.h>
#include <string>

size_t write_data(void *ptr, size_t size, size_t nmemb, FILE *stream) {
    size_t written = fwrite(ptr, size, nmemb, stream);
    return written;
}

int main(void) {
    CURL *curl;
    FILE *fp;
    CURLcode res;
    char *url = "http://localhost/some.pcap";
    char outfilename[FILENAME_MAX] = "/capture/some.pcap";
    curl = curl_easy_init();
    if (curl) {
        fp = fopen(outfilename,"wb");
        curl_easy_setopt(curl, CURLOPT_URL, "http://localhost/some.pcap");
        curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
        curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);
        res = curl_easy_perform(curl);
        /* always cleanup */
        curl_easy_cleanup(curl);
        fclose(fp);
    }
    return 0;
}

[kYroL01]

This is the standalone test to be converted in a function and called to the correct position in sharkd
https://gist.github.com/kYroL01/b92fa0d7664b92a17575227239c17076

./curl_pcap "http://www.example/file.pcap" -> download the pcap

[lmangani]

Completed in JS for now to simplify the asynchronous aspect. Thanks @kYroL01 feel free to think about this long range :)