Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

ctl-results

Results of Cyber Test Lab analysis

Caveat hackor

We've now posted a complete analysis run of the Fedora 27 packages. The CTL code is beta.

We have the two following convictions about our results:

  1. They are wrong
  2. They will get better

This will be an iterative process. We're still immature, but this project is being actively improved.

Data points collected

The Cyber Test Lab collects the following data for each ELF binary:

  • ASLR enabled or disabled (position-independant code / executable)
  • Read-only relocations (RELRO) enabled or disabled
  • Stack protected or not
  • Presence or absence of source fortified functions
  • Immediate binding or lazy resolution
  • Cyclomatic complexity
  • Cycle cost

Fedora 27 CTL scoring

Here's a graph of the scored Fedora 27 packages, and the raw data that was generated by cyber-test-lab.

scores by package

We're still dialing in our scoring technique. Expect this to change a lot in the coming days.

Fedora 27 cyclomatic complexity analysis

Here's a deeper sample of some of the data collected by this project.

afcc

This graph shows the cyclomatic complexity values for all Fedora 27 binaries (for which the CTL code cooperated, we have a bug there). It was generated with this gist. The histogram represents a probability density function indiciating the likelihood that a complexity value will fall in that range. We see that most Fedora 27 binaries have a cyclomatic complexity of less than 100, with a mean of about 35. Some binaries are much more complex, the highest in this dataset being 780. Note that we currently time out r2 after 10 minutes of trying to analyze the binary (aa command). As a result, there are many cyclomatic complexity values higher than 780 in the Fedora 27 repositories. We're tracking a bug for that too.

So what does this mean? We won't know for sure until we graduate from this static analysis to dynamic analysis in the form of fuzzing. (For an introduction to fuzzing, the Sully manual is excellent.) The theory is that more complex binaries will be more likely to crash while being fuzzed. Crashes can then be analyzed to find new 0-day vulnerabilities. So a smaller cyclomatic complexity value is better, and hopefully more "secure."

Check back for udates and more analysis.

Sample results data

When the tool works right, here's a sample result, taken from coreutils.

{
    "results": {
    "_comment": "truncated",
        "usr/bin/ls": {
            "find-libc-functions": [
                "GI___obstack_vprintf",
                "GI___vasprintf",
                "GI___vdprintf",
                "GI___vfprintf",
                "GI___vfwprintf",
                "GI___vsnprintf",
                "GI___vsprintf",
                "GI___vswprintf",
                "GI___vsyslog",
                "__longjmp",
                "_fprintf",
                "_printf",
                "_snprintf",
                "_sprintf",
                "_vfprintf",
                "_vprintf",
                "_vsnprintf",
                "_vsprintf",
                "asprintf",
                "confstr",
                "dprintf",
                "explicit_bzero",
                "fdelt",
                "fgets",
                "fgets_unlocked",
                "fgetws",
                "fgetws_unlocked",
                "fprintf",
                "fread",
                "fread_unlocked",
                "fwprintf",
                "getcwd",
                "getdomainname",
                "getgroups",
                "gethostname",
                "getlogin_r",
                "gets",
                "getwd",
                "longjmp",
                "mbsnrtowcs",
                "mbsrtowcs",
                "mbstowcs",
                "memcpy",
                "memmove",
                "mempcpy",
                "memset",
                "obstack_printf",
                "obstack_vprintf",
                "poll",
                "ppoll",
                "pread64",
                "pread",
                "printf",
                "ptsname_r",
                "read",
                "readlink",
                "readlinkat",
                "realpath",
                "recv",
                "recvfrom",
                "snprintf",
                "sprintf",
                "stpcpy",
                "stpncpy",
                "strcat",
                "strcpy",
                "strncat",
                "strncpy",
                "swprintf",
                "syslog",
                "ttyname_r",
                "vasprintf",
                "vdprintf",
                "vfprintf",
                "vfwprintf",
                "vprintf",
                "vsnprintf",
                "vsprintf",
                "vswprintf",
                "vsyslog",
                "vwprintf",
                "wcpcpy",
                "wcpncpy",
                "wcrtomb",
                "wcscat",
                "wcscpy",
                "wcsncat",
                "wcsncpy",
                "wcsnrtombs",
                "wcsrtombs",
                "wcstombs",
                "wctomb",
                "wmemcpy",
                "wmemmove",
                "wmempcpy",
                "wmemset",
                "wprintf"
            ],
            "filename": "usr/bin/ls",
            "complexity": {
                "r2aa": {
                    "afCc": -1,
                    "_comment": "this appears to be a bug, tracked at https://github.com/redteam-project/cyber-test-lab/issues/12",
                    "afC": 69
                }
            },
            "hardening-check": {
                " Read-only relocations": "yes",
                " Position Independent Executable": "yes",
                " Stack protected": "yes",
                " Fortify Source functions": "yes (some protected functions found)",
                " Immediate binding": "yes"
            },
            "report-functions": [
                "__xstat",
                "__uflow",
                "fdopen",
                "__fprintf_chk",
                "gettimeofday",
                "getgrgid",
                "calloc",
                "fscanf",
                "__freading",
                "strtoul",
                "snprintf",
                "timegm",
                "exit",
                "sigaction",
                "__fxstat",
                "strcpy",
                "iswcntrl",
                "__cxa_atexit",
                "fwrite",
                "cap_get_file",
                "strcoll",
                "__ctype_get_mb_cur_max",
                "getpwuid",
                "fputs_unlocked",
                "wcwidth",
                "ioctl",
                "_ITM_registerTMCloneTable",
                "__libc_start_main",
                "__overflow",
                "free",
                "mbsinit",
                "cap_to_text",
                "stpncpy",
                "strcmp",
                "__assert_fail",
                "fclose",
                "sigprocmask",
                "setenv",
                "fwrite_unlocked",
                "close",
                "strncmp",
                "localeconv",
                "opendir",
                "open",
                "fileno",
                "__cxa_finalize",
                "getxattr",
                "wcstombs",
                "isatty",
                "iswprint",
                "getenv",
                "strchr",
                "__ctype_toupper_loc",
                "strrchr",
                "memset",
                "memcpy",
                "closedir",
                "__snprintf_chk",
                "getfilecon",
                "malloc",
                "freecon",
                "sigismember",
                "__stack_chk_fail",
                "__errno_location",
                "__gmon_start__",
                "readlink",
                "tcgetpgrp",
                "_ITM_deregisterTMCloneTable",
                "nl_langinfo",
                "fflush",
                "cap_free",
                "ungetc",
                "gmtime_r",
                "lseek",
                "memcmp",
                "setlocale",
                "mbrtowc",
                "fnmatch",
                "__sprintf_chk",
                "textdomain",
                "wcswidth",
                "_setjmp",
                "strftime",
                "strlen",
                "__ctype_b_loc",
                "bindtextdomain",
                "raise",
                "__lxstat",
                "__strtoul_internal",
                "sigaddset",
                "_exit",
                "clock_gettime",
                "__fpending",
                "lgetfilecon",
                "tzset",
                "__printf_chk",
                "dirfd",
                "localtime_r",
                "getgrnam",
                "getpwnam",
                "realloc",
                "abort",
                "unsetenv",
                "memmove",
                "mbstowcs",
                "dcgettext",
                "signal",
                "__ctype_tolower_loc",
                "fgetfilecon",
                "fseeko",
                "sigemptyset",
                "readdir",
                "mempcpy",
                "error",
                "__memcpy_chk"
            ],
            "rpm": "coreutils-8.27-16.fc27.x86_64.rpm"
        },
        "usr/bin/pwd": {
            "find-libc-functions": [
                "GI___obstack_vprintf",
                "GI___vasprintf",
                "GI___vdprintf",
                "GI___vfprintf",
                "GI___vfwprintf",
                "GI___vsnprintf",
                "GI___vsprintf",
                "GI___vswprintf",
                "GI___vsyslog",
                "__longjmp",
                "_fprintf",
                "_printf",
                "_snprintf",
                "_sprintf",
                "_vfprintf",
                "_vprintf",
                "_vsnprintf",
                "_vsprintf",
                "asprintf",
                "confstr",
                "dprintf",
                "explicit_bzero",
                "fdelt",
                "fgets",
                "fgets_unlocked",
                "fgetws",
                "fgetws_unlocked",
                "fprintf",
                "fread",
                "fread_unlocked",
                "fwprintf",
                "getcwd",
                "getdomainname",
                "getgroups",
                "gethostname",
                "getlogin_r",
                "gets",
                "getwd",
                "longjmp",
                "mbsnrtowcs",
                "mbsrtowcs",
                "mbstowcs",
                "memcpy",
                "memmove",
                "mempcpy",
                "memset",
                "obstack_printf",
                "obstack_vprintf",
                "poll",
                "ppoll",
                "pread64",
                "pread",
                "printf",
                "ptsname_r",
                "read",
                "readlink",
                "readlinkat",
                "realpath",
                "recv",
                "recvfrom",
                "snprintf",
                "sprintf",
                "stpcpy",
                "stpncpy",
                "strcat",
                "strcpy",
                "strncat",
                "strncpy",
                "swprintf",
                "syslog",
                "ttyname_r",
                "vasprintf",
                "vdprintf",
                "vfprintf",
                "vfwprintf",
                "vprintf",
                "vsnprintf",
                "vsprintf",
                "vswprintf",
                "vsyslog",
                "vwprintf",
                "wcpcpy",
                "wcpncpy",
                "wcrtomb",
                "wcscat",
                "wcscpy",
                "wcsncat",
                "wcsncpy",
                "wcsnrtombs",
                "wcsrtombs",
                "wcstombs",
                "wctomb",
                "wmemcpy",
                "wmemmove",
                "wmempcpy",
                "wmemset",
                "wprintf"
            ],
            "filename": "usr/bin/pwd",
            "complexity": {
                "r2aa": {
                    "afCc": 9,
                    "afC": 198
                }
            },
            "hardening-check": {
                " Read-only relocations": "yes",
                " Position Independent Executable": "yes",
                " Stack protected": "yes",
                " Fortify Source functions": "yes (some protected functions found)",
                " Immediate binding": "yes"
            },
            "report-functions": [
                "getenv",
                "textdomain",
                "lseek",
                "__printf_chk",
                "strlen",
                "calloc",
                "__ctype_b_loc",
                "strncmp",
                "closedir",
                "mbsinit",
                "error",
                "__fxstat",
                "bindtextdomain",
                "open",
                "__xstat",
                "fileno",
                "free",
                "_exit",
                "fputs_unlocked",
                "__errno_location",
                "strcmp",
                "readdir",
                "iswprint",
                "setlocale",
                "close",
                "_ITM_registerTMCloneTable",
                "memcmp",
                "__uflow",
                "malloc",
                "strrchr",
                "mbrtowc",
                "strstr",
                "fscanf",
                "__fpending",
                "nl_langinfo",
                "memcpy",
                "fclose",
                "__fprintf_chk",
                "__libc_start_main",
                "realloc",
                "abort",
                "ungetc",
                "getopt_long",
                "__gmon_start__",
                "exit",
                "_ITM_deregisterTMCloneTable",
                "__cxa_finalize",
                "__ctype_get_mb_cur_max",
                "puts",
                "chdir",
                "__freading",
                "dirfd",
                "fwrite",
                "getcwd",
                "fchdir",
                "memset",
                "__lxstat",
                "fdopen",
                "fseeko",
                "fflush",
                "dcgettext",
                "__cxa_atexit"
            ],
            "rpm": "coreutils-8.27-16.fc27.x86_64.rpm"
        },
    "_comment": "truncated",
    "metadata": {
        "spec_data": {
            "Group": " System Environment/Base",
            "Name": " coreutils",
            "License": " GPLv3+",
            "URL": " https://www.gnu.org/software/coreutils/",
            "Relocations": " (not relocatable)",
            "Install Date": " (not installed)",
            "Build Host": " buildvm-05.phx2.fedoraproject.org",
            "Description": "\nThese are the GNU core utilities.  This package is the combination of\nthe old GNU fileutils, sh-utils, and textutils packages.\n",
            "Build Date": " Tue 22 Aug 2017 09:21:46 AM EDT",
            "Source RPM": " coreutils-8.27-16.fc27.src.rpm",
            "Version": " 8.27",
            "Architecture": " x86_64",
            "Signature": " RSA/SHA256, Tue 22 Aug 2017 09:36:25 AM EDT, Key ID f55e7430f5282ee4",
            "Release": " 16.fc27",
            "Vendor": " Fedora Project",
            "Packager": " Fedora Project",
            "Summary": " A set of basic GNU tools commonly used in shell scripts",
            "Size": " 5773848"
        }
    }
}

About

Results from Cyber Test Lab

Resources

License

Releases

No releases published

Packages

No packages published