R interface to the Censys "cyber"/scans search engine • https://www.censys.io/tutorial
R
Latest commit d2d95a8 Dec 31, 2016 @hrbrmstr CRAN tweaks
Permalink
Failed to load latest commit information.
R
README_cache/html for travis Dec 31, 2016
docs docs Dec 31, 2016
man pre-CRAN flight check Dec 31, 2016
tests CRAN tweaks Dec 31, 2016
.Rbuildignore CRAN tweaks Dec 31, 2016
.gitignore Initial commit Nov 7, 2015
.travis.yml fix travis config error Dec 31, 2016
CONDUCT.md inital gh commit Jan 11, 2016
DESCRIPTION changed some wording Dec 31, 2016
LICENSE Initial commit Nov 7, 2015
NAMESPACE pre-CRAN flight check Dec 31, 2016
NEWS.md docs Dec 31, 2016
README.Rmd CRAN tweaks Dec 31, 2016
README.html CRAN tweaks Dec 31, 2016
README.md CRAN tweaks Dec 31, 2016
censys.Rproj inital gh commit Jan 11, 2016
cran-comments.md CRAN tweaks Dec 31, 2016

README.md

Travis-CI Build Status CRAN_Status_Badge

censys is an R package interface to the Censys API

Censys is a search engine that enables researchers to ask questions about the hosts and networks that compose the Internet. Censys collects data on hosts and websites through daily ZMap and ZGrab scans of the IPv4 address space, in turn maintaining a database of how hosts and websites are configured. Researchers can interact with this data through a search interface, report builder, and SQL engine.

Censys tutorial.

The following functions are implemented:

  • censys_export_download: Download export job files to a specified directory
  • censys_export_job_status: Get status of a Censys export job
  • censys_get_job_result: Get results of completed Censys SQL query job
  • censys_get_job_status: Get status of a Censys SQL query job
  • censys_query: Issue SQL Queries against the Censys API
  • censys_report: Create aggregate reports on the breakdown of a field in the result set of a query
  • censys_search: Perform queries against Censys data
  • censys_series: List all series that can be queried from the SQL interface
  • censys_series_details: Get details about a series, including the list of tables and schema for the series
  • censys_start_export: Export large datasets and structured records from Censys to JSON or CSV files
  • get_series: Retrieve data on the types of scans Censys regularly performs ("series").
  • view_document: Retrieve data that Censys has about a specific host, website, or certificate.
  • view_result: Retrieve data on a particular scan "result"
  • view_series: Retrieve data that Censys has about a particular series

Installation

devtools::install_github("hrbrmstr/censys")

Usage

library(censys)

# current verison
packageVersion("censys")
#> [1] '0.1.0'

library(purrr)
library(dplyr)

res <- censys_query("
SELECT p80.http.get.headers.server, p80.http.get.headers.www_authenticate, location.country, autonomous_system.asn
FROM ipv4.20161206
WHERE REGEXP_MATCH(p80.http.get.headers.server, r'gen[56]')
")

inf <- censys_get_job_status(res$job_id)

Sys.sleep(10) # giving it some time to process for the README

map(1:3, ~censys_get_job_result(inf$job_id, .)) %>% 
  map(c("rows", "f")) %>% 
  flatten() %>% 
  map("v") %>% 
  map_df(~setNames(as.list(.), c("server", "auth", "geo", "asn"))) %>% 
  count(geo, sort=TRUE)
#> # A tibble: 54 × 2
#>               geo     n
#>             <chr> <int>
#> 1   United States   645
#> 2         Germany   338
#> 3         Vietnam   233
#> 4         Austria   135
#> 5          France   127
#> 6           Japan    77
#> 7  Czech Republic    64
#> 8           Italy    60
#> 9          Russia    57
#> 10         Canada    44
#> # ... with 44 more rows

Test Results

library(censys)
library(testthat)

date()
#> [1] "Sat Dec 31 10:14:03 2016"

test_dir("tests/")
#> testthat results ========================================================================================================
#> OK: 2 SKIPPED: 0 FAILED: 0
#> 
#> DONE ===================================================================================================================

Code of Conduct

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.