Skip to content

API

Jump to bottom
Václav Bartoš edited this page Mar 17, 2024 · 2 revisions

Application Programming Interface (API) of Network Entity Reputation Database (NERD) is based on HTTPS requests. Unless stated otherwise, all methods below are HTTPS GET requests returning a JSON document. Format of the returned JSON is specified in the description of each method.

Base URL for all methods is https://nerd.cesnet.cz/nerd/api/v1

Access to all methods requires authentication with HTTP Authorization header in the form:

Authorization: token ASDF123456789ASDF or just Authorization: ASDF123456789ASDF

The token is assigned to each user and is shown in his/her account info page.

If an error occurs, the HTTP response have appropriate error code and the body contains a JSON document with error field describing the error. For example, when there is no record in NERD for requested IP address, 404 error code is returned with the following JSON as the body: {"ip": <ip>, "error": "Not found"}

API endpoints

Query single IP address

Basic IP info

Return basic information about given IP address.

<base_url>/ip/<ip_address>

Parameters:

  • ip_address IPv4 address in dotted decimal format or IPv6 in full or shortened format.

Returned fields:

  • Only ip, rep, fmp, hostname, asn, bgppref, ipblock, geo, tags fields are returned. See "Full IP info" for description.

Example:

$ curl -H "Authorization: TOKEN" https://nerd.cesnet.cz/nerd/api/v1/ip/198.51.100.1
{
    "ip": "198.51.100.1",
    "rep": 0.65,
    "fmp": {
       "general": 0.692
    },
    "hostname": "test.example.com",
    "asn": [
        206776, 
        58222
    ],
    "bgppref": "198.51.0.0/16",
    "ipblock": "198.51.100.0 - 198.51.100.255"
    "geo": {
        "ctry": "CZ"
    },
    "bl": [
        "blacklist_name",
    ],
    "tags": [
        {"n": "tag_name", "c": 1.0},
    ]
}

Full IP info

Return all information stored in the DB for the given IP address.

<base_url>/ip/<ip_address>/full

Parameters:

  • ip_address IPv4 address in dotted decimal format or IPv6 in full or shortened format.

Returned fields:

  • ip (string) The IP address queried.
  • rep (float) Reputation score.
  • fmp (object) FMP score (Future Maliciousness Probability), probability of receiving an alert about the IP address in the next 24 hours, estimated by a machine learning algorithm.
    • general (float) General FMP score, prediction of any type of alert (in the future, it is planned to add scores specific to slected types of alerts).
  • hostname (string) Hostname as resolved by DNS PTR query.
  • asn (array of objects) Autonomous system info.
    • _id (int) Autonomous system number.
    • name (string) Autonomous system name.
    • rir (string) Name of Regional Internet Register.
    • circl_bgprank (float) BGP rank of ASN entities using BGP Ranking API (bgpranking_web).
    • eml_rank (float) Rank from The Email Laundry (EML).
    • rep (float) Reputation score.
    • ts_added (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last added.
    • ts_last_update (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last updated.
    • org (object) Organization related to the AS.
      • _id (string) Organization unique identifier.
      • name (string) Name of the organization.
      • address (string) Address of the organization.
      • ts_added (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last added.
      • ts_last_update (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last updated.
  • bgppref (object) BGP prefix in which the IP address belongs.
    • _id (string) BGP prefix. (e.g. 198.51.0.0/16)
    • rep (float) Reputation score.
    • ts_added (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last added.
    • ts_last_update (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last updated.
  • ipblock (object) IP block in which the IP address belongs
    • _id (string) NetRange. (e.g. 198.51.100.0 - 198.51.100.255)
    • rep (float) Reputation score.
    • name (string) Name of the IP block.
    • descr (string) Description of the IP block.
    • rir (string) Name of Regional Internet Register.
    • status (string) Status describing whether the block is directly assigned, reallocated atc.
    • ts_added (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last added.
    • ts_last_update (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last updated.
    • org (object) Organization related to the IP block.
      • _id (string) Organization unique identifier.
      • name (string) Name of the organization.
      • address (string) Address of the organization.
      • ts_added (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last added.
      • ts_last_update (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last updated.
  • geo (object) Geolocation information
    • ctry (string) Country (2-letter ISO code)
    • city (string) City
    • tz (string) Timezone name (e.g. Europe/Prague)
  • ts_added (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was created, this is usually the time of the first reported incident.
  • ts_last_update (string, YYYY-mm-ddTMM:HH:SS, UTC) Time the record was last updated.
  • ts_last_event (string, YYYY-mm-ddTMM:HH:SS, UTC) Time of the last event related to this IP.
  • bl (array of objects) Information about presence of IP on blacklists.
    • Array of objects with following fields (name is the key):
      • name (string) Name of the blacklist.
      • last_result (bool) true if IP was present on the blacklist when last checked, false otherwise.
      • last_check (string YYYY-mm-ddTMM:HH:SS, UTC) Time of last check.
      • history (array of time-strings) Array of times when IP was found on the blacklist.
  • events (array of objects) Number of reported events per day, node and category.
    • Array of objects with following fields ([date,cat,node] is the key):
      • date (string, YYYY-mm-dd, UTC) Date
      • cat (string) Category name (Category field from IDEA [1] messages, without dots, multiple categories joined by +)
      • node (string) Name of node reporting the events (Node.Name field from IDEA messages)
      • n (int) Number of events in given day with given category and node name.
  • events_meta Meta-information about events
    • total (int) Total number of reported events (for the time window data are kept in database, normally 90 days)
    • total{1,7,30} (int) Total number of reported events in last 1, 7, or 30 days, respectively (Note: events received in N previous whole days and in the current day are counted; for example, total1 may span 24 to 48 hours, depending on time of the query)

Some fields may not be present for all IP addresses.

More fields may be added in the future.

Reputation score only

Return only the reputation score of the IP address

<base_url>/ip/<ip_address>/rep

Parameters:

  • ip_address IPv4 address in dotted decimal format or IPv6 in full or shortened format.

Returned fields:

  • Only ip and rep fields are returned. See "Full IP info" for description.

Example:

$ curl -H "Authorization: TOKEN" https://nerd.cesnet.cz/nerd/api/v1/ip/198.51.100.1/rep
{
    "ip": "198.51.100.1",
    "rep": 0.412
}

FMP score only

Return only the FMP score of the IP address

<base_url>/ip/<ip_address>/fmp

Parameters:

  • ip_address IPv4 address in dotted decimal format or IPv6 in full or shortened format.

Returned fields:

  • Only ip and fmp fields are returned. See "Full IP info" for description.

Example:

$ curl -H "Authorization: TOKEN" https://nerd.cesnet.cz/nerd/api/v1/ip/198.51.100.1/fmp
{
    "ip": "198.51.100.1",
    "fmp": {
        "general": 0.378
    }
}

Search IPs

Return list of IP addresses matching given criteria, including basic info about each matching IP (depending on the o parameter).

<base_url>/search/ip/?<query_parameters>

Parameters can be passed via both GET and POST methods (in application/x-www-form-urlencoded format).

Parameters:

  • Filter:
    • subnet: Single CIDR prefix of IP addresses to search.
    • ip_list: List of IP addresses or CIDR prefixes to search (equivalent to "List of IPs" tab in web GUI). Comma or any whitespace can be used as a separator.
    • hostname: Suffix of hostname associated to the IP address by its reverse DNS record.
    • country: 2-letter ISO code of the country.
    • asn: Autonomous system number
    • cat: Event category, i.e. only return IP addresses with at least one alert of given category (see web form for possible values).
    • cat_op: and or or, if multiple categories are specified, this tells whether any of or apply (default: or).
    • blacklist: Name of blacklist IP must be listed on. Repeat parameter to specify multiple blacklists.
    • bl_op: and or or, if multiple blacklists are specified, this tells whether IP must be on all or any of them to match the filter (default: or).
    • tag: Name of tag that should be assigned to IP. Repeat parameter to specify multiple tags.
    • tag_op: and or or, if multiple tags are specified, this tells whether all or any of them should be present to match the filter (default: or).
    • tag_conf: Minimal confidence of tags to be considered as present (default: 0.5).
  • Other params:
    • limit: Maximum number of results returned, allowed range 1-10000, or 0 = unlimited (allowed only if o=list or with a special user permission), (default: 20 if o=json, unlimited if o=list).
    • sortby: Sort the results according to given field. Possible options:
      • events, rep, ts_added, last_activity, ip
    • asc: If set to 1, sort in ascending order (default: 0 = descending order).
    • o: Specifies output format. Possible values:
      • json (default) - JSON array with a basic IP record for each matching IP (see above what a basic record contains)
      • short - JSON array containing only _id, rep and tags fields.
      • list - Plain-text list of matching IPs is returned (one IP per line).

Examples:

$ curl -H "Authorization: TOKEN" 'https://nerd.cesnet.cz/nerd/api/v1/search/ip/?asn=2852&cat=AttemptLogin'
[
  {
    "ip": "66.240.205.34",
    "hostname": "malware-hunter.census.shodan.io",
    ...
  },
  {
    "ip": "71.6.146.186",
    "hostname": "inspire.census.shodan.io",
    ...
  },
  ...
]

$ curl -H "Authorization: TOKEN" https://nerd.cesnet.cz/nerd/api/v1/search/ip/ -d limit=100 -d o=list -d ip_list=66.240.205.0/24,185.142.236.0/24
66.240.205.34
185.142.236.38
185.142.236.35
185.142.236.34
...

Bulk querying

<base_url>/ip/bulk

Endpoint for bulk IP address queries about the reputation score. A list of IP addresses is passed as raw data in POST request. It can be passed either in binary format (4 bytes per IP address, big endian, no separator) or in a text format (ASCII, each IP separated with comma, no spaces). The format is selected by querying with header field Content-Type: text/plain for text format or Content-Type: application/octet-stream for binary format.

Returned data contain a list of reputation scores, one for each IP address queried, in the same order as IPs were passed to the API. Depending of the query format, scores are returned either as ASCII-coded decimal numbers, separated by commas (text format), or as an octet stream, where each 8 bytes represent a score in double precision data type (binary format).

Query to this endpoint consumes 1 rate-limit token for each 1000 IP addresses (0-999 = 1 token, 1000-1999 = 2 tokens, ...).

Example:

$ curl -H "Authorization: TOKEN" -H "Content-Type: text/plain" https://nerd.cesnet.cz/nerd/api/v1/ip/bulk --data "198.51.100.1,198.51.100.222,192.168.1.1"
0.8357
0.0
0.4514
Clone this wiki locally