API public documentation
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
modules
README.md

README.md

How to use BinaryEdge’s API

Note: all requests are identified by Job ID and are shown in the stream window.

Input Output
1 Connect to Data Stream
curl https://stream.api.binaryedge.io/v1/stream -H "X-Token:InsertYourClientToken"
(data stream)
2 Request a Scan Task
curl https://api.binaryedge.io/v1/tasks -d '{"type":"scan", "description": "InsertYourDescriptionHere", "options":[{"targets":["InsertAnIPAddress/IPNetwork"], "ports":[{"port":InsertPort, "protocol": "tcp or udp", "modules": ["InsertModule"]}]}]}' -v -H "X-Token:InsertYourClientToken"
{"stream_url":"stream URL","job_id":"Job ID"}

Index

Data Stream

Continuous Data Stream of data generated by our platform.

Important: The Stream might disconnect sometimes, as such, it is the client's responsibility to reconnect, as it might miss results while disconnected.

There are 2 types of Data Streams available that can be consumed, based on your client account permissions.

1. Firehose

Endpoint: https://stream.api.binaryedge.io/v1/firehose

Description: This stream contains all data generated by Binaryedge own scans, these include the continuous WorldWide Scans that target different ports every day. This stream does not contain data generated from jobs requested by clients.

2. Stream

Endpoint: https://stream.api.binaryedge.io/v1/stream

Description: This stream contains all data generated by your own requested Jobs. Clients don't have access to data generated from scans of other clients.

Tasks

GET /v1/tasks - List Requested Jobs

Retrieve a list of the latest requested jobs. This includes:

  • "status": Status of the job. Where Status can be:
    • "Requested": Job was requested successfully;
    • "Revoked": Job was revoked by user;
    • "Success": Job completed successfully;
    • "Failed": Job completed, but did not finish.
  • "requested_at": Time the job was requested;
  • "finished_at": Time the job finished;
  • "job_id": ID of the requested job;
  • "options": Job configuration options.
curl https://api.binaryedge.io/v1/tasks -H "X-Token:InsertYourClientToken"

HTTP/1.1 200 OK

[{"status": "Success", "requested_at": "2017-04-10T17:44:58.636681+00:00", "description": "Job Description 1", "finished_at": "2017-04-10T17:47:46.534544+00:00", "options": [{"targets": ["xxx.xxx.xxx.xxx"], "ports": [{"modules": ["service", "service-simple", "ssh"], "port": "80,8080"}]}], "job_id": "32637b98-8f01-46eb-a1f7-3eaee18ab1d5"}, {"status": "Success", "requested_at": "2017-04-10T17:39:53.066632+00:00", "description": "Test web", "finished_at": "2017-04-10T17:41:57.919141+00:00", "options": [{"targets": ["example.org"], "ports": [{"config": {"https": true}, "modules": ["web"], "port": 443}]}], "job_id": "73364d62-d768-4dbd-9947-aba2a453dfb7"}]

POST /v1/tasks - Create Scan Job

Create a On-Demand Job. You can specify your own targets, ports, modules and configurations.

Parameters:

  • "type": "scan" or "grab". Please refer to Supported Types.;
  • "description": Add your own description of the job. Can be a empty string, i.e. "" ;
  • "options": Configuration Options for the job, Array of JSON. 1 Job can have multiple options.
curl https://api.binaryedge.io/v1/tasks -d '{"type":"scan", "description": "InsertYourDescriptionHere", "options":[{"targets":["InsertAnIPAddress/IPNetwork"], "ports":[{"port":InsertPort, "protocol": "tcp or udp", "modules": ["InsertModule"]}]}]}' -v -H "X-Token:InsertYourClientToken"

POST /v1/tasks/job_id/revoke - Revoke Job

To cancel a requested job:

curl -XPOST https://api.binaryedge.io/v1/tasks/<JOB_ID>/revoke -H  "X-Token:InsertYourClientToken"

HTTP/1.1 200 OK
{"message": "Job revoked"}

GET /v1/replay/job_id - Replay Job

To retrieve the results from a previously requested scan job, you can replay the stream with this endpoint.

curl https://stream.api.binaryedge.io/v1/replay/<JOB_ID> -H "X-Token:InsertYourClientToken"

HTTP/1.1 200 OK
<Stream results from request job>

Job Status

In order for you to know the status of your jobs we provide information in 2 distinct ways:

1. GET /v1/tasks/job_id/status - Status Endpoint

To check the current status of a Requested job:

curl https://api.binaryedge.io/v1/tasks/<job_id>/status -H "X-Token:InsertYourClientToken"

HTTP/1.1 200 OK
{"status":"<STATUS>"}

Where Status can be:

  • "Requested": Job was requested successfully;
  • "Revoked": Job was revoked by user;
  • "Success": Job completed successfully;
  • "Failed": Job completed, but did not finish.

2. Status Messages inside stream

In your stream you will find messages providing insight on the current status of your jobs:

When a Job is created:

{
  "origin": {
    "job_id": "c4773cb-aa1e-4356eac1ad08",
    "type": "job_status",
    ...
  },
  "status": {
    "success": null,
    "started": null,
    "completed": null,
    "revoked": null
  }
}

Job is completed

{
  "origin": {
    "job_id": "c4773cb-aa1e-4356eac1ad08",
    "type": "job_status",
    ...
  },
  "status": {
    "success": true,
    "started": true,
    "completed": true,
    "revoked": false
  }
}

Meaning of the status fields:

  • "success": If the job was completed successfully, with no problems.
  • "started": When a job is requested, it's put in to a Queue;
  • "completed": If the job is completed, no more results will be sent;
  • "revoked": If the job was canceled by the user or not.

Supported Job Types

There are 2 types of requests.

1. scan

The scan type will request a scan on the targets and will launch the modules against the detected open ports. It should be used against a large number of targets and its function is to filter responding IPs.

2. grab

The grab type will try to gather information directly from the targets, without portscan phase. Should be used against a small number of targets.

Recommended for when targeting Domains, with Web and HTTP/HTTPS modules.

General Data Format

All events generated by our scanning platform, delivered via our Data Streams, have the following outline (except the Status messages presented above):

Details of the fields:

  • origin:
    • client_id:
      • Only on client stream
      • Informative, your client code;
    • job_id:
      • Only on client stream
      • Job id that event is part of;
    • type:
      • Please refer to the next section for details on each module type;
    • module:
      • 'portscan' or 'grabber'. Informative; Category of the event. If it's a portscan event or a grabber with details of the ip/port/service;
    • ip:
      • IP used by the scanner to perform the analysis;
    • port:
      • Optional, only some modules will provide this information. Currently only provided by "service-simple". We will be working to add more.
      • Port used by the scanner to perform the analysis;
    • ts:
      • Unix Timestamp with Milliseconds;
    • country:
      • ISO Code 2 of the Country the Scanner is located in.
  • target:
    • ip:
      • Target Address
    • port:
      • Target Port
    • protocol:
      • Protocol used for connection
  • result:
    • data:
      • Please refer to the next section for details on each module type.
{
  "origin": {
    "client_id": "string",
    "job_id": "string",
    "country": "string",
    "type": "string",
    "module": "string",
    "ts": integer,
    "ip": "string"
    "port": integer
  },
  "target": {
    "ip": "ip",
    "port": integer,
    "protocol": "string"
  },
  "result": {
    "data": {(...)}  
  }
}

Supported Modules Types

1. elasticsearch

Description: Extract Elasticsearch detailed information.

Detailed documentation: elasticsearch module documentation

2. http & https

Description: Extract HTTP/HTTPS information, e.g. HTTP headers, HTTP status codes, HTTP body, and redirects information. Follows up to 5 redirects.

Detailed documentation: http & https module documentation

3. memcached

Description: Extract Memcached detailed information.

Detailed documentation: memcached module documentation

4. mongodb

Description: Extract MongoDB detailed information.

Detailed documentation: mongodb module documentation

5. mqtt

Description: Grab MQTT information, including messages and topics.

Detailed documentation: mqtt module documentation

6. rdp

Description: Extract RDP details and screenshot

Detailed documentation: rdp module documentation

7. redis

Description: Extract Redis detailed information.

Detailed documentation: redis module documentation

8. service

Description: Extract detailed product specific information, e.g. product name, version, headers, scripts. If you just want product name and version, consider using the faster "service-simple".

Detailed documentation: service module documentation

9. service-simple

Description: Extract basic product specific information, e.g. product name, version. This module is much faster than "service", since it returns less information.

Detailed documentation: service-simple module documentation

10. ssh

Description: Extract SSH details, e.g. key and algorithms for SSH servers

Detailed documentation: ssh module documentation

11. ssl

Description: Extract SSL details e.g. type of encryption

Detailed documentation: ssl module documentation

12. telnet

Description: Extract Telnet information, e.g. Will, Do, Don't Won't commands.

Detailed documentation: telnet module documentation

13. vnc

Description: Extract VNC details and screenshot

Detailed documentation: vnc module documentation

14. web

Description: Extract Web technologies information and headers.

Detailed documentation: web module documentation

15. x11

Description: Extract x11 screenshot

Detailed documentation: x11 module documentation

Custom Modules

Note: If you want a custom-made module, please contact BinaryEdge.

Configurations

It is possible to set module specific configurations on the job requests. For example, the HTTP module allows the configuration of the Host and the User Agent HTTP headers. The configuration should be set in the "config" key at the same json level of the requested module.

Example:

{
   "type": "scan",
   "description": "test a bunch of networks",
   "options": [
       {
         "targets": ["xxx.xxx.x.x/xx","xxx.xxx.x.x/xx"],
         "ports": [
           {
            "port": 80,
            "modules": ["http"],
            "config":
              {
                "user_agent":"Test user Agent",
                "host_header":"google.com"
              }
           }]
       }
     ]
 }

Available configurations

Check each module's detailed documentation for the available configurations.

Query Endpoints

Historical Query

GET /v1/query/raw (deprecated)

GET /v1/query/historical - Historical IP Data Endpoint

Access our historical database. This will provide with all the raw events regarding an IP

Available options:

  • Target IP, e.g.:
    • /v1/query/historical/210.1.1.X
  • Target CIDR, e.g.:
    • /v1/query/historical/210.1.1.X/24
  • Target Range, e.g.:
    • /v1/query/historical/210.1.1.X-210.1.1.Y
curl -v https://api.binaryedge.io/v1/query/historical/222.208.xxx.xxx -H 'X-Token:InsertYourClientToken'
Response
{
  "origin": {
    "country": "uk",
    "module": "grabber",
    "ts": 1464558594512,
    "type": "service-simple"
  },
  "target": {
    "ip": "222.208.xxx.xxx",
    "protocol": "tcp",
    "port": 992
  },
  "result": {
    "data": {
      "state": {
        "state": "open|filtered"
      },
      "service": {
        "name": "telnets",
        "method": "table_default"
      }
    }
  }
}

GET /v1/query/latest - Latest IP Data Endpoint

Access our historical database. This will provide with the latest raw events regarding an IP

Available options:

  • Target IP, e.g.:
    • /v1/query/latest/210.1.1.X
  • Target CIDR, e.g.:
    • /v1/query/latest/210.1.1.X/24
  • Target Range, e.g.:
    • /v1/query/latest/210.1.1.X-210.1.1.Y
curl -v https://api.binaryedge.io/v1/query/latest/222.208.xxx.xxx -H 'X-Token:InsertYourClientToken'
Response
{
  "origin": {
    "country": "uk",
    "module": "grabber",
    "ts": 1464558594512,
    "type": "service-simple"
  },
  "target": {
    "ip": "222.208.xxx.xxx",
    "protocol": "tcp",
    "port": 992
  },
  "result": {
    "data": {
      "state": {
        "state": "open|filtered"
      },
      "service": {
        "name": "telnets",
        "method": "table_default"
      }
    }
  }
}

GET /v1/query/torrent - Torrent IP Data Endpoint

Access our historical database. This will provide with raw events related with torrent activity regarding an IP

Available options:

  • Target IP, e.g.:
    • /v1/query/torrent/210.1.1.X
  • Target CIDR, e.g.:
    • /v1/query/torrent/210.1.1.X/24
  • Target Range, e.g.:
    • /v1/query/torrent/210.1.1.X-210.1.1.Y
curl -v https://api.binaryedge.io/v1/query/torrent/222.208.xxx.xxx -H 'X-Token:InsertYourClientToken'
Response
{
  "origin":{  
    "type":"peer",
    "module":"torrent",
    "ts":1491827676263
  },
  "node":{  
    "ip":"219.88.xxx.xxx",
    "port":25923
  },
  "peer":{  
    "ip":"222.208.xxx.xxx",
    "port":30236
  },
  "torrent":{  
    "infohash":"cbe45addbb48c07ef6451bd3bee326d5cd82538f",
    "name":"NCIS Los Angeles S08E20 HDTV x264-LOL EZTV",
    "source":"EZTV",
    "category":"TV Show"
  }
}

GET /v1/query/search - Full-Text Search

Query our data, using our Text Search Engine.

curl -v https://api.binaryedge.io/v1/query/search\?query\="mysql" -H 'X-Token:InsertYourClientToken'

Or, you can use some filters:

Available options:

  • product: (string) search product names, e.g. "nginx"
  • country: (string) search using country codes, e.g. "ES"
  • port: (int) filter by port number, e.g. 80
  • Conditionals: the following conditionals are available: AND, OR. Must be UPERCASE.
curl -v https://api.binaryedge.io/v1/query/search\?query\="product:mysql%20AND%20country:ES" -H 'X-Token:InsertYourClientToken'

Error Messages in Historical Query

Querying for a malformed IP address, CIDR or range

HTTP/1.1 400 Bad Request
{"msg": "Targets with wrong format/type or range ill-defined. Please review your query.", "status": 400}

Sending invalid Token:

HTTP/1.1 401 Unauthorized
{msg": "Unauthorized.", "status": 401}

Remote Desktop Query

GET /v1/query/image

Query for a list of remote desktops found (latest first).

Available options:

  • pagesize: Maximum number of results to return per page
    • pagesize=100
  • page: Page number of the results
    • page=1
curl -v https://api.binaryedge.io/v1/query/image -H 'X-Token:InsertYourClientToken'
Response
{
    "total_records": 3,
    "events": [
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/903cb557b597d9fa29905e1908381cf90b851da945b711366686af50.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/903cb557b597d9fa29905e1908381cf90b851da945b711366686af50.jpg",
          "image_id":"903cb557b597d9fa29905e1908381cf90b851da945b711366686af50"
        },
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21.jpg",
          "image_id":"933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21"
        },
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59.jpg",
          "image_id":"9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59"
        }
    ]
}

GET /v1/query/image/<image_id>?(options)

Query details about remote desktops that were detected by BinaryEdge. This includes the following information:

  • IP: target address where the screenshot was taken;
  • Port: target port where the service was running;
  • ts: timestamp of when the screenshot was taken;
  • geoip: geographical information;
  • has_faces: Boolean, whether faces were detected or not;
  • n_faces: Integer, Number of faces detected on the image;
  • tags: Array, Strings, List of tags automatically attributed by our process;
  • height: Integer;
  • width: Integer;
  • created_at: timestamp of when the screenshot information entered the database;
  • modified_at: timestamp of when the screenshot information was last updated;
  • url: String, URL to download image;
  • thumb: String, URL to download image thumbnail;

Available options:

  • ocr: if present, shows an additional "words" field, which is a list of words obtains via our OCR process, e.g.:
    • ocr=1
curl -v https://api.binaryedge.io/v1/query/image/f1b0a311af803ea73ac48adce2378f58adce2378f5?ocr=1 -H 'X-Token:InsertYourClientToken'
Response
{
  "modified_at": 1473888487412,
  "tags": ["vnc"],
  "geoip": {
    "timezone": "Asia/Shanghai",
    "continent_code": "AS",
    "location": [120.4586, 41.5703],
    "city_name": "Chaoyang",
    "country_name": "China",
    "country_code": "CN"
  },
  "n_faces": 0,
  "words": ["administrator", "windows", "thinkpad", "enterprise"],
  "created_at": 1473544493392,
  "image_id": "f1b0a311af803ea73ac48adce2378f58adce2378f5",
  "height": 600,
  "ts": 1473544466000,
  "has_faces": false,
  "port": 5910,
  "ip": "XXX.XXX.XXX.XXX",
  "width": 800,
  "url": "https://path/to/f1b0a311af803ea73ac48adce2378f58adce2378f5.jpg",
  "thumb": "https://path/to/f1b0a311af803ea73ac48adce2378f58adce2378f5.jpg"
}

GET /v1/query/image/search?(options)

Query for a list of remote desktops according to certain filters.

Available options:

  • ip: IP, CIDR or Range you want to target, e.g.:
    • ip=127.0.0.1
  • port: Port number /range you want to target, e.g.:
    • port=5900
  • country: Search images from a certain country, e.g.:
    • country=pt
  • tag: Search images that contain a tag, e.g.:
    • tag=has_faces
    • tag=mobile
  • word: Search images that contain a word, e.g.:
    • word=microsoft
    • word=credit+card
  • pagesize: Maximum number of results to return per page
    • pagesize=100
  • page: Page number of the results
    • page=1
curl https://api.binaryedge.io/v1/query/image/search\?ip\=120.XXX.XXX.XXX  -H 'X-Token:InsertYourClientToken'
Response
{
    "total_records": 3,
    "events": [
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/903cb557b597d9fa29905e1908381cf90b851da945b711366686af50.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/903cb557b597d9fa29905e1908381cf90b851da945b711366686af50.jpg",
          "image_id":"903cb557b597d9fa29905e1908381cf90b851da945b711366686af50"
        },
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21.jpg",
          "image_id":"933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21"
        },
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59.jpg",
          "image_id":"9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59"
        }
    ]
}

GET /v1/query/image/search?similar=<image_id>

Query for a list of remote desktops that are similar to another remote desktop. Note: This option cannot be used together with the previous ones.

curl https://api.binaryedge.io/v1/query/image/search\?similar\=f1b0a311af803ea73ac48adce2378f58adce2378f5  -H 'X-Token:InsertYourClientToken'
Response
{
    "total_records": 3,
    "events": [
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/903cb557b597d9fa29905e1908381cf90b851da945b711366686af50.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/903cb557b597d9fa29905e1908381cf90b851da945b711366686af50.jpg",
          "image_id":"903cb557b597d9fa29905e1908381cf90b851da945b711366686af50"
        },
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21.jpg",
          "image_id":"933db157b280dfe236975e07073315f201e215c24cb11a3d658ba054dd21"
        },
        {
          "url":"https://d1ngxp4ef6grqi.cloudfront.net/9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59.jpg",
          "thumb":"https://d3f9qnon04ymh2.cloudfront.net/9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59.jpg",
          "image_id":"9735ad48b289c0f9338f5c1908381cf90b851da945b712386387ac59"
        }
    ]
}

Error Messages in Remote Desktop Query

Performing a malformed query:

HTTP/1.1 400 Bad Request
{"title": "Bad Request"}

Sending invalid Token:

HTTP/1.1 401 Unauthorized
{"title": "Unauthorized"}

Accessing a page that does not exist:

HTTP/1.1 404 Not Found
{"title": "Not Found"}

FAQ

Q: What is the sample parameter?

A: The Sample parameter is used to define how many open ports the platform needs to find before stopping the scan. It is useful to test modules and different configurations for each module (that we are adding in the future). This parameter is optional - by default the scan stops only after scanning the entire list of IP addresses and ports.

Q: How can I consume the stream?

A: The stream outputs to STDOUT, allowing you to consume it in different ways. For example:

  • Direct the stream to a file:
    • curl https://stream.api.binaryedge.io/v1/stream -H "X-Token:InsertYourClientToken" > file.txt
  • Pipe the stream to a custom application you developed to process it:
    • curl https://stream.api.binaryedge.io/v1/stream -H "X-Token:InsertYourClientToken" | application_name

Q: What should I do if I get a error 500?

A: In this case, you should contact support@binaryedge.io

Q: How do I scan multiple hosts with one request?

A:

options: [{
   "targets": [array of cidrs (string)],
   "ports": [{
       "port": int,
       "modules": [array of module names (string)],
       "sample": int
   }]
}]

Example:

{
   "type": "scan",
   "description": "test a bunch of networks",
   "options": [
       {
         "targets": ["xxx.xxx.x.x/xx","xxx.xxx.x.x/xx"],
         "ports": [{
            "port": 995,
            "modules": ["service"]
           },
           {
            "port": 22,
            "modules": ["ssh"]
           }]
       }, {
         "targets": ["xxx.xxx.x.x/xx"],
         "ports": [{
            "port": 5900,
            "modules": ["vnc"]
         }]
       }
     ]
 }