Skip to content
Lim plugin for Zonalizer
Perl Shell
Branch: develop
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
bin
debian
lib/Lim/Plugin
t
.gitignore
.travis.yml
COUCHDB.md
Changes
DEVELOPMENT.md
INSTALL.md
MANIFEST
MANIFEST.SKIP
Makefile.PL
README.md
tidy

README.md

API v1

This describes the API version 1 for Zonalizer. The URLs in this description assumes that Lim operates without any prefixes and all structure examples are in JSON.

Data Model Overview

The data model is structured with a top level object called analyze which contains information about the DNS tests that has been performed for a fully qualified domain name (FQDN).

An analyze is kept in memory while its queued or analyzing then it's store into a database for future access and long term storage.

All analysis are kept in spaces for seperation, see Spaces for more information.

Data Types

The following data types are used:

  • uuid: A version 4 UUID.
  • string: An UTF8 string.
  • integer: A signed big integer.
  • float: A float.
  • href: An URL pointing to another object or objects as according to HATEOAS.
  • datetime: An UTC Unix Timestamp integer.

HATEOAS

By default all URLs are HATEOAS but it can be disable via configuration or by setting base_url to false (0) in the request for any call that returns objects with URLs.

Example:

GET /zonalizer/1/analysis?base_url=0

Pagination

All calls returning a list of objects will have the capabilities to return paginated result using cursors.

The following URI query string options can be used:

  • limit: This is the number of individual objects that are returned in each page, default and max limit is configurable.
  • before: This is the cursor that points to the start of the page of data that has been returned.
  • after: This is the cursor that points to the end of the page of data that has been returned.
  • sort: The field name in the corresponding objects being returned that the result should be sorted on.
  • direction: The direction of the result in conjunction with sort, can be ascending or descending. Default is ascending.

Example:

GET /zonalizer/1/analysis?limit=100&after=cursor
GET /zonalizer/1/analysis?limit=10&sort=created&direction=descending

For each paginated result the following object is also included:

{
  ...
  "paging": {
    "cursors": {
      "after": "string",
      "before": "string"
    },
    "previous": "href",
    "next": "href"
  }
}
  • before: This is the cursor that points to the start of the page of data that has been returned.
  • after: This is the cursor that points to the end of the page of data that has been returned.
  • next: The full API query that will return the next page of data. If not included, this is the last page of data.
  • previous: The full API query that will return the previous page of data. If not included, this is the first page of data.

Configuration

Following configuration parameters exists and can be configured via Lim's YAML config files.

zonalizer

The following paramters can be configured below the root entry zonalizer.

base_url

A bool that controls if the base URL is included in the HATEOAS output.

custom_base_url

A string with a custom base URL that will be used if base_url is true, this is helpful if Zonalizer is run behind load balancer or session divider.

db_driver

A string with the database driver to use.

db_conf

A hash this the database driver configuration, see Database Configuration.

default_limit

An integer with the default number of objects to return for calls that return a list of objects.

max_limit

An integer with the maximum number of objects to return for calls that return a list of objects, the given limit may not be larger then this and if it is then the limit will be max_limit.

allow_ipv4

An integer which determines whether (1) or not (0) IPv4 is allowed to be used for analyzing.

allow_ipv6

An integer which determines whether (1) or not (0) IPv6 is allowed to be used for analyzing.

test_ipv4

An integer which determines whether (1) or not (0) to use IPv4 for analyzing if not specified by the request. Will be forced off (0) if IPv4 analyzing is not allowed (allow_ipv4).

test_ipv6

An integer which determines whether (1) or not (0) to use IPv6 for analyzing if not specified by the request. Will be forced off (0) if IPv6 analyzing is not allowed (allow_ipv6).

max_ongoing

The maximum number of ongoing analysis that can exist at the same time. If this is higher then the number of threads in the collector then the collector will queue work.

allow_undelegated

An integer which determines whether (1) or not (0) undelegated analysis are allowed to be run.

force_undelegated

An integer which determines whether (1) or not (0) undelegated information must be giving in other to run any analysis.

max_undelegated_ns

An integer with the maximum number of ns objects that can be given in an analysis request.

max_undelegated_ds

An integer with the maximum number of ds objects that can be given in an analysis request.

allow_meta_data

An integer which determines whether (1) or not (0) meta data is allowed to be added to an analysis.

max_meta_data_entries

A positive integer with the maximum number of meta data entries that can be added to an analysis.

max_meta_data_entry_size

A positive integer with the maximum size of a meta data entry, this is both key and value combined.

collector

The following parameters are available to configure the collector.

exec

The path to the Zonalizer collector (zonalizer-collector).

config

Path to the Zonemaster configuration file to use.

policy

Path to the Zonemaster policy file to use.

sourceaddr

Local IP address that the test engine should try to send its requests from.

threads

Number of threads to start.

policies

An array of different policies that can be used, each policy starts its own collector. For the optional options that are not given, if they are specified in the global collector configuration then that value will be used.

Per hash in array the following options can be used:

  • name: An unique name for the policy, must match Perl regexp [\w-]+.
  • display: A display friendly name for the policy.
  • description: A description what the policy would be used for. (optional)
  • exec: The path to the Zonalizer collector (zonalizer-collector). (optional)
  • config: Path to the Zonemaster configuration file to use. (optional)
  • policy: Path to the Zonemaster policy file to use.
  • sourceaddr: Local IP address that the test engine should try to send its requests from. (optional)
  • threads: Number of threads to start. (optional)

Example configuration:

---
zonalizer:
  collector:
    policies:
      - name: iana
        display: IANA Policy
        description: A policy specific for TLDs
        policy: /usr/share/perl5/auto/share/dist/Zonemaster/iana-profile.json

Configuration example with defaults

---
zonalizer:
  base_url: 1
  db_driver: Memory
  default_limit: 10
  max_limit: 10
  allow_ipv4: 1
  allow_ipv6: 1
  test_ipv4: 1
  test_ipv6: 1
  max_ongoing: 5
  allow_undelegated: 1
  force_undelegated: 0
  max_undelegated_ns: 10
  max_undelegated_ds: 20
  allow_meta_data: 0
  max_meta_data_entries: 42
  max_meta_data_entry_size: 512
  collector:
    exec: zonalizer-collector
    threads: 5

Spaces

A space is used as a seperation of analysis and is given as an optional option to most calls, see Calls.

If given, calls will only create or return analysis for that corresponding space.

If not given, calls will use the default configurable space or the "null" space.

The id of an analyze is unique within the corresponding space.

The "null" space is a reference to an unset space which, depending on the database backend, can be null, undef, empty string or otherwise an unset variable.

As an example; To separate a web app from a batch script, the web app may set the space to "web" while the batch script sets it to "batch". In this way they will not pollute each other spaces.

Undelegated Analyzing

Undelegated analyzing are done by manually giving the nameserver (see ns object) and delegation signer (optional, see ds object) information that will override DNS information looked up during analyzing.

Example senario taken from the Zonemaster project:

An undelegated domain test is a test performed on a domain that may, or may not, be fully published in the DNS. This can be quite useful if one is going to move one's domain from one registrar to another. For example, if you want to move your zone example.com from the nameserver "ns.example.com" to the nameserver "ns.example.org". In this scenario one could perform an undelegated domain test providing the zone (example.com) and the nameserver you are moving to (ns.example.org) BEFORE you move your domain. When the results of the test are colour coded in green one can be fairly certain that the domain's new location is supposed to be replying to queries . However there might still be other problems in the zone data itself that this test is unaware of.

Calls

GET /zonalizer/1/version

Get the version of Zonalizer, Zonemaster and all Zonemaster test modules.

{
  "version": "string",
  "zonemaster": {
    "version": "string",
    "tests": [
      { "name": "string", "version": "string" },
      { "name": "string", "version": "string" },
      { "name": "string", "version": "string" },
      ...
    ]
  }
}

GET /zonalizer/1/status

Get status about API and analysis.

{
   "api" : {
      "requests" : 501,
      "errors" : 0
   },
   "analysis" : {
      "ongoing" : 0,
      "completed" : 5,
      "failed" : 0
   }
}
  • api.requests: The number of API requests processed, this includes any kind of API call.
  • api.errors: The number of API errors.
  • analysis.ongoing: Number of currently ongoing analysis.
  • analysis.completed: Number of completed analysis.
  • analysis.failed: Number of failed analysis.

GET /zonalizer/1/analysis[?ongoing=bool&results=bool&lang=string&search=string&space=string]

Get a list of all analysis that ongoing or in the database for Zonalizer. See analyze under Objects for description of the analyze object.

When showing ongoing analysis:

  • Following options are ignored: before, after, sort, direction and search.
  • Pagination does not work.
  • Sorting of analysis is fixed on updated in a descending order.

The following fields are sortable: fqdn, created and updated.

{
  "analysis": [
    analyze,
    analyze,
    ...
  ],
  "paging": ...
}
  • ongoing: If true (1), show only ongoing analysis. Default false (0).
  • results: If true (1), include results in the analyze objects in the response. Default false (0).
  • lang: Specify the language (cc_CC) to use when generating the message in the result object and in the error object, default en_US.
  • search: A string with the "FQDN" to search/filter on. See Search for more information.
  • space: A string that identifies a unique space for analysis, see Spaces for more information. (optional)

DELETE /zonalizer/1/analysis[?space=string]

Delete all analysis. Returns HTTP Status 2xx on success and 4xx/5xx on error.

  • space: A string that identifies a unique space for analysis, see Spaces for more information. (optional)

POST /zonalizer/1/analysis?fqdn=string[&options...]

Initiate a new analysis for a given zone. See analyze under Objects for description of the analyze object.

  • fqdn: A string with the FQDN to analyze. (required)
  • policy: The policy to use for the analyze. (optional)
  • ipv4: If true (1), run the analysis over IPv4. If false (0), do not run the analysis over IPv4. (optional)
  • ipv6: If true (1), run the analysis over IPv6. If false (0), do not run the analysis over IPv6. (optional)
  • space: A string that identifies a unique space for analysis, see Spaces for more information. (optional)
  • ns: An array of nameserver objects, see ns object and Undelegated Analyzing for more information.
  • ds: An array of delegation signer objects, see ds object and Undelegated Analyzing for more information.
  • meta_data: An array of meta data objects, see meta_data object for more information.

GET /zonalizer/1/analysis/:id[?results=bool&lang=string&space=string]

Get information about an analyze. See analyze under Objects for description of the analyze object.

  • id: The ID of the analyze.
  • results: If true (1), include results in the analyze objects in the response. Default true (1).
  • lang: Specify the language (cc_CC) to use when generating the message in the result object and in the error object, default en_US.
  • space: A string that identifies a unique space for analysis, see Spaces for more information. (optional)

GET /zonalizer/1/analysis/:id[?last_results=bool&lang=string&space=string]

Get information about an analyze and include a set of the last results. See analyze under Objects for description of the analyze object.

  • id: The ID of the analyze.
  • last_results: An integer with the number of results to include.
  • lang: Specify the language (cc_CC) to use when generating the message in the result object and in the error object, default en_US.
  • space: A string that identifies a unique space for analysis, see Spaces for more information. (optional)

GET /zonalizer/1/analysis/:id/status[&space=string]

Only get status information about an analyze, this call is optimal for polling.

  • id: The ID of the analyze.
  • space: A string that identifies a unique space for analysis, see Spaces for more information. (optional)

Returns the following:

{
  "status": "string",
  "progress": integer,
  "updated": datetime
}
  • status: The status of the check, see Check Statuses.
  • progress: The progress of the check as an integer with the percent of completion.

DELETE /zonalizer/1/analysis/:id[&space=string]

Delete an analyze. Returns HTTP Status 2xx on success and 4xx/5xx on error.

  • id: The ID of the analyze.
  • space: A string that identifies a unique space for analysis, see Spaces for more information. (optional)

GET /zonalizer/1/policies

Get a list of all policies that are configured. See policy under Objects for description of the policy object.

{
  "policies": [
    policy,
    policy,
    policy,
    ...
  ]
}

GET /zonalizer/1/policy/:name

Get information about a policy. See policy under Objects for description of the policy object.

  • name: The name of the policy.

Search

The search string work in two different ways.

Search by FQDN

If the string is a FQDN (example.com, com.) then the list of analysis returned will only be for that FQDN.

Search by domain

If the string is a FQDN but includes a leading dot (.example.com, .com.) then the list of analysis returned will be for that FQDN and any other analysis that includes the domain.

As an example; .com will return all FQDNs that ends with .com including com. itself.

Objects

analyze

The main analyze object which may include all results from Zonemaster.

{
  "id": "uuid",
  "fqdn": "string",
  "policy": policy,
  "url": "href",
  "status": "string",
  "error": error,
  "progress": integer,
  "created": datetime,
  "updated": datetime,
  "results": [
    result,
    result,
    ...
  ],
  "summary": {
    "notice": integer,
    "warning": integer,
    "error": integer,
    "critical": integer
  },
  "ipv4": integer,
  "ipv6": integer,
  "ns": [
    ns,
    ns,
    ...
  ],
  "ds": [
    ds,
    ds,
    ...
  ],
  "meta_data": [
    meta_data,
    meta_data,
    ...
  ]
}
  • id: The UUID of the analyze.
  • fqdn: The FQDN of the analyze.
  • policy: The policy used for the analyze, see policy under Objects. (optional)
  • url: The URL to this object.
  • status: The status of the check, see Check Statuses.
  • error: An object describing an error, see error under Objects. (optional)
  • progress: The progress of the check as an integer with the percent of completion.
  • created: The date and time of when the object was created.
  • updated: The date and time of when the object was last updated.
  • results: An array containing result objects. (optional)
  • summary.notice: The number of NOTICE results in results.
  • summary.warning: The number of WARNING results in results.
  • summary.error: The number of ERROR results in results.
  • summary.critical: The number of CRITICAL results in results.
  • ipv4: If true (1), the analysis ran over IPv4.
  • ipv6: If true (1), the analysis ran over IPv6.
  • ns: An array of nameserver objects, see ns object and Undelegated Analyzing for more information. (optional)
  • ds: An array of delegation signer objects, see ds object and Undelegated Analyzing for more information. (optional)
  • meta_data: An array of meta data objects, see meta_data under Objects. (optional)

error

An object describing an error.

{
  "code": "string",
  "message": "string"
}
  • code: A string with the error code, see Analyze Errors.
  • message: A textual description of the error.

result

A result object which is taken unprocessed from Zonemaster, description here may vary depending on the version of Zonemaster you are running.

This documentation corresponds to version 1.0.7 of Zonemaster.

{
  "_id": integer,
  "args": {
    ...
  },
  "level": "string",
  "module": "string",
  "tag": "string",
  "timestamp": float,
  "message": "string"
}
  • _id: A basic counter for each result object in the set, starts at zero (0). This is an additional paramter which is added by Zonalizer.
  • args: An object with the arguments used for the specific result.
  • level: The serverity of the result, see Result Levels.
  • module: The Zonemaster module that produced the result.
  • tag: A describing tag of the result, this is used by Zonemaster to generate the message.
  • timestamp: A timestamp for when the result was generated, this is a float value of the number of seconds since the start of the analysis.
  • message: A describing message of the result.

ns

A nameserver object that is used for undelegated analysis.

{
  "fqdn": "string",
  "ip": "string"
}
  • fqdn: The nameserver's FQDN.
  • ip: An IP-address to use instead of doing a lookup of the nameserver's FQDN. (optional)

ds

A delegation signer records that is used for undelegated analysis. The four pieces of data should be in the same format they would have in a zone file.

For a description of the object properties below please see section 5 in RFC 4034.

{
  "keytag": "string",
  "algorithm": "string",
  "type": "string",
  "digest": "string"
}

meta_data

A meta data object. The number of object that can exist per analyze is determend by max_meta_data_entries and the total length of the key and value together is determend by max_meta_data_entry_size.

{
  "key": "string",
  "value": "string"
}
  • key: A string with the key of the meta data.
  • value: A string with the value of the meta data.

policy

A policy object that can describe what the policy is for.

{
  "name": "string",
  "display": "string",
  "description": "string"
}
  • name: An unique name of the policy.
  • display: A display friendly name of the policy.
  • description: A description what the policy would be used for. (optional)

Analyze Statuses

  • reserved: indicates that the analyze has been reserved in the database and is ongoing.
  • queued: indicates that the analyze has been queued and waiting on a worker to start processing it.
  • analyzing: indicates that the analyze has been taken up by a worker and its processing it.
  • done: indicates that the analyze is done and results are available.
  • failed: indicates that the analyze failed, check error and results for an error why it failed.
  • stopped: indicates that the analyze was stopped, check error and results for an error why it was stopped.
  • unknown: indicates that the analyze may not have any results.

Result Levels

The following result levels can be given by Zonemaster, please see Zonemaster documentation for more details.

  • DEBUG3
  • DEBUG2
  • DEBUG
  • INFO
  • NOTICE
  • WARNING
  • ERROR
  • CRITICAL

Errors

Errors that are related to the communication with the API are returned as JSON in a Lim::Error format and other errors which are related to the processing of analysis are set in the error object.

For example this is a internal server error (500):

{
  "Lim::Error" : {
    "module" : "Lim::Plugin::Zonalizer::Server",
    "code" : 500,
    "message" : null
  }
}

API Errors

These errors are returned as a string in the message value or in logs.

duplicate_id_found

A duplicated id was found.

id_not_found

The requested id was not found.

revision_missmatch

The revision of the object missmatched, the object was most likely updated out of scope.

invalid_limit

An invalid limit was supplied, limit may not be less then 0 and more the max_limit.

invalid_sort_field

An invalid field was supplied in the sort parameter.

internal_database_error

An internal database error, see logs for more information.

invalid_after

An invalid after parameter was supplied.

invalid_before

An invalid before parameter was supplied.

invalid_fqdn

An invalid fqdn or search parameter was supplied.

queue_full

The queue is full so the request has been dropped.

invalid_lang

An invalid lang parameter was supplied.

ipv4_not_allowed

The requested analysis over IPv4 is not allowed.

ipv6_not_allowed

The requested analysis over IPv6 is not allowed.

no_ip_protocol

The requested analysis has both IPv4 and IPv6 disabled, one must be enabled.

invalid_ns

Any of all of the ns objects supplied are invalid or too many.

invalid_ds

Any of all of the ds objects supplied are invalid or too many.

undelegated_not_allowed

An undelegated analysis was requested but is not allowed.

undelegated_forced

An analysis was requested without delegation information which must be supplied.

meta_data_not_allowed

Meta data was supplied but is not allowed.

invalid_meta_data

Meta data supplied is invalid.

policy_not_found

The requested policy was not found.

invalid_api_version

The API version given is invalid.

HTTP Errors

These are the HTTP status errors returned, additional errors may be returned from the framework.

400 BAD REQUEST

Indicates that some fields in the request are invalid. See message for the corresponding API error.

404 NOT FOUND

Indicates that the requested id was not found, see message for the corresponding API error.

409 CONFLICT

Indicates that an id conflict happen when trying to create database objects, this is a temporarly error and the request can be retried.

415 UNSUPPORTED MEDIA TYPE

Indicates that requested media type or languages are unsupported, see message for the corresponding API error.

500 INTERNAL SERVER ERROR

Indicates that an internal error occurred, more detailed information can be found in the logs.

This error also occurs when the framework's input and output data validation checks fail, see logs for detailed information.

501 NOT IMPLEMENTED

Indicates that the call used has not been implemented yet.

503 SERVICE UNAVAILABLE

Indicates that the service is temporarly unavailable, see message for the corresponding API error.

Analyze Errors

TODO

LICENSE AND COPYRIGHT

Copyright 2015-2016 Jerry Lundström

Copyright 2015-2016 IIS (The Internet Foundation in Sweden)

This program is free software; you can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation; or the Artistic License.

See http://dev.perl.org/licenses/ for more information.

You can’t perform that action at this time.