README.md

Wrapper generation for API routes

wrapper_table.json

The wrapper_table.json file is the canonical list of all API routes (for which wrappers need to be generated). The file contains a JSON array, all of whose elements are arrays themselves. This is what it looks like:

[
  [
    "/file-xxxx/upload",
    "fileUpload(req, objectId)",
    {
      "objectMethod": true,
      "retryable": true,
      "wikiLink": null
    }
  ],
  [
    "/file/new",
    "fileNew(req)",
    {
      "objectMethod": false,
      "retryable": false,
      "wikiLink": "https://documentation.dnanexus.com/developer/api/introduction-to-data-object-classes/files#api-method-file-new",
      "acceptsNonce": true
    }
  ],
  ...,
]

Each route is described by an array with exactly 3 elements:

1. The first element is a string giving the URL of the route (relative to apiserver).
2. The second element is a string representing how a route might be represented as a function in a dynamically typed language (its utility is limited, and it can be generated by using 1st and 3rd element of the array; it is present for historical reasons).
3. The third element is a hash with the following fields:
   • objectMethod: boolean; true for routes that are called on a specific object ID (e.g. /record-xxxx/describe), and false for routes that are not bound to a specific object ID (e.g. /system/findJobs, /record/new).
   • retryable: boolean; true if the route is "safe" to retry for failed requests. This information is used by the API wrappers to decide when to retry a failed request.
   • wikiLink: either null or a string; if provided, contains a URL for documentation for the route. This information can be used to add wiki links in api wrapper documentation.
   • acceptsNonce: boolean; (optional) if present, the route accepts a nonce that is used to uniquely identify a request.

Adding wrappers for a new language

By convention we create a file generateXXXXAPIWrappers.py in this directory for each supported language. This is a Python script which reads wrapper_table.json from stdin, and produces a language-specific wrapper file on stdout. There are also some unsupported wrappers for additional languages in the contrib directory of dx-toolkit.

To add wrappers for your favorite language (say, Ruby):

• Implement a function for making a single HTTP request to a DNAnexus API server. (You will call this function below in the implementation of each route's wrapper.) Add this and whatever other common bindings code are needed into a new directory, say, src/ruby.
  • See below for conventions regarding when HTTP requests are retried.
  • TODO: document where configuration such as the user's security context is conventionally stored.
• Create a file generateRubyAPIWrappers.py in the current directory (src/api_wrappers). This file should read a description of the routes on stdin (see wrapper_table.json), and produce a .rb file (with the relevant function definitions) on stdout. Look at src/api_wrappers/generatePythonAPIWrappers.py for an example.
• Add a new make target in src/Makefile to build the wrappers, and make the api_wrappers target depend on it. Refer to the existing examples in the Makefile.

HTTP Retry logic

An HTTP request to the API server should be retried (up to some fixed number of retries) if any of the following are true:

• A response is received from the server, and the response has an HTTP status code in 5xx range.
  • This may indicate that the server encountered a transient error.
  • If the status code is 503 (Service Unavailable) and Retry-After is set, the failure should not count against the maximum allowed number of retries.
• safe_to_retry (caller-supplied parameter; for compatibility reasons this is called always_retry in some language bindings) is True, or the request method is "GET"; and one of the following is true:
  • No response is received from the server. (For example, this may manifest in the requests library as BadStatusLine.)
  • A response is received from the server, and the content length received does not match the "Content-Length" header (possibly because the connection has been dropped). (For example, this may manifest in the requests library as ContentLengthError or ProtocolError.)
    • This indicates that the response was likely corrupted (truncated).
  • A response is received from the server, the "Content-Length" header is not set, and the response JSON cannot be parsed.
    • This is a mechanism that allows for the server to indicate a transient error encountered during a streaming response (after the headers have been sent), simply by halting output.
  • The HTTP client library signals an unhandled exception or other protocol error.
• The request method is "PUT", and one of the following is true:
  • The HTTP status code is 400, and S3 returns a RequestTimeout error. (See: Amazon S3 error codes)
  • The HTTP client library signals an unhandled exception or other protocol error.
• It is certain that the request was never received by the server (some clients may not be able to determine whether this was the case).

If a download request (i.e. against a URL returned by /file-xxxx/download) fails and is retryable under the conditions above, the client may also consider splitting the byte range into smaller byte ranges and issuing separate requests for each (with each subrange itself being retryable using the logic specified above).