Added new Debugging API

[buger]

Added new Tracing API, which allows you to debug API definition, by sending sample request to it.

As input, Tracing API require sample request object, and direct API definition.
This means that tracing API can be used even without creating API. Or for example, you can test API changes without saving them.

Example request:

POST /tyk/trace
{
    "request": {
        "method": "GET",
        "path": "/get",
        "headers": {
            "Header": ["Value"]
        }
    },
    "spec": {
        ...valid API definition...
    }
}

As output, it gives you response object, and gateway logs as JSON multi-line string.

Example output:

{
    "message": "ok"
    "response": "<raw-http-response-dump>",
    "logs": "{...}\n{...}"
}

Logging itself now way more structured, and prepared for easy parsing. Based on the logs you can see:

• API loading process: which middleware was loaded or not, issues during parsing and etc.
• Middleware logs: order of running, elapsed time, individual MW logs

Example of log output:

{"level":"debug","msg":"Default JSVM timeout used: 5s","prefix":"jsvm","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"info","msg":"Loading API","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Initializing API","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"RateCheckMW","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"VersionCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"info","msg":"Checking security policy: Token","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"AuthKey","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"KeyExpired","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"AccessRightsCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"RateLimitAndQuotaCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"GranularAccessMiddleware","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"VersionCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"KeyExpired","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Init","mw":"AccessRightsCheck","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Rate limit endpoint is: /sampletyk/rate-limits/","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Setting Listen Path: /sample","org_id":"","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"info","msg":"API Loaded","org_id":"","prefix":"gateway","server_name":"--","time":"2018-09-05T17:49:19+03:00","user_id":"--","user_ip":"--"}
{"api_id":"test","api_name":"","level":"debug","msg":"Started","mw":"RateCheckMW","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959332155740}
{"api_id":"test","api_name":"","code":200,"level":"debug","msg":"Finished","mw":"RateCheckMW","ns":27925,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Started","mw":"VersionCheck","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959332214289}
{"api_id":"test","api_name":"","code":200,"level":"debug","msg":"Finished","mw":"VersionCheck","ns":23042,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Started","mw":"AuthKey","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959332264989}
{"api_id":"test","api_name":"","level":"debug","msg":"Querying local cache","mw":"AuthKey","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Querying keystore","mw":"AuthKey","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","level":"debug","msg":"Got key","mw":"AuthKey","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","code":200,"level":"debug","msg":"Finished","mw":"AuthKey","ns":1149266,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","key":"****ecf1","level":"debug","msg":"Started","mw":"KeyExpired","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959333468462}
{"api_id":"test","api_name":"","code":200,"key":"****ecf1","level":"debug","msg":"Finished","mw":"KeyExpired","ns":23915,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","key":"****ecf1","level":"debug","msg":"Started","mw":"AccessRightsCheck","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959333521672}
{"api_id":"test","api_name":"","code":200,"key":"****ecf1","level":"debug","msg":"Finished","mw":"AccessRightsCheck","ns":18849,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","key":"****ecf1","level":"debug","msg":"Started","mw":"RateLimitAndQuotaCheck","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959333632612}
{"api_id":"test","api_name":"","code":200,"key":"****ecf1","level":"debug","msg":"Finished","mw":"RateLimitAndQuotaCheck","ns":42329,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}
{"api_id":"test","api_name":"","key":"****ecf1","level":"debug","msg":"Started","mw":"GranularAccessMiddleware","org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00","ts":1536158959333706674}
{"api_id":"test","api_name":"","code":200,"key":"****ecf1","level":"debug","msg":"Finished","mw":"GranularAccessMiddleware","ns":17514,"org_id":"","origin":"192.0.2.1","path":"/","time":"2018-09-05T17:49:19+03:00"}

Such detailed logs, accessed progmatically, allow some exciting cases like debugging VirtualEndpoint code, and seeing all JS parsing issues, or JS logging data, directly from Dashboard UI.

This API also allow you to test authentification as well. To do so, you just need to create key for the API, or JWT token, and provide authentification header to tracing API. So you can test full API cycle, include access rights and etc.

If your API is protected, and you do not provide token to Tracing API, you will get unauthorized error.

[dencoded]

implementation looks OK to except name of endpoint - I am thinking /trace vs. /debug. I feels like it returns more debug like info (what would appear in log if we do a real API call).