API Builder Not Working

[NNewar]

Axway APIM-ELK version
I have setup the environment using the docker-compose image provided in the RC-1.0.0 version

API-Manager Version
APIM - 7.6.2

Question
I am unable to fetch the information through the API Builder, however, I can see my logs inside the Kibana Dashboard. I have also used the Elastic Search APIs just to confirm if the transaction ID information is available or not.

Also, I am unable to access the API Builder Console. I am getting 404 Not Found Error.

What I've tried so far
API Builder Console Error -
I have tried allowed a list of IP addresses and disabled the allowed host configuration inside the conf/default.js, still unable to access the API Builder console.

API Builder APIs -
I have tried to change the host name of the Elasticsearch inside the "conf/elasticsearch.js" file.

Additional information
I have tried sending the request from Postman and even setup the API Node Manager configuration to call the API Builder APIs. I am still unable to see the information.

[cwiechmann]

Hey @NNewar, Is the API-Builder Docker-Container running? What is shown, when you make a docker logs on this container?

FYI:

• The API-Builder console is not part of the generated Docker-Image
• The events are processed anyway by Logstash, but some additional Lookup-Information are then not attached

I made a fix, which will make sure, that this message will appear in the Traffic-Monitor if the Lookup by Logstash fails:

[NNewar]

Hey @cwiechmann,

The API Builder Docker-Container is running:

And, I can access the API Builder through the browser as well:

I have checked the logs for the docker container and I can see that the requests are coming to the API Builder:

APIBuilder-Docker.log

As for the Lookup feature you have mentioned, I believed it was added today. I will take the latest build and will use it.

[cwiechmann]

So, you mean the API-Gateway Traffic-Monitor is not showing up the requests?

[NNewar]

Yes, Traffic-Monitor is not showing up the requests

[cwiechmann]

Okay. The most important index for that information to come up is the apigw-traffic-summary-* index.

Question: Have you deleted the index-templates and indices that might have exists before in Elasticsearch?

I recommend to enable debug for API-Builder which you can do in the docker-compose file (there is a comment).
After you have restarted, best is to filter for a known Transaction-ID to reduce the overhead and refresh the Traffic-Screen. Then you see what API-Builder is doing. Have a look for these lines:

Query Elasticsearch (elasticsearch.1) route: ["javascript.1"]
search (elasticsearch.1) Using elastic search query body: {"index":"apigw-traffic-summary-*","body":{"query":{"bool":{"must":[{"match":{"correlationId":"e2ff615f85043eab35b77de1"}},{"term":{"processInfo.serviceId":"instance-1"}},{"exists":{"field":"http"}}]}}},"size":"1000","sort":"@timestamp:desc"}
Query Elasticsearch (elasticsearch.1) route: ["javascript.1"]
Query Elasticsearch (elasticsearch.1) writing {"body":{"took":13,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":null,"hits":[{"_index":"apigw-traffic-summary-000001","_type":"_doc","_id":"e2ff615f85043eab35b77de1","_score":null,"_source":{"input":{"type":"log"},"finalStatus":"Fail","processInfo":{"serviceId":"instance-1","groupId":"group-2","gatewayName":"API-Gateway","hostname":"api-env","gatewayRegion":"N/A","groupName":"QuickStart Group","version":"7.7.20200730"},"@timestamp":"2020-09-16T12:07:28.204Z","logtype":"openlog","@version":"1","type":"summaryIndex","http":{"remoteName":"192.168.65.1","uri":"/api/v1/airports/fds","statusText":"Internal Server Error","localAddr":"192.168.65.133","status":500,"bytesReceived":368,"bytesSent":587,"localPort":"8065","wafStatus":0,"method":"GET","remotePort":"53823","remoteAddr":"192.168.65.1","authSubjectId":"Pass Through"},"correlationId":"e2ff615f85043eab35b77de1","duration":30061,"protocol":"https","protocolSrc":"8065","serviceContext":{"method":"get-code","monitor":true,"routingPolicy":"Custom routing policy","apiOrg":"API Development","responsePolicy":"Custom response policy","apiVersion":"1.0.1","service":"Airports API","apiState":"unpublished","apiDeprecated":"false","apiSecurity":"Pass Through","requestPolicy":"Custom request policy","backendBasePath":"http://192.168.65.1:8080","faulthandlerPolicy":"N/A","status":"failure"},"status":"failure"},"sort":[1600258048204]}]}},"statusCode":200,"headers":{"content-type":"application/json; charset=UTF-8","content-length":"1474","access-control-allow-credentials":"true"},"warnings":null,"meta":{"context":null,"request":{"params":{"method":"POST","path":"/apigw-traffic-summary-*/_search","body":"{\"query\":{\"bool\":{\"must\":[{\"match\":{\"correlationId\":\"e2ff615f85043eab35b77de1\"}},{\"term\":{\"processInfo.serviceId\":\"instance-1\"}},{\"exists\":{\"field\":\"http\"}}]}}}","querystring":"size=1000&sort=%40timestamp%3Adesc","headers":{"user-agent":"elasticsearch-js/7.9.0 (linux 3.10.0-1127.18.2.el7.x86_64-x64; Node.js v12.18.3)","content-type":"application/json","content-length":"160"},"timeout":60000},"options":{"ignore":[404],"maxRetries":3,"warnings":null},"id":6},"name":"apibuilder4elastic","connection":{"url":"https://elasticsearch1:9200/","id":"https://elasticsearch1:9200/","headers":{},"deadCount":0,"resurrectTimeout":0,"_openRequests":0,"status":"alive","roles":{"master":true,"data":true,"ingest":true,"ml":false}},"attempts":0,"aborted":false}} to ctx as: $.elasticsearch.result

You can take the query body from the log line and run the query manually:

GET /apigw-traffic-summary-*/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "correlationId": "e2ff615f85043eab35b77de1"
          }
        },
        {
          "term": {
            "processInfo.serviceId": "instance-1"
          }
        },
        {
          "exists": {
            "field": "http"
          }
        }
      ]
    }
  }
}

If this query doesn't return anything, then you can remove one by one of the constraints.

[cwiechmann]

@NNewar, do you made any progress and still need help or can I close this issue?

[taranwar]

Hi @cwiechmann ,

Nischal is on leave today so he should be able to test and confirm tomorrow.

Thanks
Tarique

[NNewar]

Hello @cwiechmann,

I found the issue, why the API calls from Postman was not working. I set the API Builder logging level to debug by uncommenting the line inside the docker-compose file.

• After enabling it, I sent a request from Postman and found out that the API is expecting the Cookie header (which is provided in the API Gateway Manager requests).

• I copied the Cookie header value from my local API Gateway Manager and then added it to my Postman Call.

• Then I sent the request from my Postman and got the response:

I am now sure that the docker container is properly configured and API Builder is providing the information.

I will now add the custom Elasticsearch policy inside the ANM and will test the complete solution.

Thank you for your help.

[cwiechmann]

I'm going to close this issue now as the API-Builder is obviously working. Feel free to create another issue if encounter any other issues @NNewar.

FYI: I have released 1-0.0-RC2 with a number of changes/fixes.