application-gateway-logviewer-goaccess - 404 Error with no report.html created

[rskeel]

application-gateway-logviewer-goaccess - https://github.com/Azure/azure-quickstart-templates/tree/master/application-gateway-logviewer-goaccess
Issue Details

Ran the deployment twice both times the deployment completed succesfully and VM publically accessible with the index.html page displaying ok

Issue is the report.html seems to not be creating

SSH'd into the VM installed the "locate" function and unable to find report.html

all that is in the /var/www/html folder is index.html

Attempted to manually re-install the packages listed in Github by copy and paste and all completed (apart from the SAS tokens which complained) however the report.html file still isn't created

So just to confirm, the parameters were filled in and accepted and the template deployed without error however the report.html file is not being created hence viewing http://VMpublicDNSname/report.html a 404 is displayed

[rskeel]

Looks like the logs are being parsed correctly from Go Acess however the report.html is not being created

:$ sudo /usr/local/bin/goaccess /var/log/azure/Microsoft.Azure.Networking.ApplicationGateway.LogProcessor/access.log* -o /var/www/html/report.html --real-time-html --port=8080 --log-format='"%dT%tZ"{%^:"%h",%^:"%m",%^:"%U",%^:"%q",%^:"%u",%^:"%s",%^:"%H",%^:"%b",%^:"%T",%^:%v}' --time-format='%T' --date-format='%Y-%m-%d'
(output) /var/log/azure/Microsoft.Azure.Networking.ApplicationGateway.LogProcessor/access.log
:$ ls /var/www/html/
index.html

[rskeel]

Actually on 2nd check the access.log is empty and the application log is full of

Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.

Is there a way to re-introduce the SAS URI after the install? I have created new SAS Blob URI each time and simply used the copy and paste feature in the portal to add into the parameters

[rskeel]

Rebuilt with amended SAS URI and now getting a bit further

18-12-04 19:02:54,636 ERROR [5] ?.? - RefreshBlobs: Caught Exception: System.AggregateException: One or more errors occurred. (One of the query parameters specified in the request URI is not supported.) ---> Microsoft.WindowsAzure.Storage.StorageException: One of the query parameters specified in the request URI is not supported.
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteAsyncInternal[T](RESTCommand1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token) at Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer.ListBlobsSegmentedAsync(String prefix, Boolean useFlatBlobListing, BlobListingDetails blobListingDetails, Nullable1 maxResults, BlobContinuationToken currentToken, BlobRequestOptions options, OperationContext operationContext, CancellationToken cancellationToken)
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task1.GetResultCore(Boolean waitCompletionNotification) at Microsoft.Azure.Networking.ApplicationGateway.LogProcessor.LogBlobFetcher.RefreshBlobs(Object state, ElapsedEventArgs e) in E:\src\networkmonitoring\Tools\src\AppGatewayLogProcessor\Services\LogBlobFetcher.cs:line 282 ---> (Inner Exception #0) Microsoft.WindowsAzure.Storage.StorageException: One of the query parameters specified in the request URI is not supported. at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteAsyncInternal[T](RESTCommand1 cmd, IRetryPolicy policy, OperationContext operationContext, CancellationToken token)
at Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer.ListBlobsSegmentedAsync(String prefix, Boolean useFlatBlobListing, BlobListingDetails blobListingDetails, Nullable`1 maxResults, BlobContinuationToken currentToken, BlobRequestOptions options, OperationContext operationContext, CancellationToken cancellationToken)
Request Information
RequestID:b9b09c29-001e-0001-7a03-8cf00d000000
RequestDate:Tue, 04 Dec 2018 19:02:54 GMT
StatusMessage:One of the query parameters specified in the request URI is not supported.
ErrorCode:UnsupportedQueryParameter
ErrorMessage:One of the query parameters specified in the request URI is not supported.
RequestId:b9b09c29-001e-0001-7a03-8cf00d000000
Time:2018-12-04T19:02:54.6353843Z

[allinurl]

I'd recommend opening an issue here as the error seems to be azure related. If your access log is empty, goaccess won't display anything.

[rskeel]

The gateway access logs themselves are generating ok and I can view them from log analytics and also storage explorer however the logprocessor from with the the Apache VM is repating this message

ErrorMessage:One of the query parameters specified in the request URI is not supported.

I'm generating the Blob SAS URL and using the copy option from with the azure portal to paste it into the deployment parameter field

Options I'm selecting for the Shared Access Signature I'll attach via screenshot

[rskeel]

Screenshot of the logprocessor folder showing the empty access log and the increasing count for the application log and included error

[rskeel]

Also deployed demo suite in another subscription into a a different region but still seeing the same error

https://github.com/Azure/azure-quickstart-templates/tree/master/application-gateway-demo-setup

https://azure.microsoft.com/en-gb/blog/application-gateway-analytics-via-goaccess/

using all defaults apart from U & P and the Blob SAS URL

[allinurl]

Sorry I can't help you to debug this as I'm not familiar with azure templates. Did you post your question in the azure issues page?

[rskeel]

Hi thanks for looking into this I have logged it on the other forum but no traction as yet

[rskeel]

Azure/azure-quickstart-templates#5382

[surajmb]

Hi @rskeel the issue could be with the SAS URI. The one used in your case is created from the Storage account level. However, for GoAccess to access the logs, you might need to provide container-level SAS URI for "insights-logs-applicationgatewayaccesslog".

Please check the updated ReadMe document here for further instructions on SAS URI generation: https://github.com/Azure/azure-quickstart-templates/blob/master/application-gateway-logviewer-goaccess/README.md

Once done, please replace the URI with the existing one in the /usr/share/appgatewaylogprocessor/blobsasuri.key file and restart the goaccess and appgatewaylogprocessor services. Further instructions can be found here.