Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

\n in queries are not parsed #85

Closed
drozdo opened this Issue · 9 comments

3 participants

@drozdo

It looks like queries with '\n' in them are not parsed properly. I get only "SELECT something;" as results instead of full queries. I`m using custom prefix - '%t %u@%h[%d]:'

@darold
Owner

Hi Piotr,

Could you post some lines of your log ? Even if there's no session in the prefix (%l) it should works, please post some lines.

@darold
Owner

Have you solved the issue by yourself ? Please report the solution you have found so that other people could learn from your report.

Regards,

@darold darold closed this
@theoden-dd

Confirm the problem. I use stderr format and standard prefix '%t [%p]: [%l-1] '. Attached log-file that cause pgBadger to produce select queries with 13 FROM sections in report. See slowest(N) for example.

Oh... Actually, how can I send a log-file for you to analyze?

Here are some example lines from it: (lines like first are ended with 0x0A[line feed],0x09[tab on next line] symbols in this case)

2013-06-20 04:20:42 MSK [2843]: [3-1] LOG:  duration: 22265.157 ms  statement: SELECT count(*) AS count_1 
    FROM (SELECT "DomainsActions"."Id" AS "DomainsActions_Id", "DomainsActions"."AccountId" AS "DomainsActions_AccountId", "DomainsActions"."ActionTypeId" AS "DomainsActions_ActionTypeId", "DomainsActions"."IdForUser" AS "DomainsActions_IdForUser", "DomainsActions"."CreationDate" AS "DomainsActions_CreationDate", "DomainsActions"."StartDate" AS "DomainsActions_StartDate", "DomainsActions"."CompletionDate" AS "DomainsActions_CompletionDate", "DomainsActions"."DeletionDate" AS "DomainsActions_DeletionDate", "DomainsActions"."RealMoneyPaymentDate" AS "DomainsActions_RealMoneyPaymentDate", "DomainsActions"."ResellerRealMoneyPaymentDate" AS "DomainsActions_ResellerRealMoneyPaymentDate", "DomainsActions"."Status" AS "DomainsActions_Status", "DomainsActions"."SyncHandleType" AS "DomainsActions_SyncHandleType", "DomainsActions"."ActualPayment" AS "DomainsActions_ActualPayment", "DomainsActions"."ResellerActualPayment" AS "DomainsActions_ResellerActualPayment", "DomainsActions"."ActionPrice" AS "DomainsActions_ActionPrice", "DomainsActions"."ResellerActionPrice" AS "DomainsActions_ResellerActionPrice", "DomainsActions"."BillingPersonId" AS "DomainsActions_BillingPersonId", "DomainsActions"."ManualProcessing" AS "DomainsActions_ManualProcessing", "DomainsActions"."ExecutorId" AS "DomainsActions_ExecutorId", "DomainsActions"."ActionPriceAccounting" AS "DomainsActions_ActionPriceAccounting", "DomainsActions"."ResellerActionPriceAccounting" AS "DomainsActions_ResellerActionPriceAccounting", "DomainsActions"."ActionWithdrawals" AS "DomainsActions_ActionWithdrawals", "DomainsActions"."ResellerActionWithdrawals" AS "DomainsActions_ResellerActionWithdrawals", "DomainsActions"."ActionCreditWithdrawals" AS "DomainsActions_ActionCreditWithdrawals", "DomainsActions"."ResellerActionCreditWithdrawals" AS "DomainsActions_ResellerActionCreditWithdrawals", "DomainsActions"."ActionBonusWithdrawals" AS "DomainsActions_ActionBonusWithdrawals", "DomainsActions"."ResellerActionBonusWithdrawals" AS "DomainsActions_ResellerActionBonusWithdrawals", "DomainsActions"."Data" AS "DomainsActions_Data", "DomainsActions"."ServiceProviderId" AS "DomainsActions_ServiceProviderId", "DomainsActions"."BillingCustomerPaymentStage" AS "DomainsActions_BillingCustomerPaymentStage", "DomainsActions"."BillingResellerPaymentStage" AS "DomainsActions_BillingResellerPaymentStage", "DomainsActions".cached_comission_value AS "DomainsActions_cached_comission_value" 
    FROM "DomainsActions" 
    WHERE "DomainsActions"."Status" = 'completed' ORDER BY "DomainsActions"."Id" DESC) AS anon_1
2013-06-20 04:20:42 MSK [2770]: [3-1] LOG:  duration: 87830.622 ms  statement: SELECT count(*) AS count_1 
    FROM (SELECT "DomainsActions"."Id" AS "DomainsActions_Id", "DomainsActions"."AccountId" AS "DomainsActions_AccountId", "DomainsActions"."ActionTypeId" AS "DomainsActions_ActionTypeId", "DomainsActions"."IdForUser" AS "DomainsActions_IdForUser", "DomainsActions"."CreationDate" AS "DomainsActions_CreationDate", "DomainsActions"."StartDate" AS "DomainsActions_StartDate", "DomainsActions"."CompletionDate" AS "DomainsActions_CompletionDate", "DomainsActions"."DeletionDate" AS "DomainsActions_DeletionDate", "DomainsActions"."RealMoneyPaymentDate" AS "DomainsActions_RealMoneyPaymentDate", "DomainsActions"."ResellerRealMoneyPaymentDate" AS "DomainsActions_ResellerRealMoneyPaymentDate", "DomainsActions"."Status" AS "DomainsActions_Status", "DomainsActions"."SyncHandleType" AS "DomainsActions_SyncHandleType", "DomainsActions"."ActualPayment" AS "DomainsActions_ActualPayment", "DomainsActions"."ResellerActualPayment" AS "DomainsActions_ResellerActualPayment", "DomainsActions"."ActionPrice" AS "DomainsActions_ActionPrice", "DomainsActions"."ResellerActionPrice" AS "DomainsActions_ResellerActionPrice", "DomainsActions"."BillingPersonId" AS "DomainsActions_BillingPersonId", "DomainsActions"."ManualProcessing" AS "DomainsActions_ManualProcessing", "DomainsActions"."ExecutorId" AS "DomainsActions_ExecutorId", "DomainsActions"."ActionPriceAccounting" AS "DomainsActions_ActionPriceAccounting", "DomainsActions"."ResellerActionPriceAccounting" AS "DomainsActions_ResellerActionPriceAccounting", "DomainsActions"."ActionWithdrawals" AS "DomainsActions_ActionWithdrawals", "DomainsActions"."ResellerActionWithdrawals" AS "DomainsActions_ResellerActionWithdrawals", "DomainsActions"."ActionCreditWithdrawals" AS "DomainsActions_ActionCreditWithdrawals", "DomainsActions"."ResellerActionCreditWithdrawals" AS "DomainsActions_ResellerActionCreditWithdrawals", "DomainsActions"."ActionBonusWithdrawals" AS "DomainsActions_ActionBonusWithdrawals", "DomainsActions"."ResellerActionBonusWithdrawals" AS "DomainsActions_ResellerActionBonusWithdrawals", "DomainsActions"."Data" AS "DomainsActions_Data", "DomainsActions"."ServiceProviderId" AS "DomainsActions_ServiceProviderId", "DomainsActions"."BillingCustomerPaymentStage" AS "DomainsActions_BillingCustomerPaymentStage", "DomainsActions"."BillingResellerPaymentStage" AS "DomainsActions_BillingResellerPaymentStage", "DomainsActions".cached_comission_value AS "DomainsActions_cached_comission_value" 
    FROM "DomainsActions" 
    WHERE "DomainsActions"."Status" = 'completed' ORDER BY "DomainsActions"."Id" DESC) AS anon_1 
@darold
Owner

As you can see in the report below, using you log above I'm not able to reproduce the issue.

capture

Could you try last release and if it is still showing wrong report please send your log file to my personal address for privacy gilles AT darold DOT net

Regards,

@theoden-dd

This is the screenshot of the report that I had got with pgBadger 3.4:
snimok

Have you seen? Lots of FROM sections. OS is Ubuntu Server 12.04.1 and postgres is 9.1 if it matters. I have sent the log file to you, Darold.

@darold
Owner

Hi Denis,

Ok, with your log file I was able to reproduce the issue. I think it is fixed in last commit ac81a1f with orphan lines wrongly associated to previous queries instead of temporary file and lock logged statements. Please give it a try and report any issue.

Thanks for your help.

@theoden-dd

Thanks for your responsiveness, Darold,

The upper mentioned bug was eliminated. For my usecase at least.

However one more strange kind of select arised - the select without FROM section (again, "Slowest query (N)" part):
snimok

This report was generated at 80b9676 commit with another log-file I have sent to you. I had skimmed over that log-file and didn't see something like such a trimmed query.

Suppose this is also caused by line feeds, that's why I wrote it here. If it is better to start a new issue, just say.

@darold
Owner

Thanks for sending your log file it help a lot. Please give a try to last development code, commit d39b455 should definitively fix the issue, let me know.

Regards

@theoden-dd

You are welcome.

At moment I cannot confirm the bug any more with my postgres logs.

Thank you for great software. Best wishes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.